ykman sometimes doesn't detect my yubikey

[solomon-b]

• YubiKey Manager (ykman) version: 5.1.1
• How was it installed?: via the nix package manager
• Operating system and version: Nixos 23.05
• YubiKey model and version: 5C Nano
• Bug description summary: ykman usually works fine but intermittently does not detect the key unless I remove and re-insert it.

Steps to reproduce

This is an intermittent issue without an easy reproduction.

Expected result

I would expect the yubikey to be detected every time I use ykman.

Actual results and logs

➜ ykman oath accounts code xxxx
ERROR: Failed to connect to YubiKey.

Other info

[Anything else you would like to add?]