Without other usb devices the keyboard works immediately. With them it takes 5 ish seconds for it to start responding. This is regardless of which device is discovered first (which I can manipulate), it seems to just wait for all of them. Is there a way to make it not do that?

Edit: This is specifically about linux boot. It otherwise works fine in bios / bootloader