If you actually do the calculations assuming a USB dongle with a 16-byte buffer like the 16550, you will find that at 115200 bps, the CPU has 1.4 ms to collect the data before overrun. That is an eternity for a current CPU.
A 16 byte buffer doesn’t mean that the program won’t respond to a single byte. An interrupt will be generated when one byte comes along, but the CPU has 1.4 more ms to collect the data before the buffer runs out.
Does your USB mouse jitter when you have a disk transfer going on the USB? No, because it’s a true bus, with each device getting a chance to interrupt the CPU.