Please forgive this long post which touches many complex topics while trying to answer the title’s seemingly simple question.
I currently have the following as a testbed:
- An executable compiled from C++ source code that …
- Reads a /dev/hidrawN special file …
- Connected to a secondary USB PC (not a piano/synthesizer/MIDI) keyboard …
- That has been detached from X via
xinput disable <id> - The C++ translates the key-down and key-up hidraw packets into MIDI note-on and note-off messages …
- Which it sends using the RtMidi library via ALSA …
- To a running
fluidsynth --midi-driver=alsa_seq --audio-driver=alsaprocess … - Which outputs sound through the PC’s built-in “snd_hda_intel” “sound card”'s analog 3.5mm stereo audio jack …
- To a set of self-powered computer monitor speakers.
When running, the above seems subjectively to have too much latency. Apologies for overemphasizing it, but by “latency” I mean the absolute real-time, real world elapsed time between when the PC keyboard switch is pressed to when the speaker cone begins to move. I currently have no way of objectively measuring this, but believe I need to reduce it to below 1 or 2 milliseconds. Certainly less than 5. As per this post’s title, I’d like (on any particular PC hardware) to have the lowest delay possible – if it was 1 nanosecond that would be perfectly acceptable. 
I’ve read extensively trying to understand the problem and its possible solutions, but have been confused by finding much conflicting and outdated information. This includes statements like:
- openSUSE is not suitable for realtime/production audio; instead use a Linux distribution designed for the task such as KXStudio or Bandshed
- “a kernel compiled with realtime patches and configuration is required” vs “all modern linux kernels have good realtime capabilities”
- The kernel’s “Completely Fair Scheduler” and dynamic frequency changing make the CONFIG_HZ and/or CONFIG_NO_HZ kernel parameters meaningless.
I have tried changing the fluidsynth and keyboard software’s process priorities with nice/renice, ulimit, and chrt without noticing much difference. Note that at startup fluidsynth outputs:
fluidsynth: warning: Failed to set thread to high priority
fluidsynth: warning: Failed to set thread to high priority
fluidsynth: warning: Failed to pin the sample data to RAM; swapping is possible.
loaded SoundFont has ID 1
Using ulimit -l unlimited fixes the “swapping” warning, and chrt -r 50 <pid of fluidsynth> probably fixes the “thread to high priority” ones, but again neither seem to have much effect.
Side note: How is rtkitctl used? I have found no documentation on it besides the “man” page which merely states “–reset-known : Reset real, real-time status of known threads” and “–reset-all : Reset real-time status of all threads”. What threads? How is rtkitctl told what processes’ realtime priorities to change?
I assume that in my testbed both the keyboard and fluidsynth processes are normally idle – keyboard is waiting in a “read()” system call for data from /dev/hidrawN, and fluidsynth for input on an ALSA pipe or socket (I’m very unclear about the ALSA details).
The entire system is at runlevel 5 with an X desktop, so there are certainly other processes running – top never goes to 100% idle. I’ve looked at my Leap 15.1 install and found:
$ fgrep -i hz /boot/config-4.12.14-lp151.28.13-default
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
CONFIG_NO_HZ_IDLE=y
# CONFIG_NO_HZ_FULL is not set
CONFIG_NO_HZ=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
CONFIG_MACHZ_WDT=m
(Note that the output from zgrep -i hz /proc/config.gz is identical.)
Looking only at “CONFIG_HZ=250” and ignoring “CONFIG_NO_HZ=y” (which I don’t understand), it seems that the lower bound on my worst-case latency is 4 milliseconds – the kernel only interrupts the processes currently running on the system’s cores (assuming there more runnable ones than cores) every 4 ms to run itself. At those intervals it can check the USB hardware and driver(s), see that there is input, and run the keyboard process.
Is this correct? Also, is it true that these parameters can’t be changed in a running kernel (as I have read)? Can they be changed by editing the /boot/config* file and rebooting, or by editing the kernel modeline at boot time? Or does a new kernel need to be compiled?
Alternately, does the kernel wake up asynchronously on (in this example) a hardware USB interrupt, service it (run the driver module) and upon seeing it has data for me halt the lowest priority currently running process and immediately execute my keyboard process without waiting for the CONFIG_HZ timer to elapse?
I believe the latency is caused by kernel scheduling or something associated with it because I’ve tested using zynaddsubfx and amsynth instead of fluidsynth, and with a real MIDI hardware keyboard instead of my PC keyboard program, and the latency always feels about the same.
I’ve also tried using “jack” instead of plain ALSA as the MIDI API and again found no improvement, despite the claims that jack is targeted at low-latency and adds “absolutely zero” overhead (direct quote) even though it runs as a layer on top of ALSA. (At least it doesn’t claim to be faster.) My belief is that jack is designed for synchronizing multiple MIDI (and audio) streams, as is ALSA’s “alsa-seq” vs its “alsa-raw” API. (I know for a fact that alsa-seq embeds timestamps along with other metadata into the raw MIDI messages, which for my use case is just another source of overhead and potentially increased latency.)
I think that jack tries to achieve low latency by constantly streaming data across its connections, I assume to ensure that the processes sending and receiving that data are always tagged as runnable to the kernel scheduler. I know that I always saw the jackd daemon eating at least 5% of a core when running (but doing nothing). Again, please forgive my undeducated yet extremely opinionated belief that this is a Really Bad Idea/Architecture. (Current web browsers like Firefox and Chromium/Chrome work similarly, never going fully to sleep even when not in use. Pardon my disapproval.)
The only ideas I have are to set CONFIG_HZ_1000 (or higher if that’s possible) in order to wake up my keyboard process faster, and to chrt/rtkitctl both that and fluidsynth to SCHED_RR or even SCHED_FIFO. But do those mean an infinite loop bug in the code will lock up the whole system by preventing anything else from running (like a shell to do kill)?
Lots of open-ended questions here, so any discussion or answers welcome.