Could be hardware, driver or the medium.
As the problem is more pronounced connecting to 5 GHz it looks to me that a medium problem is likely.
Can you share the output of nmcli dev wifi list? Use the command I use below to filter out BSSID/SSID:
> nmcli dev wifi list | perl -pe "s/(.{8}).{38}/\$1/"
IN-USE MODE CHAN RATE SIGNAL BARS SECURITY
Infra 4 130 Mbit/s 100 ▂▄▆█ WPA2
Infra 11 270 Mbit/s 65 ▂▄▆_ WPA2
* Infra 1 130 Mbit/s 59 ▂▄▆_ WPA2
Infra 4 270 Mbit/s 59 ▂▄▆_ WPA2
Infra 1 130 Mbit/s 45 ▂▄__ WPA2
Infra 1 130 Mbit/s 45 ▂▄__ WPA2 802.1X
Infra 1 130 Mbit/s 42 ▂▄__ WPA2
Infra 1 405 Mbit/s 39 ▂▄__ WPA2
Infra 1 270 Mbit/s 39 ▂▄__ WPA2
Infra 3 270 Mbit/s 39 ▂▄__ WPA1 WPA2
Infra 1 270 Mbit/s 35 ▂▄__ WPA2
Infra 1 130 Mbit/s 35 ▂▄__ WPA2 802.1X
Infra 1 130 Mbit/s 35 ▂▄__ WPA2
Infra 1 130 Mbit/s 35 ▂▄__ WPA2
Infra 6 540 Mbit/s 32 ▂▄__ WPA2 WPA3
Infra 8 270 Mbit/s 32 ▂▄__ WPA2
Infra 5 270 Mbit/s 29 ▂___ WPA2
Infra 6 405 Mbit/s 29 ▂___ WPA2
Infra 8 405 Mbit/s 29 ▂___ WPA1 WPA2
Infra 11 130 Mbit/s 29 ▂___ WPA2
Infra 1 540 Mbit/s 25 ▂___ WPA2 WPA3
Infra 1 270 Mbit/s 25 ▂___ WPA2
Infra 1 270 Mbit/s 25 ▂___ WPA2
Infra 6 405 Mbit/s 25 ▂___ WPA2