As I understand, 304.xx drivers add support for RandR. I briefly tested rotation using xrandr with 304.60 and it works. Did you try to use Rotate option in Monitor section?
Same problem with the manual driver. But also had the problem that the Nouveau took over when I un-installed the Nvidia driver. The nouveau would lockup my system. It would even run in init 3 mode. Had to do a find and delete nouveau in single user mode so I could get back to a network mode to put the repo driver back on. Another reason why I don’t like to use the manual driver.
There is no desktop, this is a aircraft sim. Four Video cards, three currently used. Six displays, with four rotated. The desktop is replaced with a costume test pattern, that is overlaid with instrumentation when the sim is running.