With VLC you can set the video output via Tools > Preferences > Video. What is set there. Have you tried experimenting with these. In particular, perhaps try setting VDPAU Output explicitly?
Try launching mpv (video player) from a terminal and play a video file.
mpv /path/to/your/video/file
Any error messages?