I have a “YouTube” web server on opensuse 11.4 where people are uploading videos to and are being transcoded to low and hd formats. When wowza was on the same server no problem. For production have YouTube on one server and wowza on another. Due to the codec used need “YouTube” on a 32-bit opensuse and for performance have Wowza on a separate 64-bit opensuse. The two machines are directly connected to the network and thru a separate gigabit network card they are connected to each other and communicate internally in theory thru 192.168.0.1 and .2. This internal connection is for faster direct communication of files for Wowza.
When I configured Wowz local to YouTube I told it the files are at to /srv/www/htdocs/media/videos
As Wowza is a separate server now any help on how to directly communicate between the two thru ip 192.168…
Essentially I want the Wowza server to connect to the drive of the YouTube server so it can access the video files directly for streaming and to avoid going thru network switches. Obviously I need something fast, reliable, and that can reattach on reboot.
Help is much appreciates.
Thanks
JR
EDIT: by the way I am not concerned with the link between the two machines being secure with SSH or anything as they have a direct cable connection between the two unless that’s the only way. Only reason I mention this is a nonsecure connection should in theory be faster than an encrypted one.
Why not simply use NFS to remote mount the data? It will work over the fastest link if you export and mount from the IP addresses (or the hostnames belonging to them).
I wouldn’t tend to use NFS as it seems to suffer from Stale File Handles when dealing with large amounts of data regularly, which you obviously don’t want on a server. SSHFS would keep speed high enough on a local direct connect like that and doesn’t suffer from Stale Handles.
If the business can spend money on a “real” SAN, it should solve both your immediate issues and likely provide a solid foundation for the future.
If you can’t afford a “real” SAN, then you can roll your own.
Since these are SAN requirements, I’d recommend you start reading up what a SAN does, its typical features and how it works. From that, you can start to decide what technologies to implement
You might also consider some of the new technologies being developed in the area of clustered, “virtual” storage.if you think for instance that BTFRS or VFS might be reliable enough for you now (BTFRS is likely going to start being introduced officially as part of openSUSE 12.1) you’ll continue to benefit from improvements as they’re developed and integrated.
If the business can spend money on a “real” SAN, it should solve both your immediate issues and likely provide a solid foundation for the future.
If you can’t afford a “real” SAN, then you can roll your own.
Since these are SAN requirements, I’d recommend you start reading up what a SAN does, its typical features and how it works. From that, you can start to decide what technologies to implement
You might also consider some of the new technologies being developed in the area of clustered, “virtual” storage.if you think for instance that BTFRS or VFS might be reliable enough for you now (BTFRS is likely going to start being introduced officially as part of openSUSE 12.1) you’ll continue to benefit from improvements as they’re developed and integrated.
Since it sounds like you only have two servers here: One for Apache and the other for Wowza, I’m going to have to disagree with Tony and say a “real” SAN solution is just a tad overkill - unless you just want to drop six figures for fun. SANs are great - don’t get me wrong, and you can certainly roll your own SAN for a reasonable price, but with only two servers - it’s just not what a SAN is for.
If you just need to share /srv/www/htdocs/media/videos between them, you have many options. I would investigate things in this order:
ATA of Ethernet (AoE) - very fast, no IP overhead and your direct gigabit connection is perfect for this. This is actually very easy to set up.
DRBD - just block replicate the actual files onto the Wowza server. From it’s perspective they will be local file systems with full performance.
NFS with caching. While I agree NFS has issues and overhead, unless you are dealing with huge files NFS will likely work pretty well, and if you use caching, the most often requested videos will be pulled from local cache, tremendously improving performance.
SSHFS - while I love sshfs, and while in fact you could very well use it in this simple two server instance, the encryption overhead is not needed (though of course you have encryption options). However, you may find trying to scale this with additional web and wowza servers starts to become a pain.
(I would rule out iSCSI as there is no point for the overhead of IP when you don’t need a routeable solution.)
Down the road, you may need to add additional web servers for a load balanced cluster, and additional Wowza servers if needed. Several of the above solutions could certainly support that - you don’t needed clustered storage. It is one data set being shared with many servers which actually only need read only access, and you can certainly accomplish that with the above.
I would start with AoE as it is going to be easier to implement that DRBD, and you may find it is all you need for some time to come.