So in designing a new infrastructure for our public facing sites today i found a flaw in my initial design and figured i’d use my blog to find out how others are set up.
Here’s the situation. We’re going to be moving to CF Enterprise to get all the benefits it brings and as always we’re keen to keep the costs down. We’re an MS Shop, so servers will be Windows 2003 and IIS 6.0. Db is MSSQL already in place on a SQL cluster. All our sites run in a CMS.
We’re going to use Windows network load balancing between the two servers (3km apart) with a 1gb connection between the two. This is where it starts getting messy. Applications we’ve developed are simple, we know what we need to deploy so the entire application can be deployed to both physical servers. It’s the CMS that is causing me troubles. Because of the load balancing when contributors upload content (JPGs,GIFs,PDFs etc) the incoming file must be available to both servers to serve immediately. We could sync folders across the boxes but that seems like it could get complicated if we start adding more boxes. Obviously we can’t host the files on one box since (if we made them admin the site on one node and not the load balanced address) if this fails the files are unavailable so now I’m leaning towards using a clustered fileshare so both servers now access their files from a well connected file cluster. That way, incoming files are available to both both boxes without the need for folder replication etc but with a downside of potential latency between the web server and the file cluster. With trusted caching enabled we’d be able to minimise traffic so it’s only the GIFs/JPGs going to/from the file cluster after the initial requests.
Has anyone else run into a similar situation? - i’d be keen to hear how you addressed the probem.
March 11th, 2005 at 3:53 pm
in the win2k3 toolkit ( i think that’s where it is) is a simple uet very effective program called robocopy. We have it deployed on a clients LB system. They only access the CMS from one location and it automatically copies out to the other location. It’s my understanding that it has the ability to move files from one to many locations.
look into it, hope it helps
March 11th, 2005 at 4:52 pm
robocopy is one solution but we found simple nice little app. called Mirrorfolder. We bought it and its been working beautifully in our cluster with our CF based CMS for a couple of years now.
http://www.techsoftpl.com/backup/
March 11th, 2005 at 5:19 pm
network-attached storage is the "enterprise" way to go… if you have lots of $$ then you do a Fibre-channel SAN (each server has a special card that you plug fiber into, which connects to a network of storage devices).
A less-expensive alternative is a shared SCSI disk array (multiple servers connect to one rack of drives), but that’s tough when the servers are not physically located together.
HP is now offering a "packaged cluster", which contains 2 DL380s and a MSA500 storage array (can hold up to 2Tb of disk I believe), all in a nice 8U package.
March 12th, 2005 at 12:49 am
What about using CFMX 7’s new Gateway Services, specifically the folder monitoring service? It should have the ability to ’see’ a change in files on a folder(s) and respond to it.
Per a presentation by Ben Forta to my user group this week he mentioned that the native service call does not ‘know’ when a file is complete (say, a slow upload via a user in a browser over a modem) so you would need to code a way to check the date stamps and file sizes. For instance, CF sees a new file on the server in X directory and then monitors it 3 times over say 15 seconds. If the date & file sizes remain the same then you can assume the upload is complete and you can trigger an FTP session through CF to transfer the file to the other server.
March 12th, 2005 at 5:30 am
Hey guys this is great stuff.
With regards to robocopy and mirroring folders, that to me seems like one node would be the master and that used for maintenance/admin and then uploads mirrored from there? I assume robodemo needs to be scheduled to run every x minutes?
We were orginial considering fibre cards but that would mean only one box can access the disks at one time and you can’t run NLB and Clustering on the same boxes.
I’ve looked at the gateway stuff too, but again assumes one node is the master. Else you end up will all the servers copying data everywhere and it ends up get all too complicated.
March 12th, 2005 at 8:40 am
Yes for us robodemo is in a master/slave configuration with it running copies every x seconds. i’m not sure but i think you can install the robodemo on both machines but i’m pretty sure it won’t handle conflicts properlly etc.
we have found the master/slave to be just fine and we are under the impression that we will be able to deploy it as a master/slave/slave later on down the line.
jonese
March 14th, 2005 at 1:39 pm
How about having a hidden page/fuseaction that is for receiving files? You could code it so that it only allows files from the other server, e.g. have a changed-daily code embedded. Then when a new file is uploaded it is automatically sent off to the other server. Or add it to a queue and schedule a script to be ran every five minutes to send it to the other side.
At one of my previous jobs a guy wrote a mirroring system for a file download system. You went to the site to download a file and it automatically worked out the best location to obtain it from, after checking the file existed at that location. If the file didn’t exist it was scheduled to be mirrored. Was quite nice.
Damien
March 14th, 2005 at 11:54 pm
Dave Ross is spot on here. SAN is the standard "enterprise" solution to this problem (and we use SAN at Macromedia). You can also mount a shared file system using NFS or whatever if you can’t afford SAN (but you don’t get the redundancy).
March 25th, 2005 at 10:19 am
If it’s possible for you to have release cycles it might be a good idea to simply update once every 2 - 4 weeks. It’s not a big deal if you update all servers to the lastest "version" once every two weeks on all clustered servers.
The group that I’m managing does not do updates unless we _absolutely_ need to (critical bugs after an update) and for this even we have a tool that does a remove CVS update on all of our clustered servers.
August 3rd, 2005 at 6:57 pm
I’m assuming you’ve moved to the Enterprise edition now. Were you able to set Max simultaneous requests after you moved? We are set up as enterprise, and can’t see that setting, and I’m afraid it’s going to hurt performance…
August 4th, 2005 at 2:58 am
it’s at the top of the Server Settings page - is it not there for you? There is a bug in 7 that if you use a .car file to transfer settings then the Max Simultaneous requests doesn’t come across but i’ve never seen it where you can’t see/change it.
August 4th, 2005 at 10:26 am
Very interesting… We’re running 6.1 Enterprise and it’s definitely not there. Here’s a complete description of the situation we’re seeing (including our available settings): http://www.macromedia.com/cfusion/webforums/forum/messageview.cfm?catid=143&threadid=1035172&loc=en_us
If you’re seeing it and you’re setup as enterprise, not standalone, then I’m bewildered. (Not a new state of being for me
August 11th, 2005 at 3:30 pm
It appears that the setting is in the ProxyServices section of jrun.xml for enterprise setups.