Running ColdFusion 10 in Distributed Mode
A recent discussion on CF-Talk had a member trying to find instructions for setting up ColdFusion in "Distributed mode" on a CF 11 server. If you have never heard of "distributed mode" you are not alone among CF developers. It's not a common setup in my experience. Here's a quick 101. Many processes and daemons on a given server communicate using the TCP stack. TCP provides a predictable, high capacity, mature protocol for piping requests and data in and out of server processes. In this case, IIS or Apache communicate with CF through the local loopback (localhost or 127.0.0.1) IP address and a port - usually 8012 (or 8013 or 8014) chosen at installation. Naturally you can alter the IP address to which you are connecting, changing it from the loopback to... well any IP that's listening on 8012. That means you can set up your ColdFusion servers separately from your web servers. After all it's just IP networking. Why would you do that? The Muse will let his guest handle that question.
Meanwhile, to preempt (or perhaps spur) discussion, the Muse will note that this process is similar to something called "Reverse proxy" which functions in much the same way. The difference being with CF distributed, IIS on the front end handles all the "http stuff" and passes the request to CF just like it was a local engine, whereas with reverse proxy the HTTP request is simply redirected to the alternate server. That means the alternate server needs to be a full webserver plus application server. While that increases the overhead a bit, it has some advantages - but that's a topic for a different post.
Back to our CF-Talk question, it quickly became apparent that not many folks actually knew how to accomplish this task on a CF 10 (or 11) server due to the underlying platform switch to Tomcat. After some back and forth Byron Mann chimed in with some very specific instructions on how to get this done. Byron is a lead engineer at HostMySite - which makes the Muse feel better about their ColdFusion support. :) Here's his tutorial.
Byron Mann writes...
We actually run our internal CF sites in distributed mode. ColdFusion 10 or higher (on Tomcat) works better than ColdFusion 9 or below. Previously it seems to work very slowly and the clustering (at least in our implementation) was never up to snuff. Note: you can accomplish pretty much the same thing with reverse proxy but the ColdFusion distributed mode is still pretty useful. CF can still peg CPUs from time to time, and if you have other non-ColdFusion sites on the web server (as we usually do), it is nice to have those continue to run unconcerned while the lone CF site dies a horrible death on different servers.
My best suggestion for ease of configuration would be to go ahead and install IIS on the ColdFusion server (even though you are not going to use it in production) and then run the wsconfig.exe in the c:\ColdFusion10\cfusion\runtime\bin folder. This will create the configuration files as well as the handler mappings and virtual directory for the isapi_redirect.dll. Copy the c:\ColdFusion10\config to your web server from the CF server. Then recreate the IIS handler mappings and jakarta virtual directory to mimic the CF server's IIS configuration. Don't remove the files from the CF server - you can use them as a reference for setting up your web servers the same way each time. Also, you will need to add the isapi_redirect.dll as an allowed executable. That permission can be found in IIS under the server root ISAPI instructions. Make sure the website application pool user has write permission to the 'config' directory.
Connector vs. Reverse Proxy
As an aside, here's my take on why the connector is a little more useful than reverse proxy. If you have more than one CF server on the backend, it can act as a kind of load balancer. We have 12 CF instances backing our applications. You can manually change the isapi_redirect worker files to create a pseudo cluster. You can also do things like redirect specific sites or sub-folders to a particular CF instances. One example I have is a CF service that routinely timeout and kills the server (as it calls another SOAP service off-site). I have this particular service request redirected to it's own CF instance to minimize it killing others and effecting other sites. I'm sure you can do such things with a reverse proxy as well, but the connector seems to be a more simple solution. As you might imagine there are 20 ways to catch a fish. :)
Meanwhile, the documentation for the Tomcat connector can be found at this link, but don't follow the instructions for setting it up with IIS, it has unnecessary information about registry keys etc.. I just use it as a reference for the properties in the config files.
Here is a snippet from the workers.properties file for our cluster. Keep in mind that when I say cluster, I'm not speaking of a "ColdFusion cluster" (where sessions are replicated across instances). Instead I'm speaking of balancing. The connector acts like a simple load balancer between your CF instances. Your ports may differ than 8013. I think 8009 is the default. You can see this in your CF instance in the web.xml file for tomcat under the c:\ColdFusion10\cfusion\runtime\conf\server.xml folder. Look for something like the xml node below. It may be commented out. If so you will need to un-comment it and restart CF.
tomcatAuthentication="false"/>
# workers.properties
worker.list=cluster-1
worker.cluster-1.type=lb
worker.cluster-1.balance_workers=cfusion-3,cfusion-5
worker.cluster-1.sticky_session=true
worker.cfusion-3.type=ajp13
worker.cfusion-3.host=10.78.4.57
worker.cfusion-3.port=8013
worker.cfusion-3.max_reuse_connections=250
worker.cfusion-3.lbfactor=1
worker.cfusion-3.route=cfusion-3
worker.cfusion-5.type=ajp13
worker.cfusion-5.host=10.78.4.58
worker.cfusion-5.port=8013
worker.cfusion-5.max_reuse_connections=250
worker.cfusion-5.lbfactor=1
worker.cfusion-5.route=cfusion-5
Then your uriworkermap.properties would look something like this (Note: I removed some of the default stuff):
/rest/* = cluster-1
/*.cfml/* = cluster-1
/*.mxml = cluster-1
/*.cfm = cluster-1
/*.cfm/* = cluster-1
/*.cfml = cluster-1
/*.cfc = cluster-1
/*.cfc/* = cluster-1
Finally it's a good idea to mimic the same directory structure for your web apps between the ColdFusion server and the web server. We use DFRS to replicate all the files, but other methods will work as well.
Muse Summary
Now I know there's some assumed knowledge left out of this post, but this can get you started - particularly if you migrating from CF 9 or lower to CF 10 and you are already handy with distributed mode. Meanwhile, I welcome constructive comments that can add to our joint knowledge.
6 Comments
-
Also, Check out the Clustering session from cfsummit 2014. They cover this topic, make some good recommendations, and also did some screen capture videos that can help you get this setup. -
I prototyped the same idea for my client, except we run our app servers on Ubuntu with Apache HTTP server connected to Tomcat (AJP connector) for CF. We use non-sticky sessions in our clusters so each request can be handled from any server in the cluster.
I configured Apache to know about all the Tomcat instances in the cluster even though it would use the local one by default first. But if for some reason Tomcat died on the local server, and Apache was still up, it would fall over to the next Tomcat instance, so on and so forth until it got to the end of the line trying to find a working Tomcat instance. Because of the age of some of the system we have outside of CF we actually have Tomcat 6 & 7 running on each app server, so Apache had two fallover settings, one for Tomcat 6 and the other for 7. It was pretty cool to see in operation, to bad they did not implement it. Oh well. Fun learning experience! -
@Mike,
Great! I'll see if I can scare up a link.
@Ed,
Awesome information Ed. Thank you. Sounds complex - but redundancy always is I guess :) -
I am currently in-process of setting up new servers to use Distributed Mode in our new production set up. Where I agree it is an uncommon set up, I have used in the past with UNIX and Apache and has worked well. I am running into an issue with IIS and the context root set up.
Web server talks to CF App server over 8011 port, confirmed that works with C:\{instance_name}/wwwroot/index.cfm page getting executed. But attempting to switch to D:\website\site1 situtation is not working.
So my question is two fold. Can we have a Distributed Mode set up to run more than one web site on one instance of CF 11? That is what we really need at end of day, but as it stands I can not even get one to one working with a different web root other than the default wwwroot for the installed instance. Any suggestions, please. -
One solved and another issue has come up.
We were able to resolve the Multihoming issue with CF and Distributed Mode (DM). The instance server.xml was modified to have multi virtual hosts.
The second issue we are facing, is the securing the communications between CF and IIS with the DM set up. The "packet sniffers" show the data id not encrypted. User comes in on HTTPS to site, site is severed no problem. With IIS and CF on the same server, data is encrypted no problem. DM caused the data to not be encrypted.
I was wondering if you had any suggestions or ideas on where I might look to get this resolved. -
I have one problem with apche virtual hosting + CF11.
I am using apache with CF11 and virtual hosting document root on NFS mounted folder. when I access my cf test page from that document root folder I am getting "Define the page request properties. " message on browser. No data coming.. any idea what could be the problem.
Also If I use local folder instead of NFS mounted folder for apache virtual host document root I am getting results fine.
I have verified the permissions and everything looking good.
One more thing is this is working fine in Apache + CF8
Sites to Watch
Aggregators
RSS
