Integrating Microsoft's Network Load Balancing (Windows 2003 NLB) with ColdFusion MX 6.1

This article was written by Frank DeRienzo & Brandon Purcell, the final version will be featured on the Macromedia Website in the next few weeks.

Software-based means of server load balancing have been steadily replaced by hardware-based solutions, but a significant viable niche remains for the former. The increased robustness of hardware solutions, along with their declining price has encouraged and allowed web masters and IT professionals to accelerate the transition from less-scalable (but more affordable and more easily implemented) software-based solutions to hardware-based options. The niche for software-based solutions has always been sites with two - four web servers which sustain moderate traffic, but which need to always be available. Indeed, the need for availability, scalabilty, application sensitivity, content-awareness, etc., quickly forces a sometimes painful transition from a web site based on a single server platform to one based on a server cluster or a server farm. In the two-four server niche, the preferred software-based solution for web servers running CF/JRun has been ClusterCATS, but with the advent of Microsoft's 2003 server operating systems, network load balancing (NLB), is now bundled with web servers. Previously, NLB was only offered with Microsoft's high-end operating systems, Advanced and DataCenter. The availabilty of NLB on web servers offers an economical means of software-based load-balancing and failover for ColdFusion web sites running on the 2003 operating system. By combining CF/JRun clustering with NLB, you will have a dual-cluster solution; on the first set of network interface cards (NIC), you will have a shared IP address for the web server, on the second set of NICs you will host a CF/JRun application cluster.

The following example-based article describes how to configure NLB to work with ColdFusion MX 6.1 JRun clustering, where the CF applications are co-located with the web servers. Or to put it another way, when IIS6 and CF are hosted with NLB on the same server, the following procedure is required. This article is split into two sections, the first section describes how to configure NLB to work with a CF cluster and the second section describe how to configure the CF cluster to work with NLB.

Configuring NLB for integration with a CF cluster on JRun:

The following illustration presents our working example.

 

Step One: Insure that each web server has two NIC's available, one for NLB and the other for the CF/JRun cluster.

If your web server does not have two NIC's available, one for NLB and the other for CF/JRun, then you will need to add a NIC to each web server. The JRun-based cluster will not run on the same NIC as NLB. Essentially, NLB takes over a NIC on each web server.

Step Two: Configure DNS name resolution and prepare the IP stack on each server.

For a site hosted on two web servers, you will need a minimum of five IP addresses. One address will be shared by each server; this shared address will be the fully qualified host name of the web site. For our example, we have used nlb.enterprisecluster.com, which corresponds to the IP address, 192.168.1.1. The next address and name will be different on each server. It will host the shared IP address on each server and bound to the NIC that is NLB enabled. In our example we have used www1.enterprisecluster.com corresponding to the IP 192.168.1.2 and www2.enterprisecluster.com corresponding to the IP 192.168.1.3. Finally, you will need a pair of IP addresses for a second set of NIC's to host your CF/JRun cluster. In our example we use cfjr1.enterprisecluster.com corresponding to 192.168.1.4 and cfjr2.enterprisecluster.com corresponding to 192.168.1.5 The configuration in DNS or the relevant section of the hosts file on each of these servers would look like this:

192.168.1.1    nlb.enterprisecluster.com   nlb
192.168.1.2    www1.enterprisecluster.com  www1
192.168.1.3    www2.enterprisecluster.com  www2
192.168.1.4    cfjr1.enterprisecluster.com   cfjr1
192.168.1.5    cfjr2.enterprisecluster.com   cfjr2  

Step Three: Configure the first NIC on each server to accomodate NLB.

The first NIC on each server will host two IP addresses; one of them will be shared. Let's examine the first NIC on each web server in our example NLB cluster by right-clicking on the local area connection icon and viewing the properties. Highlight the Network Load Balancing Parameter and make sure it is checked. If the Network Load Balancing parameter does not appear, then you will need to add it under the Control Panel - Add Software - Add Microsoft Components option before going on.

Edit the cluster parameters under NLB by adding the address that will be shared by each server. Following our example this is 192.168.1.1 Since this address is shared, this step is exactly the same on each server.

Edit the Host Parameters tab under NLB on each server to bind the address that will be unique on each of the NLB-enabled NIC's. Following our eample, this is 192.168.1.2, the IP for www1.enterprisecluster.com. After entering the IP address and subnet mask, click OK then open Internet Protocol (TCP/IP).

Configure the IP address for www2.enterprisecluster.com on the second server.

Under the the Internet Protocol (TCP/IP) properties, bind both the unique host address and the shared NLB address on each server. On server one as depicted below:

On the second server as depicted below:

Click the OK button and save your settings. You have configured the NLB-enabled NIC on each server.

Step Four: Configure the second NIC on each server to accomodate CF/JRun.

Right-clck on the second NIC on each server and examine the properties. Make sure that NLB is not selected by unchecking the block to the left of the NLB option as depicted below.

Edit the Internet Protocol (TCP/IP) parameters on each server; bind the unique address that will host CF/JRun on each server. Following our example, on server one this will be 192.168.1.4.

 

On the second NICon server two, it will be 192.168.1.5.

Save your settings. You have now prepared your web servers to accomodate a CF/JRun cluster with NLB. The next section will describe how to configure the CF/JRun cluster on each server's second non-NLB-enabled NIC.

Configuring CF/JRun for integration with NLB:
The most important thing to remember when deploying a cluster of JRun servers using NLB is that all communication and configuration for JRun must use the second network card that is isolated from NLB. JRun needs a static IP address and the NLB shared IP address can cause several problems with JRun Clustering.

The process of building a ColdFusion cluster on Windows using NLB is not that much different form the process where NLB is not used. I outline the process step-by-step below.

Building and Deploying a ColdFusion MX 6.1 Enterprise Cluster
To build and deploy a ColdFusion MX for J2EE cluster, use the following steps:

  1. Install JRun 4 or CFMX 6.1 (JRun install) on each server you plan to cluster.
  2. Start all servers in the cluster.
  3. Configure ColdFusion MX through the wizard.
  4. Configure server settings in the ColdFusion Administrator.
  5. Create a cluster and add the servers to the cluster.
  6. Configure session replication if necessary.
  7. Restart all servers in the cluster.
  8. Configure an external webserver to the cluster.
  9. Perform any ColdFusion Administrator configurations (mappings, database connections, and so forth) and deploy your ColdFusion application.
  10. Test the cluster.

Follow these detailed steps for building and deploying a ColdFusion MX for J2EE cluster:

  1. Install ColdFusion MX 6.1 with the J2EE Configuration (ColdFusion MX with JRun 4) on all servers that you plan to cluster

Figure X. Choose the J2EE Configuration in the Install Wizard  

  1. During the install choose the Built in webserver as shown in figure 2

Figure X. Installation Wizard choosing the webserver (choose Built-in)  

  1. Follow the prompts for the installer and complete the install for CFMX 6.1
  2. When prompted launch the Configuration Wizard after the installer finishes, enter you CF Admin password and follow the prompts in the wizard to complete the install.
  3. On windows five different services will be installed as shown below
    1. ColdFusion MX ODBC Agent
    2. ColdFusion MX ODBC Server
    3. Macromedia JRun Admin Server - JRun Management Console (JMC) htttp://127.0.0.1:8000/
    4. Macromedia JRun CFusion Server - ColdFusion Server
    5. Macromedia JRun Default Server - Default JRun Server
    Only services a, b, d are needed for CFMX to run correctly, the JRun Admin server and Default server can be shut down unless you need to use the JMC to administer JRun
  4. On unix you can use the following command to start and stop the JRun servers
    /jrun4/bin/jrun -nohup -start <servername>
    /jrun4/binjrun -stop <servername>
    /jrun4/bin/jrun -nohup -start cfusion
  5. Using either the service control panel or command line start the Admin server for each server in the cluster.
    /jrun4/bin/jrun -nohup -start admin
  6. If you want to use multiple instances on a single server or cluster instances on a single server see this article otherwise to cluster across multiple servers continue to step 9.
  7. Open the JRun Management Console (JMC) http://server:8000/ on one of your servers that you will be clustering. This will be the JMC that will be used to create and control the cluster of JRun instances.
  8. Before you can create the cluster you will need to register the remote JRun instance in the JMC. Earlier when you installed ColdFusion MX 6.1 with the J2EE option it created a JRun server named cfusion. If you are clustering JRun instances each instance has to have a unique name. You will need to rename one of the servers from cfusion to something else, in this example I use the name cfusion2.

    To change the name of a JRun server you will need to make two changes.
    1. Rename the directory {jrun-root}/servers/cfusion to {jrun-root}/servers/cfusion2
    2. Modify the cfusion entry in {jrun-root}/lib/servers.xml
      <server>
        <name>cfusion2</name>
        <directory>{jrun.home}/servers/cfusion2</directory>
      </server>

To manage remote servers from the JMC, start a server on the remote computer and register the server in the JMC. Remember to use the IP address that is bound to the second network card. You must start the remote server before registering it from the JMC

Figure 3. Registering a remote JRun application server through the JMC.

Figure X. Registering a remote JRun application server through the JMC. To register the remote server you only need the remote servers Host Name, JRun Server Name, and JNDI Port.

After registering a remote server, you can create new instances on the remote server and administer the settings for all servers from one JMC.

Figure 4. View of all servers that are available from within the JMC.

Figure X. View of all servers that are available from within the JMC.

  1. Start all JRun instances that will be part of the cluster.
  2. Log into the ColdFusion Administrator. Click the Memory Variables link under the Server Settings section.
    1. Check the box for Use J2EE Session Variables and click Submit Changes.
    2. Repeat steps 6 and 7 for all servers that you will cluster.
  3. To create the cluster, go into the JMC and click Create New Cluster. Add the appropriate servers to the cluster. Enter a name for the cluster, and choose the servers that will be a part of the cluster.

Figure 5. Creating a Cluster in the JRun JMC

Figure X. Creating a Cluster in the JRun JMC.

Figure 6. Selecting servers that will be a part of the JRun cluster.

Figure X. Selecting servers that will be a part of the JRun cluster.

  1. If the application uses session variables, you can configure session replication to synchronize session state across the cluster for failover. The following procedure explains how to configure session replication.
    1. Within the JMC, expand the clustername in the left frame.
    2. Click on one of the servers in the cluster.
    3. In the right frame, click the ColdFusion MX application. (depending on how ColdFusion is deployed you may need to first click the cfusion-ear link first then ColdFusion MX link)
    4. Click the checkbox for Enable Session Replication.
    5. Add the * to the New Replication Buddy and click Add.
    6. Click Apply.
    7. Open the {jrun-root}\servers\{server-name}\cfmx\WEB-INF\jrun-web.xml and modify the <persistence-config> entry to false, as follows:
      <persistence-config><active>false</active></persistence-config>
    8. Perform steps 14a through 14g for all servers in the cluster.
  2. If you configured session replication in step 14 then you will need to modify the classpath in {jrun-root}/bin/jvm.config file. When CFMX 6.1 is installed it makes some changes to the jvm.config files classpath that prevents session replication from working properly. To get it working remove the following entries that are highlighted in red

    # JVM classpath
    java.class.path={application.home}/servers/lib ,{application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/cfusion.jar,{application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib,{application.home}/lib/jrun.jar ,{application.home}/lib

  3. In the JMC, at the command line or in the services panel, restart all servers in the cluster.
  4. To connect your web server to the cluster, run the Web Server Configuration tool, as follows:
    1. For the installer to modify the application appropriate XML configuration files, all clustered servers must be running.
    2. If you have JRun servers on multiple computers, do not choose the default JRun host or localhost; instead use the host name of one of the servers in the cluster.
    3. Select the cluster name from the JRun Server properties pop-up menu.
    4. Select the appropriate web server.
    5. Select the site or Configuration Directory.
    6. Click OK.

      The installer makes all configuration changes and restarts the web server

    Figure 7. GUI based JRun webserver configuration tool.

Figure X. GUI based JRun webserver configuration tool.

You can use the command-line webserver configuration tool to install the webserver connector in an environment where a GUI is not available. For syntax details, refer to the ColdFusion documentation. The following example shows one way to configure Apache in a Unix environment. For all other environments see the ColdFusion documentation.

java -jar /opt/jrun4/lib/wsconfig.jar -host 
  192.168.1.4 -cluster ClusterName -ws Apache -dir /etc/httpd/conf 
  -map .cfm,.cfc,.cfml,.jsp,.jws -coldfusion -v
  1. Copy your ColdFusion Application to your webserver root of every server in the cluster. If you are running in distributed mode (JRun and the Webserver on different physical machines) you will need to copy the ColdFusion content from the web server to the application server with the same directory structure used on the web server.
  2. Next create data sources, mappings, and any other settings to the ColdFusion Administrator for every server in the cluster.
  3. Test your cluster. Browse a ColdFusion page through the external webserver (such as http://server/mypage.cfm) If you have an application that uses session variables, you can shut the server down after invoking a page and redirect your request to another running server; all while preserving the session data. To find out which server handles your request, you can add the following code, which displays the JRun servername.

    <cfobject action="create" type="java" class="jrunx.kernel.JRun" name="jr">
    <cfset servername = jr.getServerName()>
    <cfoutput>JRun Server Name: #servername#</cfoutput>

    The process routes all incoming requests through the external webserver in a round-robin fashion across all JRun servers in the cluster. In addition, the process assigns all subsequent incoming requests a jsessionid for session tracking directs them to the same JRun server by using sticky sessions.

Conclusion

Software-based means of server load balancing have a solid niche. The availabilty of of NLB on web server platforms offers a robust load-balncing solution for CF/JRun customers who are upgading their servers to the 2003 operating system. While hardware solutions are preferred for sites that sustain heavy traffic or load spikes, the solution outlined above is a viable one for sites with two-four servers that must maintain high availability.