Load balancing with Eclipse GlassFish cluster

This series of guides will lead you to set up a highly available GlassFish cluster with a load balancer that distributes incoming requests to cluster instances.

How to: Set up a GlassFish Cluster

Leave a Comment / How To ... / / / , , , ,
This entry is part 1 of 2 in the series Load balancing with Eclipse GlassFish cluster

This tutorial will guide you to set up a GlassFish cluster that serves a single application on multiple machines.

This setup doesn’t enable HTTP session replication. Session data will be available only on the original cluster instance that created it. Therefore, if you access the cluster instances through a load balancer server, you should enable sticky session support on the load balancer server. With this mechanism, a single user will be always served by the same GlassFish instance, different users (HTTP sessions) may be served by different GlassFish instances. It’s also possible to configure session replication in a GlassFish cluster so that each session is available on each instance but this is not covered by this tutorial.

Table of Contents

Step 1: Start the Default Domain

  1. Open a terminal and navigate to the bin directory of your GlassFish installation.
  2. Start the default domain:
asadmin start-domain
  1. Open GlassFish Admin Console, which is running on http://localhost:4848 by default. If you access the Admin Console from a remote machine, you need enable the secure administration first and then access Admin Console via HTTPS, e.g. https://glassfish-server.company.com:4848. To enable secure administration, refer to the Eclipse GlassFish Security Guide.

Step 2: Create a Cluster

  1. Navigate to “Clusters”
  2. Click on the “New…​” button to create a new cluster.
  3. Enter a name for the cluster (e.g., myCluster)
  4. In the “Server Instances to Be Created” table, click “New…​”, and then fill in instance1 as an instance name, keep “Node” selected to the default “localhost-domain1”
  5. Click “OK”

This will create a clustering configuration myCluster in GlassFish, with one GlassFish server instance instance1, which runs on the same machine as the GlassFish administration server (DAS).

New Cluster page

Step 3: Add routing config for sticky sessions

  1. Navigate to Clusters → myCluster
  2. Click on the “myCluster-config” link in the “Configuration” field
  3. Click on “System Properties” to open the “System Properies” configuration for the cluster
  4. Click on “Add Property” button
  5. In the new row, set the value in the “Instance Variable Name” column to “jvmRoute”
  6. Set “Default Value” to “${com.sun.aas.instanceName}” so that it’s set to the actual instance name for every cluster instance

This configuration is needed to simplify the sticky session routing mechanism in load balancers, e.g. in Apache HTTP server.

System Properties page

Step 4: Start the cluster

  1. Navigate to “Clusters”
  2. Click the checkbox in the “Select” column next to your cluster
  3. Click “Start Cluster” button and wait until the cluster is started
  4. Navigate to http://DOMAIN_NAME:28080 (e.g. http://localhost:28080), where DOMAIN_NAME is the same as in the URL of your DAS server (e.g. localhost)

This will navigate you to the welcome page of the instance1 GlassFish instance running on port 28080.

Starting the Cluster
Cluster instance running

Step 5: test the Cluster

Deploy a cluster tester application, which you can download from https://github.com/OmniFish-EE/clusterjsp/releases.

  1. In GlassFish Admin Console, navigate to Applications and click the “Deploy…”​ button
  2. In the “Targets” section, click on your cluster in the “Available Targets” column and click the “Add >” button
Deploy an application to the cluster
  1. In the “Location” section, click Browse…​ and select the application WAR file
  2. Set the “Context Root” field to “cluster”
  3. Click “OK”
  4. Navigate to http://DOMAIN_NAME:28080/cluster (e.g. http://localhost:28080/cluster)

Right now, there’s a single instance in the cluster. All requests are handled by that instance, session data is always present, and all should work as expected. As you’ll add more instances to the cluster, you can use this tester application again to verify that all works even if requests are sent to different instances.

Tester application running

Step 6: Add an SSH Node

Now, add a SSH connection to a remote server, where you want to run other GlassFish cluster instances.

This assumes that you already have a remote machine with SSH server and Java installed.

Make sure that GlassFish admin server can access the remote machine on the SSH port (port 22 by default). Then:

  1. In GlassFish Admin Console, navigate to Nodes. There’s always at least 1 node, e.g. localhost-domain1, which represents the local machine
  2. Click on the “New…​” button to add a new SSH node.
  3. Enter a name for the SSH node (e.g., sshNode1).
  4. Set the “Type” to “SSH”
  5. Set the “Host” to the IP address of the remote machine.
  6. If you want to install GlassFish on the remote machine, enable the ckeckbox “Install GlassFish Server”
  7. Set the “SSH User” to a user with sufficient privileges on the remote machine. Set to `${user.name} if it’s the same user as the one running the GlassFish admin server
  8. Set the “SSH User Authentication” based on your authentication method. Fill in the authentication details, e.g. “SSH User Password” for “Password” authentication
  9. Click “OK” to add the SSH node.
GlassFish Nodes page

Step 7: Create an Instance on the SSH Node

  1. Navigate to Clusters → myCluster.
  2. Select the “Instances” tab.
  3. Click on the “New…​” button to create a new instance.
  4. Enter a name for the new instance (e.g., remoteInstance) and select the SSH node (sshNode1) as the target “Node”.
  5. Click “OK” to create the instance.

Make sure that the admin port of the remote instance is open for connections from GlassFish admin server and isn’t blocked by a firewall. The port number is 24848 by default. You can find it in GlassFish Admin Console:

  1. Navigate to Clusters → myCluster, the tab “Instances”
  2. Click on “remoteInstance” in the “Name” column
  3. The admin port number is the first port in the “HTTP Port(s)” field
Info about the instance on the SSH node

Without this, the admin server will be able to start the instance via SSH but will not be able to communicate with it or detect that it is running.

Step 8: Start the remote instance

To start the new remote instance, you can start the cluster again, as you already did before. Starting a cluster if you already started it before will keep the “instance1” instance running, and will start all other instances which are not running.

  1. Navigate to “Clusters”
  2. Click the checkbox in the “Select” column next to your cluster
  3. Click “Start Cluster” button and wait until the cluster is started

Alternatively, you can start the remoteInstance individually:

  1. Navigate to the “Instances” tab
  2. Select remoteInstance and click on the “Start” button.

Step 9: Verify Load Balancing

This assumes that you’ve already set up a load balancer server, e.g. Apache HTTP server, and the load balancer is configured to support sticky sessions.

Access the tester application you deployed previously through the load balancer and verify that requests are load-balanced between the instances:

http://load-balancer-ip:load-balancer-port/cluster

Replace load-balancer-ip and load-balancer-port with the appropriate values for your load balancer server.

  • Verify that after an HTTP session is created, your requests are served by the same GlassFish instance and your session data remains in the session.
  • After you reset the session, it’s possible that your future requests will be served by a different GlassFish instance

After everything is working, you can undeploy the test application clusterjsp and deploy your own application. Remember to select your cluster as the deployment target, so that your application is deployed to all GlassFish instances in the cluster.

Step 10: Deploy your application to the cluster

Before you deploy your application to the cluster, make sure that all resources that your application requires are deployed to the cluster too. For example, a JDBC resource:

JDBC resource targets

Now, deploy your application to the cluster, similarly as you deployed the tester application before in the Step 5:

  1. In GlassFish Admin Console, navigate to Applications and click the “Deploy…”​ button
  2. In the “Targets” section, click on your cluster in the “Available Targets” column and click the “Add >” button
  3. Configure other deployment properties and click “OK”

Summary

That’s it! You have now created a GlassFish cluster with instances, added an SSH node, and created an instance on this node using the Admin Console. Your application is running on all instances of the cluster. Now you can set up a load balancer server (e.g. Apache HTTP Server) with sticky sessions to proxy incoming requests to GlassFish instances, which we’ll cover in a future article.

See also

Last updated 2024-01-28 13:40:07 +0100

How to: Set up a GlassFish Cluster Read More »

How to: Apache Server as a Load Balancer for your GlassFish Cluster

Leave a Comment / How To ... / / / , , , , ,
This entry is part 2 of 2 in the series Load balancing with Eclipse GlassFish cluster

How to: Apache Server as a Load Balancer for your GlassFish Cluster

Setting up a GlassFish server cluster with an Apache HTTP server as a load balancer involves several steps:

  1. Configuring GlassFish for clustering
  2. Setting up the Apache HTTP Server as a load balancer
  3. Enabling sticky sessions for session persistence

We’ll assume that you’ve already configured GlassFish for clustering. If you haven’t done so, you can follow our guide on how to set up a GlassFish cluster to prepare your own GlassFish cluster. After you’ve prepared it, this tutorial will guide you to configure Apache HTTP Server as a load balancer with sticky sessions.

Table of Contents

Step 1: Install Apache HTTP Server

Install Apache HTTP Server on a machine that will act as the load balancer. This should be a separate machine from machines that run GlassFish cluster instances, mainly for security reasons. While your cluster should be accessible through the Apache server, GlassFish cluster instances shouldn’t be accessible publicly. Therefore, you should configure your firewall or networking rules to only allow access to GlassFish instances only from the Apache server.

On Apt-based systems, like Ubuntu or Debian, you can install Apache HTTP Server from the system repository with the following commands:

sudo apt update
sudo apt install apache2

Step 2: Enable Proxy Modules in Apache HTTP Server

Enable the necessary proxy modules (proxy, proxy_http, proxy_balancer, and lbmethod_byrequests modules) in Apache server:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod proxy_balancer
sudo a2enmod lbmethod_byrequests

Step 3: Configure the load balancer mechanism

Edit the Apache configuration file (usually located at /etc/apache2/sites-available/000-default.conf) and add the following configuration:

<VirtualHost *:80>
  # Basic server configuration, use appropriate values according to your set up
  ServerAdmin webmaster@localhost
  ServerName yourdomain.com

  # access and error log files  
  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined

  # Forward requests through the load balancer named "balancer://glassfishcluster/"
  # Uses the JSESSIONID cookie to stick the sesstion to an appropriate GlassFish instance
  ProxyPass / balancer://glassfishcluster/ stickysession=JSESSIONID failontimeout=On
  ProxyPassReverse / balancer://glassfishcluster/

  # Configuration of the load balancer and GlassFish instances connected to it
  <Proxy balancer://glassfishcluster>
    ProxySet lbmethod=byrequests

    BalancerMember http://glassfish1:28080 route=instance1 timeout=10 retry=60
    BalancerMember http://glassfish2:28080 route=remoteInstance timeout=10 retry=60 
    
    # Add more BalancerMembers for additional GlassFish instances 

  </Proxy>
</VirtualHost>

Replace the following according to your actual setup:

  • yourdomain.com – the domain name of your application (the DNS record should point to the Apache server)
  • glassfish1 – hostname or IP address of your GlassFish instance with name instance1
  • glassfish2 – hostname or IP address of your GlassFish instance with name remoteInstance
  • instance1, remoteInstance – names of your GlassFish instances. If you use different names, adjust them here and make sure that the jvmRoute system property on GlassFish instances is set to the same instance name

Note that:

  • GlassFish instances must be configured with the jvmRoute system property to add GlassFish instance name to the session cookie
  • Values in the route arguments of BalancerMember must be GlassFish instance names matching that member (values set by the jvmRoute system property). The value of the jvmRoute system property should be defined in the GlassFish cluster configuration to the value of ${com.sun.aas.instanceName} to reflect the GlassFish instance name. This value will then be added to the session cookie so that the Apache loadbalancer can match a cookie with the right GlassFish instance.
  • With the configuration failontimeout=On, the load balancer waits at most 10 seconds for the request to complete. If it takes longer, it considers the GlassFish instance as unresponsive and fails over to another instance to process the request. This is to consider GlassFish instances that are stuck (e.g. when out of available memory) as inactive. If some requests take more time to complete, either disable this option, or increase the timeout arguments on BalancerMember
  • The example configuration doesn’t contain HTTPS configuration. We strongly recommend using HTTPS for the Apache virtual host in production, signed with an SSL/TLS certificate, and redirect all plain HTTP requests to equivalent HTTPS requests
  • If you use a different session cookie name, replace JSESSIONID with your custom cookie name

Step 4: Verify firewall configuration

Verify that:

  • Apache server can access the HTTP port of each GlassFish instance (port 28080 by default )
  • GlassFish instances should not be accessible from a public network for security reasons, they should only be accessible from the Apache server

Step 5: Restart Apache HTTP Server

Restart Apache to enable the new modules and apply the configuration changes.

On an operating system that uses SystemD services (e.g. Ubuntu):

sudo systemctl restart apache2

Step 6: Verify the Sticky Session routing

Test your setup by accessing your application through the Apache load balancer. Verify that sticky sessions are working, ensuring that requests within the same session are directed to the same GlassFish instance and requests without a session are routed to a random instance.

You can use the test cluster application available at https://github.com/OmniFish-EE/clusterjsp/releases:

Summary

With these steps, you should have a basic setup of a GlassFish server cluster

  • with an Apache HTTP Server acting as a load balancer
  • sticky session routing for session persistence across requests

In case of an issue with a GlassFish instance, the load balancer should stop sending requests to that instance and keep using the remaining GlassFish instances in the cluster. Once the GlassFish instance recovers from issues, it will rejoin the load balancer and will start receiving requests again.

You can also rely on this mechanism when you need to restart the cluster; instead of restarting the whole cluster at once, you can restart GlassFish instances one by one. While one of the instances is being restarted, Apache server will continue sending requests to the other instances.

How to: Apache Server as a Load Balancer for your GlassFish Cluster Read More »