Oracle Cloud – add new nodes to your loadbalancer automatically

Services that requiring you to balance requests over multiple backend services are very common. Moving away from monolith based applications and building more smaller components increases the need for good load balancing. Adding to this that within a cloud environment the number of instances can scale up and down whenever required makes it a requirement for a loadbalancer to quickly adopt to the scaling up and scaling down of nodes.

When you design your solution right the configuration of your loadbalancer should automatically adopt changes in your landscape. As an example, if a new instance of a service is created it should automatically result in the fact that this node is added to the loadbalancer configuration. Additionally, when removing an instance this should result in the fact that this instance is removed from the loadbalancer in a graceful manner.

Oracle Cloud Load balancers
As part of the cloud offerings Oracle provides a load balancing service. When deploying your cloud based applications you can leverage the Oracle Cloud load balancing service and ensure that it spans multiple availability domains.

The below diagram showcases a simple deployment with a high available load balancer service available in two availability domains which will balance the load over six machines who are spread over three availability domains

The above diagram showcases a simple implementation of the Oracle load balancer service in the Oracle cloud. You can use this blueprint as a starting point for building your more complex and sophisticated deployments for enterprise deployments in the Oracle cloud. 

Design for automation

The intention of your designs and architecture should be to support full automation of the loadbalancing process. The processes of adding and removing an instance of your backend services should never result in manual actions which need to be performed by a human. 

Taking this into account when you design your solution will give you a good starting point to ensure your solution is elastic and capable of reacting to changes in the landscape without any additional effort. 

When building the logic for your bootstrapping to ensure automated registration and de-registration of your backend nodes you can leverage the APIs from the Oracle Load Balancer. The two main choices you have are using either the REST APIs or the SDK to call out to the loadbalancer from your bootstrap logic. 

The REST APIs provide you an easy way to work with the loadbalancer in a programmatic manner, this will enable you to call the API endpoints with any programming language you like. As an example, you could use bash scripting under Oracle Linux for your bootstrap process and call out to the API endpoints to register and de-register your nodes. More information on the APIs can be found at this page. 

The SDK(s) for Oracle Cloud infrastructure, including the Oracle Cloud loadbalancer are provided in a number of programming languages such as Java, Go, Python and Ruby. As an example you can read the documentation of the Python SDK used to work with the Oracle Cloud Loadbalancer at this page. 

Bootstrap your services

A big part of ensuring that your solution is capable of automatically register and de-register instances at the Oracle Cloud Load Balancer is ensuring the bootstrapping of your instance is done right. This is especially of importance when you need to balance load over custom components that run on a virtual environment in the Oracle Compute Cloud. 

The importance in this is to ensure your services, virtual machine images, have a good bootstrapping. The bootstrapping should ensure both the registration and de-registration of an instance with the loadbalancer. 

The importance in registration and de-registration is that you take the service into account and not the running operating system. This means that part of your bootstrap will require checking if the service itself is up and running, only after the service is available you can register the service at the loadbalancer. Having the operating system up and running and not the service will result in requests being routed to the new node while it is not able to serve requests yet.

Design concept positioning

The concept of adding and removing instances automatically to your load balancer can be named as “Automatic load balancing registration” and could form a part of your enterprise architecture blueprint library. Including the concept in your enterprise architecture library and ensuring all your deployments are done in this manner will ensure a mature and unified way of working across all your solutions.