Undertanding load balancing in Spring Cloud with Ribbon and example

View more Tutorials:

1- What is Load Balancer?

Let's imagine that you have a distributed system that consists of many services (applications) running on different computers. However, when the number of users is large, a service (application) is usually created multiple replicas. Each replica runs on a different computer. At this time, " Load Balancer" appears. It helps to distribute incoming traffic equally among servers.

Bộ cần bằng tải phía Server (Server side Load Balancer)

Traditionally, Load Balancers are components placed at the Server Side. When the requests come from the Client, they will go to the load balancer, and the load balancer will designate a Server for the request. The simplest algorithm used by the load balancer is random designation. Most load balancers in this case are hardware, integrated software to control load balancing.

Client-Side Load Balancer:

When the load balancer is located at Client  side, it will actively decide which server it will send requests to based on some criteria
.
Actually, servers are different. They differ from each other in the following criteria:
  1. Availability: No server also works at all times.
  2. Performance: The speed of servers is different.
  3. Geography: Servers are located in different locations, for instance, they are located in different countries. They may be close to this Client but far from other Client.
Client-side load balancers usually send requests to servers in the same zone, or with a fast response.

2- Netflix Ribbon

Ribbon is part of  Netflix Open Source Software (Netflix OSS) family. It is a library that provides a Client-side load balancer. Because it is a member of the Netflix family, it can automatically interact with Netflix Service Discovery (Eureka).
Spring Cloud creates  APIs to help you to easily use  Ribbon libraries.
OK, We will discuss the main concepts related to the Ribbon :
  1. List of servers.
  2. Filtered list of servers.
  3. Load Balancer
  4. Ping
-
  • List Of Servers:
A list of servers that can respond to a particular service for 1 Client. For example, 1 Client needs information about the weather. There will be a list of servers being able to provide this information. This list consists of servers configured directly in the Client application, and servers discovered by the Client.
  • Filtered List of Servers:
We continue the above example. One Client needs weather information, and a list of servers can provide such information. However, not all of those servers operate, or those servers are too far from the Client; therefore, they responds very slowly. The Client will remove these servers from the list, and eventually it will have a more suitable list of servers  (a filtered list).
  • Load Balancer (Ribbon):
Ribbon is a load balancer. It is a Client -side component. It decides which server will be called (of the filtered list of servers).
There are some strategies to make a decision. But they usually rely on a " Rule Component" to make a true decision. By default, the Spring Cloud Ribbon uses the ZoneAwareLoadBalancer strategy​​​​​​​  (Servers in the same zone as Client).
Rule Component is a smart module. It creates a " Call or uncall" decision. By default, Spring Cloud uses the ZoneAvoidanceRule rule.
  • Ping:
Ping is the way that the  Client uses to rapidly check whether a server is active at that time or not? The default behavior of Spring Cloud is to delegate  Eureka to automatically check this information. However, the Spring Cloud allows you to customize to check in your way.

3- Objective of lesson

It is noted that this lesson is directly related to 2 previous lessons.
  1. Lesson (1): We have created one "Service Registration" (Eureka Server).
  2. Lesson (2): We have created one "ABC Service" application which is a Discovery Client (Eureka Client).
In this lesson, we are going to create a "XYZ Service" application. It is also a Discovery Client (Eureka Client) and it will call  "ABC Service". On the "XYZ Service" application, we will use a load balancer. 
To be able to test this application, we need to run all three applications, of which, "ABC Service" application will be created 5 replicas (simulating that it is running on 5 different computers). When the Client (XYZ Service) calls the "ABC Service", the load balancer will decide which replica of "ABC Service" is called.
OK!, It is an interesting and also quite easy job.

4- Create Spring Boot project

On the  Eclipse, create a   Spring Boot project.
Enter:
  • Name: SpringCloudLoadBalancerRibbon
  • Group: org.o7planning
  • Artifact: SpringCloudLoadBalancerRibbon
  • Description: Spring Cloud Load Balancer (Ribbon)
  • Package: org.o7planning.ribbon
  • The application being used by us will call "ABC Service". It needs a Load Balancer ; therefore, we will declare the use of Ribbon library.
  • This application should also discover other services (applications) running in the distributed system; therefore, it needs to use Discovery library. We will use Eureka Discovery (Eureka Client).
OK, the project has been created.
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
    http://maven.apache.org/xsd/maven-4.0.0.xsd">
    
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.o7planning</groupId>
    <artifactId>SpringCloudLoadBalancerRibbon</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>SpringCloudLoadBalancerRibbon</name>
    <description>Spring Cloud Load Balancer (Ribbon)</description>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.9.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
        <spring-cloud.version>Edgware.RELEASE</spring-cloud.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-eureka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-ribbon</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>


</project>

5- Configure Ribbon

application.yml
spring:
  application:
    name: XYZ-SERVICE
 
server:
  port: 5555

# -- Configure for Ribbon:
 
ping-server:
  ribbon:
    eureka:
      enabled: false # Disable Default Ping
    listOfServers: localhost:8000,localhost:8001,localhost:8002,,localhost:8003
    ServerListRefreshInterval: 15000
    
# -- Configure Discovery Client (Eureka Client).    
# Configure this application to known "Service Registration".

eureka:
  instance:
    appname: XYZ-SERVICE  # ==> This is an instance of XYZ-SERVICE
  client:    
    fetchRegistry: true
    serviceUrl:
#      defaultZone: http://my-eureka-server.com:9000/eureka
      defaultZone: http://my-eureka-server-us.com:9001/eureka
#      defaultZone: http://my-eureka-server-fr.com:9002/eureka
#      defaultZone: http://my-eureka-server-vn.com:9003/eureka   
SpringCloudLoadBalancerRibbonApplication.java
package org.o7planning.ribbon;

import org.o7planning.ribbon.config.RibbonConfiguration;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.cloud.netflix.ribbon.RibbonClient;

@RibbonClient(name = "ping-a-server", configuration = RibbonConfiguration.class)
@EnableEurekaClient
@SpringBootApplication
public class SpringCloudLoadBalancerRibbonApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringCloudLoadBalancerRibbonApplication.class, args);
    }

}
RibbonConfiguration.java
package org.o7planning.ribbon.config;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;

import com.netflix.client.config.IClientConfig;
import com.netflix.loadbalancer.IPing;
import com.netflix.loadbalancer.IRule;
import com.netflix.loadbalancer.PingUrl;
import com.netflix.loadbalancer.WeightedResponseTimeRule;

 
public class RibbonConfiguration {
    
    @Autowired
    private IClientConfig ribbonClientConfig;
 
    @Bean
    public IPing ribbonPing(IClientConfig config) {
        return new PingUrl();
    }
 
    @Bean
    public IRule ribbonRule(IClientConfig config) {
        return new WeightedResponseTimeRule();
    }
    
}

6- Controller

Example1Controller.java
package org.o7planning.ribbon.controller;

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.client.loadbalancer.LoadBalancerClient;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;

@RestController
public class Example1Controller {

    @Autowired
    private DiscoveryClient discoveryClient;

    @Autowired
    private LoadBalancerClient loadBalancer;

    @ResponseBody
    @RequestMapping(value = "/", method = RequestMethod.GET)
    public String home() {

        return "<a href='testCallAbcService'>/testCallAbcService</a>";
    }

    @ResponseBody
    @RequestMapping(value = "/testCallAbcService", method = RequestMethod.GET)
    public String showFirstService() {

        String serviceId = "ABC-SERVICE".toLowerCase();

        // (Need!!) eureka.client.fetchRegistry=true
        List<ServiceInstance> instances = this.discoveryClient.getInstances(serviceId);

        if (instances == null || instances.isEmpty()) {
            return "No instances for service: " + serviceId;
        }
        String html = "<h2>Instances for Service Id: " + serviceId + "</h2>";

        for (ServiceInstance serviceInstance : instances) {
            html += "<h3>Instance :" + serviceInstance.getUri() + "</h3>";
        }

        // Create a RestTemplate.
        RestTemplate restTemplate = new RestTemplate();

        html += "<br><h4>Call /hello of service: " + serviceId + "</h4>";

        try {
            // May be throw IllegalStateException (No instances available)
            ServiceInstance serviceInstance = this.loadBalancer.choose(serviceId);

            html += "<br>===> Load Balancer choose: " + serviceInstance.getUri();

            String url = "http://" + serviceInstance.getHost() + ":" + serviceInstance.getPort() + "/hello";

            html += "<br>Make a Call: " + url;
            html += "<br>";

            String result = restTemplate.getForObject(url, String.class);

            html += "<br>Result: " + result;
        } catch (IllegalStateException e) {
            html += "<br>loadBalancer.choose ERROR: " + e.getMessage();
            e.printStackTrace();
        } catch (Exception e) {
            html += "<br>Other ERROR: " + e.getMessage();
            e.printStackTrace();
        }
        return html;
    }

}

7- Run application

To be able to test this application fully, you need to run 2 applications of the 2 previous lessons ( "Service Registration" and  "ABC Service")
Then, you can run this application directly on the Eclipse and access the following link to view the Eureka Monitor.

Test Load Balancer:

View more Tutorials: