Performance testing tutorial: Automation, Gatling, and Jenkins
Performance testing is an integral step in the lifecycle of company applications and products. Amplifying speed, scalability, automated performance tests allows stakeholders to identify all kinds of obstacles in their applications and ensure a smooth user experience. Performance testing is in high-demand across the industry, and it’s a valuable, competitive skill you can add to your resume or profile.
If you’re looking to expand your understanding of performance testing, this article is for you. Today, we will go over the basics of performance testing and teach you the in-demand tools for automated testing: Gatling and Jenkins.
Today, we will cover the following:
- What is performance testing?
- Performance testing tools
- Automated testing with Gatling
- Running Gatling from Jenkins
- What to learn next
Learn the industry-standards for performance testing
Gain the necessary fundamentals of Gatling, Jenkins, and Lighthouse to run automated tests smoothly, including scalability testing and stress testing.
Performance Test Automation 101: Gatling, Lighthouse, & Jenkins
What is performance testing?#
Performance testing is a comprehensive process of checking for the speed, stability, response time, reliability, and resource usage of a program or product under various workloads. Performance testing allows you to identify and eliminate potential performance bottlenecks in your software.
Typically, performance testing is done through a series of quantitative tests done in a lab or a production environment. To identify bottlenecks, the typical parameters used are processing speed, data transfer rate, network bandwidth, workload efficiency, and reliability.
There are five main types of performance testing that can be used to improve different facets of an application’s performance.
- Load testing: Checks if the software can perform optimally under the expected number of users. The objective is to identify areas of performance bottlenecks and fix them before the application is live.
- Stress testing: Checks for the maximum breaking point of an application. This involves utilizing extreme workloads and high traffic to test when an application breaks.
- Soak testing: Also known as endurance testing, soak testing simulates a steady increase in users over time to test for the system’s long-term sustainability.
- Spike testing: Tests the system’s performance under a sudden large spike in users.
- Volume testing: Tests for the system’s performance under varying database volumes by populating a large amount of data and monitoring the overall software system.
Why do we need performance testing?#
Performance testing allows stakeholders to identify bottlenecks within software. A bottleneck is a single point that holds back the overall performance of an application. By identifying bottlenecks, you can highlight areas where the application might fail or lag.
With performance testing, you can identify where the software needs to improve before going public. And without it, applications could face a series of issues, such as slow runtime or inconsistencies across operating systems.
Without performance testing, we encounter all sorts of issues in our code and products, including poor response time, long load time, poor scalability, and bottlenecks.
Performance testing metrics#
Organizations use KPIs (key performance indicators) to evaluate the performance of their software during performance testing. Here are some common metrics:
- Throughput: Number of units of information a system processes over a timespan.
- Memory: The storage space available to a workload Bandwidth: the volume of data per second to move between workloads
- Latency: Amount of time between a user request and the start of a system’s response.
- CPU interrupts per second: The average number of hardware interrupts that a process receives.
- Miscellaneous Private bytes, page faults, disk queue length, maximum active sessions, network bytes total per second.
Performance testing tools#
Now that, we understand the basics of performance testing, let’s learn about the most widely used open-source performance testing tools. In this article, we will learn about Gatling and Jenkins, but it’s important to know what else is out there.
Gatling
Gatling is a load testing tool that uses Akka actors to simulate a large load of users. These tests are written in Scala and use DSL. Gatling is designed for ease of use, high performance, and maintainability. Gatling is a widely popular tool for load testing, with more than 5 million downloads and thousands of companies using it for testing. Gatling offers free versions and a paid enterprise version.
Locust
Locust is an open-source load testing tool written in Python. Locust is distributed and scalable, so it can support a large load of simultaneous users over multiple machines. With Locust, you can define user behavior in code, so there is no need for clunky UIs or XML. In a locust test, a “swarm” of locusts attacks your target site. Each locust’s behavior is configurable, and you can monitor the swarm in real-time.
Jmeter
Jmeter is an open-source load and performance testing tool written in Java. It supports a variety of applications, servers, and protocols including HTTP, TCP, SOAP, Web, LDAP, etc. It is one of the most popular performance testing tools available.
Lighthouse
Lighthouse is used for measuring website performance. It is an open-source, automated tool used to improve the quality of web pages. It offers audits for performance, accessibility, PWAs, SEO, and more. Once you give Lighthouse a URL, it will run audits against the page and generate a report. The metrics can can be used to improve the site.
Automated Testing with Gatling#
In the world of software, there are two kinds of testing: manual and automated. The process that we introduced to you above is for the most part, manual. Test automation involved running tests automatics, managing the test data, and utilizing the results to improve your software quality.
This is typically achieved by writing scripts and using automation testing tools like Gatling, a highly capable load testing tool. The architecture of Gatling is asynchronous as long as the underlying protocol, such as HTTP/HTTPS, is implemented in a non-blocking way.
This architecture allows us to utilize virtual users instead of using dedicated threading. Therefore, running thousands of concurrent virtual users is no problem compared to a thread-based load testing tool like JMeter.
Features of Gatling#
- Rich utilities: Gatling offers a collection of utilities and supports passing test data to load test scenarios from CSV, TSV, JSON, SSV, and Redis.
- UI Recorder: Gatling offers a standalone recorder able to convert HTTP web actions to Gatling test scripts.
- Akka Actors: Gatling uses Aka Actors so that the virtual users make non-block requests, providing maximum efficiency.
- Web Sockets & JMS support Gatling provides support for JMS (Java Message Service), SSE (Server-Side Events), Web Sockets, and MQTT (MQ Telemetry Transport).
- Assertions: Gatling provides assertions and checks that can be performed on the received response.
- Rich HTML report: Gatling provides a feature-rich HTML report, which contains information about the executed scenario, configuration, and statistics about the response.
- Integration with Grafana & StatsD: Gatling also provides ways to persist the execution report by exporting the results to Graphana, Influx Time Series Database, StatsD, etc.
- Jenkins CI/CD integration: Gatling allows us to override the run configuration parameters at runtime, including the reporting configurations. This allows us to easily integrate with CI/CD pipelines.
- Distributed load testing: Gatling allows us to run test scripts against multiple servers that are behind load balancers by distributing the load and making configuration changes.
Gatling Stress Test Simulation#
Stress testing, also known as endurance testing, verifies the system’s error handling capability and its reliability under extreme, prolonged conditions.
Sample:
In the example below, we send requests to https://api.coingecko.com to fetch the cryptocurrency derivatives.
import io.gatling.core.Predef._import io.gatling.http.Predef._import scala.concurrent.duration._class SampleSimulation extends Simulation {val httpProtocol = http.baseUrl("https://reqres.in/api/users").acceptHeader("*/*").doNotTrackHeader("1").userAgentHeader("Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0").disableWarmUp.disableCachingval getScenario = scenario("BasicSimulation - GET").exec(http("GET request").get("/").check(status.is(200)))setUp(getScenario.inject(rampUsers(2) during (2 seconds))).protocols(httpProtocol)}
Create the global HTTP configuration
In the code snippet below, we can create a global HTTP configuration:
- Disabling caching
- Disabling initial warm-up
- Setting the base URL for all the HTTP requests
- Setting the default content-type and accepting headers
val protocols = http.disableCaching.disableWarmUp.baseUrl("https://api.coingecko.com").contentTypeHeader("application/json").acceptHeader("application/json")
Keep the learning going.#
Learn performance testing without scrubbing through videos or documentation. Educative’s text-based courses are easy to skim and feature live coding environments - making learning quick and efficient.
Performance Test Automation 101: Gatling, Lighthouse, & Jenkins
Create the user scenario
In the code snippet below:
-
Creates a user scenario containing an HTTP
execfor making a GET request to/api/v2/derivatives/exchanges, in which the base URL is picked up from global HTTP configuration. -
Asserts the status code and checkers whether the
idfields exist in the response.
val scn = scenario("fetching all derivatives").exec(http("fetch all derivatives").get("/api/v3/derivatives/exchanges").check(status.is(200),jsonPath("$[*].id").exists))
Set up user injection profile
We create the user injection. In our case, we execute the user scenario at a constant rate of 2 times for 10 seconds. Once the simulation is done, we assert whether the number of failed requests is 0.
setUp(scn.inject(constantUsersPerSec(5) during (10 seconds))).protocols(protocols).assertions(global.failedRequests.count.is(0))
Running Gatling from Jenkins#
Continuous Integration (CI) is the first phase of the automated release pipeline. It’s a way to get early feedback on developers’ code changes using an automated test suit. The process:
- Commit code
- Build application source code
- Run automated tests
- Provide feedback
The CI process is set up by using acceptance tests for code changes. Automated performance tests in the CI process can be done using only roughly 5% of the actual load.
This allows us to receive early feedback and evaluate the impact of the newly committed code. If there are any bottlenecks or issues, we can immediately analyze and fix the problem on the spot before going live.
Jenkins setup and install#
The below guide will provide the steps for you to install Jenkins locally or on a remote machine.
- Install the Homebrew package manager
/bin/bash -c "$(curl -fsSLhttps://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
- Install Jenkins
brew install jenkins
- Start the Jenkins service
brew services start jenkins
- Once the Jenkins service starts, navigate to your browser to https://localhost:8080 to complete the installation.
The screen below will ask you for password to complete your installation along with the password location. Follow the instructions below.
Now Jenkins is set up and ready to be used with Gatling!
Install Gatling Jenkins Plugin#
Follow the below steps to install the plugin.
- Login to Jenkins as an admin user.
- Click on the
Manage Jenkinsoption on the left side panel. - Click on
Manage Pluginsoption, as shown below.
Select the available tab, search for Gatling plugin, select the Gatling plugins, and select the Download now option. Install it after restarting.
Once the plugin is installed, you can restart from the UI option as shown below or via the command line using brew, as discussed in the previous lesson.
Once the Jenkins is restarted, you can see the Gatling under installed plugins (manage plugins -> installed plugins).
Now we are all set to use the Gatling plugin for creating our load testing job, which can also be used in the CI flow!
What to learn next#
Congratulations! Now, you should have a good idea of performance testing. These skills are super valuable for any company, so adding them to your resume is a sure way to stand out as a candidate.
There’s still a lot to learn! The next steps to master Gatling and performance testing are:
- Writing and running Gatling test scripts
- Automate web page performance (WPP) with Lighthouse
- Simulating network throttling
- Gatling global assertions
- and more
To get started with these concepts and processes, check out Educative’s course Performance Test Automation 101. In this course, you’ll learn the fundamentals of Gatling for scalability testing and stress testing. You’ll then learn how to write Gatling scripts, understand web page performance, and simulate network throttling.
By the end of this course, you will have some new great skills for your resume.
Happy learning!
Free Resources
