Using a Redis feeder with Gatling

Gatling is a popular open source load testing tool that allows you to write load tests as code. It is mainly based on Scala which means you get more out of it by writing less code, and allows a great deal of flexibility in terms of how you design your load test scenarios. It can be used to send millions of requests to an application within a small amount of time, emulating different users working with different use cases.

In some cases, it is more practical to send actual data using Gatling without having to randomly generate test data. When sending millions of actual records, Gatling provides the capability of integrating one of many different data feeders, which can be wired up to provide Gatling with a constant stream of data to send to your application.

I was presented with such a need of sending millions of records pulled out of a PostgreSQL database (after some transformation) using Gatling to a set of Lamba-based web services hosted on AWS as part of a load test. Since there was a transformation step and there would be the need of using the same dataset for different scenarios on multiple test runs, I decided to store the data items in Redis after transformation for speed and quick access via Gatling (as Gatling would be running on the same box) I just made sure to use disaster recovery services just in case. 

A quick docker-compose setup for Redis and I had my database:

version: "3.7"
    image: redis:latest
    container_name: data_cache
      - '/etc/redis/data/:/data/'
      - 6379:6379
      - 6379

Now we can move on to initializing and using the Redis feeder within our Simulation class in Gatling. The following imports are needed in the Scala class:

import com.redis._
import io.gatling.redis.Predef._
import io.gatling.redis.feeder.RedisFeederBuilder

Assuming that the items that need to be sent to the application by Gatling are stored in a list on Redis, the feeder can be initialized and integrated as follows:

val dataFeederPool = new RedisClientPool("localhost", 6379)
val myDataFeeder: RedisFeederBuilder = 
  redisFeeder(dataFeederPool, "mydatalist").LPOP

Here we have used the Redis LPOP function, the Gatling Redis feeder provides the capability to use SPOP and SRANDMEMBER as well. (LPOP means items will be popped out of a list in Redis).

Integrating the feeder into a scenario is just as simple:

val myScn: ScenarioBuilder = scenario("Post an item")

After integrating the feeder, you can implement whatever checks, extra parameters or conditional statements that you need.