How we load test our API

LoadForge provides an API for it's clients, and given our current size and growth rate we wanted to load test our API capacity. API load testing (aka stress testing) is a popular use case for our platform and we wanted to take this chance to walk you through our load testing and our thoughts.

The Plan

We want to test if our API is capable of handling 250 simultaneous users. People who are new to load tests often overestimate the number of virtual users a system needs to handle to simulate their real load.

Each one of our virtual users will send an API request every 3 to 7 seconds. That means that at 250 virtual users we'll be creating about 60 API requests every second, or 3600 per minute.

We want to primarily test getting a list of Hosts, and Tests, and getting the Results for several completed Tests. This is a decision many people will be faced with when testing an API - what do you actually want to do, and does that include creating items (e.g. PUT/POST requests) or just getting items. When you create items you can create clutter on your account, and for this stage of testing we decided to just test reading results and using those as a benchmark for the API.

Creating the Test

We will (of course) be using LoadForge for this test, and we have added https://loadforge.com as a Host. The next step is creating a test. We used the following settings:

  • Workers to Launch: 1
  • Users to Simulate: 250
  • Spawn Rate: 10

This is saying launch 1 worker (this small test doesn't need many) and run 250 users, scaling up at 10 users per second. We'll hit peak load after 25 seconds in our test.

Then we create our locustfile using the LoadForge Directory for examples. It looks like this:

from locust import HttpUser, task, between

class QuickstartUser(HttpUser):
    wait_time = between(5, 9)
    
    def on_start(self):
        self.client.headers.update({'Authorization': 'Bearer xxxx'})

    @task(1)
    def index_page(self):
        self.client.get("/api/hosts")
        self.client.get("/api/tests")
        self.client.get("/api/result/1488")
        self.client.get("/api/result/1463")       

Test Results

failures.png

As you can see above, we immediately hit our rate limit on API requests per token (at least that's working well!). But you can also see the latency numbers we are getting, the results, etc. This view allows you to live monitor your test as it goes.

Post that, you are then taken to a report where you can investigate errors and more.

We advise reading our guide on Reading Load Test Results for more detail.

Tips

  1. Remember to disable any rate limiting you have on your API, as it's very likely these tests will hit it!
  2. If you create content we advise you also have the load test delete it.
  3. Start small and grow the test, rather than DoS your API into oblivion!

Ready to run that test?
Start your first test within minutes.