Efficiently Scraping Page Resources

Overview

LoadForge provides a seamless integration with the HttpUserWithResources plugin for Locust, which facilitates the automatic retrieval of crucial web page resources.

While LoadForge offers the automatic retrieval of resources, it's recommended to construct your test in a manner where resources of interest are fetched manually. This is essential because LoadForge won't execute JavaScript. Hence, for AJAX resources, you might want to manually specify them. Alternatively, you can upload a browser recording in the .har format to achieve the same result.

Crafting the Code

The provided code snippet is designed to request the root of your website (i.e., "/"). Additionally, it will download any matches found for the regular expression: .*[^(js|png|css|gif|woff)]$.

To put it simply, it fetches resources with file extensions: .js, .png, .css, .gif, or .woff.

from locust_plugins.users import HttpUserWithResources
from locust import task

class TestUserWithResources(HttpUserWithResources):
    # The following default values can be modified as per requirements:
    # bundle_resource_stats=False
    # default_resource_filter=".*[^(js)]$"

    @task
    def include_resources_default(self):
        self.client.get("/cart")

    @task
    def include_resources_true(self):
        self.client.get("/", resource_filter=".*[^(js|png|css|gif|woff)]$")

    @task
    def include_resources_false(self):
        self.client.get("/dashboard", include_resources=False)

---