Load Testing Apache Kafka

---

Overview

Apache Kafka is a distributed streaming platform for publishing and subscribing to streams of records. LoadForge supports Kafka testing via Locust by using the kafka-python library.

Locust Test Script (locust.py)

# locust.py
import time
from locust import User, between, task
from kafka import KafkaProducer, KafkaConsumer

class KafkaUser(User):
    wait_time = between(1, 3)

    def on_start(self):
        # Initialize producer and consumer
        self.producer = KafkaProducer(bootstrap_servers='localhost:9092')
        self.consumer = KafkaConsumer(
            'loadforge_test',
            bootstrap_servers='localhost:9092',
            auto_offset_reset='earliest',
            enable_auto_commit=True
        )

    @task(3)
    def produce(self):
        # Send a message
        self.producer.send('loadforge_test', b'Hello from LoadForge')
        self.producer.flush()

    @task(1)
    def consume(self):
        # Poll for messages
        msg_pack = self.consumer.poll(timeout_ms=1000)
        for tp, messages in msg_pack.items():
            for message in messages:
                _ = message.value

if __name__ == "__main__":
    from locust import run_single_user
    run_single_user(KafkaUser)

---

Notes:

• Install dependencies: pip install locust kafka-python if you wish to test locally.
• Ensure Kafka (and ZooKeeper if needed) is running and the topic loadforge_test exists.