Simulating Realistic Traffic: Building a Demo GraphQL API at Stellate

At Stellate, we build GraphQL tooling for companies operating GraphQL at scale. That means features like edge caching, rate limiting, and metrics for your GraphQL APIs.

We often need to demo these features in sales calls or at conferences. This led us to think about designing a realistic demo API that we could use for such purposes, that did not have any sensitive customer data.

This Demo API turned out to be a catalyst for boosting the productivity of our developers. It helped us to move faster, to discover bugs in our CDN, and be a central piece of our E2E tests.

This whole process had 2 outcomes:

• A GraphQL API we call the Demo API. This is a GraphQL API that simulates an ecommerce shop, with users, products, and shopping carts.

• A traffic simulation system, that sends requests to multiple Stellate services using the Demo API as the origin, this server sends thousands of requests per minute, simulating semi-realistic traffic patterns.

This blog post goes into the implementation details, it is broken up in a few sections:

1. Designing a Schema: Considerations for designing the demo GraphQL API.

2. Technical Implementation: What tech we chose to build with and how did we do it.

3. Simulating Traffic: How did we go about simulating semi-realistic traffic.

4. Benefits: What were the benefits aside from better demo data.

Designing a Schema

Our first task was deciding what kind of Demo API we wanted to create. After evaluating the use cases that benefit most from Stellate, we settled on creating an ecommerce shop API, with the following entities:

• User: The customer of our ecommerce store.

• Cart: The shopping cart that belongs to a single user and gets emptied once the user completes their order.

• Product: A product of the store with images, price, description, and stock information.

• Order: Whenever a user checks out and completes their order, their cart is converted to an order to be stored with details like prices, taxes, shipping address, etc.

• Promo Code: A promotional code, these are assigned to a user, a user can have many assigned promo codes. When checking out, if applied it will reduce the order price.

Once we had these in place, we started designing an idiomatic GraphQL API around what we wanted to show in the UI. We did not plan on building a UI right away, but we found it helped us stay as close to a real use case as possible and made it easier to design a realistic schema.

Technical Implementation

We chose to build the demo API using TypeScript since most of our engineers are familiar with it, and we went with our GraphQL server of choice, Yoga, and our schema builder of choice, Pothos.

For deployment, we were looking for something simple that needed little management and had state/storage primitives, so we went with Cloudflare Workers as these met our requirements.

The first version we did, used a combination of Durable Objects (strongly consistent KV storage) and R2 (bucket storage) to maintain state, but we eventually migrated to D1 (sqlite on the edge) which made it easier to make state mutable.

To generate data, we used fakerjs. We created a bunch of functions to generate entities, they look something like this:

function createUser(): GeneratedUser {
  return {
    id: getUserId(),
    name: faker.person.fullName(),
    email: faker.internet.email(),
    profilePicture: createImage(),
    shippingAddressStreet: faker.location.streetAddress(),
    shippingAddressCity: faker.location.city(),
    shippingAddressCountry: faker.location.country(),
    shippingAddressZip: faker.location.zipCode(),
    shippingAddressState: faker.location.state(),
    // `createPromoCode` is similar to `createUser`
    promoCodes: faker.helpers.multiple(createPromoCode, {
      count: {
        max: 3,
        min: 1,
      },
    }),
  }
}

// helper to generate uniqueIds
const getUserId = memoizeUnique(faker.string.uuid)

We have multiple of these functions to simulate creating the different entities we have. These functions get used in a script that we run whenever we change the data schema (meaning we add or remove a field from an entity). This script will do some codegen to generate a TypeScript and a SQL file:

• identifiers.ts this file includes the list of the generated product/user ids. We use when we send GQL operations from the traffic simulation server to seed the variables for the operations.

• seed.sql this is the seed file for our D1 database. It creates the DB schema and seeds it with a bunch of data including users, products, carts, etc. Whenever something changes in the schema, we can reset the whole DB again, as this is not mission critical data it is fine to drop the DB and recreate.

For generating these files, we just output strings and use NodeJS fs.writeFile to write the contents, we are creating really simple files so we decided to keep it simple.

As this data is stored in D1, we can use SQL queries in our Demo API resolvers to read and modify the data. This keeps the data stateful and makes simulating changes possible.

We choose to make the API randomly respond with an error to make it more realistic - real systems do sadly have errors. This was especially important for our use case as it allowed us to exercise the features related to errors in the Stellate Dashboard like alerting, error aggregation and error replay.

Simulating Traffic

Now having our demo API in place, we went ahead and created a few Stellate services using this API as the origin. The data simulation service is a separate service from the demo API. We have a list of GraphQL operations we want to simulate, and then we have a JS config object that states how many requests we should sent per operation.

This server runs on fly.io, it is a simple NodeJS app running 24/7, it uses a cron schedule to send minutely requests to our demo API. It will send a random amount of requests per minute, and it has min/max boundaries that depend on the hour of the day, simulating day/night shopping cycles, so that some hours have higher traffic than others.

We considered using Cloudflare Workers scheduled jobs for this task, but as workers limit how many outgoing requests you can send to 1000 per scheduled cron job, this limit does not work for us. This would be ok if we only had a single Stellate service bound to this demo API, but we simulate traffic in a few different environments meaning we can send > 1000 of requests per minute to different Stellate services. This limitation led us to just run a good ‘ol NodeJS server for the job.

Additionally to this server, we build a small CLI that allows us to simulate traffic to our Demo API from our laptops using the same logic. The CLI allows us to send a set amount of requests or simulate specific requests such as requests that will cause errors. We can use this tool to both simulate traffic in the production Demo APIs, or also to simulate traffic locally when developing.

Benefits

The initial goal of making it easier to show the product off in demo/sales calls with realistic data that makes use of most of the dashboard features has certainly been achieved 🎉.

But after almost a year of usage of this system we’ve found a few unexpected benefits:

• Public demo: Seeing how well customers respond to the demo’s we’re also working on making the demo dashboard public and including it in our onboarding flow 👌 This will likely increase our conversion rate using these assets we already built.

• Improved internal documentation: Now when we implement a new feature we’ve gotten in the habit of implementing it for the demo API. It’s lowered the bar for dog fooding and also means that anyone brushing up on a feature can pull up the demo API and easily see it in action. This allows our internal documentation to easily become interactive.

• Faster local development: Previously we would have to manually create traffic on our local machine to test the features we are working on. Now we no longer need to manually construct and send relevant GraphQL request, but can instead just run a quick command in our terminals and get thousands of request that exercise all our features to test with.

• Better E2E tests: We also use this API in our E2E tests and it works great to test specific scenarios (e.g. introducing artificial delays and testing that our CDN layer is properly handling GraphQL defer/stream). We actually caught a few bugs this way!

Our demo service hasn’t entirely eliminated the need to make requests manually during development and testing nor will it ever. But rather it’s become extra tool in our toolbelt to aid in local development and makes it easier to test specific scenarios and spot issues before we go live.

If you want to see how this demo API looks like, we recently launched a public demo of our dashboard at stellate.co/demo.

We hope you enjoyed this read and that it might inspire you to make your own demo service for your multi tenant product. if you want to learn more, do not hesitate to reach out!