Implementation Guides

Respecting rate limits

Rate limits help API providers, like InsightIQ, maintain service availability while preventing resource overuse by a single API consumer. This guide explains how to handle rate limits effectively when working with InsightIQ or similar APIs.

Breaching a rate limit

When integrating with an API, it is important to stay mindful of the rate limits enforced by the provider. These limits help ensure stability for all users of the platform. However, there are scenarios where exceeding the rate limit might be unavoidable, especially during high-traffic periods or when processing events in real-time workflows.

Example Scenario

At InsightIQ, we recommend fetching the details of a resource as soon as you receive a webhook update about it. Since webhooks typically provide only the resource IDs, you will need to make additional API requests to retrieve the full details.

Now, consider a situation where multiple webhooks arrive in quick succession, each containing multiple resource IDs. As your system processes these webhooks, the number of API requests can increase rapidly, potentially breaching your rate limits.

What to expect after breaching a rate limit

When you exceed a rate limit, the API responds with an HTTP status code 429 Too Many Requests. This response indicates you should pause and retry later.

If your system does not account for rate limits, it may interpret the 429 Too Many Requests response as a standard 4xx error and retry immediately. This can create a feedback loop, worsening the problem and overloading your system.

Handling Rate Limits

Implementing strategies that respect limits and optimize request processing. These practices apply not only to InsightIQ but are essential for interacting with any API that enforces rate limiting.

Backing off

Backing off means waiting before retrying the request after hitting a rate limit. There are several algorithms to determine how long to wait:

  • Constant Backoff: Retry at fixed intervals, such as 2s, 4s, 6s, 8s and so on.

  • Fibonacci Backoff: Retry intervals follow the Fibonacci sequence, such as 1s, 1s, 2s, 3s, 5s, 8s, and so on.

  • Exponential Backoff: Retry intervals double with each attempt, such as 2s, 4s, 8s, 16s, and so on.

While these strategies have their merits, InsightIQ offers a more efficient method: using the Retry-After header to guide your retry timing.

Retry-After

A more effective way to handle rate limits is by using the Retry-After header, which eliminates guesswork and provides precise instructions on when to retry your request.

The Retry-After header can return one of two values:

  1. Seconds: The number of seconds to wait before retrying the request.

  2. HTTP Date: A timestamp indicating when the request can be retried (e.g., Wed, 21 Oct 2023 07:28:00 GMT). Learn more about this format in Mozilla's documentation.

Why Use Retry-After?

  • No Guesswork: The API directly tells you how long to wait, ensuring optimal retries.

  • Simpler Maintenance: No need to hard-code retry intervals; changes to rate limits are automatically handled by the API.

  • Dynamic Flexibility: The API can adapt limits based on traffic patterns, ensuring fair resource distribution. All of this without the client having to worry about it.

Implementation

At InsightIQ, we use the Retry-After header with a value in seconds. Assuming you have a method named call_api to process the requests, here is some pseudo-code for how you might want to handle retries.

fetch_and_retry(request) {
    response = await call_api(request) // the method that actually calls the API and processes the response
    while(response.status == 429) {
        retry_after = response.headers['Retry-After'] // value in seconds from API header
        sleep(retry_after) // time in seconds or sleep(retry_after*1000) if sleep time is in ms
        await call_api(request)
    }
    return response
}

Now instead of calling the call_api method directly every time to process an API request, you would instead pass on the request to the fetch_and_retry method and let it take care of the rate limit.

Conclusion

If you prefer not to build your own rate-limiting logic, there are various libraries and frameworks available in most backend languages to handle rate limits.

Handling rate limits is a critical part of working with APIs. Whether using a custom solution or a library, ensure your implementation respects limits and manages retries efficiently. For further assistance, contact [email protected].

Was this helpful?
Was this helpful?