Rate Limiting

Understand and handle API rate limits effectively.

How Rate Limiting Works

Rate limits protect the platform and ensure fair usage:

Limit Type	Scope	Reset	Refused with
Requests per minute	Per API key / user	Sliding 60-second window	`429`
Tokens per minute	Per API key / user	Sliding 60-second window	`429`
Tokens per day	Per API key / user	Daily	`429`
Concurrent requests	Per tenant / user	Active connections	`429`
Cost per day	Per tenant / user	Daily	`402`
Per-API-key daily spend cap	One API key	Daily	`402`
Cost per month	Per tenant / user	Monthly	`402`

Daily limits reset at 00:00 UTC; monthly limits reset at 00:00 UTC on the first day of the next calendar month. These reset instants are global and do not move with a user's display timezone.

Money-denominated limits answer `402`, not `429`

The ceilings measured in currency are refused with 402 insufficient_quota, in the billing envelope, not with a 429. That now includes every one that used to answer 429:

the daily cost limit -- your own personal one, or the account-wide one
the per-API-key daily spend cap
the monthly cost limit -- the account-wide one, or the one set on your own user

They join no_credit_line, credit_limit_exceeded and payg_spend_cap_exceeded, so a money refusal has one spelling on this platform, with no exception left.

The monthly ceiling is the last one to change, and it is worth calling out separately: it carries error.code of monthly_cost_cap_exceeded and a limit_type of cost_per_month (account-wide) or cost_per_month_user (your own). It only comes into play where the billing path is not governing the period; for accounts on a wallet or a credit limit, the credit limit and the pay-as-you-go cap govern instead, and both answer 402 as well.

This changes what your SDK raises

An OpenAI-compatible SDK maps 429 to RateLimitError and has no mapping for 402, so an except RateLimitError that used to catch a daily or monthly spend stop no longer does. See migrating your error handling.

The daily money ceilings still clear on the clock at 00:00 UTC, and the monthly one at 00:00 UTC on the first day of the next month; each 402 carries a reset_at saying so, so backing off until then genuinely works. Note how far away a monthly reset_at can be, up to about four weeks: schedule against it rather than looping. The credit limit and the pay-as-you-go cap do not clear on a clock; someone has to settle a balance or change a limit. All of them are documented under Error Handling.

The gates counted in requests, tokens or concurrent connections are unchanged and still answer 429 in the envelope below.

Sliding Minute Windows

Both per-minute limits -- requests and tokens -- are enforced as true sliding windows. The window is kept as 60 one-second buckets and the total is recomputed on every read, so "the last minute" always means the 60 seconds immediately behind the current request, never the current wall-clock minute.

This matters because a fixed minute counter is exploitable at the boundary: a client could spend a full minute's budget in the last second of one minute, and the whole budget again in the first second of the next, putting twice the configured limit through in a two-second span. A sliding window cannot be walked over that way -- capacity is only returned as individual seconds age out of the window.

The practical consequence for your client is that capacity comes back gradually rather than all at once. After a burst that fills the window, the first slot frees up 60 seconds after the burst started, not at the top of the next minute.

Concurrent Limit Accuracy

The concurrent-request limit is enforced atomically: every admission decision (read current count → compare to cap → increment) runs as a single Redis Lua step, so parallel callers cannot squeeze past the cap by hitting the gateway in the same millisecond. A tenant with concurrent_requests=10 will admit exactly 10 in-flight requests at once; the 11th gets a 429 until one of the in-flight ones finishes. The x-ratelimit-active-concurrent header reflects the same atomic counter.

Streams Hold Their Slot

A streaming request holds its concurrency slot for the entire response body, not just until the first token arrives. A stream that takes three minutes to finish occupies one of your concurrent slots for three minutes.

Size concurrent_requests against how many streams you expect to have open at once, not against your request rate. An application with a modest request rate but long streaming responses can sit permanently at its concurrency ceiling while its per-minute limits stay almost untouched. A client that disconnects before the first byte releases the slot immediately, so an aborted request costs nothing further. See the Streaming guide.

Rate Limit Headers

Rate limit headers are included in API responses, but their format differs between successful and rate-limited responses.

On Successful Responses (200)

Successful responses include OpenAI-compatible headers with Unix timestamp reset values, plus additional platform-specific headers:

# OpenAI-compatible headers
x-ratelimit-limit-requests: 100
x-ratelimit-remaining-requests: 95
x-ratelimit-limit-tokens: 100000
x-ratelimit-remaining-tokens: 85000
x-ratelimit-reset-requests: 1704067281
x-ratelimit-reset-tokens: 1704067263

# Platform-specific headers
x-ratelimit-limit-concurrent: 50
x-ratelimit-active-concurrent: 3
x-ratelimit-limit-cost: 1000.00
x-ratelimit-used-cost: 12.50
x-ratelimit-limit-tpd: 10000000
x-ratelimit-used-tpd: 150000

Header	Description
`x-ratelimit-limit-*`	Maximum allowed
`x-ratelimit-remaining-*`	Currently available
`x-ratelimit-reset-*`	Unix timestamp (whole seconds) at which capacity next returns
`x-ratelimit-limit-requests`	Requests per minute limit
`x-ratelimit-remaining-requests`	Requests still available in the sliding minute
`x-ratelimit-limit-concurrent`	Max concurrent requests
`x-ratelimit-active-concurrent`	Currently active concurrent requests
`x-ratelimit-limit-cost`	Cost limit (daily or monthly)
`x-ratelimit-used-cost`	Cost consumed so far
`x-ratelimit-limit-tpd`	Tokens per day limit
`x-ratelimit-used-tpd`	Tokens consumed today

All header names are lowercase.

What the reset headers actually mean

Both reset headers report the sliding window's real shed instant -- the moment the next slice of capacity frees up -- rather than a calendar boundary:

x-ratelimit-reset-tokens is the oldest live bucket plus 60 seconds: the instant the first tokens age out of the window and become spendable again.
x-ratelimit-reset-requests is the current time plus 60 seconds: the point by which everything counted right now has aged out.

They are Unix epoch seconds and they will not land on a wall-clock :00 boundary. A client that rounds them to the next minute, or assumes they mark the start of a fresh quota period, will wait longer than it needs to. Use them as an instant to wait until, and re-read them on the next response. The same values are reported on 200 and on 429, so you can steer proactively rather than only reacting to a refusal.

On Rate-Limited Responses (429)

When a rate limit is exceeded, the response includes the same reset values plus a Retry-After header, and the ceiling that was breached is reported in both header families:

x-ratelimit-limit-requests: 100
x-ratelimit-remaining-requests: 0
x-ratelimit-reset-requests: 1704067281
x-ratelimit-limit-tokens: 100000
x-ratelimit-remaining-tokens: 0
x-ratelimit-reset-tokens: 1704067263
x-ratelimit-limit-tpd: 10000000
x-ratelimit-used-tpd: 10000000
Retry-After: 30

Which headers carry the breach

The breached ceiling is written into the header family whose unit matches it. A tokens-per-day breach is reported in tokens-per-day headers, not smuggled into the per-minute token headers where it would read as an absurdly large minute limit.

Breach (`limit_type`)	Headers carrying the limit and the consumption
`requests_per_minute`	`x-ratelimit-limit-requests` / `x-ratelimit-remaining-requests`
`tokens_per_minute`	`x-ratelimit-limit-tokens` / `x-ratelimit-remaining-tokens`
`tokens_per_day`	`x-ratelimit-limit-tpd` / `x-ratelimit-used-tpd`
`concurrent_requests`	`x-ratelimit-limit-concurrent` / `x-ratelimit-active-concurrent`

The money ceilings follow the same rule on their 402: the breach is written into x-ratelimit-limit-cost / x-ratelimit-used-cost, the family whose unit is dollars.

Both OpenAI-compatible header families are filled on every 429 from a single measurement snapshot, so the request and token numbers you read are consistent with each other and with the moment of refusal. Headers are emitted only for limits that were actually measured -- a limit the platform did not evaluate is left out rather than reported as a fabricated 0.

The cost headers on a `402`

Changing the status of the money ceilings did not take their headers away. A 402 for cost_per_day, cost_per_day_user, api_key_spend_cap, cost_per_month or cost_per_month_user carries x-ratelimit-limit-cost / x-ratelimit-used-cost for the breached ceiling and the request and token families alongside them, exactly as the 429 did -- so a client that reads its remaining budget off a refusal keeps working across the change.

The 402s that are not per-window budgets -- credit_limit and payg_spend_cap -- carry no x-ratelimit-* headers at all. A credit ceiling is not a rate limit, and advertising x-ratelimit-limit-cost for one would describe a window that does not exist.

Rate Limit Errors

When one of the 429 limits is exceeded, you receive an OpenAI-style error envelope. The money ceilings use the 402 envelope documented in Error Handling instead, and it carries the same extra fields.

{
  "error": {
    "message": "Rate limit exceeded for requests_per_minute. Try again in 30 seconds.",
    "type": "rate_limit_error",
    "param": null,
    "code": "too_many_requests",
    "limit_type": "requests_per_minute",
    "limit": 100,
    "current": 100,
    "retry_after": 30,
    "reset_at": "2026-08-20T00:00:21Z"
  }
}

HTTP Status: 429 Too Many Requests

Field	Type	Description
`error.message`	string	`Rate limit exceeded for {limit_type}. Try again in {n} seconds.`
`error.type`	string	Always `rate_limit_error` for 429
`error.param`	null	Always present, always `null` for 429
`error.code`	string	`too_many_requests`
`error.limit_type`	string	Which limit was exceeded (e.g. `requests_per_minute`, `tokens_per_minute`, `tokens_per_day`, `concurrent_requests`)
`error.limit`	number	The configured limit value
`error.current`	number	Current consumption that triggered the violation
`error.retry_after`	integer	Seconds to wait before retrying
`error.reset_at`	string	Exact instant capacity returns, in ISO 8601 UTC (`Z`) form, when the limit has one. For the sliding minute limits this is the shed instant, not a clock boundary
`Retry-After` header	integer	Same value, in HTTP header form

error.message is always a plain string. Older versions of the platform sometimes nested a {message, retry_after} object inside error.message; that is no longer the case. SDKs and UIs that render error.message directly always receive a clean string.

Use error.reset_at when displaying a reset time or scheduling a retry. Convert that instant to the user's IANA timezone for display, but keep UTC as the source of truth. Do not round it to the next minute -- for the sliding minute limits it is a mid-minute instant, and rounding only makes your client wait longer. If reset_at is absent, fall back to retry_after; concurrent-request limits, for example, clear when another request finishes and do not have one fixed reset instant.

The Unix reset header and error.reset_at represent the same whole-second instant. Existing integrations can continue using the headers without changes.

Handling Rate Limits

These retry loops cover 429 only

RateLimitError is raised for the 429s on this page and nothing else. A spend ceiling now answers 402 and will fall straight through these handlers, which is the right outcome -- backing off does not add credit -- but your client needs a branch for it. See Error Handling.

Python with Retry

import time
from openai import OpenAI, RateLimitError

client = OpenAI(
    api_key="sk-proj-your-api-key",
    base_url="https://api.bulutistan.ai/v1"
)

def chat_with_retry(messages, max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="your-chat-model",
                messages=messages
            )
        except RateLimitError as e:
            if attempt == max_retries - 1:
                raise

            # Exponential backoff
            wait_time = min(2 ** attempt, 60)
            print(f"Rate limited. Waiting {wait_time}s...")
            time.sleep(wait_time)

Using tenacity Library

from tenacity import (
    retry,
    stop_after_attempt,
    wait_exponential,
    retry_if_exception_type
)
from openai import RateLimitError

@retry(
    retry=retry_if_exception_type(RateLimitError),
    wait=wait_exponential(multiplier=1, min=1, max=60),
    stop=stop_after_attempt(5)
)
def chat_completion(messages):
    return client.chat.completions.create(
        model="your-chat-model",
        messages=messages
    )

Node.js with Retry

async function chatWithRetry(messages, maxRetries = 5) {
    for (let attempt = 0; attempt < maxRetries; attempt++) {
        try {
            return await client.chat.completions.create({
                model: 'your-chat-model',
                messages
            });
        } catch (error) {
            if (error instanceof OpenAI.RateLimitError) {
                if (attempt === maxRetries - 1) throw error;

                const waitTime = Math.min(Math.pow(2, attempt) * 1000, 60000);
                console.log(`Rate limited. Waiting ${waitTime}ms...`);
                await new Promise(r => setTimeout(r, waitTime));
            } else {
                throw error;
            }
        }
    }
}

Proactive Rate Management

Monitor Remaining Limits

def check_rate_limits(response):
    headers = response.headers
    remaining = int(headers.get('x-ratelimit-remaining-requests', 0))
    reset_timestamp = int(headers.get('x-ratelimit-reset-requests', '0'))

    # Calculate seconds until reset from Unix timestamp
    wait_seconds = max(reset_timestamp - int(time.time()), 0)

    if remaining < 10:
        print(f"Low on requests. {remaining} left. Resets in {wait_seconds}s")
        if remaining < 5:
            time.sleep(max(wait_seconds, 1))

Request Throttling

import time
from collections import deque

class RateLimiter:
    def __init__(self, max_requests_per_minute=60):
        self.max_rpm = max_requests_per_minute
        self.requests = deque()

    def wait_if_needed(self):
        now = time.time()

        # Remove old requests
        while self.requests and self.requests[0] < now - 60:
            self.requests.popleft()

        if len(self.requests) >= self.max_rpm:
            sleep_time = 60 - (now - self.requests[0])
            if sleep_time > 0:
                print(f"Throttling: waiting {sleep_time:.1f}s")
                time.sleep(sleep_time)

        self.requests.append(time.time())

# Usage
limiter = RateLimiter(max_requests_per_minute=50)

for message in messages:
    limiter.wait_if_needed()
    response = client.chat.completions.create(...)

Batch Processing

For high-volume processing, use batching:

import asyncio
from openai import AsyncOpenAI

async_client = AsyncOpenAI(
    api_key="sk-proj-your-api-key",
    base_url="https://api.bulutistan.ai/v1"
)

async def process_batch(messages_list, batch_size=10, delay=1):
    results = []

    for i in range(0, len(messages_list), batch_size):
        batch = messages_list[i:i + batch_size]

        # Process batch concurrently
        tasks = [
            async_client.chat.completions.create(
                model="your-chat-model",
                messages=msgs
            )
            for msgs in batch
        ]

        batch_results = await asyncio.gather(*tasks, return_exceptions=True)
        results.extend(batch_results)

        # Delay between batches
        if i + batch_size < len(messages_list):
            await asyncio.sleep(delay)

    return results

Best Practices

Implement exponential backoff: Start small, increase wait time
Monitor rate limit headers: Adjust behavior proactively
Use request queuing: Smooth out burst traffic
Cache responses: Avoid repeated identical requests
Use async jobs: For batch processing, use async job API
Distribute across keys: Use multiple API keys for higher limits

Your Rate Limits

Rate limits are configured per tenant. Check your current limits:

curl https://api.bulutistan.ai/api/v1/profile/rate-limits \
  -H "Authorization: Bearer <your-jwt-token>"

Or view in the Portal Dashboard.

The money ceilings are largely self-service: your personal daily spending limit, the organization spending cap and a key's own daily spend cap are all set from the portal -- see Cost Limits. The monthly cost ceiling is the exception: the portal's spending limit is a daily one, so a monthly ceiling is set by your administrator and only they can raise it. Contact your administrator for that, and for the request, token and concurrency limits.

Rate Limits API

Get Your Effective Rate Limits

Retrieve your current rate limits with source information:

curl https://api.bulutistan.ai/api/v1/profile/rate-limits \
  -H "Authorization: Bearer <your-jwt-token>"

GET /api/v1/rate-limits is the same payload under a shorter path, and either can be used:

curl https://api.bulutistan.ai/api/v1/rate-limits \
  -H "Authorization: Bearer <your-jwt-token>"

note

Both endpoints also accept a management API key with the rate_limits:read scope in place of a JWT.

Both report your limits after hierarchical resolution, not the platform's process defaults. If you built against an earlier version of GET /api/v1/rate-limits, re-read it: it used to return fixed defaults that ignored the caller entirely, and the numbers it gave -- concurrency in particular -- could be far below the limit actually being enforced for you.

Response:

{
  "requests_per_minute": {
    "value": 1000,
    "source": "platform",
    "is_custom": false
  },
  "tokens_per_minute": {
    "value": 100000,
    "source": "platform",
    "is_custom": false
  },
  "tokens_per_day": {
    "value": 10000000,
    "source": "platform",
    "is_custom": false
  },
  "concurrent_requests": {
    "value": 100,
    "source": "tenant",
    "is_custom": true
  },
  "cost_per_day": {
    "value": 10000.0,
    "source": "platform",
    "is_custom": false
  },
  "cost_per_month": {
    "value": 2000.0,
    "source": "platform",
    "is_custom": false
  }
}

Understanding Limit Sources

Rate limits follow a hierarchy, and the nearest level that sets a value wins -- not the smallest value in the chain:

API Key Level        (consulted first)
    └── User Level
            └── Organization Level
                    └── Plan Level
                            └── Platform Level   (consulted last)

Source	Description
`api_key`	Limit set on a specific API key
`user`	Your personal limit
`tenant`	A limit set for your organization
`plan`	The baseline that comes with your organization's plan
`platform`	System default

Each limit also includes an is_custom field indicating whether the limit was explicitly configured for you (true, for api_key, user and tenant) or is a baseline you inherit (false, for plan and platform).

A limit set for your organization is not capped by the plan or the default

A value set for your organization is bound only by what the platform can store. It is not clamped to the plan's baseline, and it is not clamped to the number the platform ships as its default -- so a limit negotiated for your organization can be many times the default and will resolve, and be enforced, at the negotiated number.

This is a change in resolved values. Any override above the plan's baseline used to be clamped back down to the plan value every time limits were resolved, which meant a negotiated increase over a plan baseline had no effect and this endpoint reported the plan's number. That clamp is gone. If you sized a client against limits you read before this change, re-read them: concurrent_requests in particular may now resolve considerably higher.

Two consequences worth knowing:

A plan change does not move a limit set for your organization. After a plan change the negotiated value stands until an administrator clears it, and clearing the override is what hands the organization back to its plan baseline.
Your personal limit is still bounded by your organization's. A user-level value above the organization's effective limit is refused. A user is a member of an organization, not a party to its contract.

Overrides are also cleared one dimension at a time, so a mixture of sources in a single response is normal and expected: concurrent_requests can read tenant while tokens_per_day beside it reads plan and requests_per_minute reads platform. When one override is cleared, only that dimension falls back to what it inherits; the others keep the values set for them.

The concurrent realtime session limit is the exception to all of this and is unchanged: it is still bound by a fixed platform maximum.

Per-API-key overrides are not reported here

The resolution these endpoints report runs user > tenant > plan > platform. A limit set on the individual API key is enforced but does not appear in the response, so a key carrying its own tighter limit can be refused at a number below the one reported here. The refusal itself is always accurate: read error.limit off the 429 (or the x-ratelimit-limit-* headers) when the two disagree.

Python Example

import requests

jwt_token = "<your-jwt-token>"
base_url = "https://api.bulutistan.ai"

response = requests.get(
    f"{base_url}/api/v1/profile/rate-limits",
    headers={"Authorization": f"Bearer {jwt_token}"}
)

limits = response.json()

print("Your Rate Limits:")
for limit_name, info in limits.items():
    custom = " (custom)" if info['is_custom'] else ""
    print(f"  {limit_name}: {info['value']} (from {info['source']}){custom}")

How Rate Limiting Works​

Money-denominated limits answer 402, not 429​

Sliding Minute Windows​

Concurrent Limit Accuracy​

Streams Hold Their Slot​

Rate Limit Headers​

On Successful Responses (200)​

What the reset headers actually mean​

On Rate-Limited Responses (429)​

Which headers carry the breach​

The cost headers on a 402​

Rate Limit Errors​

Handling Rate Limits​

Python with Retry​

Using tenacity Library​

Node.js with Retry​

Proactive Rate Management​

Monitor Remaining Limits​

Request Throttling​

Batch Processing​

Best Practices​

Your Rate Limits​

Rate Limits API​

Get Your Effective Rate Limits​

Understanding Limit Sources​

A limit set for your organization is not capped by the plan or the default​

Python Example​