Background Jobs

info

This page covers the API for background job processing. For the portal Jobs dashboard, see Jobs Dashboard.

Process long-running inference requests as background jobs.

Overview

Background jobs allow you to:

• Submit requests that may take longer to process
• Continue working while requests are processed
• Track status and retrieve results later
• Avoid keeping a browser or HTTP connection open for long-running inference

Background jobs are recent, Redis-backed job records. Results are retained for 24 hours. Usage and billing records are stored separately by the Usage Service.

Creating Background Jobs

From the Playground

1. Go to Playground
2. Configure your request (model, prompt, parameters)
3. In the Request Mode card, check Background
4. Click Send
5. You'll receive a job ID

Background is a row in the Request Mode card, alongside Stream. It is disabled (with the reason stated on the row) when the model has no async support ("This model does not support async"), or when Stream is currently on ("Turn off Stream to run this as a background job."). The two are mutually exclusive.

From the API

Use background: true on supported inference requests:

response = client.post("/v1/chat/completions", json={
    "model": "model-id",
    "messages": [
        {"role": "user", "content": "Write a detailed analysis..."}
    ],
    "max_tokens": 4096,
    "background": True
})

job_id = response.json()["job_id"]

async: true is still accepted as a compatibility alias, but new integrations should use background: true.

The background body is the synchronous body

background: true does not narrow what a request may contain. A body that is valid synchronously is valid in the background, and vice versa.

That includes messages whose content is an array of content parts rather than a plain string — text, image_url and file parts are all accepted — and messages carrying the developer role. PDF file parts are prepared for the model on the background path exactly as they are on the synchronous one, so an attachment that works in a normal call works as a job with the same body.

Unsupported combinations

Background jobs currently do not support:

• stream: true, since the two are mutually exclusive
• Image generation
• Any audio endpoint. background: true (or the async alias) against /v1/audio/transcriptions, /v1/audio/translations or /v1/audio/speech is rejected with a 400: "Background jobs are not supported for audio endpoints." Audio requests are synchronous
• Some provider-backed models
• callback_url webhooks; sending one is rejected with a 400 (callback_url_unsupported)

Use polling through the Jobs API to retrieve results.

Job Status

Jobs progress through these states:

Status	Color	Description
Pending	Amber	Queued, waiting to start
Retrying	Orange	Retrying after a transient failure
Running	Blue	Currently processing
Completed	Green	Finished successfully
Failed	Red	Error occurred
Timed out	Red	Exceeded the job time limit
Cancelled	Gray	Cancelled by user

Viewing Jobs

Jobs Dashboard

Navigate to Jobs in the sidebar to see:

• Status Overview: Cards showing counts for each status
• Filter by Status: Click a status card to filter
• Job List: All jobs with expandable details

Job Details

Click on a job to see:

• Job ID: Unique identifier
• Created: Timestamp
• Model: Model used
• Input: Your messages and system prompt
• Parameters: All inference settings
• Response: Generated output (when completed)
• Token Usage: Input, output, and total tokens

Managing Jobs

Cancel a Job

For pending or running jobs:

1. Find the job in the list
2. Click the Cancel button
3. Confirm cancellation

Cancelled jobs cannot be resumed.

Auto-Refresh

Toggle Auto-refresh to automatically update job status:

• Useful when waiting for jobs to complete
• Shows a spinner animation when active
• Updates every few seconds

Manual Refresh

Click the Refresh button to manually update the job list.

Token Usage

Completed jobs show token usage:

Input: 150 tokens
Output: 892 tokens
Total: 1,042 tokens

This helps you:

• Track consumption
• Estimate the cost of chat, completion and embedding work
• Optimize future requests

note

Tokens are the billing unit for text work only. Audio is metered differently: per second of input audio for transcription, and per million characters of input text for speech synthesis. Token counts are therefore not a cost estimate for it. Audio cannot be run as a background job in any case; see Models for audio rates.

A job that is already running is still billed

If a background job pushes you past a spend cap, the job completes and is charged. The cap is enforced before a request is accepted, so the refusal lands on your next request rather than on the one already in flight. Plan cap headroom with your largest queued job in mind.

API Reference

Create Background Job

POST /v1/chat/completions

{
  "model": "model-id",
  "messages": [...],
  "max_tokens": 1024,
  "temperature": 0.7,
  "background": true
}

List Jobs

GET /api/v1/jobs
GET /api/v1/jobs?status=completed

Get Job

GET /api/v1/jobs/{job_id}

Cancel Job

POST /api/v1/jobs/{job_id}/cancel

Get Job Stats

GET /api/v1/jobs/stats

Best Practices

When to Use Background Jobs

• Long-form content generation (articles, reports)
• Complex reasoning tasks
• Large context inputs
• Batch processing multiple requests

When to Use Sync/Stream

• Quick questions and answers
• Interactive chat
• Real-time applications
• Short responses

Monitoring Tips

1. Use auto-refresh for active monitoring
2. Filter by "Running" to see in-progress jobs
3. Check "Failed" jobs for error details
4. Export results from completed jobs

Retention

Background job results are retained for 24 hours. If a job result is no longer available, submit a new request. Download important outputs promptly.