Background Jobs

info

This page covers the API for background job processing. For the portal Jobs dashboard, see Jobs Dashboard.

Process long-running inference requests as background jobs.

Overview

Background jobs allow you to:

• Submit requests that may take longer to process
• Continue working while requests are processed
• Track status and retrieve results later
• Avoid keeping a browser or HTTP connection open for long-running inference

Background jobs are recent, Redis-backed job records. Results are retained for 24 hours. Usage and billing records are stored separately by the Usage Service.

Creating Background Jobs

From the Playground

1. Go to Playground
2. Configure your request (model, prompt, parameters)
3. Toggle Background on
4. Click Send
5. You'll receive a job ID

From the API

Use background: true on supported inference requests:

response = client.post("/v1/chat/completions", json={
    "model": "model-id",
    "messages": [
        {"role": "user", "content": "Write a detailed analysis..."}
    ],
    "max_tokens": 4096,
    "background": True
})

job_id = response.json()["job_id"]

async: true is still accepted as a compatibility alias, but new integrations should use background: true.

Unsupported combinations

Background jobs currently do not support:

• stream: true
• Image generation
• Some provider-backed models
• callback_url webhooks

Use polling through the Jobs API to retrieve results.

Job Status

Jobs progress through these states:

Status	Color	Description
Pending	Amber	Queued, waiting to start
Retrying	Orange	Retrying after a transient failure
Running	Blue	Currently processing
Completed	Green	Finished successfully
Failed	Red	Error occurred
Timed out	Red	Exceeded the job time limit
Cancelled	Gray	Cancelled by user

Viewing Jobs

Jobs Dashboard

Navigate to Jobs in the sidebar to see:

• Status Overview: Cards showing counts for each status
• Filter by Status: Click a status card to filter
• Job List: All jobs with expandable details

Job Details

Click on a job to see:

• Job ID: Unique identifier
• Created: Timestamp
• Model: Model used
• Input: Your messages and system prompt
• Parameters: All inference settings
• Response: Generated output (when completed)
• Token Usage: Input, output, and total tokens

Managing Jobs

Cancel a Job

For pending or running jobs:

1. Find the job in the list
2. Click the Cancel button
3. Confirm cancellation

Cancelled jobs cannot be resumed.

Auto-Refresh

Toggle Auto-refresh to automatically update job status:

• Useful when waiting for jobs to complete
• Shows a spinner animation when active
• Updates every few seconds

Manual Refresh

Click the Refresh button to manually update the job list.

Token Usage

Completed jobs show token usage:

Input: 150 tokens
Output: 892 tokens
Total: 1,042 tokens

This helps you:

• Track consumption
• Estimate costs
• Optimize future requests

API Reference

Create Background Job

POST /v1/chat/completions

{
  "model": "model-id",
  "messages": [...],
  "max_tokens": 1024,
  "temperature": 0.7,
  "background": true
}

List Jobs

GET /api/v1/jobs
GET /api/v1/jobs?status=completed

Get Job

GET /api/v1/jobs/{job_id}

Cancel Job

POST /api/v1/jobs/{job_id}/cancel

Get Job Stats

GET /api/v1/jobs/stats

Best Practices

When to Use Background Jobs

• Long-form content generation (articles, reports)
• Complex reasoning tasks
• Large context inputs
• Batch processing multiple requests

When to Use Sync/Stream

• Quick questions and answers
• Interactive chat
• Real-time applications
• Short responses

Monitoring Tips

1. Use auto-refresh for active monitoring
2. Filter by "Running" to see in-progress jobs
3. Check "Failed" jobs for error details
4. Export results from completed jobs

Retention

Background job results are retained for 24 hours. If a job result is no longer available, submit a new request. Download important outputs promptly.