Making Your First Request

A detailed guide to making inference requests.

Request Structure

All inference requests follow the OpenAI API format.

Endpoint

POST /v1/chat/completions

Headers

Header	Value	Required
Content-Type	application/json	Yes
Authorization	Bearer {api_key}	Yes

Request Body

{
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ],
  "max_tokens": 100,
  "temperature": 0.7,
  "stream": false
}

Parameters

Parameter	Type	Default	Description
model	string	required	Model identifier
messages	array	required	Conversation messages
max_tokens	integer	none	Maximum tokens to generate. If omitted, the model generates until a stop token or the context window is reached
temperature	float	1.0	Sampling temperature (0-2)
top_p	float	1.0	Nucleus sampling parameter
stream	boolean	false	Enable streaming responses
stop	array	null	Stop sequences

note

The defaults above are OpenAI/vLLM conventions, not values enforced by the platform. The request body is forwarded to the model as-is — any parameter you omit is left for the serving engine to default.

Message Roles

Role	Description
system	Sets the behavior of the assistant
user	Messages from the user
assistant	Previous assistant responses

Response

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1704067200,
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 9,
    "total_tokens": 29
  }
}

Available Models

Check available models:

curl https://api.bulutistan.ai/v1/models \
  -H "Authorization: Bearer sk-proj-your-api-key"

Error Handling

from openai import OpenAI, APIError

client = OpenAI(
    api_key="sk-proj-your-api-key",
    base_url="https://api.bulutistan.ai/v1"
)

try:
    response = client.chat.completions.create(
        model="meta-llama/Llama-3.1-8B-Instruct",
        messages=[{"role": "user", "content": "Hello"}]
    )
except APIError as e:
    print(f"API Error: {e.status_code} - {e.message}")

See Error Handling for complete error reference.