Python (OpenAI SDK)

Bulutistan LLMaaS is fully compatible with the official OpenAI Python SDK. Simply change the base_url to use Bulutistan LLMaaS's models.

Installation

pip install openai

Configuration

from openai import OpenAI

client = OpenAI(
    api_key="sk-proj-your-api-key",
    base_url="https://api.bulutistan.ai/v1"
)

Environment Variables

export OPENAI_API_KEY="sk-proj-your-api-key"
export OPENAI_BASE_URL="https://api.bulutistan.ai/v1"

from openai import OpenAI

# Automatically reads from environment
client = OpenAI()

Chat Completions

Basic Usage

response = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is Python?"}
    ]
)

print(response.choices[0].message.content)

With Parameters

response = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[
        {"role": "user", "content": "Write a haiku about programming"}
    ],
    max_tokens=100,
    temperature=0.7,
    top_p=0.9
)

Streaming

stream = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[
        {"role": "user", "content": "Tell me a story"}
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Async Usage

from openai import AsyncOpenAI
import asyncio

async_client = AsyncOpenAI(
    api_key="sk-proj-your-api-key",
    base_url="https://api.bulutistan.ai/v1"
)

async def main():
    response = await async_client.chat.completions.create(
        model="meta-llama/Llama-3.1-8B-Instruct",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    print(response.choices[0].message.content)

asyncio.run(main())

Embeddings

response = client.embeddings.create(
    model="BAAI/bge-large-en-v1.5",
    input="Text to embed"
)

embedding = response.data[0].embedding
print(f"Dimension: {len(embedding)}")

Batch Embeddings

response = client.embeddings.create(
    model="BAAI/bge-large-en-v1.5",
    input=[
        "First text",
        "Second text",
        "Third text"
    ]
)

for item in response.data:
    print(f"Index {item.index}: {len(item.embedding)} dimensions")

Error Handling

from openai import (
    OpenAI,
    APIError,
    RateLimitError,
    AuthenticationError,
    BadRequestError
)

client = OpenAI(
    api_key="sk-proj-your-api-key",
    base_url="https://api.bulutistan.ai/v1"
)

try:
    response = client.chat.completions.create(
        model="meta-llama/Llama-3.1-8B-Instruct",
        messages=[{"role": "user", "content": "Hello"}]
    )
except AuthenticationError:
    print("Invalid API key")
except RateLimitError:
    print("Rate limit exceeded - implement backoff")
except BadRequestError as e:
    print(f"Bad request: {e.message}")
except APIError as e:
    print(f"API error: {e.status_code} - {e.message}")

Retry with Backoff

import time
from openai import RateLimitError

def chat_with_retry(client, messages, max_retries=3):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="meta-llama/Llama-3.1-8B-Instruct",
                messages=messages
            )
        except RateLimitError:
            if attempt == max_retries - 1:
                raise
            wait = 2 ** attempt
            print(f"Rate limited. Waiting {wait}s...")
            time.sleep(wait)

Context Manager

from openai import OpenAI

with OpenAI(
    api_key="sk-proj-your-api-key",
    base_url="https://api.bulutistan.ai/v1"
) as client:
    response = client.chat.completions.create(
        model="meta-llama/Llama-3.1-8B-Instruct",
        messages=[{"role": "user", "content": "Hello"}]
    )

Timeout Configuration

client = OpenAI(
    api_key="sk-proj-your-api-key",
    base_url="https://api.bulutistan.ai/v1",
    timeout=60.0,  # 60 seconds
    max_retries=2
)

Type Hints

The SDK includes full type hints:

from openai.types.chat import ChatCompletion, ChatCompletionMessage

response: ChatCompletion = client.chat.completions.create(...)
message: ChatCompletionMessage = response.choices[0].message
content: str = message.content

Image Generation

Generate images using compatible models:

response = client.images.generate(
    model="your-image-model",
    prompt="A sunset over mountains",
    n=1,
    size="1024x1024"
)

image_data = response.data[0].b64_json

The response contains base64-encoded image data. You can decode and save it:

import base64

with open("output.png", "wb") as f:
    f.write(base64.b64decode(image_data))

Best Practices

1. Use environment variables for API keys
2. Implement retry logic for rate limits
3. Use async for concurrent requests
4. Set appropriate timeouts for your use case
5. Handle errors gracefully in production