Skip to main content

Node.js (OpenAI SDK)

Bulutistan LLMaaS is fully compatible with the official OpenAI Node.js SDK. Simply change the baseURL to use Bulutistan LLMaaS's models.

Installation​

npm install openai
# or
yarn add openai
# or
pnpm add openai

Configuration​

import OpenAI from 'openai';

const client = new OpenAI({
    apiKey: 'sk-proj-your-api-key',
    baseURL: 'https://api.bulutistan.ai/v1'
});

Environment Variables​

export OPENAI_API_KEY="sk-proj-your-api-key"
export OPENAI_BASE_URL="https://api.bulutistan.ai/v1"
import OpenAI from 'openai';

// Automatically reads from environment
const client = new OpenAI();

Chat Completions​

Basic Usage​

const response = await client.chat.completions.create({
    model: 'meta-llama/Llama-3.1-8B-Instruct',
    messages: [
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user', content: 'What is Node.js?' }
    ]
});

console.log(response.choices[0].message.content);

With Parameters​

const response = await client.chat.completions.create({
    model: 'meta-llama/Llama-3.1-8B-Instruct',
    messages: [
        { role: 'user', content: 'Write a haiku about JavaScript' }
    ],
    max_tokens: 100,
    temperature: 0.7,
    top_p: 0.9
});

Streaming​

const stream = await client.chat.completions.create({
    model: 'meta-llama/Llama-3.1-8B-Instruct',
    messages: [
        { role: 'user', content: 'Tell me a story' }
    ],
    stream: true
});

for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content;
    if (content) {
        process.stdout.write(content);
    }
}

Express.js Streaming​

import express from 'express';
import OpenAI from 'openai';

const app = express();
const client = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY,
    baseURL: process.env.OPENAI_BASE_URL
});

app.post('/chat', async (req, res) => {
    res.setHeader('Content-Type', 'text/event-stream');
    res.setHeader('Cache-Control', 'no-cache');
    res.setHeader('Connection', 'keep-alive');

    const stream = await client.chat.completions.create({
        model: 'meta-llama/Llama-3.1-8B-Instruct',
        messages: req.body.messages,
        stream: true
    });

    for await (const chunk of stream) {
        const content = chunk.choices[0]?.delta?.content;
        if (content) {
            res.write(`data: ${JSON.stringify({ content })}\n\n`);
        }
    }

    res.write('data: [DONE]\n\n');
    res.end();
});

Embeddings​

const response = await client.embeddings.create({
    model: 'BAAI/bge-large-en-v1.5',
    input: 'Text to embed'
});

const embedding = response.data[0].embedding;
console.log(`Dimension: ${embedding.length}`);

Batch Embeddings​

const response = await client.embeddings.create({
    model: 'BAAI/bge-large-en-v1.5',
    input: [
        'First text',
        'Second text',
        'Third text'
    ]
});

response.data.forEach(item => {
    console.log(`Index ${item.index}: ${item.embedding.length} dimensions`);
});

Error Handling​

import OpenAI from 'openai';

const client = new OpenAI({
    apiKey: 'sk-proj-your-api-key',
    baseURL: 'https://api.bulutistan.ai/v1'
});

try {
    const response = await client.chat.completions.create({
        model: 'meta-llama/Llama-3.1-8B-Instruct',
        messages: [{ role: 'user', content: 'Hello' }]
    });
} catch (error) {
    if (error instanceof OpenAI.AuthenticationError) {
        console.error('Invalid API key');
    } else if (error instanceof OpenAI.RateLimitError) {
        console.error('Rate limit exceeded');
    } else if (error instanceof OpenAI.BadRequestError) {
        console.error('Bad request:', error.message);
    } else if (error instanceof OpenAI.APIError) {
        console.error(`API error ${error.status}:`, error.message);
    } else {
        throw error;
    }
}

Retry with Backoff​

async function chatWithRetry(client, messages, maxRetries = 3) {
    for (let attempt = 0; attempt < maxRetries; attempt++) {
        try {
            return await client.chat.completions.create({
                model: 'meta-llama/Llama-3.1-8B-Instruct',
                messages
            });
        } catch (error) {
            if (error instanceof OpenAI.RateLimitError) {
                if (attempt === maxRetries - 1) throw error;
                const wait = Math.pow(2, attempt) * 1000;
                console.log(`Rate limited. Waiting ${wait}ms...`);
                await new Promise(r => setTimeout(r, wait));
            } else {
                throw error;
            }
        }
    }
}

Timeout Configuration​

const client = new OpenAI({
    apiKey: 'sk-proj-your-api-key',
    baseURL: 'https://api.bulutistan.ai/v1',
    timeout: 60000, // 60 seconds
    maxRetries: 2
});

TypeScript​

Full TypeScript support included:

import OpenAI from 'openai';
import { ChatCompletion, ChatCompletionMessageParam } from 'openai/resources';

const client = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY!,
    baseURL: process.env.OPENAI_BASE_URL!
});

const messages: ChatCompletionMessageParam[] = [
    { role: 'user', content: 'Hello' }
];

const response: ChatCompletion = await client.chat.completions.create({
    model: 'meta-llama/Llama-3.1-8B-Instruct',
    messages
});

const content: string | null = response.choices[0].message.content;

Image Generation​

Generate images using compatible models:

const response = await client.images.generate({
    model: "your-image-model",
    prompt: "A sunset over mountains",
    n: 1,
    size: "1024x1024",
});

const imageData = response.data[0].b64_json;

Save the base64-encoded image to a file:

import fs from 'fs';

const buffer = Buffer.from(imageData, 'base64');
fs.writeFileSync('output.png', buffer);

Best Practices​

  1. Use environment variables for API keys
  2. Implement retry logic for production
  3. Use TypeScript for type safety
  4. Handle streams properly to avoid memory issues
  5. Set appropriate timeouts for your use case