Authentication

All API requests require an API key. Create one from the API Keys page.

Pass your key using either header:

Authorization: Bearer YOUR_API_KEY
# or
X-Api-Key: YOUR_API_KEY

Credits

Paid chat completions and video generations require a positive prepaid balance. Usage is deducted after a successful response. Video jobs are billed on the first poll that returns status: "completed" — submitting a job (HTTP 202) is not billed. Free-tier models keep working at $0. GET https://api.fastinfra.ai/v1/models is not gated.

When the wallet is empty, paid chat requests return HTTP 402:

HTTP/1.1 402 Payment Required

{
  "error": {
    "message": "Insufficient API credits. Add funds on the Billing page, or use a free-tier model.",
    "type": "insufficient_quota",
    "code": "insufficient_quota"
  }
}

Rate Limits

Chat completion requests are rate limited to protect GPU capacity and keep latency stable for all users. Production integrations must handle 429 Too Many Requests and honor the Retry-After response header.

Current limits

What is limited

• Rate limited: POST https://api.fastinfra.ai/v1/chat/completions and POST https://api.fastinfra.ai/v1/videos/generations
• Not rate limited: GET https://api.fastinfra.ai/v1/models, GET https://api.fastinfra.ai/v1/videos/jobs/{id}, account pages, and other non-inference routes

429 responses

These conditions return HTTP 429 with Retry-After: 60 and type: "rate_limit_error":

Per-minute limit exceeded — too many chat requests on one key within 60 seconds:

HTTP/1.1 429 Too Many Requests
Retry-After: 60

{
  "error": {
    "message": "Rate limit exceeded. Maximum 120 requests per minute per API key.",
    "type": "rate_limit_error"
  }
}

Concurrent limit exceeded — too many in-flight chat requests on one key, or server-wide capacity is full:

HTTP/1.1 429 Too Many Requests
Retry-After: 60

{
  "error": {
    "message": "Too many concurrent requests. Wait for in-flight requests to finish before retrying.",
    "type": "rate_limit_error"
  }
}

Video queue full — more than a handful of LTX jobs are already waiting on the GPU. Extra submits should stay queued; this 429 only fires when that queue itself is full. Poll existing job ids; retry the POST after Retry-After.

HTTP/1.1 429 Too Many Requests
Retry-After: 60

{
  "error": {
    "message": "Video generation queue is full. Retry shortly.",
    "type": "rate_limit_error"
  }
}

Integration best practices

• On 429, wait at least the number of seconds in Retry-After (typically 60) before retrying.
• Avoid firing many parallel chat calls on the same API key; queue or serialize when possible.
• Use "stream": true for long generations — see Streaming.
• Batch related prompts where your app allows, instead of many tiny back-to-back calls.

Retry example (Python)

import time
from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.fastinfra.ai/v1")

def chat_with_retry(messages, max_retries=3):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="llama3.1:8b",
                messages=messages,
            )
        except Exception as exc:
            if getattr(exc, "status_code", None) != 429 or attempt == max_retries - 1:
                raise
            retry_after = int(getattr(exc, "response", {}).headers.get("Retry-After", 60))
            time.sleep(retry_after)

Chat Completions

Create a chat completion using the OpenAI-compatible endpoint. Pass any model ID from GET https://api.fastinfra.ai/v1/models or the pricing catalog.

Request body

{
  "model": "llama3.1:8b",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello!" }
  ],
  "temperature": 0.7
}

Example (local / free-tier model)

curl https://api.fastinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "messages": [{"role": "user", "content": "Summarize quantum computing in one sentence."}]
  }'

Example (catalog model)

Any model ID from the catalog works the same way — routing is handled automatically. See Model Routing.

curl https://api.fastinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "agentica-org/DeepCoder-14B-Preview",
    "messages": [{"role": "user", "content": "Explain model routing in one sentence."}]
  }'

Response

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "llama3.1:8b",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Quantum computing uses quantum bits..."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 24,
    "total_tokens": 36
  }
}

Streaming (recommended for production)

Streaming is a client choice, not something FastInfra forces. Your app must send "stream": true in the JSON body (or use an SDK with streaming enabled). FastInfra does not turn on streaming for you on non-streaming requests.

• Your side (API caller): set "stream": true on POST /v1/chat/completions. Tokens arrive as Server-Sent Events (data: … lines).
• FastInfra side (automatic): when self-hosted models omit max_tokens, the gateway caps at 2048; sets reasoning_effort: low on Qwen3.6 and gpt-oss-120b unless you opt into thinking; keeps the connection alive with SSE pings during slow streams; fails over to wholesale routes if the primary GPU hangs before the first token.

curl https://api.fastinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -N \
  -d '{
    "model": "gpt-oss-120b",
    "stream": true,
    "messages": [{"role": "user", "content": "Explain inference routing briefly."}]
  }'

Use streaming for any response expected to take more than a few seconds — especially gpt-oss-120b and qwen3.6-27b. It reduces silent wait time, keeps proxies from treating the connection as idle, and lets your UI show partial output immediately.

Video Generations

Text-to-video with synced audio, served on FastInfra GPUs. Use model lightricks/ltx-2.5 (aliases: ltx-2.5, ltx-2.5-distilled). This is not a chat-completions model — call the video endpoint below. Generation takes minutes, so the API is asynchronous: POST returns HTTP 202 with a job id in about a second, then poll GET until status is completed. Waiting on POST until the MP4 is ready will 524 through Cloudflare (~100s).

Pricing (input / output tokens)

Billed like every other FastInfra model: USD per token. Video length is mapped to output tokens so the catalog stays consistent with chat SKUs. Competition (fal.ai LTX-2.5 fast) charges per second of video ($0.09/s at 720p, $0.13/s at 1080p). FastInfra’s default 1280x704 clip is priced at $0.10 per second of output.

Example: a 5-second clip is 5,000 output tokens → $0.50 plus a few cents of prompt tokens.

Request body

{
  "model": "lightricks/ltx-2.5",
  "prompt": "A woman looks at the camera and says, welcome to FastInfra, cinematic lighting.",
  "seconds": 5,
  "size": "1280x704",
  "seed": 42
}

curl

curl https://api.fastinfra.ai/v1/videos/generations \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "lightricks/ltx-2.5",
    "prompt": "A woman looks at the camera and says, welcome to FastInfra, cinematic lighting.",
    "seconds": 5,
    "size": "1280x704"
  }'
# HTTP 202 — copy "id", then poll:
curl https://api.fastinfra.ai/v1/videos/jobs/JOB_ID \
  -H "Authorization: Bearer YOUR_API_KEY"

Python

import base64, json, time, urllib.request

headers = {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
}
req = urllib.request.Request(
    "https://api.fastinfra.ai/v1/videos/generations",
    data=json.dumps({
        "model": "lightricks/ltx-2.5",
        "prompt": "A golden retriever running through a sunny meadow, cinematic, birdsong.",
        "seconds": 5,
        "size": "1280x704",
    }).encode(),
    headers=headers,
    method="POST",
)
with urllib.request.urlopen(req, timeout=60) as resp:
    job = json.load(resp)
job_id = job["id"]

while True:
    time.sleep(2)
    poll = urllib.request.Request(
        f"https://api.fastinfra.ai/v1/videos/jobs/{job_id}",
        headers={"Authorization": "Bearer YOUR_API_KEY"},
    )
    with urllib.request.urlopen(poll, timeout=60) as resp:
        payload = json.load(resp)
    if payload["status"] == "completed":
        break
    if payload["status"] == "failed":
        raise SystemExit(payload.get("error") or "video job failed")

open("clip.mp4", "wb").write(base64.b64decode(payload["data"][0]["b64_json"]))
print(payload["id"], payload["usage"])

Submit response (HTTP 202)

{
  "id": "video_...",
  "object": "video.job",
  "created": 1234567890,
  "model": "lightricks/ltx-2.5",
  "seconds": 5,
  "size": "1280x704",
  "status": "queued"
}

Completed poll (HTTP 200)

{
  "id": "video_...",
  "object": "video",
  "created": 1234567890,
  "model": "lightricks/ltx-2.5",
  "seconds": 5,
  "size": "1280x704",
  "status": "completed",
  "data": [{ "b64_json": "<mp4-bytes-as-base64>", "format": "mp4" }],
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 5000,
    "total_tokens": 5018
  }
}

GPU Pods

Rent dedicated GPU machines with any Docker image — training, fine-tuning, ComfyUI, vLLM, Jupyter, or your own stack. Pods are managed from the GPU Cloud dashboard (not the OpenAI API). Billing is per minute from your prepaid credit balance; stop the pod and compute billing stops immediately.

Getting started

1. Sign in and add credits on the Billing page.
2. Browse the GPU catalog for hourly rates (RTX 4090, A100, H100, H200, and more).
3. Open Deploy, pick a GPU tier, Docker image, ports, and disk size.
4. When status is RUNNING, copy SSH or HTTP endpoints from the pod detail page and connect.
5. Stop when finished — billing for GPU compute ends the moment the pod stops.

Deploy options

Pricing & billing

GPU pod prices include a 8% platform markup on underlying compute and storage. Catalog hourly rates are rounded up so displayed prices are always what you pay.

• Compute — billed per minute while the pod is RUNNING (GPU hourly rate × GPU count, plus running disk).
• Stopped volume — if you attached a persistent volume and stop (but do not terminate) the pod, a lower storage-only rate applies until you terminate or start again.
• Deploy gate — you need enough credits for at least 1 hour(s) of runtime at the pod's hourly rate before deploy or start.
• Auto-stop — if your balance hits zero while a pod is running, it is stopped automatically to prevent overdraft. Add credits and start it again.
• Terminate — permanently deletes the pod and its volume; all billing ends.

Per-minute charges appear on the pod detail page under Billing history. Inference API usage (chat completions) is billed separately — see Credits.

Pod lifecycle

Connecting

When a pod is RUNNING, the detail page lists connection endpoints:

• SSH — ssh root@HOST -p PORT (when SSH is enabled and port 22 is exposed).
• HTTP services — Jupyter, Gradio, or custom apps on exposed HTTP ports appear as HOST:PORT.
• Endpoints refresh — the detail page polls status every 15 seconds while provisioning or running; reload if endpoints are still assigning.

# Example after deploy (values shown on your pod detail page)
ssh [email protected] -p 22001

# Jupyter in browser (if enabled)
http://203.0.113.42:8888

Popular Docker images

• runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404 — PyTorch + CUDA (default on deploy form)
• runpod/tensorflow:2.2.0-py3 — TensorFlow
• Community templates for ComfyUI, Ollama, vLLM — use any public registry image your workload needs

Model Routing

FastInfra serves hundreds of models through one OpenAI-compatible API. You send the same request shape for every model; the gateway automatically routes each call to the cheapest healthy capacity for that model, with automatic failover when a route degrades.

Discover models

• GET https://api.fastinfra.ai/v1/models — full catalog of available model IDs
• GET https://api.fastinfra.ai/v1/models/count — total model count
• Pricing page — searchable catalog with per-token pricing

Routing behavior

For every request, the gateway resolves the route in this order:

1. Free-tier models — served at $0 on Hetzner Ollama capacity (llama3.1:8b, mistral:7b, etc.)
2. Dedicated GPU primary — e.g. gpt-oss-120b on RunPod vLLM, qwen3.6-27b on H200
3. Fallback chain — wholesale providers tried automatically if the primary fails or is saturated

Routing is automatic — your integration stays identical as capacity changes. Enable wholesale API keys in the admin panel so fallbacks activate when self-hosted GPUs are busy or offline.

Wholesale fallbacks (cheapest first)

When a self-hosted route fails or hits its concurrency cap, these wholesale providers are tried in order (requires API keys in admin):

You are billed per token only when traffic actually routes to a wholesale provider. Self-hosted RunPod / Hetzner capacity is used first whenever healthy.

Python example

from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.fastinfra.ai/v1")

response = client.chat.completions.create(
    model="agentica-org/DeepCoder-14B-Preview",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Errors

• 502 — No inference provider available for the requested model. The model is not in the catalog or is not currently enabled.
• 503 — Inference provider is not reachable. Upstream capacity is temporarily offline; retry shortly.

If a model does not appear in GET https://api.fastinfra.ai/v1/models, it may not be enabled on this platform yet. Browse the pricing catalog for models available on this deployment.

List Models

Returns the full model catalog available on this deployment. Each entry includes a stable id to use in chat completion requests.

Response shape

{
  "object": "list",
  "data": [
    { "id": "anthropic/claude-3.5-sonnet", "object": "model" },
    { "id": "deepseek/deepseek-v4-flash-0731", "object": "model" },
    { "id": "llama3.1:8b", "object": "model" }
  ],
  "total_count": 3
}

Use GET https://api.fastinfra.ai/v1/models/count for a lightweight count without downloading the full list.

Currently available (787)

• glm-5p2
• kimi-k2p6
• kimi-k2p7-code
• minimax-m2p7
• nemotron-3-ultra-nvfp4
• nemotron-lightning-3p5-30b-a3b
• qwen3p7-plus
• qwen3p8-2p4t-a95b
• qwen3p8-max
• glm-5p2-fast
• kimi-k3-fast
• agentica-org/DeepCoder-14B-Preview
• aion-labs/aion-2.0
• aion-labs/aion-3.0
• aion-labs/aion-3.0-mini
• aion-labs/aion-rp-llama-3.1-8b
• alibaba/happyhorse-1.0-i2v
• alibaba/happyhorse-1.0-r2v
• alibaba/happyhorse-1.0-t2v
• alibaba/happyhorse-1.1-i2v
• alibaba/happyhorse-1.1-r2v
• alibaba/happyhorse-1.1-t2v
• all-minilm:latest
• allenai/Molmo-7B-D-0924
• amazon/nova-2-lite-v1
• amazon/nova-lite-v1
• amazon/nova-micro-v1
• amazon/nova-premier-v1
• amazon/nova-pro-v1
• anthracite-org/magnum-v4-72b
• anthropic/claude-3-haiku
• anthropic/claude-fable-5
• anthropic/claude-fable-5.1
• anthropic/claude-fable-5.1:batch
• anthropic/claude-fable-5:batch
• anthropic/claude-haiku-4-5
• anthropic/claude-haiku-4.5:batch
• anthropic/claude-opus-4
• anthropic/claude-opus-4-7
• anthropic/claude-opus-4-8
• anthropic/claude-opus-4.1
• anthropic/claude-opus-4.1:batch
• anthropic/claude-opus-4.5
• anthropic/claude-opus-4.5:batch
• anthropic/claude-opus-4.6
• anthropic/claude-opus-4.6:batch
• anthropic/claude-opus-4.7:batch
• anthropic/claude-opus-4.8:batch
• anthropic/claude-opus-5
• anthropic/claude-opus-5:batch

Showing 50 of 787 models. Call GET https://api.fastinfra.ai/v1/models for the full list.

Code Examples

C# (.NET)

var client = new OpenAIClient(
    new ApiKeyCredential("YOUR_API_KEY"),
    new OpenAIClientOptions { Endpoint = new Uri("https://api.fastinfra.ai/v1") });

var chat = client.GetChatClient("llama3.1:8b");
var response = await chat.CompleteChatAsync("Hello!");
Console.WriteLine(response.Value.Content[0].Text);

Python (free-tier / local model)

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.fastinfra.ai/v1"
)

response = client.chat.completions.create(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Python (provider model)

from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.fastinfra.ai/v1")

response = client.chat.completions.create(
    model="agentica-org/DeepCoder-14B-Preview",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

JavaScript

const response = await fetch("https://api.fastinfra.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "agentica-org/DeepCoder-14B-Preview",
    messages: [{ role: "user", content: "Hello!" }]
  })
});

const data = await response.json();
console.log(data.choices[0].message.content);