LLM API pricing per million tokens explained

How token counting works

LLM pricing is calculated in tokens, not characters or words. A token typically represents a subword unit, ranging from a single character to a full word, depending on the tokenizer. FastInfra normalizes these counts across 785 models using their native tokenizers, ensuring you pay for actual computational units rather than approximations.

Input and output tokens are priced separately. Input tokens include the system prompt, user message, and any retrieved context. Output tokens are the tokens generated by the model in its response. Most models charge more for output tokens because generation requires higher computational intensity than processing input context.

Tokenization variations

Different models use different tokenizers. The same English sentence may tokenize to 10 tokens in one model and 12 in another. Non-English languages and code often fragment into more tokens due to lower vocabulary coverage. Always verify token counts for your specific workload using the model's official tokenizer library or FastInfra's documentation at /docs.

Context window limits

Every model has a maximum context window, defined in tokens. Exceeding this limit causes the request to fail or truncates the input. When your context approaches the limit, you must compress prompts or retrieve only the most relevant documents. Some newer models support windows exceeding 100,000 tokens, but latency and cost increase linearly with input size.

Checking token usage

FastInfra returns token counts in every API response. You can parse these fields to track usage per request or aggregate them for billing estimates.

{
  "choices": [...],
  "usage": {
    "prompt_tokens": 1024,
    "completion_tokens": 256,
    "total_tokens": 1280
  }
}

Use these exact values to calculate costs. Do not rely on character counts or word processors, as they do not align with model tokenization rules. For detailed cost examples, see /blog/topic/pricing-guides.

Median prices across the catalog

Median prices establish a baseline for budgeting before you optimize for specific workloads. The catalog includes hundreds of models ranging from small, fast open-weight architectures to large, high-capability proprietary systems. This variance creates a wide price distribution, but the medians offer a stable reference point for standard chat applications. Current median input pricing across the active catalog is 0.2835 per million tokens. Output pricing averages 0.882 per million tokens. The ratio between output and input costs is approximately 3.1. This disparity matters because output tokens typically require more compute resources to generate than input tokens to process.

Use these medians to estimate costs for mixed workloads. If your application generates long responses, output costs will dominate your bill. If you are processing large documents with short summaries, input costs will be higher.

Median pricing benchmarks for standard chat models

Metric	Value
Median Input Price	0.2835 / 1M tokens
Median Output Price	0.882 / 1M tokens
Output/Input Ratio	3.1

These figures apply to on-demand inference via the standard API. They do not include discounts from reserved capacity or dedicated hardware. For a complete breakdown of individual model rates, see the pricing page.

When comparing models, ignore the median and look at the specific model card. A model priced below the median might lack the context window or reasoning capabilities your application requires. Conversely, a model above the median might handle complex logic with fewer retries, reducing total cost despite higher per-token rates.

Track your actual token usage against these medians for the first month. Most applications deviate from the median due to specific prompt structures or response lengths. Use the API documentation to implement precise token counting on your client side.

Cheapest paid chat models

Production workloads require predictable latency and availability. The cheapest paid chat models on FastInfra balance cost against performance guarantees. These models handle standard instruction following, summarization, and code assistance without the overhead of frontier-class parameters. The input price for the most economical paid chat model is 0.0179. Output tokens typically cost slightly more than input tokens. This pricing structure incentivizes concise prompts and structured outputs.

The 10 cheapest paid chat models on FastInfra by combined input and output price (USD per 1M tokens)

Model	Input $/1M	Output $/1M	Routes
Qwen/Qwen2-1.5B-Instruct	$0.021	$0.021	2
mistralai/mistral-nemo	$0.0305	$0.0315	1
mistralai/Mistral-Nemo-Instruct-2407	$0.0305	$0.0315	1
meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo	$0.021	$0.042	2
inclusionai/ling-3.0-flash	$0.0221	$0.0662	2
Sao10K/L3-8B-Lunaris-v1-Turbo	$0.042	$0.0525	1
sao10k/l3-lunaris-8b	$0.042	$0.0525	1
openai/gpt-oss-20b	$0.0189	$0.0945	4
meta-llama/llama-3.2-1b-instruct	$0.063	$0.063	2
meta-llama/llama-3.2-3b-instruct	$0.063	$0.063	2

Lower cost often correlates with smaller context windows or reduced reasoning depth. Evaluate your use case against these trade-offs. If your application relies on simple classification or short-form generation, the lowest-tier models provide sufficient accuracy at a fraction of the cost of larger models. To use a specific model, specify it in the API request body. The routing layer automatically selects the lowest-cost provider that matches the model name.

{
  "model": "llama-3.1-8b-instruct",
  "messages": [
    {"role": "user", "content": "Summarize this text."}
  ]
}

Verify the exact model name and availability in the model catalog. Some models may have rate limits or availability constraints during peak hours. For high-throughput applications, consider reserving capacity or using dedicated instances to avoid contention. The pricing page provides detailed breakdowns for all models. Compare input and output rates to estimate total costs for your expected request volume. Remember that token counting varies slightly between models, so monitor actual usage during integration.

Free tier model limits

The free tier provides access to 12 models without requiring a credit card. These endpoints are useful for prototyping, unit testing, or building demos where cost is not a factor. The API structure remains identical to paid models, ensuring your code works seamlessly when you switch to a paid model later. Free models share compute resources with paid traffic. This introduces variable latency and lower rate limits compared to dedicated or paid shared instances. Do not use free models for production workloads that require strict latency SLAs or high throughput. Key constraints for the free tier include:

• Rate limits: Requests are throttled to prevent abuse. Exact limits vary by model but are significantly lower than paid tiers.
• No SLA: Availability is best-effort. Endpoints may be temporarily unavailable during peak load or maintenance.
• Context window: Some free models enforce shorter maximum context lengths than their paid counterparts.
• Priority: Paid requests are processed first. Free requests wait in a lower-priority queue.

To test a free model, use the standard OpenAI-compatible endpoint. Replace the model identifier with one of the free options, such as `llama-3-8b`.

curl https://api.fastinfra.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3-8b",
    "messages": [{"role": "user", "content": "Say hello"}],
    "max_tokens": 50
  }'

The response structure matches paid models. You can verify the model used in the response object. If you encounter rate limit errors, reduce your request frequency or switch to a paid model. For a complete list of available free models and their specific limits, see the model catalog. When you are ready to move to production, you can switch to a paid model by changing the model identifier. Your existing code does not need modification. The pricing page details the cost per token for each paid model.

Calculating cost per request

Estimating API costs requires separating input and output tokens, as most models charge differentials apply to generation. Input tokens are the prompt, context window, and system instructions. Output tokens are the model’s response. FastInfra applies per-token pricing to both streams, so you must calculate them independently before summing the total.

Use this formula for a single request:

total_cost = (input_tokens * input_price) + (output_tokens * output_price)

For example, if a model charges 0.2835 for input and 0.882 for output, a request with 1,000 input tokens and 500 output tokens costs:

• Input: 1,000 × 0.2835
• Output: 500 × 0.882
• Total: Sum of both values

Output tokens typically cost more than input tokens. The average ratio across the catalog is 3.1. If your application generates long responses, output costs will dominate the bill. Short prompts with concise answers keep costs lower.

Batching multiple requests into a single API call does not reduce per-token prices. Each token is billed individually. However, batching reduces overhead from HTTP headers and latency, which matters for throughput, not direct token spend.

To verify your estimates, use the pricing calculator at /pricing. It lets you input expected token volumes and select specific models from the 785 available. This helps you compare costs before deploying.

If your usage exceeds the free tier limits for the 12 free models, you will incur charges. Monitor your dashboard to track actual spend against estimates. Discrepancies often arise from unexpected context window growth or recursive tool calls that generate additional input tokens.

When to rent GPU pods

API pricing scales linearly with usage. Renting GPU pods shifts the cost structure to a fixed hourly rate. This trade-off favors high-throughput workloads, private model fine-tuning, or latency-sensitive applications where network overhead matters. Use GPU pods when:

• You process more than 49 million tokens per hour.
• You need to run custom models not available via the public API.
• You require deterministic latency without queueing delays.
• You must keep data on isolated hardware for compliance.

For lower volumes, the per-token API remains cheaper because you pay only for what you use. Pods charge for reserved capacity regardless of utilization. To estimate the break-even point, compare the hourly pod cost against your average daily token volume. If your daily spend on the API exceeds the monthly equivalent of the pod rental, switch to dedicated hardware. You can spin up a pod via the API:

curl -X POST https://api.fastinfra.ai/v1/pods \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"model": "llama-3-70b", "gpu_type": "a100", "count": 1}'

The response returns an endpoint URL and credentials. Route your inference traffic to this private endpoint. Monitor utilization through the dashboard. Scale down when demand drops to avoid idle costs. For detailed cost comparisons, see LLM pricing guides.

Routing for lowest cost

Automatic routing shifts cost optimization from manual selection to infrastructure-level logic. You define the quality floor; the platform handles the provider negotiation. This prevents overpaying for marginal latency improvements when a cheaper model meets your accuracy requirements. The routing engine evaluates three constraints for every request: 1. **Latency SLA**: The maximum acceptable time to first token and total completion time. 2. **Quality Threshold**: Minimum acceptance rate or specific model capabilities (e.g., function calling, context window size). 3. **Cost Ceiling**: The hard limit on price per million tokens. If a request targets a specific model, routing respects that choice. If you pass `auto` or omit the model parameter, the engine scans the active catalog of 785 options. It selects the provider offering the lowest 0.2835 for input and 0.882 for output that satisfies your SLA. This often means swapping between providers mid-session if one experiences congestion or price spikes. Configure the behavior via the `routing` parameter in your API call:

curl https://api.fastinfra.com/v1/chat/completions \
  -H "Authorization: Bearer $FASTINFRA_API_KEY" \
  -d '{
    "model": "auto",
    "routing": {
      "max_latency_ms": 2000,
      "min_quality_score": 0.85,
      "prefer_lowest_cost": true
    },
    "messages": [{"role": "user", "content": "Summarize this text."}]
  }'

This approach eliminates the need to maintain a static list of fallback models. When the primary low-cost provider throttles, the request automatically shifts to the next available option within your budget. You pay for the actual tokens processed, not for reserved capacity or unused headroom. Routing works best for non-deterministic workloads like summarization, translation, or general chat. For tasks requiring strict reproducibility, pin the model name. Review your actual usage patterns on /pricing to identify which requests benefit most from dynamic selection.

Related FastInfra resources

• See live model prices
• More llm pricing guides
• Live model prices per million tokens
• API documentation and quickstart