NewDeepSeek V4.1 Flash 1017 is now availableTry it now

Blog

Launches, research and engineering notes from the Inference.net team.

We made vLLM tokenization 120x faster with Elixir, over a line graph of vLLM req/s with our token IDs vs raw chat
Harry Bairstow

We made vLLM tokenization 120x faster with Elixir

How we built ChatTokenizer, an Elixir and Rust tokenizer that matches vLLM and SGLang token for token and is up to 120x faster than their own tokenization

Read post
MiniMax M3.1 Flash, available now on Inference on Day 0
Amar Singh

MiniMax M3.1 Flash is live on Inference: Day 0

Inference is an official Day 0 provider for MiniMax M3.1 Flash, a 428B-parameter Mixture-of-Experts model built for coding, agents, and complex workflows. Run it today on our serverless API at $0.10 / $0.50 per million tokens.

Read post
Introducing AutoEvals: Automatically find the best model for your task
Amar Singh

Introducing AutoEvals: Automatically find the best model for your task

Automatically discover which model is best suited for your task, with the data you already generate. AutoEvals replays your production traffic across candidate models and tells you which one to run, with real numbers on quality, cost, and latency.

Read post
Schematron V2: Frontier HTML-to-JSON extraction at a fraction of the cost
Amar Singh

Schematron V2: Frontier HTML-to-JSON extraction at a fraction of the cost

Schematron is a family of small, purpose-built models that transform messy HTML into clean, structured JSON, delivering frontier-level extraction quality at a fraction of the cost and latency of large general-purpose LLMs.

Read post
Introducing Inference platform: Monitor, train, and deploy self-improving AI models
Sam Hogan

Introducing Inference platform: Monitor, train, and deploy self-improving AI models

Build self-improving AI-systems from production data

Read post
How Inference.net trains Specialized Language Models that cut AI costs by up to 50x
Sam Hogan

How Inference.net trains Specialized Language Models that cut AI costs by up to 50x

Learn how Inference.net trains Specialized Language Models using the NVIDIA NeMO framework to delivery frontier accuracy at up to 50x lower cost.

Read post
Specialized LLMs: The model you need doesn't exist yet
Sam Hogan

Specialized LLMs: The model you need doesn't exist yet

Specialized LLMs trained on your own user data can match frontier quality for a fraction of the cost

Read post
Project OSSAS: Custom LLMs to process 100 Million Research Papers
Sam Hogan

Project OSSAS: Custom LLMs to process 100 Million Research Papers

Project OSSAS is a large-scale open-science initiative to make the world’s scientific knowledge accessible through AI-generated summaries of research papers.

Read post
LOGIC: Trustless Inference through Log-Probability Verification
Amar Singh

LOGIC: Trustless Inference through Log-Probability Verification

A practical method for verifying LLM inference requests in trustless environments.

Read post
Hybrid-Attention models are the future for SLMs
Amar Singh

Hybrid-Attention models are the future for SLMs

Hybrid attention delivers up to 3x cost reduction compared to traditional transformer models

Read post