Dedicated inference · in development
Your capacity. In Switzerland. No queue.
Shared endpoints are cheap until everyone computes at the same time. Dedicated inference reserves throughput on open models for you alone — with data that does not leave the country and does not flow into any training.
How it works
The technology behind it, without the marketing fog.
Requests are not processed one after another but together, token by token. The graphics card stays fully utilized without one long answer blocking all the short ones.
The attention cache is managed in pages, like main memory. This allows long contexts and many concurrent conversations on the same card.
Modern accelerators compute natively with 4-bit floating-point numbers and fine-grained scaling. A model of the 27-billion class thus fits on a single card and stays fast.
We commit to throughput per request and in aggregate — measured, not estimated — and report it.
The research field: efficient inference
A large share of current LLM research is devoted not to training but to the question of how to run them more cheaply. Low-bit quantization reduces memory and bandwidth, but costs varying amounts of accuracy depending on the task — mathematics and code react more sensitively than summaries.
Speculative decoding lets a small model propose several tokens that the large one verifies in a single step. Running preparation (prefill) and generation (decode) on separate cards exploits the fact that the two phases have entirely different hardware bottlenecks.
Open questions
- Where is the limit of quantization before tasks measurably suffer?
- How much energy does a token cost — and how low can it go?
- How do you share a card fairly between latency and throughput customers?
If the breakthrough succeeds
Outlook, not a promise: what would become possible once the open questions are answered.
Lossless low-bit models
If 4-bit inference succeeded without measurable quality loss for all task types, a single workstation would serve an entire office. Law firms, medical practices and public authorities could run strong models in-house without outsourcing data.
Energy per token as a key metric
If the energy required per token fell by an order of magnitude, local inference would be not only more privacy-friendly but also cheaper than the cloud.
Where we stand
The same evidence as on the overview — every number with its kind.
- Open weights (27B class, embeddings), fixed capacity
- Data stays in Switzerland, no training on customer data
- Tokens per second as a commitment, measured and published
Who it is for
- Companies required to keep their data in Switzerland
- Products with predictable load and fixed response times
- Teams that want to use open models without operating them
Work on it with us.
We open pilots one by one. Briefly describe what you would like to compute — no payment, no commitment, an honest assessment.
Request a pilot