Hetzner Inference: OpenAI-Compatible EU AI API

Hetzner Inference is an experimental, OpenAI-compatible API for running language-model requests on Hetzner infrastructure. It is interesting because it reduces provider migration to a base-URL and API-key change, while putting European infrastructure, data sovereignty, and low-friction experimentation ahead of production guarantees.

The important qualification is that this is not yet a normal managed AI product. During the current experiment, Hetzner does not publish token billing, an SLA, stable capacity guarantees, or production commitments. The practical recommendation is simple: use Hetzner Inference for prototypes, synthetic or anonymized data, internal tooling, and provider portability tests—not for regulated production workloads or customer data until the contractual and operational pieces are documented.

Direct answer: Hetzner Inference is a free experimental LLM endpoint with an OpenAI-compatible interface. Public independent testing and Hetzner’s own integration tutorial identify Qwen/Qwen3.6-35B-A3B-FP8 as the available model. You create a token in the Hetzner Experiments dashboard, configure an OpenAI client with Hetzner’s endpoint, and keep your application code largely unchanged.

Key takeaways

• Hetzner Inference is experimental; do not treat the endpoint as an SLA-backed production service.
• The current public model is Qwen/Qwen3.6-35B-A3B-FP8, a 35B-total / approximately 3B-active mixture-of-experts model.
• The model accepts text and image input and is reported with a 262,144-token context window, but model capabilities and limits can change while the experiment evolves.
• OpenAI compatibility makes migration and A/B testing straightforward.
• The service is currently free because public reporting indicates that billing has not been introduced yet; there is no announced token price.
• Hetzner’s public GPU servers are a useful infrastructure comparison, not proof of the hardware behind the managed API.
• The best early workloads are summarization, classification, extraction, RAG components, internal assistants, and development environments.
• API-key hygiene, request timeouts, retries, bounded output, redaction, and a provider fallback are essential because experimental capacity is not a reliability contract.

What Hetzner Inference actually is

Hetzner Inference sits between two familiar deployment models:

1. A hosted model API: your application sends requests to a provider-managed endpoint and does not operate GPUs.
2. A dedicated GPU server: you rent a machine, install a serving stack such as vLLM or SGLang, and own the operational burden.

Hetzner’s experiment takes the first approach. You receive an API token and call a managed OpenAI-compatible endpoint. The service abstracts away model loading, GPU scheduling, batching, upgrades, and capacity management. That is materially easier than renting a GPU server, but it also means you do not control the backend, model rollout, queueing policy, or regional placement.

This distinction matters when comparing the experiment with OpenAI-compatible self-hosted serving using SGLang. Self-hosting gives you control over weights, networking, observability, and data handling. Hetzner Inference gives you a faster path to testing without buying or operating a GPU.

Product status and what is not promised

Independent infrastructure coverage describes the service as an experiment intended to learn about usage, scale, and the features developers want. The public material does not establish a stable rate limit, a guaranteed uptime percentage, a published quota, a data-processing agreement, or a long-term price.

Treat these as unknowns, not as implied defaults:

Current model: Qwen3.6-35B-A3B-FP8

The model identified in current public coverage and Hetzner’s OpenCode tutorial is Qwen/Qwen3.6-35B-A3B-FP8.

Its name communicates several useful details:

• 35B: approximately 35 billion total parameters are present in the model.
• A3B: a mixture-of-experts design activates roughly 3 billion parameters for an individual token path, rather than using every parameter on every token.
• FP8: the weights use 8-bit floating-point quantization, reducing memory pressure compared with full-precision weights.

The model is reported as multimodal, accepting text and images, with a native context window of 262,144 tokens. Those capabilities should still be tested against the exact endpoint behavior you receive: context limits, image encoding formats, reasoning defaults, maximum output tokens, and unsupported OpenAI fields are all implementation details that can change during an experiment.

A model with a relatively small active parameter count can be attractive for latency-sensitive utility work. That does not make it a frontier-model replacement. Early independent testing found it useful for ordinary instruction following and image understanding, but also exposed failures on simple arithmetic. That is a normal reminder to benchmark the task you actually care about rather than relying on the model label or parameter count.

Architecture: where the endpoint fits

The following conceptual architecture separates what is publicly observable from what is inferred. Hetzner has published the API integration surface, but it has not publicly documented every internal serving component or the exact hardware pool behind the experiment.

The fallback path is not optional architecture for a production application. It is how you prevent an experimental provider’s availability, capacity, or model change from becoming your application’s outage.

Quick start with the Python OpenAI SDK

Install the SDK in an isolated environment:

1python -m venv .venv
2source .venv/bin/activate
3python -m pip install --upgrade pip openai

Create an API token in the Hetzner Experiments dashboard. Store it as an environment variable rather than committing it to source control:

1export HETZNER_INFERENCE_API_KEY='replace-with-your-token'

Then call the compatible endpoint. The base URL below is the endpoint used in Hetzner’s community tutorial and independent examples:

1import os
2from openai import OpenAI
3
4client = OpenAI(
5    base_url="https://inference.hetzner.com/api/v1",
6    api_key=os.environ["HETZNER_INFERENCE_API_KEY"],
7    timeout=60.0,
8    max_retries=2,
9)
10
11response = client.chat.completions.create(
12    model="Qwen/Qwen3.6-35B-A3B-FP8",
13    messages=[
14        {
15            "role": "user",
16            "content": "Explain mixture-of-experts models in three short bullet points.",
17        }
18    ],
19    max_tokens=256,
20    temperature=0.2,
21)
22
23print(response.choices[0].message.content)

The exact model identifier and endpoint should be rechecked in the dashboard before deployment. An experimental service can change its model inventory without preserving every compatibility detail.

Why OpenAI compatibility matters

The value is not only convenience. A stable client abstraction lets you compare providers under the same application contract:

• Keep message construction and response parsing in one adapter.
• Change the base URL and credentials through configuration.
• Run the same evaluation set against multiple providers.
• Route sensitive or high-reliability requests elsewhere.
• Move from a prototype to self-hosting without rewriting every call site.

Do not confuse API compatibility with behavioral compatibility. Providers can differ in tool calling, structured outputs, reasoning controls, streaming events, token accounting, image formats, safety filters, and error schemas. Keep provider-specific features behind an adapter and test them explicitly.

OpenCode and agent workflows

Hetzner publishes an OpenCode integration and systemd sandbox tutorial. The integration configures OpenCode with an OpenAI-compatible provider and the Qwen model. The tutorial’s security message is as important as its setup instructions: a coding agent can read files, modify a repository, and execute commands, so model-provider integration should be paired with least-privilege execution.

A safer pattern is:

1. Give the agent access only to a disposable or explicitly selected Git repository.
2. Keep API credentials in environment variables or a secret manager.
3. Run the agent under a restricted user or systemd sandbox.
4. Block access to unrelated home-directory files and sockets.
5. Review commands and patches before applying them to production systems.
6. Rotate the Hetzner token if it appears in logs, shell history, screenshots, or a prompt transcript.

This connects naturally with the operational lessons in OpenCode’s release and session-management changes: long-running coding agents need controlled context, safe recovery, and careful handling of imported data—not just a fast model endpoint.

Performance: useful first measurements, not an SLA

Independent tests published in July 2026 reported approximately 153 ms median time to first token over a small set of short requests on an already-open connection, and approximately 224 output tokens per second across several longer generations capped at 512 tokens.

Those numbers are useful as an early signal, not as a capacity promise. They do not answer:

• How the service behaves at peak concurrency.
• Whether a cold request has materially higher latency.
• How image inputs affect time to first token.
• Whether queueing changes by account or request size.
• What error rate occurs during model reloads or maintenance.
• Whether the same result holds from your users’ geography.

Build a small benchmark before choosing the endpoint. Record p50, p95, and p99 latency; time to first token; output throughput; HTTP errors; timeout rate; and tokens per request. Test at the concurrency your application expects, then repeat periodically because an experiment is a moving target.

1import time
2from openai import OpenAI
3
4client = OpenAI(
5    base_url="https://inference.hetzner.com/api/v1",
6    api_key=os.environ["HETZNER_INFERENCE_API_KEY"],
7    timeout=60,
8)
9
10started = time.perf_counter()
11response = client.chat.completions.create(
12    model="Qwen/Qwen3.6-35B-A3B-FP8",
13    messages=[{"role": "user", "content": "Return the word READY."}],
14    max_tokens=16,
15)
16elapsed_ms = (time.perf_counter() - started) * 1000
17print({"elapsed_ms": round(elapsed_ms, 1), "text": response.choices[0].message.content})

This measures end-to-end non-streaming latency. For interactive applications, add a streaming benchmark and measure time to the first received chunk separately.

Security, privacy, and compliance boundaries

European data-center location can be valuable for residency and governance, but location alone does not establish compliance. Before sending personal, confidential, or regulated data, verify the current Hetzner terms, retention behavior, subprocessors, transfer terms, and whether a data-processing agreement is available for this specific experiment.

Until those details are documented for your use case, use the following boundary:

Appropriate for early testing:

• Synthetic prompts and generated fixtures.
• Public documents.
• Anonymized text with re-identification risk assessed.
• Internal experiments that contain no personal or regulated data.
• Development and staging environments.

Do not send without a formal review:

• Customer names, emails, support tickets, or identifiers.
• Health, financial, employment, or legal records.
• Secrets, private keys, access tokens, or production logs containing credentials.
• Data covered by contractual residency, retention, or deletion obligations.

Use a redaction layer before the provider adapter. Log request metadata, not raw prompts by default. Encrypt secrets at rest, restrict who can create tokens, and set an explicit retention policy for model outputs.

Reliability pattern for an experimental endpoint

A resilient client should fail closed and fail over deliberately:

1import os
2from openai import OpenAI, APIConnectionError, APITimeoutError, RateLimitError
3
4hetzner = OpenAI(
5    base_url="https://inference.hetzner.com/api/v1",
6    api_key=os.environ["HETZNER_INFERENCE_API_KEY"],
7    timeout=30,
8    max_retries=1,
9)
10
11backup = OpenAI(api_key=os.environ["BACKUP_OPENAI_API_KEY"], timeout=30, max_retries=1)
12
13messages = [{"role": "user", "content": "Classify this document as invoice, receipt, or other."}]
14
15try:
16    result = hetzner.chat.completions.create(
17        model="Qwen/Qwen3.6-35B-A3B-FP8",
18        messages=messages,
19        max_tokens=32,
20        temperature=0,
21    )
22except (APIConnectionError, APITimeoutError, RateLimitError):
23    result = backup.chat.completions.create(
24        model=os.environ["BACKUP_MODEL"],
25        messages=messages,
26        max_tokens=32,
27        temperature=0,
28    )
29
30print(result.choices[0].message.content)

In a real system, also add idempotency where supported, a circuit breaker, bounded queue depth, request cancellation, metrics, and an explicit policy for whether fallback data may leave the approved region. Never silently route regulated data to an unapproved provider.

Hetzner Inference versus renting a GPU

Hetzner’s public dedicated GPU range provides useful context. Its GPU page lists the GEX44 with an NVIDIA RTX 4000 SFF Ada GPU and 20 GB VRAM, and the GEX131 with an NVIDIA RTX PRO 6000 Blackwell Max-Q GPU and 96 GB VRAM. Hetzner describes GEX44 as suited to efficient inference and GEX131 as suited to demanding training workloads. The page also states that public GEX servers use one GPU per server and do not support multi-GPU configurations.

That catalog does not prove which hardware powers the managed Inference API. Treating the API as “an RTX PRO 6000 endpoint” would be an unsupported claim. The real comparison is operational:

For a second managed comparison, mature providers such as Fireworks publish per-token pricing, multiple model families, serving paths, and rate-limit documentation. That makes them easier to budget and operate in production, while Hetzner’s current advantage is low-friction experimentation and potential European infrastructure alignment. The trade-off is not “which provider is fastest” but “how much uncertainty can this workload tolerate?”

Cost and capacity planning

The current experiment is reported as free and does not publish token pricing. That is excellent for discovering whether a use case works, but it is not a long-term cost model. Before building deep coupling, estimate:

• Input and output tokens per request.
• Requests per minute and peak concurrency.
• Context-window growth over time.
• Image requests and their encoded size.
• Retry amplification during transient failures.
• Evaluation and fallback traffic.
• The cost of moving to a dedicated GPU or another API later.

Keep your provider configuration external. If the experiment introduces billing, rate limits, a new model, or a new endpoint, you should be able to update configuration and tests without changing business logic.

Common errors and debugging checklist

401 or 403 authentication errors

Check that the token was created in the Experiments dashboard, copied without whitespace, and loaded into the same environment used by the process. Do not print the token while debugging.

404 model or endpoint errors

Confirm the current base URL and model identifier from the dashboard or current official documentation. Experimental model inventories can change. Avoid hard-coding an old model ID in multiple services.

Requests time out

Start with a 30–60 second client timeout, then measure rather than setting an unlimited timeout. Reduce max_tokens, bound concurrency, and check whether your request is sending a very large context or image payload.

Output quality is inconsistent

Pin temperature where appropriate, define a task-specific evaluation set, and validate structured results with a schema. A compatible API does not guarantee identical reasoning, tool use, or formatting across providers.

Agent actions are unsafe

The model provider is not a sandbox. Restrict the agent runtime, isolate the repository, remove unnecessary environment variables, and require review for destructive commands. Hetzner’s own OpenCode tutorial demonstrates this separation with a systemd-based sandbox pattern.

Streaming or advanced fields fail

Test basic chat completions first. Then add streaming, vision, tools, JSON mode, or provider-specific fields one at a time. Keep optional features behind capability checks instead of assuming every OpenAI field is supported.

Who should try Hetzner Inference?

Good candidates:

• Developers evaluating an EU-hosted inference path.
• Teams prototyping summarization, extraction, classification, or RAG.
• AI-agent builders who want an OpenAI-compatible development endpoint.
• Organizations comparing managed inference with MCP and multi-agent architectures.
• Engineers testing whether a smaller MoE model is sufficient before committing to a larger model.

Poor candidates right now:

• Customer-facing systems that require an uptime commitment.
• Regulated workloads needing a signed DPA and documented retention controls.
• Applications requiring several model families or guaranteed regional capacity.
• High-volume production systems without a tested fallback.
• Workloads that need frontier-level reasoning without a task-specific evaluation.

FAQ

Is Hetzner Inference free?

Public independent coverage reports that the experiment is free and that billing has not yet been introduced. Hetzner has not published a future token price, so do not assume the current cost will continue.

Is Hetzner Inference OpenAI-compatible?

Yes. The public integration examples use an OpenAI-compatible endpoint and the standard OpenAI SDK shape. Compatibility should be tested for the exact features your application uses, especially tools, structured outputs, streaming, and vision.

Which model does Hetzner Inference provide?

Current public materials identify Qwen/Qwen3.6-35B-A3B-FP8 as the available model. Verify the live dashboard before relying on the identifier because this is an active experiment.

Can I use it with OpenCode?

Yes. Hetzner provides a community tutorial showing OpenCode configuration with the Inference API and a systemd sandbox. The sandbox is important because OpenCode can edit files and execute commands.

Can I send customer data to the endpoint?

Do not assume that European hosting automatically makes a workload compliant. Review the current experiment terms, retention, subprocessors, and DPA availability. Until those are documented and approved for your case, use synthetic, anonymized, or non-personal data.

Does Hetzner Inference use Hetzner’s public GEX GPUs?

The public API documentation does not establish that. Hetzner’s dedicated GPU catalog describes GEX hardware, but the managed endpoint’s backend should be treated as undisclosed unless Hetzner confirms it directly.

Is it production-ready?

Not based on the currently published information. The lack of a documented SLA, stable rate limits, billing model, and production guarantees means it is best treated as an experimental and development endpoint.

Conclusion

Hetzner Inference is strategically more interesting than its current one-model catalog suggests. OpenAI compatibility makes it easy to try, the current experiment removes initial cost friction, and Hetzner’s European infrastructure creates a credible alternative for teams that want to test AI workloads outside the major US API vendors.

But the responsible engineering conclusion is equally clear: this is an experiment, not a production contract. Build an adapter, benchmark your real tasks, redact data, keep credentials out of logs, sandbox agents, and maintain a tested fallback. If Hetzner later adds billing, a broader catalog, documented limits, an SLA, and a DPA, the migration path from prototype to production will already be in place.

Sources and further reading

• Hetzner Experiments Inference documentation
• Hetzner Community: OpenCode with Hetzner Experiments Inference API + systemd Sandbox
• Hetzner dedicated GPU servers
• Qwen3.6-35B-A3B-FP8 on Hugging Face
• Sliplane: Hetzner Inference — First Look
• EMIT Solution: Hetzner Inference API
• Fireworks AI serverless pricing

Visual credit: original Mermaid architecture diagram by Essam Amdani; product facts linked to official Hetzner documentation and independent sources above.