What Is It?
The IONOS CLOUD AI Model Hub is a fully managed, serverless inference platform that gives you API access to a curated selection of open-weight foundation models. You use it for text generation, coding, image generation and editing, embeddings, reranking, OCR, and tool calling, without provisioning, scaling, or managing GPU infrastructure. Its REST API is OpenAI-compatible, so tools and SDKs built for OpenAI endpoints can connect to it.
Quick Facts
| Aspect | Details |
|---|---|
| Type | Fully managed serverless inference (API-first) |
| Model families | Large Language Models, Coding Models, Text-to-Image Models, OCR Models, Embedding Models, Reranking Models |
| OpenAI-compatible endpoint | https://openai.inference.de-txl.ionos.com/v1 |
| API paths | GET /v1/models, POST /v1/chat/completions, POST /v1/responses, POST /v1/embeddings, POST /v1/rerank, POST /v1/images/generations, POST /v1/images/edits |
| API docs | https://api.ionos.com/docs/inference-openai/v1 |
| Authentication | Bearer Token. Tokens are JSON Web Tokens (JWT) with an expiration date; the how-tos assume an IONOS_API_TOKEN environment variable |
| Token lifetime | 1 hour to 365 days, set at creation. The token value is shown only once |
| Required privilege | Group privilege Access and manage AI Model Hub (contract owners have access by default) |
| Data residency | All data processing and inference occur exclusively in Germany (Berlin) |
| API reachability | Global: the endpoints are reachable from any location over the public internet |
| Statelessness | Stateless per request. No session or conversation is kept; prompts and outputs are never logged, not written to persistent storage, and not used to train any model |
| Certifications | ISO 27001-certified data centers in Germany; AI Model Hub is within the scope of BSI C5 |
| GDPR | Designed to comply with the GDPR; no third-party sub-processors |
| Uptime SLA | 99.9% monthly average, scoped to the AI Model Hub API endpoints only (not per-model availability or inference quality) |
| SLA credits | 10% of the monthly bill below 99.9%, 25% below 99.0%, 40% below 95.0% |
| Maintenance | Up to 4 hours per quarter, announced 2 working days ahead, not counted as unavailability |
| Support response | Disruption: under 1 hour. Service or information request: under 72 hours. Coverage: German business hours |
Language and Coding Models
| Model | Model identifier | Input | Context window | Reasoning | Tool calling |
|---|---|---|---|---|---|
| Qwen3.5-9B | Qwen/Qwen3.5-9B |
Text, image, video | 262,000 tokens | Yes | Yes |
| Mistral Small 24B | mistralai/Mistral-Small-24B-Instruct |
Text, image | 128,000 tokens | No | Yes |
| Qwen3.8-27B | Qwen/Qwen3.8-27B |
Text, image, video (video in beta) | 262,000 tokens | Yes | Yes |
| Meta Llama 3.3 70B | meta-llama/Llama-3.3-70B-Instruct |
Text | 128,000 tokens | No | Yes |
| OpenAI GPT-OSS 120B | openai/gpt-oss-120b |
Text | 128,000 tokens | Yes | Yes |
| Qwen3.5 397B A17B | Qwen/Qwen3.5-397B-A17B |
Text, image | 262,000 tokens | Yes | Yes |
| Qwen3-Coder-Next 80B (coding) | Qwen/Qwen3-Coder-Next |
Text | 256,000 tokens | No | Yes |
All language and coding models support streaming. Qwen3.5-9B and Qwen3.8-27B accept up to 4 images per request.
Embedding, Reranking, Image and OCR Models
| Model | Model identifier | Input | Key spec |
|---|---|---|---|
| BGE m3 | BAAI/bge-m3 |
Text | 1024 dimensions, 8192-token max input |
| Qwen3 VL Embedding 8B | Qwen/Qwen3-VL-Embedding-8B |
Text, image | 4096 dimensions, 32768-token max input |
| Qwen3 VL Reranker 8B | Qwen/Qwen3-VL-Reranker-8B |
Text, image | Cross-encoder reranker, 32768-token max input |
| FLUX.2 [klein] 4B | black-forest-labs/FLUX.2-klein-4B |
Text, image | Generation and editing; width and height 64 to 2048 (rounded down to a multiple of 16); n from 1 to 10, max 12,582,912 output pixels per request |
| LightOnOCR-2-1B | lightonai/LightOnOCR-2-1B |
Image | 16,000-token context, 16 MiB max request payload, Markdown output |
Rate Limits
| Endpoint/Scope | Base Limit | Burst Limit | Notes |
|---|---|---|---|
| Each model | 5 requests/second | 10 requests/second | Applies separately to each model, including image generation models |
| Listing models | 5 requests/second | 10 requests/second | Applies to GET /v1/models |
The burst limit is the maximum number of requests in a 2-second window. Limits are enforced per contract (not per user), separately for each model, and your contract can have custom limits. Exceeding a limit returns HTTP 429 Too Many Requests.
Rate-Limit Response Headers
| Header | Description |
|---|---|
X-RateLimit-Limit-Requests |
Your base limit: the number of requests allowed per interval, for example 5 for 5 requests per second, without the burst allowance. The actual value depends on your contract. |
X-RateLimit-Remaining-Requests |
The number of requests remaining in the base limit, excluding the burst allowance. This value counts down from X-RateLimit-Limit-Requests and will not exceed it. To avoid a 429 Too Many Requests error, pause or throttle your requests when this reaches 0. |
X-RateLimit-Reset-Requests |
The time until the rate limit is back at its initial state, with the full base limit and burst allowance available again. The value is a duration such as 200ms, 1.5s, or 6m0s. |
Retry-After |
The number of seconds to wait before retrying. This header appears on 429 and 529 responses. |
X-Request-Id |
Identifier for this request, mirroring X-Trace-ID. Include it when reporting a problem to support. |
X-Gateway-Version |
The build version of the API that served the response. |
Image Input Request Limits
| Limit | Value |
|---|---|
| Total request body | 16 MiB |
Each inline base64 data URI, after decoding |
12 MiB |
| Content parts per message | 2048 |
What You Can Do
Text Generation with Chat Completions
Send prompts to /v1/chat/completions to generate conversational answers, summaries, and contextual responses. Prefix caching is active for all language and coding models, so requests that start with the same content can respond faster.
Use the Responses API
/v1/responses is an alternative to chat completions for text generation with every LLM. It takes a flat list of input items and returns typed output items (message, reasoning, function_call), so reasoning and tool calls arrive as separate objects. Use instructions for a system prompt.
Control Reasoning
OpenAI GPT-OSS 120B takes reasoning_effort low, medium (default), or high. Qwen3.5-9B and Qwen3.5 397B A17B reason by default; set reasoning_effort to none to turn it off. Qwen3.8-27B takes xhigh, medium (default), low, or none.
Tool Calling
Define tools with JSON schemas in the request. The model decides whether to answer directly or to return a structured tool call that your code executes, for example to fetch live data or trigger a workflow.
Image Generation and Editing
Create images from text with FLUX.2 [klein] 4B through /v1/images/generations, or edit an existing image through /v1/images/edits (multipart/form-data). Output formats are jpg, jpeg, png, and webp.
Image Input
Attach images as a public URL or a base64 data URI on /v1/chat/completions, /v1/responses, /v1/embeddings, /v1/rerank, and /v1/images/edits, for models whose model card advertises image input.
Embeddings, Reranking and RAG
Turn text (and, with Qwen3 VL Embedding 8B, images) into vectors for semantic search, similarity, and clustering. Use Qwen3 VL Reranker 8B through POST /v1/rerank as a second retrieval stage. For Retrieval Augmented Generation (RAG), the how-to uses DBaaS for PostgreSQL with the vector extension (pgvector) as the vector store.
OCR and PDF-to-Text
Convert document images to Markdown text (including LaTeX for math) with LightOnOCR-2-1B. For PDFs, render each page as an image first; Mistral Small 24B can also extract and interpret PDF content from page images.
Connect Existing Tools
Tools such as AnythingLLM and OpenCode (through @ai-sdk/openai-compatible) connect through the OpenAI-compatible API. The n8n package @ionos-cloud/n8n-nodes-ionos-cloud adds an IONOS CLOUD Chat Model node for AI Agents and an IONOS CLOUD Embeddings node for vector-store RAG workflows.
Best For
| Scenario | Why It Fits |
|---|---|
| Chatbots and assistants with data-sovereignty needs | Inference runs only in Germany, requests are stateless, and your data is never used for training |
| Semantic search and RAG over private documents | Embedding, reranking, and chat models in one API, with pgvector as a standard PostgreSQL vector store |
| AI-assisted software development | A dedicated coding model with tool calling, usable from OpenCode and other OpenAI-compatible tools |
| Content and marketing visuals | A text-to-image model generates and edits images from prompts |
| Workflow automation through natural language | Tool calling lets models trigger APIs and business systems |
| Document digitization | A dedicated OCR model turns scans and document images into structured text |
Consider Alternatives If
| If You Need... | Consider | Why |
|---|---|---|
| Fine-tuning a model on your own data | Cloud GPU VM | GPU servers provide the compute to customize foundation models with proprietary datasets |
| Steady 24/7 inference at a fixed rate | Cloud GPU VM | Dedicated GPU resources at a fixed rate instead of per-request pricing |
| AI-driven management of your cloud infrastructure | IONOS CLOUD MCP Server | It gives an AI client tools for the IONOS CLOUD API; the AI Model Hub can supply the model |
Key Considerations
Billing & Costs
- Main billing: Per request, by model type. Language, coding, and OCR models: per million input and output tokens. Embedding and reranking models: per million input tokens at one rate. FLUX.2 [klein] 4B: per megapixel. Prices in EUR (net):
| Model | Input per 1M tokens | Output per 1M tokens |
|---|---|---|
| Qwen3.5-9B | 0.10 | 0.15 |
| Mistral Small 24B | 0.10 | 0.30 |
| Qwen3.8-27B | 0.40 | 2.70 |
| Meta Llama 3.3 70B | 0.65 | 0.65 |
| OpenAI GPT-OSS 120B | 0.15 | 0.65 |
| Qwen3.5 397B A17B | 0.60 | 3.60 |
| Qwen3-Coder-Next 80B | 0.15 | 0.80 |
| LightOnOCR-2-1B | 0.15 | 0.30 |
- Embeddings and reranking (per 1M tokens): BGE m3 0.02, Qwen3 VL Embedding 8B 0.10, Qwen3 VL Reranker 8B 0.04. For reranking, the query and all submitted documents count as input
- Images: FLUX.2 [klein] 4B costs 0.013 for the first output megapixel, 0.001 per additional output megapixel, and 0.001 per input megapixel (1 MP = 1024 x 1024; output is rounded up; each input image is billed as 1 MP)
- What counts as input: System prompts and tool definitions count as input tokens. Images sent to Mistral Small 24B are billed as input tokens
- Retries and caching: Each retry is billed separately. Cached or repeated input is not discounted
- When idle: You pay only for consumed tokens and megapixels. Playground requests in the DCD are billed at standard rates
- Monitoring: Cost & Usage in the DCD (contract administrators and owners on regular contracts; owners on reseller contracts) and the Billing API utilization endpoints, broken down by model and unit
Limitations
- Model lifecycle: IONOS CLOUD adds new models and retires older ones; query
GET /v1/modelsfor the current list - Responses API:
store,previous_response_id,conversation,background,stream, and built-in tools such asweb_searchandfile_searchare not available (a400error names the field).tool_choiceacceptsautoandnone - Request concurrency: Requests larger than 4 MiB and chunked requests are accepted six at a time; a further one gets
503withRetry-After: 1 - Platform overload: Under very high demand across all customers, the API can return
HTTP 529 Site is Overloadedwith aRetry-Afterheader - OCR input: LightOnOCR-2-1B accepts images only (no PDF), and text prompts do not change its Markdown output
- No pseudonymization: Request content is not pseudonymized by IONOS CLOUD; apply it in your application if you need it
- New users: A public cloud contract is required, and activation of a new contract can take up to 24 hours
- EU AI Act: IONOS CLOUD is the provider of the AI system for every model on the Hub, and the model developers are the providers of the models. You are the Deployer (or the Provider of your own AI system) and do your own risk assessment (Limited-Risk or High-Risk). No pre-configured AI Act logging template is provided
Management Options
- Data Center Designer (DCD): Users & Groups for the privilege, Token Manager for tokens, AI > AI Model Hub (Models and Playground tabs), and Cost & Usage
- REST API: The OpenAI-compatible API
- SDKs and tools: OpenAI-compatible SDKs, plus integrations such as AnythingLLM, OpenCode, and n8n