Product reference pages are available in English only.
AI

AI Model Hub

What Is It?

The IONOS CLOUD AI Model Hub is a fully managed, serverless inference platform that gives you API access to a curated selection of open-weight foundation models. You use it for text generation, coding, image generation and editing, embeddings, reranking, OCR, and tool calling, without provisioning, scaling, or managing GPU infrastructure. Its REST API is OpenAI-compatible, so tools and SDKs built for OpenAI endpoints can connect to it.

Quick Facts

Aspect Details
Type Fully managed serverless inference (API-first)
Model families Large Language Models, Coding Models, Text-to-Image Models, OCR Models, Embedding Models, Reranking Models
OpenAI-compatible endpoint https://openai.inference.de-txl.ionos.com/v1
API paths GET /v1/models, POST /v1/chat/completions, POST /v1/responses, POST /v1/embeddings, POST /v1/rerank, POST /v1/images/generations, POST /v1/images/edits
API docs https://api.ionos.com/docs/inference-openai/v1
Authentication Bearer Token. Tokens are JSON Web Tokens (JWT) with an expiration date; the how-tos assume an IONOS_API_TOKEN environment variable
Token lifetime 1 hour to 365 days, set at creation. The token value is shown only once
Required privilege Group privilege Access and manage AI Model Hub (contract owners have access by default)
Data residency All data processing and inference occur exclusively in Germany (Berlin)
API reachability Global: the endpoints are reachable from any location over the public internet
Statelessness Stateless per request. No session or conversation is kept; prompts and outputs are never logged, not written to persistent storage, and not used to train any model
Certifications ISO 27001-certified data centers in Germany; AI Model Hub is within the scope of BSI C5
GDPR Designed to comply with the GDPR; no third-party sub-processors
Uptime SLA 99.9% monthly average, scoped to the AI Model Hub API endpoints only (not per-model availability or inference quality)
SLA credits 10% of the monthly bill below 99.9%, 25% below 99.0%, 40% below 95.0%
Maintenance Up to 4 hours per quarter, announced 2 working days ahead, not counted as unavailability
Support response Disruption: under 1 hour. Service or information request: under 72 hours. Coverage: German business hours

Language and Coding Models

Model Model identifier Input Context window Reasoning Tool calling
Qwen3.5-9B Qwen/Qwen3.5-9B Text, image, video 262,000 tokens Yes Yes
Mistral Small 24B mistralai/Mistral-Small-24B-Instruct Text, image 128,000 tokens No Yes
Qwen3.8-27B Qwen/Qwen3.8-27B Text, image, video (video in beta) 262,000 tokens Yes Yes
Meta Llama 3.3 70B meta-llama/Llama-3.3-70B-Instruct Text 128,000 tokens No Yes
OpenAI GPT-OSS 120B openai/gpt-oss-120b Text 128,000 tokens Yes Yes
Qwen3.5 397B A17B Qwen/Qwen3.5-397B-A17B Text, image 262,000 tokens Yes Yes
Qwen3-Coder-Next 80B (coding) Qwen/Qwen3-Coder-Next Text 256,000 tokens No Yes

All language and coding models support streaming. Qwen3.5-9B and Qwen3.8-27B accept up to 4 images per request.

Embedding, Reranking, Image and OCR Models

Model Model identifier Input Key spec
BGE m3 BAAI/bge-m3 Text 1024 dimensions, 8192-token max input
Qwen3 VL Embedding 8B Qwen/Qwen3-VL-Embedding-8B Text, image 4096 dimensions, 32768-token max input
Qwen3 VL Reranker 8B Qwen/Qwen3-VL-Reranker-8B Text, image Cross-encoder reranker, 32768-token max input
FLUX.2 [klein] 4B black-forest-labs/FLUX.2-klein-4B Text, image Generation and editing; width and height 64 to 2048 (rounded down to a multiple of 16); n from 1 to 10, max 12,582,912 output pixels per request
LightOnOCR-2-1B lightonai/LightOnOCR-2-1B Image 16,000-token context, 16 MiB max request payload, Markdown output

Rate Limits

Endpoint/Scope Base Limit Burst Limit Notes
Each model 5 requests/second 10 requests/second Applies separately to each model, including image generation models
Listing models 5 requests/second 10 requests/second Applies to GET /v1/models

The burst limit is the maximum number of requests in a 2-second window. Limits are enforced per contract (not per user), separately for each model, and your contract can have custom limits. Exceeding a limit returns HTTP 429 Too Many Requests.

Rate-Limit Response Headers

Header Description
X-RateLimit-Limit-Requests Your base limit: the number of requests allowed per interval, for example 5 for 5 requests per second, without the burst allowance. The actual value depends on your contract.
X-RateLimit-Remaining-Requests The number of requests remaining in the base limit, excluding the burst allowance. This value counts down from X-RateLimit-Limit-Requests and will not exceed it. To avoid a 429 Too Many Requests error, pause or throttle your requests when this reaches 0.
X-RateLimit-Reset-Requests The time until the rate limit is back at its initial state, with the full base limit and burst allowance available again. The value is a duration such as 200ms, 1.5s, or 6m0s.
Retry-After The number of seconds to wait before retrying. This header appears on 429 and 529 responses.
X-Request-Id Identifier for this request, mirroring X-Trace-ID. Include it when reporting a problem to support.
X-Gateway-Version The build version of the API that served the response.

Image Input Request Limits

Limit Value
Total request body 16 MiB
Each inline base64 data URI, after decoding 12 MiB
Content parts per message 2048

What You Can Do

Text Generation with Chat Completions

Send prompts to /v1/chat/completions to generate conversational answers, summaries, and contextual responses. Prefix caching is active for all language and coding models, so requests that start with the same content can respond faster.

Use the Responses API

/v1/responses is an alternative to chat completions for text generation with every LLM. It takes a flat list of input items and returns typed output items (message, reasoning, function_call), so reasoning and tool calls arrive as separate objects. Use instructions for a system prompt.

Control Reasoning

OpenAI GPT-OSS 120B takes reasoning_effort low, medium (default), or high. Qwen3.5-9B and Qwen3.5 397B A17B reason by default; set reasoning_effort to none to turn it off. Qwen3.8-27B takes xhigh, medium (default), low, or none.

Tool Calling

Define tools with JSON schemas in the request. The model decides whether to answer directly or to return a structured tool call that your code executes, for example to fetch live data or trigger a workflow.

Image Generation and Editing

Create images from text with FLUX.2 [klein] 4B through /v1/images/generations, or edit an existing image through /v1/images/edits (multipart/form-data). Output formats are jpg, jpeg, png, and webp.

Image Input

Attach images as a public URL or a base64 data URI on /v1/chat/completions, /v1/responses, /v1/embeddings, /v1/rerank, and /v1/images/edits, for models whose model card advertises image input.

Embeddings, Reranking and RAG

Turn text (and, with Qwen3 VL Embedding 8B, images) into vectors for semantic search, similarity, and clustering. Use Qwen3 VL Reranker 8B through POST /v1/rerank as a second retrieval stage. For Retrieval Augmented Generation (RAG), the how-to uses DBaaS for PostgreSQL with the vector extension (pgvector) as the vector store.

OCR and PDF-to-Text

Convert document images to Markdown text (including LaTeX for math) with LightOnOCR-2-1B. For PDFs, render each page as an image first; Mistral Small 24B can also extract and interpret PDF content from page images.

Connect Existing Tools

Tools such as AnythingLLM and OpenCode (through @ai-sdk/openai-compatible) connect through the OpenAI-compatible API. The n8n package @ionos-cloud/n8n-nodes-ionos-cloud adds an IONOS CLOUD Chat Model node for AI Agents and an IONOS CLOUD Embeddings node for vector-store RAG workflows.

Best For

Scenario Why It Fits
Chatbots and assistants with data-sovereignty needs Inference runs only in Germany, requests are stateless, and your data is never used for training
Semantic search and RAG over private documents Embedding, reranking, and chat models in one API, with pgvector as a standard PostgreSQL vector store
AI-assisted software development A dedicated coding model with tool calling, usable from OpenCode and other OpenAI-compatible tools
Content and marketing visuals A text-to-image model generates and edits images from prompts
Workflow automation through natural language Tool calling lets models trigger APIs and business systems
Document digitization A dedicated OCR model turns scans and document images into structured text

Consider Alternatives If

If You Need... Consider Why
Fine-tuning a model on your own data Cloud GPU VM GPU servers provide the compute to customize foundation models with proprietary datasets
Steady 24/7 inference at a fixed rate Cloud GPU VM Dedicated GPU resources at a fixed rate instead of per-request pricing
AI-driven management of your cloud infrastructure IONOS CLOUD MCP Server It gives an AI client tools for the IONOS CLOUD API; the AI Model Hub can supply the model

Key Considerations

Billing & Costs

  • Main billing: Per request, by model type. Language, coding, and OCR models: per million input and output tokens. Embedding and reranking models: per million input tokens at one rate. FLUX.2 [klein] 4B: per megapixel. Prices in EUR (net):
Model Input per 1M tokens Output per 1M tokens
Qwen3.5-9B 0.10 0.15
Mistral Small 24B 0.10 0.30
Qwen3.8-27B 0.40 2.70
Meta Llama 3.3 70B 0.65 0.65
OpenAI GPT-OSS 120B 0.15 0.65
Qwen3.5 397B A17B 0.60 3.60
Qwen3-Coder-Next 80B 0.15 0.80
LightOnOCR-2-1B 0.15 0.30
  • Embeddings and reranking (per 1M tokens): BGE m3 0.02, Qwen3 VL Embedding 8B 0.10, Qwen3 VL Reranker 8B 0.04. For reranking, the query and all submitted documents count as input
  • Images: FLUX.2 [klein] 4B costs 0.013 for the first output megapixel, 0.001 per additional output megapixel, and 0.001 per input megapixel (1 MP = 1024 x 1024; output is rounded up; each input image is billed as 1 MP)
  • What counts as input: System prompts and tool definitions count as input tokens. Images sent to Mistral Small 24B are billed as input tokens
  • Retries and caching: Each retry is billed separately. Cached or repeated input is not discounted
  • When idle: You pay only for consumed tokens and megapixels. Playground requests in the DCD are billed at standard rates
  • Monitoring: Cost & Usage in the DCD (contract administrators and owners on regular contracts; owners on reseller contracts) and the Billing API utilization endpoints, broken down by model and unit

Limitations

  • Model lifecycle: IONOS CLOUD adds new models and retires older ones; query GET /v1/models for the current list
  • Responses API: store, previous_response_id, conversation, background, stream, and built-in tools such as web_search and file_search are not available (a 400 error names the field). tool_choice accepts auto and none
  • Request concurrency: Requests larger than 4 MiB and chunked requests are accepted six at a time; a further one gets 503 with Retry-After: 1
  • Platform overload: Under very high demand across all customers, the API can return HTTP 529 Site is Overloaded with a Retry-After header
  • OCR input: LightOnOCR-2-1B accepts images only (no PDF), and text prompts do not change its Markdown output
  • No pseudonymization: Request content is not pseudonymized by IONOS CLOUD; apply it in your application if you need it
  • New users: A public cloud contract is required, and activation of a new contract can take up to 24 hours
  • EU AI Act: IONOS CLOUD is the provider of the AI system for every model on the Hub, and the model developers are the providers of the models. You are the Deployer (or the Provider of your own AI system) and do your own risk assessment (Limited-Risk or High-Risk). No pre-configured AI Act logging template is provided

Management Options

  • Data Center Designer (DCD): Users & Groups for the privilege, Token Manager for tokens, AI > AI Model Hub (Models and Playground tabs), and Cost & Usage
  • REST API: The OpenAI-compatible API
  • SDKs and tools: OpenAI-compatible SDKs, plus integrations such as AnythingLLM, OpenCode, and n8n