Unit 2.5: Database and Streaming Service Provisioning
Introduction
TaskBoard's transactional data still lives on a Block Storage volume of the API server. You want a managed PostgreSQL cluster with backups and failover, and a cache in front of it for sessions and hot reads, without operating either yourself. Both must be created from the same Terraform project as the rest of the stack, so that every branch environment gets its own database tier.
In this unit you provision that database tier as code. You size and configure the PostgreSQL cluster, connect it to the private DB LAN from Unit 2.2, add an In-Memory DB cluster, and output the connection details your application needs. MariaDB, MongoDB and Event Streams for Apache Kafka follow the same pattern, and you see their resource names and key differences. In the Code Lab you deploy TaskBoard's database tier and connect to both clusters.
1. PostgreSQL with ionoscloud_pg_cluster_v2
resource "ionoscloud_pg_cluster_v2" "taskboard" {
name = "taskboard-pg"
description = "TaskBoard transactional data"
version = var.pg_version
location = ionoscloud_datacenter.taskboard.location
replication_mode = "ASYNCHRONOUS"
instances = {
count = 3
cores = 2
ram = 4
storage_size = 50
}
connections = {
datacenter_id = ionoscloud_datacenter.taskboard.id
lan_id = ionoscloud_lan.db.id
primary_instance_address = "10.10.2.6/24"
}
backup = {
location = var.backup_location
retention_days = 7
}
maintenance_window = {
day_of_the_week = "Sunday"
time = "03:00:00"
}
credentials = {
username = "taskboard"
password = var.pg_password
password_version = "1"
database = "taskboard"
}
}
The v2 resource talks to the regional PostgreSQL v2 API that matches location, and changing location destroys the cluster and creates a new one. In instances, ram and storage_size are in GB, unlike the MB of a server's ram.
1.1 Sizing and Replication
instances.count is the total number of instances: one primary and n-1 standbys. A cluster has a maximum of five instances, and IONOS CLOUD recommends at least three for production. replication_mode is ASYNCHRONOUS or STRICTLY_SYNCHRONOUS:
| Mode | Behaviour | Trade-off |
|---|---|---|
ASYNCHRONOUS |
The primary does not wait for a standby to confirm a write | Better performance; a small amount of data may be lost on failover |
STRICTLY_SYNCHRONOUS |
Each commit waits for a synchronous standby; with no standby available, no writes are accepted | No data loss on failover; sacrifices availability for durability |
As the table shows, STRICTLY_SYNCHRONOUS stops writes when no standby is available, so it needs at least three instances. TaskBoard's development environment uses ASYNCHRONOUS.
Standby instances serve high availability: one takes over when the primary fails. Your application connects through the cluster's single dns_name. Plan read scaling with the cache in Section 2, not with the standbys; Module 4 builds that caching layer.
1.2 Backups, Maintenance and Versions
backup.location is the Object Storage location for backups and is immutable. List the valid values with GET /backup-locations of the PostgreSQL v2 API, for example eu-central-3. Retention defaults to 7 days, and you can restore to a point in time within the retention window through restore_from_backup. The maintenance_window is a weekly 4-hour window; its time is in UTC.
Supported major versions are 14, 15, 16, 17 and 18. Read the exact version strings for your location from the data source:
data "ionoscloud_pg_versions_v2" "available" {
location = ionoscloud_datacenter.taskboard.location
}
output "pg_versions" {
value = [for v in data.ionoscloud_pg_versions_v2.available.versions : "${v.version} ${v.status}"]
}
You can only upgrade to a higher version; downgrades are not supported.
1.3 Credentials Without State Leaks
credentials.password is write-only: it is never stored in Terraform state and requires Terraform 1.11 or later. Because Terraform cannot detect a change to a value it does not store, you increment password_version (for example from "1" to "2") whenever you rotate the password. Read the password from your secret store, never from a .tfvars file in Git:
variable "pg_password" {
type = string
sensitive = true
ephemeral = true
}
export TF_VAR_pg_password="$(vault kv get -field=pg_password secret/taskboard/dev)"
The user names postgres, admin, root, sys, replication and names starting with pg_ are reserved, so choose an application-specific name such as taskboard.
2. In-Memory DB with ionoscloud_inmemorydb_cluster_v2
In-Memory DB v2 runs Valkey, which is compatible with Redis clients, in versions 9, 8.1 and 8.
resource "ionoscloud_inmemorydb_cluster_v2" "cache" {
name = "taskboard-cache"
location = ionoscloud_datacenter.taskboard.location
version = var.cache_version
eviction_policy = "allkeys-lru"
persistence_mode = "None"
instances = {
count = 1
cores = 1
ram = 4
}
connections = {
datacenter_id = ionoscloud_datacenter.taskboard.id
lan_id = ionoscloud_lan.db.id
primary_instance_address = "10.10.2.7/24"
}
snapshot = {
location = var.backup_location
retention_days = 7
}
maintenance_window = {
day_of_the_week = "Sunday"
time = "04:00:00"
}
credentials = {
username = "taskboard"
password = {
algorithm = "SHA-256"
hash = sha256(var.cache_password)
}
}
}
2.1 Cache-Specific Arguments
- Password hash: the API accepts only a pre-hashed SHA-256 value as a 64-character lowercase hexadecimal string. Terraform's
sha256()function produces exactly that, so the plain password never reaches the API or the state. eviction_policy: required.allkeys-lrusuits a read cache;noevictionrejects writes when memory is full.persistence_mode:None,AOF,RDBorRDB_AOF. A pure cache can run without persistence; sessions you do not want to lose after a restart needRDBorAOF.instances:countfrom 1 to 5,ramfrom 4 to 240 GB. RAM cannot be downgraded, and storage is derived from RAM and the persistence mode.connections: only one connection is allowed per cluster.
Read the available version strings with the ionoscloud_inmemorydb_versions_v2 data source, in the same way as for PostgreSQL. Clients connect with redis-cli or any Redis-compatible driver, optionally over TLS, and the cluster's dns_name points to its primary instance.
3. Network Placement and Connection Outputs
3.1 Addresses in the DB LAN
Every managed database gets a static address in a private LAN that you choose in code. Two rules keep these addresses safe:
- Take them from the range reserved for managed services,
.2to.9(Unit 2.2, Section 1.3). TaskBoard uses.4for the NAT gateway,.6for PostgreSQL and.7for the cache. - Avoid the ranges that DBaaS blocks:
10.208.0.0/12,10.233.0.0/18,192.168.230.0/24and10.233.64.0/18.
The clusters sit in the DB LAN, so the app servers need a NIC in that LAN to reach them:
resource "ionoscloud_nic" "api_db" {
datacenter_id = ionoscloud_datacenter.taskboard.id
server_id = ionoscloud_server.api.id
lan = ionoscloud_lan.db.id
name = "db"
dhcp = true
ips = ["10.10.2.11"]
firewall_active = false
}
Add the same NIC with 10.10.2.21 to the worker. The DB-tier NSG rules from Unit 2.3 then admit these two addresses.
3.2 Outputs for the Application
output "pg_host" {
value = ionoscloud_pg_cluster_v2.taskboard.dns_name
}
output "pg_url" {
value = "postgresql://taskboard@${ionoscloud_pg_cluster_v2.taskboard.dns_name}:5432/taskboard?sslmode=verify-full"
sensitive = true
}
output "cache_host" {
value = ionoscloud_inmemorydb_cluster_v2.cache.dns_name
}
The URL deliberately has no password: your application reads it from the same secret store that fed Terraform. All client connections to PostgreSQL are encrypted with TLS; the default SSL mode is prefer and clients cannot disable encryption. For verify-ca or verify-full, the client needs the root certificate ISRG Root X1, because the server certificates are issued by Let's Encrypt for the cluster's DNS name. Client libraries must support SCRAM-SHA-256 authentication.
4. Same-Pattern References: MariaDB, MongoDB and Kafka
These services share the core building blocks: a location, a private LAN connection and sizing. MariaDB and MongoDB also take a maintenance window, and MariaDB takes credentials.
resource "ionoscloud_mariadb_cluster_v2" "reports" {
name = "taskboard-reports"
location = ionoscloud_datacenter.taskboard.location
version = var.mariadb_version
instances = { count = 1, cores = 2, ram = 4, storage_size = 20 }
connections = {
datacenter_id = ionoscloud_datacenter.taskboard.id
lan_id = ionoscloud_lan.db.id
primary_instance_address = "10.10.2.8/24"
}
backup = { location = var.backup_location, retention_days = 7 }
maintenance_window = { day_of_the_week = "Sunday", time = "05:00:00" }
credentials = { username = "app", password = var.mariadb_password, database = "app" }
}
| Service | Resource | Key differences |
|---|---|---|
| DBaaS for MariaDB | ionoscloud_mariadb_cluster_v2 |
Port 3306; versions 10.6, 10.11, 11.4, 11.8 and 12.3; replication is asynchronous only; password is a sensitive argument, null in state after import |
| MongoDB | ionoscloud_mongo_cluster |
edition (playground, business or enterprise), mongodb_version (6.0 or 7.0), instances and connections changes force recreation; connections.cidr_list takes the list of IPs with subnet for the cluster; computed connection_string uses mongodb+srv |
| Event Streams for Apache Kafka | ionoscloud_kafka_cluster, ionoscloud_kafka_cluster_topic |
size instead of instance sizing; three brokers whose broker_addresses you set in the LAN; mTLS authentication with client certificates valid for 365 days; topics set replication_factor and number_of_partitions |
As the table shows, the shape is the same, but recreation rules and authentication differ. Check the RequiresReplace and "forces re-creation" notes before you change an argument on a live cluster.
resource "ionoscloud_kafka_cluster" "events" {
name = "taskboard-events"
location = ionoscloud_datacenter.taskboard.location
version = "4.0.0"
size = "S"
connections {
datacenter_id = ionoscloud_datacenter.taskboard.id
lan_id = ionoscloud_lan.db.id
broker_addresses = ["10.10.2.2/24", "10.10.2.3/24", "10.10.2.5/24"]
}
}
resource "ionoscloud_kafka_cluster_topic" "task_events" {
cluster_id = ionoscloud_kafka_cluster.events.id
location = ionoscloud_kafka_cluster.events.location
name = "task-events"
replication_factor = 3
number_of_partitions = 6
}
A replication factor of 3 is the default and the recommended value. Broker addresses must avoid 10.224.0.0/11. Your clients fetch their certificates and key from GET /clusters/{clusterId}/users/{userId}/access; Unit 4.3 covers the client side.
API Reference Quick Card
Key API endpoints behind this unit's resources:
| Method | Endpoint | Description |
|---|---|---|
POST |
/clusters |
Created by ionoscloud_pg_cluster_v2 (PostgreSQL v2 API, https://postgresql.de-txl.ionos.com/v2) |
GET |
/backup-locations |
Lists valid backup.location values (PostgreSQL v2 API) |
GET |
/versions |
Lists available versions (PostgreSQL v2 and In-Memory DB v2 APIs) |
POST |
/clusters |
Created by ionoscloud_inmemorydb_cluster_v2 (In-Memory DB v2 API, https://in-memory-db.de-txl.ionos.com/v2) |
GET |
/clusters/{clusterId}/users/{userId}/access |
Kafka client certificates and key (Kafka API) |
Base URL: regional per service, for example https://postgresql.de-txl.ionos.com/v2
Authentication: Authorization: Bearer <token>
Code Lab
Objective: Deploy TaskBoard's database tier with Terraform (a PostgreSQL cluster and an In-Memory DB cluster in the DB LAN), output the connection details, and connect to both from the API server.
Prerequisites:
- The project from Units 2.1 to 2.4, applied, with
IONOS_TOKENset and Terraform 1.11 or later - Two passwords in your secret store, exported as
TF_VAR_pg_passwordandTF_VAR_cache_password psqlandredis-cli(orvalkey-cli) on the API server
Step 1: List the available versions
Add the ionoscloud_pg_versions_v2 data source and output from Section 1.2, plus the same pattern for ionoscloud_inmemorydb_versions_v2 (its list attribute is items).
terraform apply -auto-approve -target=data.ionoscloud_pg_versions_v2.available >/dev/null
terraform output pg_versions
Expected output:
[
"14 ...",
...
"18 ..."
]
Step 2: Set the versions and the backup location
variable "pg_version" { default = "17" } # one of the versions from Step 1
variable "cache_version" { type = string } # one of the In-Memory DB versions
variable "backup_location" { default = "eu-central-3" }
Expected output:
(file saved)
Step 3: Add the clusters, NICs and outputs
Add the PostgreSQL cluster (Section 1), the cache (Section 2), the api_db NIC and the worker's DB NIC (Section 3.1), and the outputs (Section 3.2). Then apply:
terraform apply -auto-approve | grep "Apply complete"
terraform output -raw pg_host; echo
Expected output:
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
pg-<id>.postgresql.de-txl.ionos.com
Cluster creation takes several minutes; the provider waits up to the resource timeouts.
Step 4: Connect to PostgreSQL from the API server
PGHOST=$(terraform output -raw pg_host)
ssh root@"$(terraform output -raw api_public_ip)" \
"PGPASSWORD='$TF_VAR_pg_password' psql 'host=$PGHOST dbname=taskboard user=taskboard sslmode=require' -c 'select version();'"
Expected output:
PostgreSQL 17...
(1 row)
Step 5: Connect to the cache
CACHE=$(terraform output -raw cache_host)
ssh root@"$(terraform output -raw api_public_ip)" \
"redis-cli -h $CACHE --user taskboard --pass '$TF_VAR_cache_password' PING"
Expected output:
PONG
Validation Checklist:
- [ ]
terraform applycreated both clusters and the two DB-LAN NICs - [ ]
psqlconnects over TLS to the cluster's DNS name - [ ] The cache answers
PONGwith the configured user - [ ]
terraform state show ionoscloud_pg_cluster_v2.taskboardshows no password
Cleanup:
terraform destroy -auto-approve
Common Pitfalls
Developer mistakes to avoid with this unit's topic:
-
Primary address in a blocked or DHCP range
- Problem: Cluster creation fails, or the database collides with a server address.
- Why it happens: The address lies in a range DBaaS blocks, or outside the
.2to.9managed-services range, where DHCP may hand it to a NIC. - Fix: Plan all managed-service addresses per LAN in one
localsmap, from.2to.9, outside the blocked ranges (Section 3.1).
-
Password rotation that never happens
- Problem: You change
TF_VAR_pg_password,terraform planshows no changes, and the old password still works. - Why it happens: The password is write-only and not stored in state, so Terraform cannot see that it changed.
- Fix: Increment
credentials.password_versiontogether with the new password.
- Problem: You change
-
Connections that break on failover
- Problem: After maintenance or a failover, the API logs connection errors until it restarts.
- Why it happens: During a failover the client must reconnect. In an uncontrolled failure the connection may not be closed cleanly.
- Fix: Reconnect with an exponential back-off, starting with an immediate retry, and use a connection pool that validates connections.
Summary
You can now provision TaskBoard's database tier as code. The PostgreSQL v2 cluster has a primary and standbys for failover, a replication mode you choose deliberately, backups in an Object Storage location, and a write-only password that never lands in state. The In-Memory DB cluster runs Valkey with a SHA-256 hashed password and an eviction policy that suits a cache. Both connect to the private DB LAN on addresses from the managed-services range, and your application gets their DNS names from Terraform outputs.
Key Points:
- Use the v2 resources
ionoscloud_pg_cluster_v2,ionoscloud_inmemorydb_cluster_v2andionoscloud_mariadb_cluster_v2 - A PostgreSQL cluster has up to five instances: one primary and n-1 standbys for high availability
- PostgreSQL connections are always TLS-encrypted; use ISRG Root X1 for
verify-full - Keep passwords out of state: write-only for PostgreSQL,
sha256()hashes for In-Memory DB - Give managed databases static addresses from
.2to.9and avoid the DBaaS blocked ranges
Important Terminology:
- Standby instance: A PostgreSQL instance that replicates the primary and takes over on failover
- Replication mode:
ASYNCHRONOUSorSTRICTLY_SYNCHRONOUS, the trade-off between write speed and durability - Write-only argument: A Terraform argument sent to the API but never stored in state, such as the PostgreSQL password
- Eviction policy: The rule that decides which keys In-Memory DB removes when memory is full
- Primary instance address: The static IP and netmask a managed database gets in your private LAN
Next Steps
Continue Learning: Unit 2.6: Knowledge Check - Infrastructure as Code
Related Topics: