Structum Configuration Philosophy¶
Version: 3.0
Status: Stable
Index¶
1. Core Principle¶
Structum adopts a layered configuration approach to separate responsibilities:
Developers → Define structure and defaults
DevOps/SRE → Configure infrastructure and secrets
End Users → Customize runtime experience
1.1 “Configuration as Code”¶
Configuration follows the same principles as code:
Versioned (defaults in Git)
Validated (Pydantic schemas)
Testable (mock providers for unit tests)
Overridable (override layers without modifying sources)
1.2 Separation of Concerns¶
Application code doesn’t know where configuration comes from.
# ✅ Business code decoupled
config = get_config()
db_host = config.get("database.host")
# Can come from: TOML, ENV, Secrets, Runtime, ...
# ❌ Anti-pattern: coupling
db_host = os.getenv("DB_HOST", "localhost")
# Hard-coded to override mechanism
2. The 5-Layer Model¶
Structum uses a priority hierarchy to resolve configuration:
┌──────────────────────────────────────────────────┐
│ Layer 1: RUNTIME (In-Memory) │ ← HIGHEST PRIORITY
│ • Changes via config.set() not persisted │
│ • Valid only for current process │
├──────────────────────────────────────────────────┤
│ Layer 2: USER PERSISTENT (Runtime Saved) │
│ • File: ~/.structum/{namespace}_saved.json │
│ • Changes persisted via config.save() │
│ • User/instance specific │
├──────────────────────────────────────────────────┤
│ Layer 3: ENVIRONMENT VARIABLES │
│ • Format: STRUCTUM_{NAMESPACE}__{KEY} │
│ • Infrastructure config (Docker/K8s) │
│ • Set by DevOps/deployment scripts │
├──────────────────────────────────────────────────┤
│ Layer 4: SECRETS │
│ • File: config/.secrets.toml (git-ignored) │
│ • Contains password, API keys, tokens │
│ • Auto-merged with Layer 5 │
├──────────────────────────────────────────────────┤
│ Layer 5: APPLICATION DEFAULTS │ ← LOWEST PRIORITY
│ • File: config/app/{namespace}.toml │
│ • Defaults declared by developer │
│ • Versioned in Git, baseline for all │
└──────────────────────────────────────────────────┘
2.1 Resolution: “First Found Wins”¶
# Pseudo-code of resolver
def resolve_value(key):
if key in runtime_layer:
return runtime_layer[key] # Layer 1
if key in user_persistent_layer:
return user_persistent_layer[key] # Layer 2
if key in environment_variables:
return environment_variables[key] # Layer 3
if key in secrets_layer:
return secrets_layer[key] # Layer 4
if key in application_defaults:
return application_defaults[key] # Layer 5
return default_value
First value found wins. Upper layers override lower ones.
3. Separation of Responsibilities¶
Layer 5: Developer Layer (Application Defaults)¶
Who: Software Engineer
Format: TOML
Location: config/app/{namespace}.toml (versioned)
Purpose: Define structure, valid keys, safe defaults
# config/app/database.toml
[default]
host = "localhost"
port = 5432
pool_size = 10
[production]
host = "db.prod.example.com"
pool_size = 50
Layer 4: Secrets Layer¶
Who: DevOps, Security Team
Format: TOML (local) or Secret Manager (production)
Location: config/.secrets.toml (git-ignored)
Purpose: Isolate sensitive credentials from code
# config/.secrets.toml (DO NOT version!)
[database.auth]
password = "SuperSecretPassword123"
[api_client]
api_key = "sk-proj-abc123xyz"
⚠️ Security Checklist:
[ ] Added to
.gitignore[ ] File permissions:
chmod 600[ ] In production: use Vault/K8s Secrets instead
Layer 3: Environment Variables¶
Who: DevOps, SRE, CI/CD
Format: Environment Variables
Location: Shell, Docker Compose, K8s ConfigMaps
Purpose: Override per-environment without modifying files
# Override single value
export STRUCTUM__DATABASE__HOST=prod-db.example.com
# Nested value (double underscore)
export STRUCTUM__DATABASE__POOL__MAX_SIZE=100
Layer 2: User Persistent¶
Who: End User, Application Runtime
Format: JSON
Location: ~/.structum/{namespace}_saved.json
Purpose: Save user preferences across restarts
config.set("ui.theme", "dark_mode")
config.save() # Creates ~/.structum/ui_saved.json
Layer 1: Runtime (In-Memory)¶
Who: Application Runtime
Format: In-Memory Dict
Purpose: Temporary changes without persistence
config.set("debug.verbose", True) # Only for this run
# NOT persisted unless config.save() is called
4. Resolution Flow¶
Example: Multi-Layer Resolution¶
Setup:
# config/app/database.toml
[default]
host = "localhost"
port = 5432
# Environment
export STRUCTUM__DATABASE__HOST=staging-db.com
// ~/.structum/database_saved.json
{"host": "my-custom-db.local"}
Resolution:
config.get("database.host")
# Layer 1 (Runtime): ❌ Not set
# Layer 2 (User): ✅ "my-custom-db.local" ← WINS
config.get("database.port")
# Layer 1-2: ❌ Not set
# Layer 3 (ENV): ❌ Not set
# Layer 4 (Secrets): ❌ Not set
# Layer 5 (Defaults): ✅ 5432 ← WINS
5. Diagrams¶
5.1 Architecture Overview¶
┌────────────────────────────────────────────────────────────────┐
│ APPLICATION CODE │
│ config.get("db.host") │
└────────────────────────┬───────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────┐
│ CONFIG FACADE (Singleton) │
│ • Manages layer priority │
│ • Resolves dot-notation (a.b.c → nested dict) │
│ • In-memory cache for performance │
└────────────────────────┬───────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────┐
│ DYNACONF CONFIG PROVIDER (Strategy) │
│ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ LAYER RESOLVER (Priority Stack) │ │
│ ├─────────────────────────────────────────────────┤ │
│ │ 1. Runtime (volatile) │ │
│ │ 2. User Persistent (~/.structum/*.json) │ │
│ │ 3. Environment Variables (STRUCTUM__*) │ │
│ │ 4. Secrets (.secrets.toml) │ │
│ │ 5. Application Defaults (.toml) │ │
│ └─────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────┘
6. Use Cases¶
6.1 Local Development¶
Setup: Only config/app/*.toml (versioned defaults)
config.get("database.host") # "localhost" (from TOML)
6.2 Staging Environment¶
Setup: Defaults + ENV vars from deployment
STRUCTUM__DATABASE__HOST=staging-db.internal
config.get("database.host") # "staging-db.internal" (ENV wins)
6.3 Production with Secrets¶
Setup: Defaults + ENV + .secrets.toml
config.get("database.host") # "prod-db" (from ENV)
config.get("database.auth.password") # "..." (from .secrets.toml)
6.4 End User Customization¶
Setup: User modifies preferences
config.set("ui.theme", "dark")
config.save() # Persists to ~/.structum/ui_saved.json
# After restart:
config.get("ui.theme") # "dark" (from saved.json)
7. Best Practices¶
7.1 What Goes Where¶
Layer |
Content |
Examples |
|---|---|---|
Defaults (TOML) |
Safe values, structure |
|
Secrets |
Sensitive credentials |
|
Environment |
Infrastructure config |
|
User Persistent |
User preferences |
|
Runtime |
Temporary overrides |
|
7.2 Anti-Patterns to Avoid¶
❌ Secrets in versioned TOML
# config/app/database.toml
password = "admin123" # ❌ Exposed in Git!
❌ Magic values without default
db_host = config.get("database.host") # ❌ May raise error!
# ✅ Always provide default
db_host = config.get("database.host", default="localhost")
7.3 Production Security Checklist¶
[ ]
.secrets.tomlin.gitignore[ ] File permissions:
chmod 600[ ] Use Vault/K8s Secrets in production
[ ] Periodic credential rotation
[ ] Pydantic validation enabled