Structum Observability Stack¶

Complete guide to Structum’s dual-mode logging and metrics system.

Feature	Status	Details
Version	0.1.0	Alpha
Last Updated	2025-01-12

Index¶

Architectural Overview
Logging System
Metrics System
Context Propagation
Plugin Observability (Enterprise)
Setup Guide
Web Framework Integration
Best Practices
Troubleshooting
API Reference

1. Architectural Overview¶

1.1 Dual-Mode Design¶

Structum Observability follows the same pattern as configuration: works out-of-the-box with simple fallback, scales to enterprise with plugins.

        graph TD
    classDef app fill:#e1f5fe,stroke:#01579b
    classDef core fill:#fff3e0,stroke:#ff6f00
    classDef mode fill:#f3e5f5,stroke:#7b1fa2
    classDef back fill:#e8f5e9,stroke:#2e7d32

    APP[Application Code]:::app --> |log.info / metrics.inc| FACADE

    subgraph "Observability Facade (Core)"
        FACADE[Facade]:::core
        FACADE --> LOG[Logging]:::core
        FACADE --> MET[Metrics]:::core
    end

    LOG --> L_MODE{"Mode Detection"}:::mode
    MET --> M_MODE{"Mode Detection"}:::mode

    L_MODE --> |"No Plugin"| L_FALL["Fallback: stdlib"]:::back
    L_MODE --> |"Plugin"| L_ENT["Enterprise: structlog"]:::back

    M_MODE --> |"No Plugin"| M_NOOP["No-Op"]:::back
    M_MODE --> |"Plugin"| M_PROM["Prometheus"]:::back

1.2 Mode Detection (Automatic)¶

The system automatically detects which backend to use:

Logging:

# On import of structum_lab.logging
try:
    import structlog
    _LOGGER_BACKEND = "structlog"  # Enterprise Mode
except ImportError:
    _LOGGER_BACKEND = "stdlib"     # Fallback Mode

Metrics:

# On import of structum_lab.monitoring
try:
    from prometheus_client import Counter, Histogram
    _METRICS_BACKEND = "prometheus"
except ImportError:
    _METRICS_BACKEND = "noop"

⚠️ Note: Detection happens at import-time, not runtime. If you install plugins after importing, a restart is required.

2. Logging System¶

2.1 Unified API¶

All loggers implement LoggerInterface:

from typing import Protocol, Any

class LoggerInterface(Protocol):
    """Contract for logging backends."""
    
    def debug(self, message: str, **kwargs: Any) -> None: ...
    def info(self, message: str, **kwargs: Any) -> None: ...
    def warning(self, message: str, **kwargs: Any) -> None: ...
    def error(self, message: str, **kwargs: Any) -> None: ...
    def critical(self, message: str, **kwargs: Any) -> None: ...

Usage:

from structum_lab.logging import get_logger

log = get_logger(__name__)

# Works IDENTICALLY in Fallback and Enterprise mode
log.info("User logged in", user_id=123, ip_address="192.168.1.1")

2.2 Fallback Mode (Default)¶

Activation: Automatic if structlog is not installed.

Implementation: Thin wrapper around stdlib logging.Logger.

Output:

2025-01-11 10:30:45 INFO     myapp.service: User logged in

Features:

✅ Zero extra dependencies
✅ Compatible with stdlib logging (can coexist)
❌ **kwargs end up in extra dict (hidden by default)
❌ Fixed format, not customizable
❌ No structured output (JSON)

Configuration:

from structum_lab.logging import configure_logging

# Setup fallback logger
configure_logging(
    level="INFO",               # Global log level
    format="text",              # "text" or "json" (limited in fallback)
    handlers=["console"],       # Output destinations
    capture_warnings=True       # Capture warnings.warn()
)

Accessing kwargs in Fallback:

import logging

# Custom formatter to show kwargs
formatter = logging.Formatter(
    '%(asctime)s %(levelname)s %(name)s: %(message)s [%(user_id)s]'
)

# Extra kwargs are accessible as %(name)s
log.info("User login", extra={"user_id": 123})  # ⚠️ Different syntax than Enterprise

2.3 Enterprise Mode (Plugin)¶

Activation: Automatic when structum-observability is installed.

Implementation: Powered by structlog with optimized processors.

Output:

{
  "timestamp": "2025-01-11T10:30:45.123456Z",
  "level": "info",
  "logger": "myapp.service",
  "message": "User logged in",
  "user_id": 123,
  "ip_address": "192.168.1.1",
  "request_id": "req-abc123",
  "process": 12345,
  "thread": "MainThread"
}

Features:

✅ Structured JSON output (machine-readable)
✅ kwargs integrated into log event (first-class)
✅ Automatic context (request_id, user, correlation_id)
✅ Performance: ~3x faster than stdlib for high-throughput
✅ Customizable processor chain

Configuration:

from structum_lab.logging import configure_logging

configure_logging(
    level="INFO",
    format="json",                    # Structured JSON
    processors=[                      # Custom processor chain
        "timestamp",                  # ISO8601 timestamp
        "add_log_level",              # Normalize "level" key
        "add_logger_name",            # Add "logger" key
        "contextvars",                # Inject request_id, user, etc
        "stack_info",                 # Traceback on error
        "json_renderer"               # Final JSON encoding
    ],
    context_class="dict",             # Internal context format
    wrapper_class="BoundLogger"       # Custom logger class
)

Available Processors:

Processor	Function
`timestamp`	Adds ISO8601 timestamp
`add_log_level`	Normalizes level name
`add_logger_name`	Adds logger name
`contextvars`	Injects context from `contextvars`
`stack_info`	Captures stack trace
`exception_formatter`	Formats exceptions
`json_renderer`	Encodes final JSON
`console_renderer`	Pretty-print for dev

2.4 Detailed Comparison¶

Feature	Fallback Mode	Enterprise Mode
Backend	`logging` (stdlib)	`structlog`
Output Format	Human-readable text	Structured JSON
kwargs Handling	`extra` dict (hidden)	First-class fields
Performance	Standard	~3x faster
Context Injection	Manual	Automatic (`contextvars`)
Customization	Limited (Formatter)	Complete (Processors)
Dependencies	Zero	`structlog>=23.1.0`
Use Case	CLI, scripts, dev	Production, microservices
Parsing Tools	Grep, awk	`jq`, ELK, Splunk

2.5 Logger Naming Convention¶

# ✅ Use __name__ for auto-hierarchy
log = get_logger(__name__)
# myapp.services.user → Logger("myapp.services.user")

# ✅ Explicit component namespace
log = get_logger("myapp.database")

# ❌ Avoid generic hardcoded strings
log = get_logger("logger")  # Untraceable

2.6 Verifying Active Mode¶

from structum_lab.logging import get_logger_backend

backend = get_logger_backend()
print(f"Active logging backend: {backend}")
# Output: "stdlib" or "structlog"

3. Metrics System¶

3.1 Unified API¶

from typing import Protocol, Any

class MetricsInterface(Protocol):
    """Contract for metrics backend."""
    
    def increment(
        self, 
        name: str, 
        value: int = 1, 
        tags: dict[str, str] | None = None
    ) -> None:
        """Increment counter."""
    
    def gauge(
        self, 
        name: str, 
        value: float, 
        tags: dict[str, str] | None = None
    ) -> None:
        """Set gauge value."""
    
    def timing(
        self, 
        name: str, 
        value: float, 
        tags: dict[str, str] | None = None
    ) -> None:
        """Record duration (converted to histogram)."""
    
    def histogram(
        self, 
        name: str, 
        value: float, 
        tags: dict[str, str] | None = None
    ) -> None:
        """Record value distribution."""

Usage:

from structum_lab.monitoring import get_metrics

metrics = get_metrics("myapp")

# Counter: only increases
metrics.increment("requests.total", tags={"method": "GET", "status": "200"})

# Gauge: can increase/decrease
metrics.gauge("connections.active", 42)

# Timing: operation duration
metrics.timing("request.duration", 0.123, tags={"endpoint": "/api/users"})

# Histogram: value distribution
metrics.histogram("payload.size_bytes", 1024)

3.2 No-Op Mode (Default)¶

Activation: Automatic if prometheus-client is not installed.

Behavior:

All methods are no-op (do nothing)
No errors raised
⚠️ No warning that metrics are ignored

How to detect:

from structum_lab.monitoring import get_metrics_backend

backend = get_metrics_backend()
if backend == "noop":
    print("⚠️ WARNING: Metrics are disabled (No-Op mode)")

3.3 Prometheus Mode (Plugin)¶

Activation: Automatic when structum-observability is installed.

Registry: Uses global prometheus_client.REGISTRY by default.

Metric Conversion:

API Call	Prometheus Type	Notes
`increment()`	`Counter`	Only increases
`gauge()`	`Gauge`	Can increase/decrease
`timing()`	`Histogram`	With auto-configured buckets
`histogram()`	`Histogram`	Custom distribution

Namespace Prefix:

metrics = get_metrics("myapp")
metrics.increment("requests.total")

# Prometheus metric name:
# myapp_requests_total
#
# Pattern: {namespace}_{metric_name}

Tags → Labels:

metrics.increment(
    "requests.total",
    tags={"method": "GET", "status": "200"}
)

# Prometheus:
# myapp_requests_total{method="GET", status="200"} 1

3.4 Built-in Structum Metrics¶

Configuration Provider (`structum.config`)¶

If you use DynaconfConfigProvider, you automatically get:

# Counter: config operations
structum_config_operations_total{
    operation="get|set|has",
    status="success|error",
    cache="hit|miss"
}

# Histogram: operation latency
structum_config_operation_duration_seconds_bucket{
    operation="get|set|has"
}

# Gauge: cache size
structum_config_cache_size

# Gauge: cache hit rate (0.0-1.0)
structum_config_cache_hit_rate

Sample Prometheus Query:

# Operations per second
rate(structum_config_operations_total[5m])

# P99 latency
histogram_quantile(
    0.99,
    rate(structum_config_operation_duration_seconds_bucket[5m])
)

# Cache effectiveness
structum_config_cache_hit_rate > 0.8

3.5 Best Practices¶

✅ Naming Convention:

# Pattern: namespace.component.metric_name
metrics = get_metrics("myapp.api")
metrics.increment("requests.total")  # myapp_api_requests_total

# ❌ Avoid redundant prefixes
metrics.increment("myapp_api_requests_total")  # becomes myapp_api_myapp_api_...

✅ Tags vs Metric Names:

# ✅ Use tags for dimensions
metrics.increment("requests.total", tags={"method": "GET", "status": "200"})
metrics.increment("requests.total", tags={"method": "POST", "status": "201"})

# ❌ Do not create separate metrics
metrics.increment("requests.get.200")
metrics.increment("requests.post.201")

✅ Cardinality Control:

# ✅ Low cardinality (controlled)
tags = {"method": "GET", "status": "200"}  # ~10 methods * ~10 status = 100 series

# ❌ High cardinality (combinatorial explosion)
tags = {"user_id": user_id, "request_id": req_id}  # Millions of time series!

✅ Counter vs Gauge:

# ✅ Counter: monotonically increasing values
metrics.increment("requests_total")      # Only goes up
metrics.increment("errors_total")        # Only goes up

# ✅ Gauge: fluctuating values
metrics.gauge("connections_active", 42)  # Up/Down
metrics.gauge("queue_size", 100)         # Up/Down

# ❌ Error: gauge for counter
metrics.gauge("requests_total", total_requests)  # Loses rate info

4. Context Propagation¶

4.1 What is Context?¶

Context is a set of key-value pairs that automatically propagates through:

Function calls
Threads (with limitations)
Log events
Metrics tags (optional)

Example:

HTTP Request → request_id="abc123", user="alice"
    ↓
Service Layer
    ↓ (context propagates)
Database Call
    ↓ (context present in logs)
Log: "Query executed" + request_id="abc123" + user="alice"

4.2 Implementation (`contextvars`)¶

Structum uses contextvars (stdlib Python 3.7+) for context propagation.

⚠️ Important: contextvars is thread-local but async-safe.

from contextvars import ContextVar

# Define context vars
request_id_var: ContextVar[str] = ContextVar("request_id", default=None)
user_var: ContextVar[str] = ContextVar("user", default=None)

4.3 Setting Context¶

Manually:

from structum_lab.logging import set_context, get_logger

log = get_logger(__name__)

# Set context for this request
set_context(request_id="req-abc123", user="alice")

# All subsequent logs will include these values
log.info("Processing order")
# {... "request_id": "req-abc123", "user": "alice", ...}

log.info("Order completed", order_id=456)
# {... "request_id": "req-abc123", "user": "alice", "order_id": 456, ...}

With Context Manager:

from structum_lab.logging import bind_context, get_logger

log = get_logger(__name__)

with bind_context(request_id="req-xyz", user="bob"):
    log.info("Inside context")  # Has request_id and user
    
log.info("Outside context")  # request_id and user not present

With Decorator:

from structum_lab.logging import with_context

@with_context(user="system")
def background_job():
    log.info("Job started")  # Automatically has user="system"

4.4 Reading Context¶

from structum_lab.logging import get_context

context = get_context()
print(context)  # {"request_id": "...", "user": "..."}

# Access single value
request_id = context.get("request_id")

4.5 Clearing Context¶

from structum_lab.logging import clear_context

clear_context()  # Removes all context
clear_context("request_id")  # Removes only request_id

4.6 Thread Safety¶

✅ Async-Safe:

import asyncio
from structum_lab.logging import set_context, get_logger

log = get_logger(__name__)

async def handler():
    set_context(request_id="req-123")
    log.info("Async handler")  # Context OK
    
    await other_async_func()  # Context propagates
    
asyncio.run(handler())

⚠️ Thread-Local (Does not propagate between threads):

import threading
from structum_lab.logging import set_context, get_logger

log = get_logger(__name__)

def worker():
    log.info("Worker thread")  # ❌ Context LOST (different thread)

set_context(request_id="req-456")
log.info("Main thread")  # ✅ Context OK

thread = threading.Thread(target=worker)
thread.start()

Solution for Threading:

from contextvars import copy_context

def worker():
    log.info("Worker")  # ✅ Context present

set_context(request_id="req-789")

# Copy context to new thread
ctx = copy_context()
thread = threading.Thread(target=ctx.run, args=(worker,))
thread.start()

5. Plugin Observability (Enterprise)¶

5.1 Installation¶

pip install structum-observability

Dependencies included:

structlog>=23.1.0 (logging)
prometheus-client>=0.18.0 (metrics)
python-json-logger>=2.0.0 (JSON formatting)

5.2 Automatic Activation¶

The plugin activates automatically on import:

# main.py
from structum_lab.logging import get_logger, configure_logging
from structum_lab.monitoring import get_metrics

# ✅ If structum-observability is installed, uses Enterprise mode
# ❌ Otherwise, uses Fallback/No-Op mode

log = get_logger(__name__)
metrics = get_metrics("myapp")

⚠️ Note: Avoid explicit import of structum_lab.plugins.observability.

5.3 Enterprise Configuration¶

from structum_lab.logging import configure_logging

configure_logging(
    level="INFO",
    format="json",
    processors=[
        "timestamp",
        "add_log_level",
        "add_logger_name",
        "contextvars",           # ← Context injection
        "stack_info",
        "exception_formatter",
        "json_renderer"
    ],
    context_vars=[               # ← Define which context vars to track
        "request_id",
        "user",
        "correlation_id",
        "tenant_id"
    ]
)

5.4 Decorator: `track_operation`¶

Automatically logs + metrics per operation:

from structum_lab.plugins.observability import track_operation

@track_operation("process_order")
def process_order(order_id: int):
    # Logic here
    return {"status": "completed"}

# Automatically:
# 1. Log "Starting process_order" (with timestamp)
# 2. Metrics: myapp_process_order_total{status="success"}
# 3. Metrics: myapp_process_order_duration_seconds (histogram)
# 4. Log "Completed process_order" (with duration)

With context:

@track_operation("checkout", include_args=["user_id"])
def checkout(user_id: int, cart_id: int):
    # ...
    pass

# Log includes: user_id=123 (cart_id excluded)

5.5 Tracing Support (Future)¶

⚠️ Alpha: Support for OpenTelemetry tracing is in roadmap.

# Future API (not yet available)
from structum_lab.plugins.observability import trace

@trace(span_name="database_query")
def query_user(user_id):
    # Automatically creates span + exports to OTLP
    pass

6. Setup Guide¶

6.1 Quick Start (Fallback Mode)¶

For: CLI tools, scripts, rapid prototyping.

# No extra installation needed
pip install structum

# app.py
from structum_lab.logging import configure_logging, get_logger

# Setup (optional, defaults OK)
configure_logging(level="INFO")

log = get_logger(__name__)
log.info("Application started")

Output:

2025-01-11 10:45:00 INFO     __main__: Application started

6.2 Upgrade to Enterprise Mode¶

# Install plugin
pip install structum-observability

# app.py (SAME CODE, no changes)
from structum_lab.logging import configure_logging, get_logger

configure_logging(
    level="INFO",
    format="json"  # ← Now supported
)

log = get_logger(__name__)
log.info("Application started")

Output:

{"timestamp": "2025-01-11T10:45:00.123Z", "level": "info", "logger": "__main__", "message": "Application started"}

6.3 Production Setup (Complete)¶

# config/observability.py
from structum_lab.logging import configure_logging
from structum_lab.monitoring import configure_metrics
import os

def setup_observability():
    """Setup production-grade observability."""
    
    # Environment
    env = os.getenv("ENV", "development")
    log_level = os.getenv("LOG_LEVEL", "INFO")
    
    # Logging
    configure_logging(
        level=log_level,
        format="json" if env == "production" else "console",
        processors=[
            "timestamp",
            "add_log_level",
            "add_logger_name",
            "contextvars",
            "stack_info",
            "exception_formatter",
            "json_renderer" if env == "production" else "console_renderer"
        ],
        context_vars=["request_id", "user", "tenant_id"]
    )
    
    # Metrics (if plugin available)
    try:
        configure_metrics(
            namespace="myapp",
            enable_default_metrics=True,  # Process/Runtime metrics
            histogram_buckets=[0.001, 0.01, 0.1, 0.5, 1.0, 5.0, 10.0]
        )
    except ImportError:
        print("⚠️ Metrics plugin not available (No-Op mode)")

# main.py
from config.observability import setup_observability

if __name__ == "__main__":
    setup_observability()
    
    # App code
    from structum_lab.logging import get_logger
    log = get_logger(__name__)
    log.info("Application initialized")

7. Web Framework Integration¶

7.1 Flask¶

# app.py
from flask import Flask, request, g
from structum_lab.logging import get_logger, set_context
from structum_lab.monitoring import get_metrics
import uuid
import time

app = Flask(__name__)
log = get_logger(__name__)
metrics = get_metrics("myapp.api")

@app.before_request
def before_request():
    """Inject context per-request."""
    # Generate request ID
    request_id = request.headers.get("X-Request-ID", str(uuid.uuid4()))
    
    # Set context (propagates in all logs)
    set_context(
        request_id=request_id,
        method=request.method,
        path=request.path
    )
    
    # Save timestamp for latency
    g.start_time = time.perf_counter()
    
    log.info("Request started")

@app.after_request
def after_request(response):
    """Emit metrics post-request."""
    # Latency
    duration = time.perf_counter() - g.start_time
    
    metrics.timing(
        "request.duration",
        duration,
        tags={
            "method": request.method,
            "endpoint": request.endpoint or "unknown",
            "status": response.status_code
        }
    )
    
    # Counter
    metrics.increment(
        "requests.total",
        tags={
            "method": request.method,
            "status": response.status_code
        }
    )
    
    log.info("Request completed", status=response.status_code, duration=duration)
    
    return response

@app.route("/users/<int:user_id>")
def get_user(user_id):
    log.info("Fetching user", user_id=user_id)
    # ... business logic ...
    return {"user_id": user_id, "name": "Alice"}

@app.route("/metrics")
def metrics_endpoint():
    """Expose Prometheus metrics."""
    from prometheus_client import generate_latest, CONTENT_TYPE_LATEST
    from flask import Response
    
    return Response(generate_latest(), mimetype=CONTENT_TYPE_LATEST)

if __name__ == "__main__":
    from config.observability import setup_observability
    setup_observability()
    
    app.run()

7.2 FastAPI¶

# app.py
from fastapi import FastAPI, Request
from structum_lab.logging import get_logger, set_context, clear_context
from structum_lab.monitoring import get_metrics
import uuid
import time

app = FastAPI()
log = get_logger(__name__)
metrics = get_metrics("myapp.api")

@app.middleware("http")
async def observability_middleware(request: Request, call_next):
    """Inject context + metrics per request."""
    # Setup context
    request_id = request.headers.get("x-request-id", str(uuid.uuid4()))
    set_context(
        request_id=request_id,
        method=request.method,
        path=request.url.path
    )
    
    log.info("Request started")
    
    # Track timing
    start = time.perf_counter()
    
    try:
        response = await call_next(request)
        duration = time.perf_counter() - start
        
        # Metrics
        metrics.timing(
            "request.duration",
            duration,
            tags={
                "method": request.method,
                "path": request.url.path,
                "status": response.status_code
            }
        )
        
        metrics.increment(
            "requests.total",
            tags={"method": request.method, "status": response.status_code}
        )
        
        log.info("Request completed", status=response.status_code, duration=duration)
        
        return response
        
    finally:
        # Cleanup context
        clear_context()

@app.get("/users/{user_id}")
async def get_user(user_id: int):
    log.info("Fetching user", user_id=user_id)
    return {"user_id": user_id, "name": "Bob"}

@app.get("/metrics")
async def metrics_endpoint():
    """Prometheus metrics endpoint."""
    from prometheus_client import generate_latest, CONTENT_TYPE_LATEST
    from fastapi.responses import Response
    
    return Response(content=generate_latest(), media_type=CONTENT_TYPE_LATEST)

if __name__ == "__main__":
    import uvicorn
    from config.observability import setup_observability
    
    setup_observability()
    uvicorn.run(app, host="0.0.0.0", port=8000)

7.3 Django (Middleware)¶

# middleware.py
from structum_lab.logging import set_context, clear_context, get_logger
from structum_lab.monitoring import get_metrics
import uuid
import time

log = get_logger(__name__)
metrics = get_metrics("myapp.api")

class ObservabilityMiddleware:
    """Django middleware for logging + metrics."""
    
    def __init__(self, get_response):
        self.get_response = get_response
    
    def __call__(self, request):
        # Setup context
        request_id = request.META.get("HTTP_X_REQUEST_ID", str(uuid.uuid4()))
        set_context(
            request_id=request_id,
            method=request.method,
            path=request.path
        )
        
        log.info("Request started")
        start = time.perf_counter()
        
        try:
            response = self.get_response(request)
            duration = time.perf_counter() - start
            
            # Metrics
            metrics.timing(
                "request.duration",
                duration,
                tags={
                    "method": request.method,
                    "status": response.status_code
                }
            )
            
            metrics.increment(
                "requests.total",
                tags={"method": request.method, "status": response.status_code}
            )
            
            log.info("Request completed", status=response.status_code, duration=duration)
            
            return response
            
        finally:
            clear_context()

# settings.py
MIDDLEWARE = [
    # ... other middleware ...
    "myapp.middleware.ObservabilityMiddleware",
]

8. Best Practices¶

8.1 Logging Best Practices¶

✅ Structured Data over String Formatting:

# ❌ String formatting (loses structure)
log.info(f"User {user_id} logged in from {ip}")

# ✅ Structured kwargs
log.info("User logged in", user_id=user_id, ip_address=ip)

✅ Log Levels:

# DEBUG: Dev info, verbose
log.debug("Cache lookup", key="user:123", found=True)

# INFO: Significant business events
log.info("Order placed", order_id=456, user_id=123)

# WARNING: Anomalous but handled situations
log.warning("Rate limit approached", user_id=789, current_rate=95)

# ERROR: Errors requiring attention
log.error("Payment failed", order_id=456, error="Insufficient funds")

# CRITICAL: Catastrophic failures
log.critical("Database connection lost", attempts=3)

✅ Exception Logging:

try:
    risky_operation()
except Exception as e:
    log.error("Operation failed", exc_info=True)  # Includes traceback
    # In Enterprise mode, exception is formatted automatically

❌ Avoid Log Spam:

# ❌ Log in loop
for item in items:
    log.info("Processing item", item_id=item.id)  # Too verbose!

# ✅ Aggregated log
log.info("Processing batch", item_count=len(items))
# ... process ...
log.info("Batch completed", processed=len(items), failed=failed_count)

8.2 Metrics Best Practices¶

✅ Counter for Events:

# ✅ Use counter for event counts
metrics.increment("orders.placed")
metrics.increment("emails.sent", tags={"type": "welcome"})

✅ Gauge for States:

# ✅ Use gauge for instant values
metrics.gauge("queue.depth", current_queue_size)
metrics.gauge("connections.active", active_connections)

✅ Histogram for Distributions:

# ✅ Use histogram for latency/size
metrics.histogram("request.duration", duration_seconds)
metrics.histogram("response.size_bytes", response_size)

❌ Do Not Abuse Tags:

# ❌ High cardinality (millions of time series)
metrics.increment("views", tags={"user_id": user_id, "page_url": url})

# ✅ Low cardinality (controlled)
metrics.increment("views", tags={"page_type": "article", "category": "tech"})

8.3 Context Best Practices¶

✅ Set Context at Entry:

# ✅ Setup context as early as possible
@app.before_request
def setup_context():
    set_context(request_id=generate_id(), user=get_current_user())

✅ Clear Context at Exit:

# ✅ Cleanup to avoid leaks
@app.after_request
def cleanup_context(response):
    clear_context()
    return response

❌ Do Not Mutate Context Midway:

# ❌ Confusion: context changes mid-request
def process():
    log.info("Start")  # user="alice"
    set_context(user="bob")
    log.info("End")  # user="bob" ← Inconsistent!

# ✅ Use nested context if needed
with bind_context(subprocess="worker"):
    log.info("Subprocess work")  # user="alice", subprocess="worker"

9. Troubleshooting¶

9.1 “Logging not working”¶

Symptom: log.info() prints nothing.

Checklist:

Log level too high?

configure_logging(level="DEBUG")  # Lower the level

Handler configured?

import logging
logging.basicConfig()  # Fallback stdlib

Stdout buffering?

python -u app.py  # Unbuffered output
# or
export PYTHONUNBUFFERED=1

9.2 “kwargs not appearing in logs”¶

Cause: You are in Fallback mode, kwargs end up in extra.

Verify mode:

from structum_lab.logging import get_logger_backend
print(get_logger_backend())  # "stdlib" or "structlog"?

Fix: Install enterprise plugin:

pip install structum-observability

9.3 “Metrics not being collected”¶

Symptom: /metrics endpoint empty or missing metrics.

Checklist:

Backend active?

from structum_lab.monitoring import get_metrics_backend
print(get_metrics_backend())  # "noop" or "prometheus"?

Correct namespace?

metrics = get_metrics("myapp")
metrics.increment("test")

# Look in Prometheus for: myapp_test

Shared registry?

from prometheus_client import REGISTRY
print(list(REGISTRY._collector_to_names.keys()))  # Shows collectors

9.4 “Context not propagating”¶

Cause: Threading without copy_context().

Fix:

from contextvars import copy_context
import threading

def worker():
    log.info("Worker")  # Context present

set_context(request_id="123")

ctx = copy_context()
thread = threading.Thread(target=ctx.run, args=(worker,))
thread.start()

9.5 “Plugin does not activate”¶

Symptom: structum-observability installed but still using Fallback.

Cause: Import happened before installation.

Fix: Restart python process:

# ❌ Not enough to reinstall
pip install structum-observability
python app.py  # Still Fallback

# ✅ Restart needed
pip install structum-observability
# Stop Python
# Restart
python app.py  # Now uses Enterprise mode

10. API Reference¶

10.1 Logging API¶

# Get logger
from structum_lab.logging import get_logger
log = get_logger(name: str) -> LoggerInterface

# Configure
from structum_lab.logging import configure_logging
configure_logging(
    level: str = "INFO",
    format: str = "text",  # "text", "json", "console"
    processors: list[str] | None = None,
    handlers: list[str] = ["console"],
    capture_warnings: bool = True,
    context_vars: list[str] | None = None
) -> None

# Context management
from structum_lab.logging import (
    set_context,
    get_context,
    clear_context,
    bind_context,
    with_context
)

set_context(**kwargs) -> None
get_context() -> dict[str, Any]
clear_context(key: str | None = None) -> None

@bind_context(**kwargs)  # Context manager
@with_context(**kwargs)  # Decorator

# Backend detection
from structum_lab.logging import get_logger_backend
get_logger_backend() -> str  # "stdlib" | "structlog"

10.2 Metrics API¶

# Get metrics emitter
from structum_lab.monitoring import get_metrics
metrics = get_metrics(namespace: str) -> MetricsInterface

# Configure
from structum_lab.monitoring import configure_metrics
configure_metrics(
    namespace: str,
    enable_default_metrics: bool = True,
    histogram_buckets: list[float] | None = None,
    registry: Any | None = None  # Prometheus Registry
) -> None

# Backend detection
from structum_lab.monitoring import get_metrics_backend
get_metrics_backend() -> str  # "noop" | "prometheus"

10.3 Plugin API¶

# Decorator
from structum_lab.plugins.observability import track_operation

@track_operation(
    operation_name: str,
    include_args: list[str] | None = None,
    include_result: bool = False,
    metric_tags: dict[str, str] | None = None
)

Changelog¶

v0.1.0 (Alpha)¶

📝 Unified documentation (logging + metrics + context)
✨ Complete examples for Flask/FastAPI/Django
🐛 Clarified mode detection and plugin activation
📊 Best practices and troubleshooting guide

References¶

Maintainer: Structum Observability Team
Repository: https://github.com/PythonWoods/structum
Issues: https://github.com/PythonWoods/structum/issues

Structum Observability Stack¶

Index¶

1. Architectural Overview¶

1.1 Dual-Mode Design¶

1.2 Mode Detection (Automatic)¶

2. Logging System¶

2.1 Unified API¶

2.2 Fallback Mode (Default)¶

2.3 Enterprise Mode (Plugin)¶

2.4 Detailed Comparison¶

2.5 Logger Naming Convention¶

2.6 Verifying Active Mode¶

3. Metrics System¶

3.1 Unified API¶

3.2 No-Op Mode (Default)¶

3.3 Prometheus Mode (Plugin)¶

3.4 Built-in Structum Metrics¶

Configuration Provider (structum.config)¶

3.5 Best Practices¶

4. Context Propagation¶

4.1 What is Context?¶

4.2 Implementation (contextvars)¶

4.3 Setting Context¶

4.4 Reading Context¶

4.5 Clearing Context¶

4.6 Thread Safety¶

5. Plugin Observability (Enterprise)¶

5.1 Installation¶

5.2 Automatic Activation¶

5.3 Enterprise Configuration¶

5.4 Decorator: track_operation¶

5.5 Tracing Support (Future)¶

6. Setup Guide¶

6.1 Quick Start (Fallback Mode)¶

6.2 Upgrade to Enterprise Mode¶

6.3 Production Setup (Complete)¶

7. Web Framework Integration¶

7.1 Flask¶

7.2 FastAPI¶

7.3 Django (Middleware)¶

8. Best Practices¶

8.1 Logging Best Practices¶

8.2 Metrics Best Practices¶

8.3 Context Best Practices¶

9. Troubleshooting¶

9.1 “Logging not working”¶

9.2 “kwargs not appearing in logs”¶

9.3 “Metrics not being collected”¶

9.4 “Context not propagating”¶

9.5 “Plugin does not activate”¶

10. API Reference¶

10.1 Logging API¶

10.2 Metrics API¶

10.3 Plugin API¶

Changelog¶

v0.1.0 (Alpha)¶

References¶

Configuration Provider (`structum.config`)¶

4.2 Implementation (`contextvars`)¶

5.4 Decorator: `track_operation`¶