Testing Commands

The test command provides pytest integration with coverage reporting and threshold enforcement for comprehensive testing workflows.

Overview

Testing commands are provided by the structum-cli-tools plugin:

pip install structum-cli-tools

This adds the test command with full pytest and coverage.py integration.

Command

test

Run pytest with optional coverage reporting and threshold enforcement.

Arguments:

path

Test directory or file path (default: tests/)

Options:

--cov

Enable coverage measurement

--cov-report FORMAT

Coverage report format (default: term-missing)

Supported formats: - term-missing - Terminal with missing lines - html - HTML report in htmlcov/ - xml - XML report for CI/CD - json - JSON format

--cov-fail-under N

Minimum coverage percentage required (e.g., 90)

--verbose / -v

Verbose test output

--markers / -m EXPR

Run tests matching marker expression

--keyword / -k EXPR

Run tests matching keyword expression

--maxfail / -x N

Stop after N test failures

Examples:

Basic Usage

# Run all tests
structum test

# Run specific directory
structum test tests/unit/

# Run specific file
structum test tests/test_auth.py

Coverage Reporting

# Basic coverage
structum test --cov

# With HTML report
structum test --cov --cov-report=html
# Opens htmlcov/index.html

# Multiple formats
structum test --cov --cov-report=term-missing --cov-report=xml

# Enforce 90% minimum
structum test --cov --cov-fail-under=90

Selective Testing

# Run only unit tests (requires @pytest.mark.unit)
structum test -m unit

# Skip slow tests
structum test -m "not slow"

# Run tests containing "auth" in name
structum test -k auth

# Multiple keywords (OR)
structum test -k "auth or database"

Debugging

# Verbose output
structum test -v

# Stop at first failure
structum test --maxfail=1

# Combine options
structum test -v --maxfail=1 -k "test_login"

Configuration

Coverage Configuration (.coveragerc)

Create .coveragerc in project root:

[run]
source = packages/
omit =
    */tests/*
    */demo/*
    */__pycache__/*

[report]
precision = 2
show_missing = True
skip_covered = False

[html]
directory = htmlcov

pytest Configuration (pyproject.toml)

Configure pytest in pyproject.toml:

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]

# Markers
markers = [
    "unit: Unit tests",
    "integration: Integration tests",
    "slow: Tests that take > 1s",
]

# Coverage
addopts = [
    "--strict-markers",
    "--tb=short",
]

CI/CD Integration

GitHub Actions Example

name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.11", "3.12"]

    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install dependencies
        run: |
          pip install structum-cli-tools
          pip install -e .

      - name: Run tests with coverage
        run: structum test --cov --cov-fail-under=90 --cov-report=xml

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage.xml

Codecov Integration

Add to codecov.yml:

coverage:
  status:
    project:
      default:
        target: 90%
        threshold: 1%
    patch:
      default:
        target: 95%

Best Practices

Test Organization

tests/
├── unit/           # Fast, isolated tests
│   ├── test_auth.py
│   ├── test_config.py
│   └── test_database.py
├── integration/    # Multi-component tests
│   ├── test_api.py
│   └── test_workflow.py
├── fixtures/       # Shared test data
│   └── conftest.py
└── conftest.py     # Root fixtures

Marker Usage

import pytest

@pytest.mark.unit
def test_fast_operation():
    assert True

@pytest.mark.integration
@pytest.mark.slow
def test_database_workflow():
    # Integration test
    pass

# Run with:
# structum test -m unit          # Only unit tests
# structum test -m "not slow"    # Skip slow tests

Coverage Goals

  • Unit tests: Aim for 95%+ coverage

  • Integration tests: Focus on critical paths

  • Overall project: Minimum 90% recommended

  • New code: 100% coverage for new modules

Exit Codes

  • 0 - All tests passed (and coverage met if --cov-fail-under used)

  • 1 - Tests failed or coverage below threshold

Troubleshooting

Low Coverage

  1. Run with --cov-report=html to see missing lines

  2. Check htmlcov/index.html for visual report

  3. Add tests for uncovered branches

  4. Use # pragma: no cover ONLY for defensive code

Test Discovery Issues

Ensure: - Test files named test_*.py or *_test.py - Test functions start with test_ - Test classes start with Test

Import Errors

If tests can’t import modules:

# Install package in editable mode
pip install -e .

# Or update PYTHONPATH
export PYTHONPATH="$PWD/packages:$PYTHONPATH"

See Also