Skip to content

Configuration Storage

Referenced Files in This Document

Table of Contents

  1. Introduction

  2. Project Structure

  3. Core Components

  4. Architecture Overview

  5. Detailed Component Analysis

  6. Dependency Analysis

  7. Performance Considerations

  8. Troubleshooting Guide

  9. Conclusion

  10. Appendices

Introduction

This document explains the configuration storage and loading system used by Nexo. It covers how settings are sourced from environment variables, configuration files, and runtime overrides; how they are validated using Pydantic models; and how they are applied across services, database layers, and UI components. It also documents security practices for sensitive values, migration strategies, validation rules, and debugging techniques.

Project Structure

Configuration-related code is primarily located under:

  • src/config.py: Core configuration model and loader (Pydantic-based).

  • apps/service/shared/config.py: Service-layer configuration model and helpers.

  • apps/service/routes/config.py: REST endpoints to read/update configuration via the API.

  • apps/dashboard/src/pages/Config.tsx: Dashboard UI page that reads/writes configuration through the API.

  • apps/service/app.py, apps/service/deps.py, apps/service/main.py: Application bootstrap and dependency injection wiring for configuration.

  • alembic/env.py: Database migrations integration with configuration.

  • Legacy or alternate entry points (e.g., src/server/video_api.py, src/ui/dashboard_web.py) may import configuration depending on deployment mode.

graph TB

A["Environment Variables"] --> C["Config Loader<br/>src/config.py"]

B["Config File(s)<br/>.env / config.yaml / JSON"] --> C

C --> D["Pydantic Models<br/>Validation & Defaults"]

D --> E["Service Layer Config<br/>apps/service/shared/config.py"]

E --> F["API Routes<br/>apps/service/routes/config.py"]

E --> G["DB Migrations<br/>alembic/env.py"]

E --> H["App Bootstrap<br/>apps/service/app.py / deps.py / main.py"]

I["Dashboard UI<br/>apps/dashboard/src/pages/Config.tsx"] --> F

Diagram sources

Section sources

Core Components

  • Configuration Model and Loader: Centralized Pydantic models define typed settings, defaults, and validation rules. The loader merges environment variables, file-based settings, and runtime overrides into a single configuration object.

  • Service Configuration Module: Exposes strongly-typed accessors for service-level features such as database connections, API endpoints, feature flags, and user preferences.

  • API Configuration Endpoints: Provide secure read/write operations for configuration management, typically guarded by authentication and authorization.

  • UI Configuration Page: Renders editable fields bound to the API endpoints for non-technical users.

  • Dependency Injection: Services receive configuration instances via FastAPI dependencies to ensure consistent access throughout request handling.

  • Migration Integration: Alembic reads database connection details from configuration to apply schema changes safely.

Key responsibilities:

  • Source precedence: Environment variables override file-based settings, which override defaults. Runtime overrides take highest precedence within a process.

  • Validation: Pydantic enforces types, ranges, enums, and custom validators at load time.

  • Security: Sensitive values are loaded from environment variables or encrypted stores and never persisted in plaintext unless explicitly allowed.

Section sources

Architecture Overview

The configuration architecture follows a layered approach:

  • Sources: Environment variables, configuration files, and runtime overrides.

  • Validation: Pydantic models enforce schema correctness and provide default values.

  • Distribution: Configuration objects are injected into services and routes.

  • Persistence: Some settings are stored in persistent storage (e.g., database or encrypted files) and exposed via API.

  • UI: Dashboard allows editing configuration through a web interface backed by the API.

sequenceDiagram

participant Env as "Environment"

participant File as "Config Files"

participant Loader as "Config Loader<br/>src/config.py"

participant Model as "Pydantic Models"

participant Service as "Service Layer<br/>shared/config.py"

participant API as "Routes<br/>routes/config.py"

participant UI as "Dashboard UI<br/>Config.tsx"

Env->>Loader : Provide env vars

File->>Loader : Provide file settings

Loader->>Model : Merge and validate

Model-->>Service : Typed config instance

UI->>API : GET/PUT /api/config

API->>Service : Read/Update settings

Service-->>API : Validated response

API-->>UI : JSON payload

Diagram sources

Detailed Component Analysis

Configuration Model and Loader (src/config.py)

  • Purpose: Define the canonical configuration schema and implement the loader that merges multiple sources.

  • Key behaviors:

  • Defines Pydantic models for each configuration category (database, API endpoints, feature flags, user preferences).

  • Implements default values and validation rules (types, constraints, enums).

  • Loads environment variables and maps them to model fields.

  • Supports optional file-based configuration (YAML/JSON) with fallbacks.

  • Applies runtime overrides when provided programmatically.

  • Error handling:

  • Raises validation errors early if required fields are missing or invalid.

  • Provides clear error messages indicating the failing field and rule.

flowchart TD

Start(["Load Config"]) --> LoadEnv["Read Environment Variables"]

LoadEnv --> LoadFile["Read Config File(s)"]

LoadFile --> Merge["Merge Sources<br/>Defaults < File < Env < Runtime"]

Merge --> Validate["Pydantic Validation"]

Validate --> Valid{"Valid?"}

Valid --> |No| RaiseError["Raise ValidationError"]

Valid --> |Yes| ReturnCfg["Return Typed Config Object"]

Diagram sources

Section sources

Service Configuration Module (apps/service/shared/config.py)

  • Purpose: Provide service-specific configuration accessors and helpers.

  • Key behaviors:

  • Re-exports or wraps core configuration for service modules.

  • Adds convenience methods for common tasks (e.g., building database URLs, checking feature flags).

  • Ensures consistent usage across routes and background jobs.

Section sources

API Configuration Endpoints (apps/service/routes/config.py)

  • Purpose: Expose configuration management over HTTP.

  • Key behaviors:

  • GET /api/config: Returns current configuration (with sensitive fields masked).

  • PUT /api/config: Updates configuration after validation and authorization checks.

  • PATCH /api/config/{key}: Partial updates for specific keys.

  • Security:

  • Requires authentication and appropriate roles.

  • Masks secrets in responses.

  • Validates all inputs against Pydantic schemas before persistence.

sequenceDiagram

participant Client as "Client"

participant Auth as "Auth Middleware"

participant Route as "config route"

participant Svc as "Shared Config"

participant Store as "Persistent Store"

Client->>Route : GET /api/config

Route->>Auth : Verify token/role

Auth-->>Route : Authorized

Route->>Svc : Read config

Svc-->>Route : Config (secrets masked)

Route-->>Client : JSON response

Client->>Route : PUT /api/config {settings}

Route->>Auth : Verify token/role

Auth-->>Route : Authorized

Route->>Svc : Validate settings

Svc-->>Route : Validated settings

Route->>Store : Persist updated config

Store-->>Route : OK

Route-->>Client : Updated config (masked)

Diagram sources

Section sources

Dashboard Configuration UI (apps/dashboard/src/pages/Config.tsx)

  • Purpose: Provide a user-friendly interface to view and edit configuration.

  • Key behaviors:

  • Fetches current configuration via API.

  • Presents editable fields grouped by categories.

  • Submits updates back to the API with client-side validation hints.

  • UX considerations:

  • Masks sensitive fields in display.

  • Shows validation errors returned by the server.

Section sources

App Bootstrap and Dependency Injection (apps/service/app.py, apps/service/deps.py, apps/service/main.py)

  • Purpose: Initialize application and inject configuration into routes and services.

  • Key behaviors:

  • app.py sets up FastAPI app, middleware, and lifespan events.

  • deps.py defines dependency providers that return configuration instances.

  • main.py wires startup tasks and ensures configuration is available globally.

Section sources

Database Migrations Integration (alembic/env.py)

  • Purpose: Use configuration to connect to the database during migrations.

  • Key behaviors:

  • Reads database URL and options from configuration.

  • Applies Alembic revisions consistently with runtime settings.

Section sources

Legacy Entry Points (Optional)

  • src/server/video_api.py and src/ui/dashboard_web.py may import configuration depending on deployment mode. Ensure these paths use the same configuration loader to avoid inconsistencies.

Section sources

Dependency Analysis

Configuration flows from loader to services and routes, with UI interacting via API.

graph LR

Loader["src/config.py"] --> Shared["apps/service/shared/config.py"]

Shared --> Routes["apps/service/routes/config.py"]

Shared --> Deps["apps/service/deps.py"]

Deps --> App["apps/service/app.py"]

App --> Main["apps/service/main.py"]

UI["apps/dashboard/src/pages/Config.tsx"] --> Routes

Shared --> Alembic["alembic/env.py"]

Diagram sources

Section sources

Performance Considerations

  • Minimize repeated parsing: Load configuration once at startup and reuse the typed object.

  • Avoid heavy file I/O: Prefer environment variables for critical settings in production.

  • Cache derived values: Compute expensive transformations (e.g., URL builders) once and reuse.

  • Lazy initialization: Defer resource creation until first use to reduce startup time.

[No sources needed since this section provides general guidance]

Troubleshooting Guide

Common issues and resolutions:

  • Missing required fields: Check environment variables and config files; review validation error messages for exact field names.

  • Type mismatches: Ensure environment variable values match expected types (strings vs integers vs booleans).

  • Secret exposure: Confirm sensitive fields are masked in API responses and not logged.

  • Migration failures: Verify database connection settings in configuration and test connectivity before running migrations.

  • Inconsistent behavior across processes: Ensure all entry points use the same configuration loader and precedence rules.

Debugging tips:

  • Log configuration source precedence at startup to verify overrides.

  • Add endpoint health checks that report configuration validity without exposing secrets.

  • Use structured logging to capture validation errors with context.

Section sources

Conclusion

Nexo’s configuration system combines robust Pydantic-based validation with flexible sourcing from environment variables, files, and runtime overrides. By centralizing configuration in typed models and distributing it via dependency injection, the application maintains consistency, security, and clarity. The API and UI enable safe management of settings, while Alembic integrates seamlessly for database migrations. Following the documented practices ensures reliable operation across environments and simplifies troubleshooting.

[No sources needed since this section summarizes without analyzing specific files]

Appendices

Configuration Categories

  • Database Connections: Host, port, credentials, pool sizes, SSL options.

  • API Endpoints: Base URLs, timeouts, retry policies, rate limits.

  • Feature Flags: Enable/disable experimental features per environment.

  • User Preferences: UI themes, language, notification toggles.

[No sources needed since this section provides general guidance]

Security Practices

  • Sensitive Values: Load from environment variables or encrypted stores; never hardcode.

  • Masking: Hide secrets in API responses and logs.

  • Access Control: Require authentication and role checks for configuration writes.

  • Encryption: Consider encrypting stored secrets at rest if persisted.

[No sources needed since this section provides general guidance]

Common Scenarios and Defaults

  • Local Development: Minimal defaults, local database, debug flags enabled.

  • Staging: Strict validation, external APIs with sandbox endpoints.

  • Production: Strong defaults, secrets via environment, reduced verbosity.

[No sources needed since this section provides general guidance]

Migration Strategies

  • Backward Compatibility: Keep deprecated fields with warnings.

  • Schema Evolution: Use Alembic to migrate database schema alongside configuration changes.

  • Rollback Plan: Maintain reversible migrations and configuration versioning.

[No sources needed since this section provides general guidance]