Content Matching¶

Referenced Files in This Document

Table of Contents¶

Introduction
Project Structure
Core Components
Architecture Overview
Detailed Component Analysis
Dependency Analysis
Performance Considerations
Troubleshooting Guide
Conclusion
Appendices

Introduction¶

This document explains Nexo’s content matching system: how it extracts features from media, computes similarity across different types (images, documents), and ranks results to detect duplicates or near-duplicates. It covers matcher configuration, thresholds, performance tuning, data structures, indexing strategies, memory management for batch operations, and troubleshooting guidance for false positives/negatives and bottlenecks.

Project Structure¶

The matching subsystem is primarily implemented under the core package and exposed via service routes and use cases. The key files are:

Core matching logic and types
Feature extraction for images and documents
Pipeline orchestration and decision handling
Configuration and API exposure

graph TB

subgraph "Core"

M["matcher.py"]

T["types.py"]

P["pipeline.py"]

C["confidence.py"]

D["decision.py"]

U["usecases.py"]

IP["image_parser.py"]

DP["document_parser.py"]

end

subgraph "Service"

A["app.py"]

R["routes/tools.py"]

end

subgraph "Config"

CFG["shared/config.py"]

end

subgraph "Tests"

TM["test_matcher.py"]

end

A --> R

R --> U

U --> M

M --> T

M --> IP

M --> DP

U --> P

P --> D

P --> C

CFG --> A

TM --> M

Diagram sources

Section sources

Core Components¶

Matcher: Implements similarity computation and ranking across feature vectors. Supports multiple backends and pluggable algorithms.
Types: Defines shared data models for features, matches, and scoring metadata.
Parsers: Extract features from images and documents into normalized representations suitable for comparison.
Pipeline: Orchestrates scanning, parsing, matching, confidence calculation, and decision recording.
Confidence: Aggregates scores and applies thresholding to produce actionable decisions.
Use Cases: Expose high-level operations for matching workflows.
Service Routes: Provide HTTP endpoints to trigger matching and retrieve results.

Key responsibilities:

Feature extraction: Convert raw media into comparable vectors or signatures.
Similarity computation: Apply distance/similarity metrics per type.
Thresholding and ranking: Filter and order candidate matches by relevance.
Decision integration: Persist outcomes and audit trails.

Section sources

Architecture Overview¶

The matching workflow spans extraction, comparison, scoring, and decisioning.

sequenceDiagram

participant Client as "Client"

participant API as "tools.py"

participant UC as "usecases.py"

participant Match as "matcher.py"

participant Img as "image_parser.py"

participant Doc as "document_parser.py"

participant Pipe as "pipeline.py"

participant Conf as "confidence.py"

participant Dec as "decision.py"

Client->>API : "POST /match"

API->>UC : "invoke match operation"

UC->>Pipe : "prepare pipeline context"

Pipe->>Img : "extract image features"

Pipe->>Doc : "extract document features"

Pipe-->>UC : "feature set"

UC->>Match : "compute similarities"

Match-->>UC : "ranked candidates"

UC->>Conf : "apply thresholds and aggregate"

Conf-->>UC : "finalized scores"

UC->>Dec : "record decisions"

Dec-->>API : "results"

API-->>Client : "JSON response"

Diagram sources

Detailed Component Analysis¶

Matcher Engine¶

The matcher implements the core similarity algorithms and ranking logic. It supports:

Multiple similarity metrics (e.g., cosine similarity, Jaccard, Euclidean) depending on feature type.
Pluggable backends for vector comparisons.
Configurable thresholds and top-k selection.
Batch processing with streaming-friendly patterns.

Typical responsibilities:

Normalize inputs and validate feature shapes.
Compute pairwise or query-vs-index similarities.
Rank candidates and apply filters based on thresholds.
Return structured match results with metadata.

classDiagram

class Matcher {

+configure(options)

+compute(query_features, index_features) MatchResult[]

+batch_compute(batch) BatchResult[]

-normalize(features)

-select_metric(type)

-apply_thresholds(scores, thresholds)

}

class ImageMatcher {

+compute(query_img, index_imgs) MatchResult[]

}

class DocumentMatcher {

+compute(query_doc, index_docs) MatchResult[]

}

Matcher <|-- ImageMatcher : "extends"

Matcher <|-- DocumentMatcher : "extends"

Diagram sources

matcher.py

Section sources

matcher.py

Data Types and Structures¶

Shared types define the contracts between components:

Feature objects: Encapsulate extracted representations (vectors, hashes, signatures).
MatchResult: Contains target identifiers, similarity score, metric used, and optional metadata.
BatchResult: Aggregates multiple MatchResult entries with summary statistics.

These types ensure consistent serialization and interoperability across parsers, matcher, and decision modules.

Section sources

types.py

Feature Extraction: Images and Documents¶

Image parser: Converts images into compact descriptors (e.g., perceptual hashes, color histograms, embeddings). Normalization and resizing may be applied to improve robustness.
Document parser: Extracts textual or structural features (e.g., token frequencies, TF-IDF-like vectors, semantic embeddings) for comparison.

Extraction outputs conform to the shared types consumed by the matcher.

Section sources

Pipeline Orchestration and Decisions¶

The pipeline coordinates scanning, parsing, matching, confidence aggregation, and decision persistence:

Prepares contexts and batches items efficiently.
Invokes extractors and matcher.
Applies confidence rules and thresholds.
Records decisions and logs outcomes.

flowchart TD

Start(["Start"]) --> Scan["Scan Items"]

Scan --> Parse["Extract Features"]

Parse --> Match["Compute Similarities"]

Match --> Rank["Rank Candidates"]

Rank --> Threshold{"Above Threshold?"}

Threshold --> |Yes| Decide["Record Decision"]

Threshold --> |No| Skip["Skip or Log"]

Decide --> End(["End"])

Skip --> End

Diagram sources

Section sources

Service Exposure and Configuration¶

API route exposes a matching endpoint that accepts payloads and returns ranked matches.
Use cases wrap the pipeline and matcher calls, providing a clean interface for callers.
Configuration centralizes matcher options, thresholds, and performance parameters.

sequenceDiagram

participant Client as "Client"

participant Tools as "tools.py"

participant UC as "usecases.py"

participant Pipe as "pipeline.py"

participant Match as "matcher.py"

participant Conf as "confidence.py"

participant Dec as "decision.py"

Client->>Tools : "POST /match"

Tools->>UC : "match(items, config)"

UC->>Pipe : "run_pipeline(items)"

Pipe->>Match : "compute_similarity()"

Match-->>Pipe : "candidates"

Pipe->>Conf : "aggregate_scores()"

Conf-->>Pipe : "final_scores"

Pipe->>Dec : "persist_decisions()"

Dec-->>Tools : "results"

Tools-->>Client : "JSON"

Diagram sources

Section sources

Dependency Analysis¶

The matching subsystem has clear layering:

Service routes depend on use cases.
Use cases depend on pipeline and matcher.
Matcher depends on feature types and parsers.
Pipeline depends on confidence and decision modules.

graph LR

Tools["tools.py"] --> UC["usecases.py"]

UC --> Pipe["pipeline.py"]

Pipe --> Match["matcher.py"]

Match --> Types["types.py"]

Match --> Img["image_parser.py"]

Match --> Doc["document_parser.py"]

Pipe --> Conf["confidence.py"]

Pipe --> Dec["decision.py"]

App["app.py"] --> Tools

Config["config.py"] --> App

Diagram sources

Section sources

Performance Considerations¶

Indexing strategies:
For vector-based features, consider approximate nearest neighbor indexes to reduce O(n) scans.
For hash-based features, use inverted indices keyed by signature segments.
Batch sizing:
Tune batch sizes to balance throughput and memory usage; smaller batches reduce peak memory but increase overhead.
Parallelism:
Use process pools or async I/O for feature extraction and similarity computations where appropriate.
Memory management:
Stream large datasets instead of loading all at once.
Reuse buffers and avoid unnecessary copies during normalization.
Metric selection:
Choose metrics aligned with feature semantics (cosine for embeddings, Jaccard for sets, etc.).
Threshold calibration:
Calibrate per-feature-type thresholds using validation sets to minimize false positives/negatives.

[No sources needed since this section provides general guidance]

Troubleshooting Guide¶

Common issues and remedies:

False positives:
Lower similarity thresholds or add secondary checks (e.g., temporal or metadata constraints).
Improve feature robustness (e.g., normalize lighting for images, tokenize consistently for documents).
False negatives:
Raise thresholds cautiously; introduce multi-metric fusion to capture broader similarity.
Enrich features with additional signals (e.g., audio fingerprints, OCR text).
Performance bottlenecks:
Profile CPU-bound steps (similarity computation) and GPU-accelerate if applicable.
Reduce dimensionality of embeddings or quantize vectors for faster comparisons.
Increase batch size gradually while monitoring memory pressure.
Reproducibility:
Ensure deterministic preprocessing and hashing seeds.
Validate input schemas against shared types before matching.

Section sources

Conclusion¶

Nexo’s content matching system combines modular feature extraction, configurable similarity metrics, and robust pipeline orchestration to detect duplicates and near-duplicates across media types. By tuning thresholds, selecting appropriate indexes, and managing batch/memory carefully, you can achieve accurate and scalable matching. The provided diagrams and references help you navigate implementation details and extend the system with custom matchers and metrics.

Appendices¶

Implementing a Custom Matcher¶

Steps:

Define your feature extractor to output types compatible with the shared schema.
Implement a matcher subclass that selects an appropriate similarity metric and applies thresholds.
Integrate via the pipeline by registering your matcher and configuring thresholds in the centralized configuration.
Add tests to validate behavior on representative samples.

Section sources

Configuring Similarity Thresholds¶

Set per-type thresholds in configuration to control sensitivity.
Use confidence aggregation to combine multiple signals before final decisions.
Monitor precision/recall trade-offs and adjust thresholds iteratively.

Section sources

Optimizing for Large Datasets¶

Precompute and cache stable features to avoid repeated extraction.
Employ approximate nearest neighbor libraries for vector spaces.
Shard indexes by domain or time windows to limit search scope.
Stream queries and results to keep memory footprint low.

[No sources needed since this section provides general guidance]