Core Features

Knowledge Graph

Copy page

Cortex automatically builds a knowledge graph from your documents using GraphRAG (Graph Retrieval-Augmented Generation). This enables semantic understanding, relationship discovery, and enhanced search capabilities.

What is GraphRAG?

GraphRAG uses Large Language Models to:

1. Extract Entities - Identify people, organizations, concepts, technologies, locations
2. Find Relationships - Discover how entities relate to each other
3. Resolve Semantics - Merge duplicate entities with different names
4. Build Structure - Create a traversable graph stored in Neo4j

Two-Phase Extraction Pipeline

Cortex uses a two-phase approach for building the knowledge graph:

Phase A (per-document, during upload): Entity extraction from each document using a dedicated extraction model (can be a smaller, faster model). Entities are fuzzy-matched to chunks for provenance tracking. Entity resolution deduplicates entities at storage time using embedding-based vector similarity (when ENABLE_SEMANTIC_ENTITY_RESOLUTION=true) to catch semantic matches like "Museum of Crypto Art" / "MOCA", with Levenshtein 85% as a fallback. This merges variants like "OpenAI" and "Open AI" into a single node with aliases. Entity types are strictly enforced to 10 allowed types (Person, Organization, Location, Concept, Technology, Event, Product, Document, System, Process) via fuzzy matching. Entity-extraction responses that hit the output-token cap are automatically split and retried, so entity-dense documents don't lose entities to truncation; a document summary is generated as extraction context only when a document spans multiple extraction batches. After entity extraction and chunk linking, per-chunk relationship extraction runs: chunks with 2+ linked entities are analyzed by the relationship model (falls back to extraction model, then primary model) to extract relationships using the chunk text as direct evidence — by default several chunks are packed into one LLM call (ENABLE_BATCHED_CHUNK_RELATIONSHIPS, RELATIONSHIP_CHUNKS_PER_CALL), with individual retries only for chunks missing from a response. Entity names in per-chunk relationships are automatically mapped to their canonical (dedup-resolved) names before storage, ensuring relationships reference the correct merged entities. Self-referential relationships (where source and target are the same entity) are automatically filtered out. Per-chunk extraction uses retry with exponential backoff (tenacity, 4 attempts, 2-30s wait) for rate limit errors, and concurrency is controlled by CONCURRENT_RELATIONS (default 3). These relationships are stored with extraction_method='per_chunk' — including the model's confidence score alongside description and weight — and provide high-confidence, evidence-grounded connections before Phase B runs.

Phase B (per-collection, on demand): Cross-document relationship analysis using the relationship model (falls back to extraction model, then primary model). RELATIONSHIP_DISCOVERY_MODE selects between two discovery engines:

Targeted mode (default, RELATIONSHIP_DISCOVERY_MODE=targeted): candidate entity pairs are generated without the LLM; the LLM only verifies and classifies them:

1. Embedding backfill — Entities missing an embedding are embedded and bulk-written first, so the Neo4j entity_embedding vector index covers the whole entity set (skipped when no embedding API key is configured — candidates then come from co-mention only).
2. Candidate generation (no LLM) — Two Neo4j-side generators: kNN over the vector index (RELATIONSHIP_KNN_K neighbors per entity, minimum similarity RELATIONSHIP_KNN_MIN_SIMILARITY, already-connected pairs excluded in-query), and document co-mention (unconnected pairs mentioned together in at least RELATIONSHIP_MIN_SHARED_DOCS documents; entities appearing in more than RELATIONSHIP_DOC_FREQ_CAP documents are skipped as hub anchors).
3. Ranking & caps — Pairs are scored (weighted blend of kNN similarity and shared-document count), deduplicated across directions and sources, then capped per entity (RELATIONSHIP_CANDIDATES_PER_ENTITY) and in total (RELATIONSHIP_MAX_CANDIDATE_PAIRS).
4. LLM verification — Ranked pairs are verified in small batched calls of RELATIONSHIP_PAIRS_PER_CALL (default 40) pairs, each with up to RELATIONSHIP_PAIR_CONTEXT_TOKENS of chunk context. The model returns structured XML relationships with confidence scores; the confidence < 0.5 filter and the RELATIONSHIP_MAX_PER_ENTITY cap apply as usual. Concurrency is controlled by PARALLEL_RELATIONSHIP_BATCHES.

Because the LLM only sees ranked pairs with actual signal, analysis time and token cost stay proportional to the candidate budget rather than to the full entity space — a substantial efficiency gain on large graphs. Targeted mode runs a single pass (no multi-round discovery); RELATIONSHIP_MAX_HOURS is still enforced between verification batches.

Legacy mode (RELATIONSHIP_DISCOVERY_MODE=llm_scan): the previous full-batch LLM scan, with a two-phase approach per batch:

1. Candidate scanning — The relationship model scans entity pairs with their co-occurring chunk context and proposes candidate relationships. Includes few-shot good and bad examples to guide the LLM, plus anti-hub negative instructions ("If no clear relationship exists, do not create one") with bad examples showing co-occurrence pairs to avoid.
2. Structured confirmation — The relationship model confirms candidates with structured XML output including confidence scores (0.0-1.0), grounding each relationship in source text. Relationships with confidence < 0.5 are filtered before storage. Self-referential relationships (where source and target are the same entity) are also automatically filtered out at both the extraction and storage levels.

In legacy mode, entities are grouped into batches using Union-Find co-occurrence clustering — entities that share chunks are grouped together, with high/low connection count interleaving to prevent hub entities from concentrating in early batches. Batch overlap is 5% with degree-aware selection (entities already in 2+ batches are excluded from overlap). Chunk context is filled dynamically per batch using a 60/40 token split (60% for chunk text, 40% for entity descriptions and output budget), with greedy entity-coverage-diversity selection that maximizes coverage of different entities rather than always picking chunks dominated by hub entities.

Legacy mode supports multi-round discovery: initial analysis runs multiple rounds (default 3, configurable via RELATIONSHIP_MAX_ROUNDS), "Find more" (re-analyze) always does 1 round. Progress is tracked cumulatively across all rounds (e.g., "Batch 400/861" for 3 rounds of 287 batches). Guided by a target ERR (Entity-Relationship Ratio) metric (default 1.0). RELATIONSHIP_TARGET_RATIO and RELATIONSHIP_MAX_ROUNDS only apply in legacy mode; the ERR itself is still displayed on the Knowledge Graph page as a quality indicator in both modes.

Anti-hub protections: A per-entity relationship cap (RELATIONSHIP_MAX_PER_ENTITY, default 50) prevents any single entity from accumulating disproportionate connections — relationships are skipped when both endpoints exceed the cap. Existing relationships shown to the LLM between rounds are capped at 20 per entity (highest weight first) to prevent hub reinforcement. Prompts emphasize direct, evidence-based relationships and explicitly discourage star patterns through common intermediary entities.

Both discovery engines support two run modes: incremental (builds on existing relationships) and rebuild (deletes only batch-analysis relationships, preserving per-chunk relationships from Step 1, then re-analyzes from scratch — with multi-round in legacy mode). Relationship types are constrained to 14 standard types via fuzzy matching (MENTIONS was removed as it was a lazy co-occurrence catch-all). A plaintext fallback parser handles arrow-format output (e.g., Entity1 -> RELATION -> Entity2) when XML parsing fails. Triggered via API or the Knowledge Graph page after document processing.

Generate Graph (One-Click 3-Step Pipeline)

The Knowledge Graph page (/extract) exposes a single Generate Graph / Regenerate Graph button that runs all three steps — entity extraction, cross-document relationship analysis, and community detection — server-side as a chained background flow.

Under the hood, the frontend issues one request:

Code
 
curl -X POST "http://localhost:8000/api/documents/reprocess?chain=relationship_analysis,community_detection" \
  -H "X-API-Key: your-api-key" \
  -d '{"document_ids": [...]}'

The chain query parameter (also accepted on /api/documents/process-pending and /api/graph/relationships/analyze) tells the backend to automatically spawn the next pipeline step when each task finishes. Each step still produces its own task with its own task_id, task_type, and progress messages — Step 1's task is held in running state until background image analysis also completes, so the chain only advances when image-derived entities have landed.

Why this matters: the full flow survives navigation away from the Knowledge Graph page, page reloads, and even closing the browser. The user can return at any time and the page re-attaches to whichever pipeline task is currently running. Plain single-step buttons ("Extract Entities", "Analyze Relationships", "Detect Communities") never auto-chain; only the Generate Graph flow sets the chain string.

The chain also survives server restarts and redeploys: every pipeline task persists its resume parameters (including the remaining chain) alongside its task record, and on startup Cortex resumes the interrupted step — Step 1 picks its pending documents back up and still continues into Steps 2 and 3, while an interrupted Step 2 or 3 restarts directly (quota-guarded, disable with AUTO_RESUME_PENDING_ON_STARTUP=false).

Entity Types

Cortex extracts various entity types:

Type	Examples
Person	Names, authors, researchers
Organization	Companies, institutions, teams
Concept	Ideas, theories, methodologies
Technology	Software, tools, frameworks
Location	Places, regions, countries
Event	Conferences, releases, meetings
Product	Products, services, offerings
Document	Papers, reports, articles
System	Platforms, infrastructure, OS
Process	Workflows, procedures, methods

Entity types are strictly enforced during extraction — non-standard types are automatically fuzzy-matched to the nearest allowed type (defaulting to Concept if no match above 75%).

Relationship Types

Common relationship types discovered:

Relationship	Description
RELATED_TO	Generic/fallback relationship
CREATED_BY	Creator/creation relationship
WORKS_FOR	Employment relationship
PART_OF	Containment/membership
USES	Utilization relationship
LOCATED_IN	Geographic relationship
IMPLEMENTS	Implementation relationship
DEPENDS_ON	Dependency relationship
IS_A	Classification relationship
HAS_PROPERTY	Attribute relationship
FOUNDED_BY	Founding relationship
FEATURES	Feature/capability relationship
CONTAINS	Containment relationship
INTERACTS_WITH	Interaction relationship

Relationship types are strictly enforced — the LLM is instructed to only use these 14 types, and any non-standard types in the output are fuzzy-matched to the nearest allowed type (80% threshold, fallback to RELATED_TO). The MENTIONS type was intentionally excluded as it was being used as a lazy catch-all for co-occurrence without meaningful semantic content.

Graph Visualization

API Usage

Get Graph Statistics

Code
 
curl "http://localhost:8000/api/stats" \
  -H "X-API-Key: your-api-key"

Code
 
{
  "document_count": 156,
  "chunk_count": 4280,
  "entity_count": 1542,
  "relationship_count": 3891,
  "per_chunk_relationship_count": 1204,
  "community_count": 23,
  "collection_count": 5
}

Get Graph Visualization Data

Code
 
curl "http://localhost:8000/api/graph/visualization" \
  -H "X-API-Key: your-api-key"

Code
 
{
  "nodes": [
    {"id": "ent_1", "label": "OpenAI", "type": "Organization"},
    {"id": "ent_2", "label": "GPT-4", "type": "Technology"}
  ],
  "edges": [
    {"source": "ent_1", "target": "ent_2", "type": "CREATED"}
  ]
}

Get Entity Details

Code
 
curl "http://localhost:8000/api/graph/entities/ent_abc123" \
  -H "X-API-Key: your-api-key"

Code
 
{
  "id": "ent_abc123",
  "name": "OpenAI",
  "type": "Organization",
  "description": "AI research company",
  "mention_count": 45,
  "related_documents": ["doc_1", "doc_2", "doc_3"],
  "relationships": [
    {"target": "GPT-4", "type": "CREATED"},
    {"target": "ChatGPT", "type": "CREATED"},
    {"target": "Sam Altman", "type": "LED_BY"}
  ]
}

Get Entity Relationships

Code
 
curl "http://localhost:8000/api/graph/entities/ent_abc123/relationships" \
  -H "X-API-Key: your-api-key"

Query Subgraph

Get a subgraph starting from a specific entity:

Code
 
curl -X POST "http://localhost:8000/api/graph/subgraph" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "entity_name": "Machine Learning",
    "max_depth": 2,
    "limit": 50
  }'

Search Entities

Code
 
curl "http://localhost:8000/api/graph/entities?search=neural&type=Concept&limit=20" \
  -H "X-API-Key: your-api-key"

Entity Editing

You can edit an entity's name or description directly from the Explore > Entities tab. Click any entity to open the detail modal, then click the pencil icon next to the name or description to edit inline.

• Name changes preserve the old name in the entity's aliases array, so searches for the old name still work
• Duplicate names are rejected — the new name must be unique
• Graph integrity is maintained — all relationships and chunk mentions remain intact because Neo4j edges connect to nodes, not name strings

Update Entity via API

Code
 
curl -X PATCH "http://localhost:8000/api/graph/entity/OpenAI" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "OpenAI, Inc.",
    "description": "AI research and deployment company"
  }'

Code
 
{
  "name": "OpenAI, Inc.",
  "type": "Organization",
  "description": "AI research and deployment company",
  "aliases": ["OpenAI"]
}

Semantic Entity Resolution

Cortex automatically merges entities that refer to the same thing:

When ENABLE_SEMANTIC_ENTITY_RESOLUTION=true (default), entity resolution uses embedding-based vector similarity via Neo4j's vector index to catch semantic matches that string similarity misses (e.g., "Museum of Crypto Art" and "MOCA"). Levenshtein string matching is used as a fallback.

This applies symmetrically to entities extracted from text and from image descriptions. The image-analysis pipeline now batch-embeds each image's extracted entities (one embeddings call per image, gracefully falling back to Levenshtein on failure) before storing them — so the same entity_embedding vector index covers both surfaces and cross-source duplicates (e.g. an "MOCA" entity from an image caption and a "Museum of Crypto Art" entity from text) collapse at write time.

Configuration:

Code
 
ENABLE_SEMANTIC_ENTITY_RESOLUTION=true
ENTITY_SIMILARITY_THRESHOLD=0.85

Entity Deduplication

While Cortex automatically resolves many duplicates during extraction (via embedding-based semantic matching with Levenshtein 85% fallback), some near-duplicates may still slip through -- especially across large document sets or when entity names vary in subtle ways (e.g., "Machine Learning" vs "machine learning (ML)"). The Entity Deduplication feature provides tools to find and merge these remaining duplicates.

Quick Access from Entities

You can jump directly to deduplication for any entity from the Explore > Entities browser. Each entity card has a merge icon button that navigates to /deduplicate?entity=EntityName, which auto-scans and filters results to that entity. The entity detail modal also has a Deduplicate button in the footer.

Find Duplicate Candidates

Scan the knowledge graph for groups of entities that appear to refer to the same thing:

Code
 
curl "http://localhost:8000/api/entities/duplicates?threshold=0.85&limit=50" \
  -H "X-API-Key: your-api-key"

Code
 
{
  "groups": [
    {
      "canonical": "Machine Learning",
      "duplicates": [
        {"name": "machine learning", "type": "Concept", "similarity": 0.95, "mention_count": 12},
        {"name": "ML", "type": "Concept", "similarity": 0.87, "mention_count": 5}
      ],
      "similarity": 0.91
    },
    {
      "canonical": "OpenAI",
      "duplicates": [
        {"name": "Open AI", "type": "Organization", "similarity": 0.92, "mention_count": 3}
      ],
      "similarity": 0.92
    }
  ],
  "total_groups": 2,
  "threshold": 0.85
}

The threshold parameter (0.5 to 1.0) controls how similar entity names must be to be considered duplicates. Lower values return more candidates but may include false positives.

On large graphs the scan can take a while: one scan runs at a time (identical requests join it), and if it outlasts the inline wait window the endpoint answers 202 {"status": "running", "progress": 0.42} — poll the same URL until it returns "status": "complete". Completed results are cached server-side (entity merges invalidate the cache); pass refresh=true to force a fresh scan. The web interface handles this polling automatically and shows scan progress.

Merge Entities

Once you have identified true duplicates, merge them into a single canonical entity:

Code
 
curl -X POST "http://localhost:8000/api/entities/merge" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "canonical": "Machine Learning",
    "merge": ["machine learning", "ML"]
  }'

Code
 
{
  "canonical": "Machine Learning",
  "merged": ["machine learning", "ML"],
  "relationships_transferred": 8,
  "mentions_transferred": 17,
  "aliases_added": ["machine learning", "ML"]
}

When entities are merged:

• All relationships from the merged entities are transferred to the canonical entity
• All chunk mentions (MENTIONS links) are moved to the canonical entity
• The merged entity names are added as aliases on the canonical entity for future resolution
• The merged entities are removed from the graph

View Merge History

Review past merge operations:

Code
 
curl "http://localhost:8000/api/entities/merge-history?limit=20" \
  -H "X-API-Key: your-api-key"

Code
 
{
  "entries": [
    {
      "canonical": "Machine Learning",
      "merged": ["machine learning", "ML"],
      "relationships_transferred": 8,
      "mentions_transferred": 17,
      "merged_at": "2026-03-18T14:30:00Z"
    }
  ],
  "total": 1
}

Deduplication Workflow

Tips for effective deduplication:

• Start with the default threshold (0.85) to find obvious duplicates first
• Lower the threshold gradually to catch more subtle variants
• Use the inspect button (eye icon) on each entity in a group to view its full details before deciding to merge
• Always review candidates before merging -- false positives at low thresholds can incorrectly combine unrelated entities
• After merging, re-run community detection to reflect the updated graph structure

Graph in Search

During search, Cortex uses the knowledge graph to:

1. Expand queries - Find related entities
2. Traverse relationships - Discover connected information
3. Add context - Include entity descriptions in prompts

Configure graph usage:

Code
 
MAX_GRAPH_HOPS=2
GRAPH_WEIGHT=0.2

Visualization

The Knowledge Graph page (under Manage) provides a one-click "Generate Graph" (or "Regenerate Graph") button that runs the full 3-step pipeline in sequence:

1. Entity Extraction & Relationship Discovery — Process uploaded documents to extract entities and discover relations (per-chunk) grounded in the source text. Shows entity and relation counts. This step is also image-analysis-aware: documents that have finished text processing but still have background image analysis running (e.g., "Analyzed 3/67 images") are treated as in-progress. The step displays an aggregate image analysis progress bar and stays in "In Progress" state until all images across all documents have been analyzed. This ensures entities from images are included before advancing to relationship analysis.
2. Deep Relationship Analysis — Discover cross-document relations between entities using the relationship model. By default (RELATIONSHIP_DISCOVERY_MODE=targeted) candidate pairs come from entity-embedding kNN and document co-mention, and the LLM only verifies them in small batched calls, keeping analysis fast on large graphs. Shows only cross-document relation counts (excludes per-chunk). The "Find more" button runs an additional round of incremental analysis. The ERR (Entity-Relationship Ratio) indicator is displayed to 2 decimal places in both discovery modes (the ERR target only drives extra rounds in legacy llm_scan mode).
3. Community Detection — Group related entities into communities.

The button label changes to "Regenerate Graph" when a graph already exists. When regenerating, Step 1 first deletes all communities, relationships, and entities (in that order) before reprocessing documents, ensuring a true from-scratch rebuild of the entire knowledge graph. The full cleanup order is: DELETE /api/graph/communities → DELETE /api/graph/relationships → DELETE /api/graph/entities → reprocess all documents → relationship analysis (rebuild mode) → community detection.

The pipeline is resilient to page refreshes — progress is persisted via sessionStorage with a saved task ID (regenerateTaskId) for the active step's backend task. On resume, the step runner checks the saved task's status: running → resume polling, completed → advance to the next step, failed → abort, not found → start fresh. This explicit task-based approach eliminates false step-skipping that could occur from stale data. Each step also tracks staleness via persisted timestamps, ensuring users are always guided to keep the knowledge graph up to date.

The Documents page also provides a "Generate Graph" button. It navigates to the Knowledge Graph page and auto-starts the pipeline on arrival — one click on Documents, no second click needed. The button appends ?autostart=1 to the destination URL; the Knowledge Graph page detects the flag, waits for its initial data fetch, fires the chain once, and clears the URL parameter so a refresh doesn't re-trigger. The destructive-action confirmation dialog still appears if you already have entities in the graph.

The Explore section provides an interactive graph visualization (Knowledge Graph tab) plus read-only browsers for Entities, Relationships, and Communities:

• Node Colors - Different colors for entity types
• Node Size - Based on mention frequency
• Edge Labels - Relationship types
• Clustering - Related entities grouped together
• Filtering - Filter by entity type or relationship
• Search - Find specific entities

Configuration

Code
 
# Enable/disable graph extraction
ENABLE_GRAPH_EXTRACTION=true

# Model for entity extraction and community summarization (defaults to OPENAI_MODEL)
# Recommended: Qwen3.6 27B — reasoning suppressed so it behaves like a fast instruct model (no overthinking)
# GRAPH_EXTRACTION_MODEL=qwen3-6-27b
# GRAPH_EXTRACTION_API_BASE=http://localhost:11434/v1
# GRAPH_EXTRACTION_API_KEY=ollama

# Model for relationship extraction (defaults to GRAPH_EXTRACTION_MODEL → OPENAI_MODEL)
# Recommended: Qwen3.6 27B — reasoning suppressed so it behaves like a fast instruct model (no overthinking)
# Used for both per-chunk (Step 1) and batch analysis (Step 2)
# Runs on a separate rate limit from entity extraction
# RELATIONSHIP_EXTRACTION_MODEL=qwen3-6-27b
# RELATIONSHIP_EXTRACTION_API_BASE=http://localhost:11434/v1
# RELATIONSHIP_EXTRACTION_API_KEY=ollama

# Concurrent per-chunk relationship extractions per document (default: 3)
# CONCURRENT_RELATIONS=3

# Graph traversal depth
MAX_GRAPH_HOPS=2

# Context + output budgets cascade from primary (OPENAI_MAX_*) when set to 0.
# See "Budget Fallback Chain" in /pages/configuration for the full chain.
# GRAPH_EXTRACTION_MAX_CONTEXT=0              # 0 = inherit min(OPENAI_MAX_CONTEXT, 48000); recommended explicit 24000
# EXTRACTION_MAX_OUTPUT_TOKENS=12000    # default 12000 — ≈ half the extraction input budget (entity-dense docs overflow smaller caps); 0 = inherit OPENAI_MAX_OUTPUT_TOKENS (=8000)

# Entity resolution
ENABLE_SEMANTIC_ENTITY_RESOLUTION=true
ENTITY_SIMILARITY_THRESHOLD=0.85

# Relationship analysis (Phase B / Step 2)
# RELATIONSHIP_DISCOVERY_MODE=targeted          # 'targeted' (default) | 'llm_scan' (legacy full-batch scan)
# RELATIONSHIP_MAX_CONTEXT=0                    # 0 = inherit GRAPH_EXTRACTION_MAX_CONTEXT → primary
# RELATIONSHIP_MAX_OUTPUT_TOKENS=0              # per-chunk + candidate scan (in chain)
# RELATIONSHIP_BATCH_MAX_OUTPUT_TOKENS=16000    # Phase 2 batch (standalone, NOT in chain)
# PARALLEL_RELATIONSHIP_BATCHES=5       # Batches / verification calls processed in parallel
# RELATIONSHIP_MAX_HOURS=0              # Max hours for relationship analysis (0 = no limit)
# RELATIONSHIP_MAX_PER_ENTITY=50        # Soft cap on relationships per entity (0 = no cap)

# Targeted mode only:
# RELATIONSHIP_KNN_K=8                  # Nearest neighbors per entity (vector index)
# RELATIONSHIP_KNN_MIN_SIMILARITY=0.80  # Min vector-index score for a kNN candidate pair
# RELATIONSHIP_MIN_SHARED_DOCS=2        # Min docs co-mentioning a pair (0 = disable generator)
# RELATIONSHIP_DOC_FREQ_CAP=30          # Skip hub entities mentioned in more docs than this
# RELATIONSHIP_MAX_CANDIDATE_PAIRS=15000  # Total candidate-pair budget (top-ranked kept)
# RELATIONSHIP_CANDIDATES_PER_ENTITY=10 # Max candidate pairs per entity (hub guard)
# RELATIONSHIP_PAIRS_PER_CALL=40        # Candidate pairs verified per LLM call
# RELATIONSHIP_PAIR_CONTEXT_TOKENS=3000 # Chunk-context tokens per verification call

# Legacy llm_scan mode only:
# RELATIONSHIP_TARGET_RATIO=1.0         # Target entity-to-relationship ratio (ERR)
# RELATIONSHIP_MAX_ROUNDS=3             # Max discovery rounds per batch (initial analysis)

# Reasoning Control — lets you use reasoning-capable models for ingestion
# (GPT-5/5.1, Claude 4.x, Qwen3, DeepSeek-R1, etc.) with their thinking
# forced OFF. Reasoning hurts structured extraction (drift, hidden-token
# cost, latency, malformed JSON). No-op for pure instruct models.
# Values: off | minimal | auto | low | medium | high
EXTRACTION_REASONING_MODE=off    # extraction + summaries + communities
RELATIONSHIP_REASONING_MODE=off  # candidate scan + relationship extraction
VISION_REASONING_MODE=off        # vision-model image descriptions (e.g. Qwen3-VL-27B)
DEFAULT_REASONING_MODE=auto      # Q&A / researcher agent (stays AUTO)
# REASONING_MODEL_OVERRIDES=gpt-5.8:none,custom:minimal  # escape hatch

Reasoning across providers. Cortex auto-detects provider from base_url (OpenAI, OpenRouter, Venice, Anthropic, self-hosted vLLM) and model family by regex on the model string, then injects the right kwargs: reasoning_effort for OpenAI direct, extra_body.reasoning.effort for OpenRouter, extra_body.venice_parameters.disable_thinking for Venice, extra_body.thinking={type:disabled} for Anthropic, extra_body.chat_template_kwargs.enable_thinking=false for vLLM (Qwen3-style). New minor releases route automatically via regex (e.g. gpt-5.8 works like gpt-5.1). For models the heuristic misclassifies, use REASONING_MODEL_OVERRIDES. Runtime fallback: if a model rejects the param with a 400, the wrapper strips it on retry and caches the model so subsequent calls skip the param.

Caveats. gpt-5-pro is hard-pinned to high by OpenAI (OFF is logged + ignored). gpt-5-codex auto-downgrades minimal→low. Anthropic Opus 4.7+ uses adaptive thinking (helper omits thinking). OpenRouter exclude:true does NOT save tokens; we use effort:"none". The researcher/Q&A agent stays on AUTO because reasoning_effort=minimal disables parallel tool calls on OpenAI.

Recommended Workflow

1. Upload documents - entities are extracted per-document (Phase A)
2. Analyze relationships - POST /api/graph/relationships/analyze (Phase B)
3. Detect communities - POST /api/graph/communities/detect

Graph Cleanup

When documents are deleted, Cortex automatically cleans up the knowledge graph:

What Gets Cleaned

Item	Behavior
Active Processing	Cancelled immediately before deletion
Chunks	All chunks from the document are removed
Entities	Removed if only mentioned by this document
Shared Entities	Preserved if mentioned by other documents
Relationships	Removed with their entities (DETACH DELETE)
Communities	Removed if no member entities remain

Cleanup API

Clean up orphaned data manually:

Code
 
# Clean orphaned entities and communities
curl -X POST "http://localhost:8000/api/cleanup/orphaned-entities" \
  -H "X-API-Key: your-api-key"

Response:

Code
 
{
  "message": "Cleanup completed",
  "orphaned_entities_removed": 42,
  "orphaned_communities_removed": 3
}

The StatsBar in the UI also shows a cleanup banner when it detects orphaned graph data (entities exist but no documents).

Entity Management

Delete all entities and their connections (DETACH DELETE). This removes every entity node and all relationships attached to them:

Code
 
# Delete ALL entities and their connections
curl -X DELETE "http://localhost:8000/api/graph/entities" \
  -H "X-API-Key: your-api-key"

Response:

Code
 
{
  "entities_deleted": 1542
}

Relationship Management

Delete all entity relationships (useful before re-running relationship analysis):

Code
 
# Delete ALL relationships between entities
curl -X DELETE "http://localhost:8000/api/graph/relationships" \
  -H "X-API-Key: your-api-key"

Response:

Code
 
{
  "relationships_deleted": 142
}

Community Management

Delete individual or all communities:

Code
 
# Delete a specific community (entities are unlinked, not deleted)
curl -X DELETE "http://localhost:8000/api/graph/communities/5" \
  -H "X-API-Key: your-api-key"

# Delete ALL communities
curl -X DELETE "http://localhost:8000/api/graph/communities" \
  -H "X-API-Key: your-api-key"

System Reset

For a complete wipe of all data, use the System Reset feature in Settings → Danger Zone, or call the API directly:

Code
 
curl -X POST "http://localhost:8000/api/admin/reset" \
  -H "X-API-Key: your-admin-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "delete_documents": true,
    "delete_uploaded_files": true,
    "delete_custom_inputs": true,
    "delete_collections": true,
    "delete_api_keys": false
  }'

When documents are deleted, the system also cleans up all associated data:

Data	Cleaned Up
Documents, Chunks	All document and chunk nodes
Entities, Relationships	All entity nodes and their relationships
Communities	All community nodes
Merge History	All deduplication audit trail (MergeHistory nodes)
System Metadata	Staleness timestamps (SystemMeta nodes)
Client Cache	Dismissed dedup suggestions, regeneration flow state

Best Practices

1. Quality Documents - Better source documents = better entities
2. Review Entities - Periodically review extracted entities for accuracy
3. Tune Threshold - Adjust ENTITY_SIMILARITY_THRESHOLD based on your domain
4. Deduplicate Periodically - After large ingestion batches, use GET /api/entities/duplicates to find and merge remaining duplicates that automatic resolution missed
5. Use Communities - Enable community detection for large graphs
6. Safe Deletion - Delete documents safely knowing the graph will be cleaned automatically
7. Clean Up After Bulk Deletion - After deleting many documents, run cleanup or use the UI banner to remove orphaned graph data