Core Features

Document Upload

Copy page

Cortex supports various document formats and provides flexible upload options for both single files and bulk operations. In the web UI, uploading is done directly from the Documents page via the Upload button, which opens a drag-and-drop modal with collection selection. The modal closes immediately after files are selected — upload progress is shown inline in the document list. Once uploaded, use the graph button on the Documents page to bring the knowledge graph up to date. The button navigates to the Knowledge Graph page under Manage and auto-starts the right flow on arrival — no second click needed. On a fresh instance it reads Generate Graph and builds the full pipeline (entity extraction, relationship analysis, community detection). Once a graph exists it reads Update Graph and only extracts entities from the newly uploaded documents — the existing graph is never rebuilt from the Documents page; a full rebuild stays behind the explicit Regenerate Graph button (and its confirmation) on the Knowledge Graph page.

Supported Formats

Format	Extension	Speed	Notes
PDF	.pdf	~1 s/page	Per-page ML layout analysis — fine for papers/reports, minutes for books; scanned PDFs slower still (OCR)
EPUB	.epub	Seconds	E-books parsed natively — much faster than the same book as PDF; prefer uploading the EPUB when you have both. Kindle formats (.mobi, .azw) are not supported — convert them to EPUB first (e.g. with Calibre)
Plain Text	.txt	Instant	Direct text ingestion
Markdown	.md	Instant	Preserves formatting, headers, code blocks
Word	.docx	Seconds	Parsed natively — upload the .docx, not an "Export as PDF" of it

Rule of thumb: upload the source format, not a rendering of it. Only PDFs and standalone images go through per-page ML analysis; every other format is parsed directly from its markup in seconds regardless of length. Books → EPUB, Office documents → the office file itself, web content → Web Import or HTML/Markdown. Reserve PDF for content that only exists as PDF.

Upload Methods

Single File Upload

Upload one file at a time with immediate processing:

Code
 
# collection_id, start_processing, and source are query parameters
curl -X POST "http://localhost:8000/api/upload?collection_id=default&start_processing=true" \
  -H "X-API-Key: your-api-key" \
  -F "file=@document.pdf"

With a custom source (for API integrations):

Code
 
curl -X POST "http://localhost:8000/api/upload?source=youtube-transcriber" \
  -H "X-API-Key: your-api-key" \
  -F "file=@transcript.md"

The source parameter lets you tag documents by origin. Defaults to "upload" for UI uploads. Set it to your app's identifier (e.g. "slack-bot", "notion-sync") when building integrations. Documents can be filtered by source in the UI.

Response:

Code
 
{
  "filename": "document.pdf",
  "doc_id": "doc_abc123",
  "status": "processing",
  "message": "Document uploaded and processing started",
  "source": "upload"
}

Bulk Upload

For uploading many files (100+), disable immediate processing and batch process later:

Code
 
# Step 1: Upload files without processing (start_processing is a query parameter)
for file in *.pdf; do
  curl -X POST "http://localhost:8000/api/upload?start_processing=false" \
    -H "X-API-Key: your-api-key" \
    -F "file=@$file"
done

# Step 2: Start batch processing
curl -X POST "http://localhost:8000/api/documents/process-pending" \
  -H "X-API-Key: your-api-key"

Processing Pipeline

When a document is uploaded, it goes through these stages:

Tracking Progress

Get Document Status

Code
 
curl "http://localhost:8000/api/documents/doc_abc123" \
  -H "X-API-Key: your-api-key"

Response:

Code
 
{
  "id": "doc_abc123",
  "filename": "document.pdf",
  "status": "completed",
  "chunk_count": 42,
  "entity_count": 18,
  "created_at": "2024-01-15T10:30:00Z",
  "processed_at": "2024-01-15T10:32:15Z",
  "collection_id": "default"
}

Processing Status Values

Status	Description
pending	Uploaded, waiting for processing
processing	Currently being processed
completed	Successfully processed (text extraction done; images may still be analyzing in the background — check image_progress_current vs image_progress_total)
failed	Processing failed (check logs)

Custom Inputs

In addition to file uploads, you can add knowledge manually:

Q&A Pairs

Code
 
curl -X POST "http://localhost:8000/api/custom-input" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "qa",
    "title": "Company FAQ",
    "question": "What is Cortex?",
    "answer": "Cortex is an agentic knowledge base..."
  }'

Plain Text

Code
 
curl -X POST "http://localhost:8000/api/custom-input" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "text",
    "title": "Product Description",
    "content": "Cortex is a powerful..."
  }'

Markdown

Code
 
curl -X POST "http://localhost:8000/api/custom-input" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "markdown",
    "title": "Technical Documentation",
    "content": "# Overview\n\nCortex provides..."
  }'

Viewing Documents

You can view the original uploaded file for any document. The behavior depends on the file type:

• Markdown files (.md): Rendered in an in-app viewer with full Markdown formatting (headings, tables, code blocks, etc.)
• PDF files: Opened in a new browser tab for inline viewing
• Other file types: Opened in a new tab — the browser decides whether to display or download the file

Code
 
# Get the original file (returns the file with appropriate Content-Type)
curl "http://localhost:8000/api/documents/doc_abc123/file" \
  -H "X-API-Key: your-api-key" \
  -o document.pdf

In the UI, click the eye icon on any document card to view it.

Document Management

List All Documents

Code
 
curl "http://localhost:8000/api/documents" \
  -H "X-API-Key: your-api-key"

Delete a Document

When you delete a document, Cortex automatically:

1. Cancels active processing - If the document is being processed, the task is stopped
2. Removes chunks - All text chunks are deleted
3. Cleans up entities - Orphaned entities (only in this document) are removed
4. Cleans up communities - Empty communities are deleted

Code
 
curl -X DELETE "http://localhost:8000/api/documents/doc_abc123" \
  -H "X-API-Key: your-api-key"

Response:

Code
 
{
  "message": "Document deleted successfully",
  "processing_cancelled": true,
  "orphaned_entities_removed": 15,
  "orphaned_communities_removed": 2
}

Reprocess a Document

Useful after updating extraction settings:

Code
 
curl -X POST "http://localhost:8000/api/documents/doc_abc123/reprocess" \
  -H "X-API-Key: your-api-key"

Bulk Download (ZIP)

Download multiple documents as a ZIP archive. Supports large instances (1000+ documents) with ZIP64 and streaming:

Code
 
curl -X POST "http://localhost:8000/api/documents/download-zip" \
  -H "Content-Type: application/json" \
  -d '{"document_ids": ["doc_abc123", "doc_def456"]}' \
  -o documents.zip

Response: A streamed application/zip file containing the original uploaded files. Duplicate filenames are automatically disambiguated (e.g., report.pdf, report (1).pdf).

In the UI, select documents on the Documents page and click the Download button in the bulk actions toolbar.

Bulk Delete

Delete multiple documents at once. All active processing is cancelled before deletion:

Code
 
curl -X POST "http://localhost:8000/api/documents/delete" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{"document_ids": ["doc_abc123", "doc_def456"]}'

Response:

Code
 
{
  "message": "Successfully deleted 2 document(s)",
  "deleted_count": 2,
  "processing_cancelled": 1,
  "orphaned_entities_removed": 28,
  "orphaned_communities_removed": 3
}

Delete All Documents

Delete the entire knowledge base (use with caution):

Code
 
curl -X DELETE "http://localhost:8000/api/documents" \
  -H "X-API-Key: your-api-key"

This cancels all active processing tasks and removes all documents, chunks, entities, and communities.

Configuration

Control document processing behavior via environment variables:

Uploads are streamed in bounded chunks and rejected mid-stream with HTTP 413 once they exceed MAX_FILE_SIZE_MB, so an oversized file is never fully buffered into memory first. A multipart upload with no filename returns 400.

Code
 
# Maximum file size (uploads exceeding this are rejected with HTTP 413)
MAX_FILE_SIZE_MB=50

# Chunking settings
CHUNK_SIZE=500
CHUNK_OVERLAP=50
CHUNK_BY=sentence
SENTENCES_PER_CHUNK=5

# Processing concurrency
BATCH_PROCESSING_CONCURRENCY=2
CONCURRENT_EXTRACTIONS=3