Core Features

Ask AI

Copy page

Ask questions about your documents and get intelligent answers with source citations. Cortex's Ask AI feature uses Retrieval-Augmented Generation (RAG) with optional agentic reasoning for complex queries. The primary interface is the streaming endpoint, which delivers results in real time via Server-Sent Events (SSE).

Frontend UI

In the Cortex frontend, Ask AI is accessed via the Explore section with two dedicated tabs:

Tab	Description	Best For
Deep Research	Multi-step agentic reasoning that breaks down complex questions into sub-queries	Complex questions requiring thorough analysis
Chat	Quick hybrid search with knowledge graph context	Fast lookups and straightforward questions

Both modes share a simplified input interface with:

• A text input field (placeholder varies by mode)
• A Settings button (cog icon) to configure collection scope and streaming
• A Send button that activates when you type
• A collection indicator below the input showing the active search scope

The settings dropdown allows you to:

• Toggle Stream responses on/off (for debugging)
• Select a Collection Scope — defaults to All Collections so searches span the entire knowledge base. Select a specific collection to narrow results to that collection's documents only.

Response Layout

In the UI, responses are rendered in this order:

1. Research Process (Deep Research only) — Sub-Questions, Thinking Steps, and Reasoning Steps are displayed above the main answer content. The research process section auto-scrolls to the bottom during streaming so the latest activity is always visible.
2. Content — The main answer from the writer LLM.
3. Graph Context — Related entities, relationships, and communities.
4. Sources — Retrieved source documents with citations.

Source Citation Viewer

Clicking a source opens a modal that displays the full document content with the cited chunk highlighted. The surrounding text is shown at 60% opacity while the cited chunk appears at full opacity with an accent-colored left border. The modal auto-scrolls to bring the highlighted chunk into view.

How It Works

The researcher/writer agent architecture uses an LLM-driven researcher to gather information through tool-calling, followed by a writer that synthesizes a streamed answer. Deep Research selects this architecture with ENABLE_AGENT_RESEARCH=true (default); false selects the legacy fixed decompose/search/synthesize pipeline. Chat selects its agent with ENABLE_AGENT_CHAT; its legacy path is a single-pass researched answer.

The researcher agent has access to these tools:

Tool	Speed Mode	Quality Mode	Description
knowledge_search	1 call	3-5+ calls	Hybrid RRF search (vector + keyword + graph) with cross-encoder reranking. Up to 3 queries per call; the pooled candidates are deduplicated and about twice RERANK_TOP_K deep, and query entities are resolved to stored entities (including aliases) before graph traversal. The graph leg follows entity-to-entity relationships and ranks its passages by how many query entities they mention.
community_search	-	Yes	Search entity community summaries for thematic context (honors the collection scope)
entity_lookup	-	Yes	Look up specific entities by name for deeper exploration (honors the collection scope)
reasoning	-	Yes	Plan next research step (streamed to UI as thinking events)
done	Yes	Yes	Signal research completion with a summary for the writer

Streaming Endpoint

Code
 
POST /api/ask/stream

This is the primary endpoint for asking questions. It returns a stream of Server-Sent Events containing sources, graph context, and answer tokens as they are generated.

Authentication

All requests require the X-API-Key header:

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{"question": "What is GraphRAG?"}' \
  --no-buffer

Request Schema (RAGRequest)

Parameter	Type	Default	Description
question	string	required	The question to ask
top_k	integer	5	Number of results to retrieve (1–20)
use_reranking	boolean	true	Apply cross-encoder reranking to improve relevance
use_graph	boolean	true	Include knowledge graph context in retrieval
max_hops	integer	2	Graph traversal depth (1–3 hops)
use_agentic	boolean	false	Enable deep research mode with multi-step reasoning
use_fast_search	boolean	false	Use fast vector-only search (disables hybrid/reranking)
collection_id	string | null	null	Scope search to a specific collection
conversation_history	ConversationMessage[]	null	Previous messages for multi-turn context
conversation_memory	object | null	null	Opt-in client-carried memory blob (see Conversation Memory). Omit for stateless behavior.

ConversationMessage Schema

Code
 
{
  "role": "user" | "assistant",
  "content": "Message text"
}

The backend keeps the most recent messages (configured by MAX_CONVERSATION_HISTORY, default 6).

Two Modes

1. Chat Mode (Hybrid + Reranking)

The default mode for quick questions. Performs hybrid search (vector + keyword + graph), applies cross-encoder reranking, enriches with graph context, then streams the LLM answer.

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What are the main benefits of knowledge graphs?",
    "top_k": 5,
    "use_reranking": true,
    "use_graph": true
  }' --no-buffer

Event sequence: sources → graph_context → content (multiple) → done

Code
 
data: {"sources": [{"document_id": "doc_abc", "chunk_id": "chunk_1", "content": "Knowledge graphs provide...", "score": 0.94, "metadata": {"document_title": "Overview.pdf"}}]}

data: {"graph_context": {"entities": [...], "relationships": [...], "chunks": [...]}}

data: {"content": "Knowledge graphs "}
data: {"content": "provide several "}
data: {"content": "key benefits..."}

data: {"done": true}

2. Deep Research Mode (Agentic)

For complex questions requiring multi-angle investigation. The researcher agent iteratively searches, explores communities, looks up entities, and cross-references information before handing off to the writer for a comprehensive answer (up to 4000 tokens).

The reflect-then-search rhythm is enforced by the loop, not left to the model: a search round the model didn't reason about triggers a forced reflection step (RESEARCHER_FORCE_REFLECTION), consecutive rounds that mostly re-surface already-seen sources end research early (RESEARCHER_NOVELTY_MIN_NEW_RATIO / RESEARCHER_NOVELTY_STALE_ROUNDS), and a wall-clock budget (RESEARCHER_WALL_CLOCK_SECONDS, default 60s) guarantees the answer starts even when the LLM provider is queueing. Answer quality is consistent across tool-calling models — including ones that don't reflect on their own.

Requires ENABLE_AGENTIC_RAG=true and ENABLE_AGENT_RESEARCH=true on the server.

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Compare the methodologies used in papers A and B",
    "use_agentic": true
  }' --no-buffer

Event sequence: thinking (reasoning + search progress) → retrieval (per search) → sources → graph_context → retrieval_stats → content (multiple) → done

Code
 
data: {"thinking": "Starting research..."}

data: {"thinking": "The user wants to compare methodologies in papers A and B. I'll start by searching for each paper's methodology separately."}
data: {"thinking": "Searching: Paper A methodology, Paper B methodology, methodology comparison"}
data: {"retrieval": "Found 8 sources"}

data: {"thinking": "Good results on Paper A. I should also check for community-level context on research methodologies."}
data: {"retrieval": "Found 2 relevant communities"}

data: {"thinking": "Let me search for specific differences and limitations."}
data: {"thinking": "Searching: Paper A limitations, Paper B strengths weaknesses, methodology comparison criteria"}
data: {"retrieval": "Found 6 sources"}

data: {"thinking": "Comprehensive coverage achieved. Ready to wrap up."}

data: {"sources": [...]}
data: {"graph_context": {"entities": [...], "relationships": [...], "communities": [...]}}
data: {"retrieval_stats": {"total_sources_considered": 14, "unique_sources": 11, "search_calls": 2, "communities_used": 2}}

data: {"content": "Both papers "}
data: {"content": "approach the problem "}
data: {"content": "differently..."}

data: {"done": true, "communities_used": [1, 4]}

SSE Event Reference

Every event is a JSON object on a data: line. Each event contains exactly one of these keys:

Event Key	Type	Mode	Description
status	{stage, message}	All (if stream_reasoning_steps)	Current pipeline stage — analyzing/searching/reranking/generating. Drives a live "working" indicator; removes the silent pre-token window
content	string	All	A token of the streamed answer
sources	SearchResult[]	All	Retrieved source documents with scores
graph_context	object	All	Knowledge graph entities, relationships, and community data
thinking	string	Deep Research	Status message describing the current reasoning step
sub_questions	string[]	Deep Research	The decomposed research sub-questions
retrieval	string	Deep Research	Per-sub-question retrieval progress
retrieval_stats	object	Deep Research	Summary: total_sources, unique_sources, communities_used
done	boolean	All	true when the answer is complete. On memory-carrying turns it includes pending_memory: true — a memory_update event still follows before the stream closes
pending_memory	boolean	On the done event	Present (true) when a memory_update will follow the done event — keep reading the stream
refused	boolean	On the refusal content frame and its done event	Present (true) when the prompt-injection defenses answered with the safe refusal instead of knowledge — rephrase as a plain question rather than showing the canned text as an answer
truncated	boolean	On the done event	Present (true) when the answer hit its output-token cap. On the agent writer the answer also ends with a visible "cut short" note. The flag-off legacy streaming writers set the flag from their writer stream's provider reason without a note — the deep synthesis call has a fixed 2,000-token cap, and the standard chat writer's cap is WRITER_MAX_TOKENS_SPEED (default 1,200). The fast streaming writer does the same (its own branch, selected by depth: "fast"/use_fast_search, independent of the agent-chat flag) with a literal 600-token cap
error	string	All	Error message if something went wrong
communities_used	number[]	Deep Research	Community IDs used, included in the done event
memory_update	object	When conversation_memory sent	Updated memory blob to store and replay next turn — emitted after done by default (see Conversation Memory)

Each source object in sources also carries a stable sid (string) for cross-turn citation continuity.

Event order note: done fires as soon as the last answer token is streamed (finalize your UI there), but it is not necessarily the final frame: with EMIT_DONE_BEFORE_MEMORY=true (the default) the post-answer memory compaction runs afterwards and memory_update arrives 1–4 s later, then the stream closes. Consume the stream to its actual end — a client that stops reading at done loses memory continuity. Set EMIT_DONE_BEFORE_MEMORY=false to restore the legacy order (memory_update → done).

During silent windows (≥ 8 s with no event), the stream also emits SSE comment keep-alives (: ping) — ignored by the SSE spec and clients, present only to prevent proxy idle-timeouts.

Source Object Shape

Code
 
{
  "document_id": "abc123",
  "chunk_id": "chunk_456",
  "content": "The relevant text from the document...",
  "score": 0.94,
  "metadata": {
    "document_title": "Report.pdf",
    "collection_id": "research"
  }
}

Conversation History

Maintain context across multiple questions by passing previous messages:

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Can you elaborate on the third point?",
    "conversation_history": [
      {"role": "user", "content": "What are the benefits of knowledge graphs?"},
      {"role": "assistant", "content": "Knowledge graphs provide: 1) Structured relationships, 2) Semantic understanding, 3) Discovery of hidden connections."}
    ]
  }' --no-buffer

The backend automatically trims conversation history to the most recent messages (default: 6, configurable via MAX_CONVERSATION_HISTORY).

Conversation Memory

For long conversations, raw history trimming silently forgets older turns. Conversation Memory is an opt-in, client-carried alternative: the backend stays stateless, the client round-trips an opaque conversation_memory blob, and the agent curates a bounded, knowledge-grounded context from it each turn. Omit the field for today's stateless behavior — it's fully backward-compatible.

How to use it:

1. Start with "conversation_memory": {} (or omit on turn 1).
2. Read the memory_update SSE event emitted at the end of each turn — by default it arrives after the done event (which carries pending_memory: true), so keep consuming the stream until it actually closes.
3. Send that blob back as conversation_memory on the next turn (and keep sending the full conversation_history).

Code
 
// memory_update event payload — store it client-side, send it back next turn
{
  "memory_update": {
    "version": 3,
    "transcript": { "summary": "…rolling summary of older turns…", "summarized_count": 8 },
    "facts": ["Project codename: BlueFalcon", "Deadline: Friday"],
    "open_questions": [],
    "intent": "The user wants concise answers in German.",
    "source_ledger": [{ "sid": "s_7fd8dcf10785", "filename": "doc.pdf", "gist": "…" }],
    "kg_context": { "entities": [], "communities": [] }
  }
}

What it gives you:

• No amnesia — older turns fold into a rolling summary plus durable facts / open_questions / intent, rebuilt into a small fixed context each turn (lower cost and latency than ever-growing history fed twice).
• Citation continuity — every source in the sources event carries a conversation-stable sid, accumulated in source_ledger, so a source cited in turn 2 keeps its identity in turn 5.
• Memory fast-path — follow-ups answerable from memory alone ("summarize that", "why?", "in German") skip retrieval entirely for a fast, cheap response (toggle with ENABLE_MEMORY_FAST_PATH).

Compaction runs after the answer streams (a cheap fast-model call), so it adds no user-visible latency. Treat the blob as opaque — its shape may grow; only ever store and replay what the server returns.

Collection Scope

Without an explicit collection, retrieval uses the caller's API-key collection scope. An unrestricted key can search all collections; a restricted key supplies its allowed collections, including an empty allowed set.

To narrow results to a specific collection, pass collection_id in your API request:

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What were the Q4 results?",
    "collection_id": "financial-reports"
  }' --no-buffer

When collection_id is omitted (or null), the key's scope applies; an explicit permitted collection narrows it. Chunk queries apply the effective scope in vector, keyword and graph retrieval. The default Deep Research agent also supplies it to its community and entity tools. Scoped vector search over-fetches its candidate depth (10×, capped at 200, never below that depth) before filtering; post-filter ANN retrieval does not guarantee a full candidate set.

Local unreleased legacy scope repairs: with ENABLE_AGENT_RESEARCH=false, both streaming endpoints (agentic_rag_stream) and non-streaming /api/ask (rag_query → _agentic_rag_query) forward the effective scalar/allowlist, including an empty allowed set, to every sub-question's chunk queries and community selection/summary access. Only the non-streaming path has the no-LLM-key recursive fallback, which also preserves scope; its ~28s deadline does not apply to SSE. Collection scoping does not establish complete graph-context privacy: traversal metadata remains global, and a shared full community summary can mention inaccessible members once one member is accessible.

Local unreleased finish-reason propagation: on the flag-off non-streaming path, the answer synthesis completion's provider finish_reason is now projected to the public finish_reason/truncated fields — stop → finish_reason: "stop", truncated: false; length → finish_reason: "length", truncated: true. Both fields already exist in the response schema, so no API shape or client change is required, and the sub-question decomposition call's reason is never used as the answer's public reason.

Local unreleased optional projection: on the same flag-off non-streaming legacy deep-research path (use_agentic: true or the equivalent depth: "deep", routed through rag_query → _agentic_rag_query under the ~28s deadline), the response now projects the helper's optional research metadata into the existing RAGResponse fields — sub_questions (string list), communities_used (integer community IDs) and retrieval_stats (exactly four keys: total_sources_considered, unique_sources, sub_questions_researched, communities_referenced). Nonempty and literal empty ([]) lists are preserved; the schema accepts an empty object for retrieval_stats, but the real helper always returns all four keys populated. Missing or null producer values project to public null — which is also what the standard chat path and the no-LLM-key fallback return. Refusals are not uniform: the input-screen refusals (pattern validator / prompt-guard classifier) are constructed before retrieval and leave all three null, while a model refusal (the writer emitted the canned deflection) still runs the real helper, so the populated values are projected alongside refused: true/refusal_source: "model". Earlier inspected handler snapshots omitted these values; check the version/capability rather than assuming. The repair is a local handler-local .get projection; the streaming Deep Research events keep their own separate retrieval statistics, which this projection does not touch.

Local unreleased legacy streaming done-flag: on the flag-off streaming path (ENABLE_AGENT_RESEARCH=false, both /api/ask/stream and /api/ask/stream/thinking), the legacy synthesis stream's provider finish_reason is projected onto the public done frame as truncated — length → truncated: true; stop, a null reason, and a missing reason attribute never set the flag, and the sub-question decomposition call's reason is never used. The flag is additive — no new SSE field is required and no public finish_reason is exposed on SSE — and this legacy path does not append the agent writer's visible "cut short" note; its synthesis call carries its own 2,000-token output cap.

Local unreleased standard streaming done-flag: on the standard chat branch of POST /api/ask/stream (ENABLE_AGENT_CHAT=false, the default — reached with depth: "standard" or the equivalent agreeing legacy flags), the standard writer's provider finish_reason is captured from the synthesis stream before the text-only security filter and projected onto the public done frame as truncated — length → truncated: true; stop, a null reason, and a missing reason attribute never set the flag. The flag is additive — no new SSE field is required, no public finish_reason is exposed on SSE, and this path appends no visible "cut short" note. Its writer cap is WRITER_MAX_TOKENS_SPEED (default 1,200) — distinct from the flag-off deep synthesis call's literal 2,000 and the non-streaming chat path's literal 1,200; the deep legacy streaming path, the agent writer and the fast streaming path are separate implementations and are unchanged by this repair.

Local unreleased fast streaming done-flag: on the fast branch of POST /api/ask/stream (reached with depth: "fast" or use_fast_search: true — always its own branch, independent of ENABLE_AGENT_CHAT; /api/ask/stream/thinking has no fast branch), the fast writer's provider finish_reason is captured from the synthesis stream before the text-only security filter and projected onto the public done frame as truncated — length → truncated: true; stop, a null reason, and a missing reason attribute never set the flag. The flag is additive — no new SSE field is required, no public finish_reason is exposed on SSE, and this path appends no visible "cut short" note. Its output cap is a literal 600 tokens — not WRITER_MAX_TOKENS_SPEED — and its model is the Fast Mode model (OPENAI_MODEL_FAST_MODE, default OPENAI_MODEL), not the WRITER_MODEL override; the first-turn context is built from the top 3 retrieved chunks trimmed to 600 characters each, and a follow-up turn with conversation history skips retrieval entirely.

In the UI: Click the Settings (cog) icon next to the send button and use the Collection Scope dropdown. The dropdown defaults to All Collections and shows a persistent indicator below the input confirming the active scope. Select a specific collection to filter, or select "All Collections" to search everything.

Frontend Integration (JavaScript/TypeScript)

Consume the SSE stream using fetch and ReadableStream:

Code
 
async function* askStream(
  baseUrl: string,
  apiKey: string,
  question: string,
  options: {
    topK?: number;
    useReranking?: boolean;
    useGraph?: boolean;
    useAgentic?: boolean;
    useFastSearch?: boolean;
    conversationHistory?: { role: string; content: string }[];
    collectionId?: string;
  } = {}
) {
  const res = await fetch(`${baseUrl}/api/ask/stream`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-API-Key": apiKey,
    },
    body: JSON.stringify({
      question,
      top_k: options.topK ?? 5,
      use_reranking: options.useReranking ?? true,
      use_graph: options.useGraph ?? true,
      use_agentic: options.useAgentic ?? false,
      use_fast_search: options.useFastSearch ?? false,
      conversation_history: options.conversationHistory,
      ...(options.collectionId
        ? { collection_id: options.collectionId }
        : {}),
    }),
  });

  if (!res.ok) {
    const error = await res.json().catch(() => ({ detail: "Stream failed" }));
    throw new Error(error.detail || `HTTP ${res.status}`);
  }

  const reader = res.body?.getReader();
  if (!reader) throw new Error("No response body");

  const decoder = new TextDecoder();
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split("\n");
    buffer = lines.pop() || "";

    for (const line of lines) {
      if (line.startsWith("data: ")) {
        try {
          yield JSON.parse(line.slice(6));
        } catch {
          // skip malformed events
        }
      }
    }
  }
}

// Usage
for await (const event of askStream("http://localhost:8000", "your-api-key", "What is GraphRAG?")) {
  if (event.sources) console.log("Sources:", event.sources);
  if (event.graph_context) console.log("Graph:", event.graph_context);
  if (event.content) process.stdout.write(event.content);
  if (event.thinking) console.log("[Thinking]", event.thinking);
  if (event.done) console.log("\n--- Done ---");
  if (event.error) console.error("Error:", event.error);
}

Python Integration

Using httpx (recommended)

Code
 
import httpx
import json

def ask_stream(base_url: str, api_key: str, question: str, **kwargs):
    """Stream answers from Cortex's Ask AI endpoint."""
    payload = {"question": question, **kwargs}

    with httpx.stream(
        "POST",
        f"{base_url}/api/ask/stream",
        headers={"X-API-Key": api_key, "Content-Type": "application/json"},
        json=payload,
        timeout=60.0,
    ) as response:
        response.raise_for_status()
        buffer = ""
        for chunk in response.iter_text():
            buffer += chunk
            while "\n" in buffer:
                line, buffer = buffer.split("\n", 1)
                if line.startswith("data: "):
                    event = json.loads(line[6:])
                    yield event

# Standard mode
for event in ask_stream("http://localhost:8000", "your-api-key", "What is GraphRAG?"):
    if "content" in event:
        print(event["content"], end="", flush=True)
    elif "sources" in event:
        print(f"\n[{len(event['sources'])} sources retrieved]")
    elif "done" in event:
        print("\n--- Complete ---")

# Deep research mode
for event in ask_stream(
    "http://localhost:8000",
    "your-api-key",
    "Compare the approaches in these papers",
    use_agentic=True,
    top_k=10,
):
    if "thinking" in event:
        print(f"  [{event['thinking']}]")
    elif "sub_questions" in event:
        for i, q in enumerate(event["sub_questions"], 1):
            print(f"  Sub-Q {i}: {q}")
    elif "retrieval" in event:
        print(f"  {event['retrieval']}")
    elif "content" in event:
        print(event["content"], end="", flush=True)
    elif "done" in event:
        print("\n--- Complete ---")

Using requests

Code
 
import requests
import json

def ask_stream(base_url: str, api_key: str, question: str, **kwargs):
    payload = {"question": question, **kwargs}

    with requests.post(
        f"{base_url}/api/ask/stream",
        headers={"X-API-Key": api_key, "Content-Type": "application/json"},
        json=payload,
        stream=True,
        timeout=60,
    ) as response:
        response.raise_for_status()
        for line in response.iter_lines(decode_unicode=True):
            if line and line.startswith("data: "):
                yield json.loads(line[6:])

Multi-Turn Conversation

Code
 
history = []

def chat(question: str):
    """Ask a follow-up question with conversation context."""
    answer_parts = []
    for event in ask_stream(
        "http://localhost:8000",
        "your-api-key",
        question,
        conversation_history=history,
    ):
        if "content" in event:
            answer_parts.append(event["content"])
            print(event["content"], end="", flush=True)
        elif "done" in event:
            print()

    # Update history for next turn
    answer = "".join(answer_parts)
    history.append({"role": "user", "content": question})
    history.append({"role": "assistant", "content": answer})

chat("What are the main findings?")
chat("Can you elaborate on the second point?")
chat("How does that compare to the introduction?")

Configuration

Code
 
# Agentic RAG (Deep Research)
ENABLE_AGENTIC_RAG=true          # Enable deep research mode

# Agent-Based Research Pipeline
ENABLE_AGENT_RESEARCH=true       # Use agent pipeline for deep research (set false for legacy)
ENABLE_AGENT_CHAT=true           # Use agent pipeline for standard chat (required for skills)
# RESEARCHER_MAX_ITERATIONS_SPEED=3    # Agent iterations for chat mode
# RESEARCHER_MAX_ITERATIONS_QUALITY=5  # Agent iterations for deep research
# WRITER_MAX_TOKENS_SPEED=1200         # Max output tokens for chat
# WRITER_MAX_TOKENS_QUALITY=4000       # Max output tokens for deep research

# Deep research: reflection & convergence (all default on)
# RESEARCHER_FORCE_REFLECTION=true       # Force a reasoning step after unreflected search rounds
# RESEARCHER_NOVELTY_MIN_NEW_RATIO=0.35  # Search round below this share of new sources counts as stale
# RESEARCHER_NOVELTY_STALE_ROUNDS=2      # Consecutive stale rounds before the answer is written
# RESEARCHER_WALL_CLOCK_SECONDS=60      # Research time budget; on expiry the writer starts (0 = unlimited)

# Conversation
MAX_CONVERSATION_HISTORY=6       # Messages to keep for context

# Visibility
STREAM_REASONING_STEPS=true      # Show thinking steps in deep research mode
SHOW_RETRIEVAL_STATS=true        # Include retrieval_stats events

# Prompt security
PROMPT_SECURITY=true             # Sanitize input and filter harmful output

Agent vs Legacy Pipeline

The agent pipeline (ENABLE_AGENT_RESEARCH=true, default) provides adaptive research depth and reasoning transparency but has specific requirements and trade-offs:

	Agent Pipeline	Legacy Pipeline
LLM requirement	Must support function calling / tool use (OpenAI tools parameter)	Any OpenAI-compatible chat endpoint
Compatible models	GPT-4o, GPT-4o-mini, Claude, Mistral Large, Command R+	Any model (including local Ollama/vLLM)
Token usage	3-5x higher (multiple researcher iterations)	Lower (2 LLM calls: decompose + synthesize)
Latency	15-30s typical (4-8 LLM round-trips)	5-10s typical (2 LLM calls)
Research depth	Adaptive — agent decides when to dig deeper	Fixed — always decomposes into N sub-questions
Behavior	Non-deterministic — agent chooses search queries dynamically	Deterministic — fixed decompose → search → synthesize path
Transparency	Reasoning tool streams agent's thought process	Hard-coded status messages

Set ENABLE_AGENT_RESEARCH=false if your model doesn't support function calling, or if you prefer lower cost/latency with predictable behavior.

Prompt Security

Cortex includes built-in protection against prompt injection attacks:

Code
 
PROMPT_SECURITY=true

When enabled:

• Validates and sanitizes user input
• Injects anti-manipulation instructions into the system prompt
• Filters potentially harmful outputs
• Returns safe refusal messages when attacks are detected — flagged with refused: true on the streaming content/done frames and as a top-level field of the non-streaming response, so clients can distinguish a refusal from an answer without matching the text

The non-streaming POST /api/ask response additionally carries finish_reason and truncated (set when the answer hit the writer's output-token cap — 1,200 tokens on the standard non-streaming chat path, 2,000 on the flag-off legacy deep-research synthesis call), and a failure before an answer is produced returns a structured 500 {"detail": {"error": "ask_failed", "use_endpoint": "/api/ask/stream"}} — like the 504 deadline_exceeded, a signal to retry on the streaming endpoint.