Core Features

Ask AI

Copy page

Ask questions about your documents and get intelligent answers with source citations. Cortex's Ask AI feature uses Retrieval-Augmented Generation (RAG) with optional agentic reasoning for complex queries. The primary interface is the streaming endpoint, which delivers results in real time via Server-Sent Events (SSE).

Frontend UI

In the Cortex frontend, Ask AI is accessed via the Explore section with two dedicated tabs:

Tab	Description	Best For
Deep Research	Multi-step agentic reasoning that breaks down complex questions into sub-queries	Complex questions requiring thorough analysis
Chat	Quick hybrid search with knowledge graph context	Fast lookups and straightforward questions

Both modes share a simplified input interface with:

• A text input field (placeholder varies by mode)
• A Settings button (cog icon) to configure collection scope and streaming
• A Send button that activates when you type
• A collection indicator below the input showing the active search scope

The settings dropdown allows you to:

• Toggle Stream responses on/off (for debugging)
• Select a Collection Scope — defaults to All Collections so searches span the entire knowledge base. Select a specific collection to narrow results to that collection's documents only.

Response Layout

In the UI, responses are rendered in this order:

1. Research Process (Deep Research only) — Sub-Questions, Thinking Steps, and Reasoning Steps are displayed above the main answer content. The research process section auto-scrolls to the bottom during streaming so the latest activity is always visible.
2. Content — The main answer from the writer LLM.
3. Graph Context — Related entities, relationships, and communities.
4. Sources — Retrieved source documents with citations.

Source Citation Viewer

Clicking a source opens a modal that displays the full document content with the cited chunk highlighted. The surrounding text is shown at 60% opacity while the cited chunk appears at full opacity with an accent-colored left border. The modal auto-scrolls to bring the highlighted chunk into view.

How It Works

Both modes use a researcher/writer agent architecture. An LLM-driven researcher agent iteratively gathers information from the knowledge base using tool-calling, then a separate writer LLM synthesizes the gathered context into a streamed answer.

The researcher agent has access to these tools:

Tool	Speed Mode	Quality Mode	Description
knowledge_search	1 call	3-5+ calls	Hybrid RRF search (vector + keyword + graph) with cross-encoder reranking. Up to 3 queries per call.
community_search	-	Yes	Search entity community summaries for thematic context
entity_lookup	-	Yes	Look up specific entities by name for deeper exploration
reasoning	-	Yes	Plan next research step (streamed to UI as thinking events)
done	Yes	Yes	Signal research completion with a summary for the writer

Streaming Endpoint

Code
 
POST /api/ask/stream

This is the primary endpoint for asking questions. It returns a stream of Server-Sent Events containing sources, graph context, and answer tokens as they are generated.

Authentication

All requests require the X-API-Key header:

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{"question": "What is GraphRAG?"}' \
  --no-buffer

Request Schema (RAGRequest)

Parameter	Type	Default	Description
question	string	required	The question to ask
top_k	integer	5	Number of results to retrieve (1–20)
use_reranking	boolean	true	Apply cross-encoder reranking to improve relevance
use_graph	boolean	true	Include knowledge graph context in retrieval
max_hops	integer	2	Graph traversal depth (1–3 hops)
use_agentic	boolean	false	Enable deep research mode with multi-step reasoning
use_fast_search	boolean	false	Use fast vector-only search (disables hybrid/reranking)
collection_id	string | null	null	Scope search to a specific collection
conversation_history	ConversationMessage[]	null	Previous messages for multi-turn context
conversation_memory	object | null	null	Opt-in client-carried memory blob (see Conversation Memory). Omit for stateless behavior.

ConversationMessage Schema

Code
 
{
  "role": "user" | "assistant",
  "content": "Message text"
}

The backend keeps the most recent messages (configured by MAX_CONVERSATION_HISTORY, default 6).

Two Modes

1. Chat Mode (Hybrid + Reranking)

The default mode for quick questions. Performs hybrid search (vector + keyword + graph), applies cross-encoder reranking, enriches with graph context, then streams the LLM answer.

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What are the main benefits of knowledge graphs?",
    "top_k": 5,
    "use_reranking": true,
    "use_graph": true
  }' --no-buffer

Event sequence: sources → graph_context → content (multiple) → done

Code
 
data: {"sources": [{"document_id": "doc_abc", "chunk_id": "chunk_1", "content": "Knowledge graphs provide...", "score": 0.94, "metadata": {"document_title": "Overview.pdf"}}]}

data: {"graph_context": {"entities": [...], "relationships": [...], "chunks": [...]}}

data: {"content": "Knowledge graphs "}
data: {"content": "provide several "}
data: {"content": "key benefits..."}

data: {"done": true}

2. Deep Research Mode (Agentic)

For complex questions requiring multi-angle investigation. The researcher agent iteratively searches, explores communities, looks up entities, and cross-references information before handing off to the writer for a comprehensive answer (up to 4000 tokens).

The reflect-then-search rhythm is enforced by the loop, not left to the model: a search round the model didn't reason about triggers a forced reflection step (RESEARCHER_FORCE_REFLECTION), consecutive rounds that mostly re-surface already-seen sources end research early (RESEARCHER_NOVELTY_MIN_NEW_RATIO / RESEARCHER_NOVELTY_STALE_ROUNDS), and a wall-clock budget (RESEARCHER_WALL_CLOCK_SECONDS, default 120s) guarantees the answer starts even when the LLM provider is queueing. Answer quality is consistent across tool-calling models — including ones that don't reflect on their own.

Requires ENABLE_AGENTIC_RAG=true and ENABLE_AGENT_RESEARCH=true on the server.

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Compare the methodologies used in papers A and B",
    "use_agentic": true
  }' --no-buffer

Event sequence: thinking (reasoning + search progress) → retrieval (per search) → sources → graph_context → retrieval_stats → content (multiple) → done

Code
 
data: {"thinking": "Starting research..."}

data: {"thinking": "The user wants to compare methodologies in papers A and B. I'll start by searching for each paper's methodology separately."}
data: {"thinking": "Searching: Paper A methodology, Paper B methodology, methodology comparison"}
data: {"retrieval": "Found 8 sources"}

data: {"thinking": "Good results on Paper A. I should also check for community-level context on research methodologies."}
data: {"retrieval": "Found 2 relevant communities"}

data: {"thinking": "Let me search for specific differences and limitations."}
data: {"thinking": "Searching: Paper A limitations, Paper B strengths weaknesses, methodology comparison criteria"}
data: {"retrieval": "Found 6 sources"}

data: {"thinking": "Comprehensive coverage achieved. Ready to wrap up."}

data: {"sources": [...]}
data: {"graph_context": {"entities": [...], "relationships": [...], "communities": [...]}}
data: {"retrieval_stats": {"total_sources_considered": 14, "unique_sources": 11, "search_calls": 2, "communities_used": 2}}

data: {"content": "Both papers "}
data: {"content": "approach the problem "}
data: {"content": "differently..."}

data: {"done": true, "communities_used": [1, 4]}

SSE Event Reference

Every event is a JSON object on a data: line. Each event contains exactly one of these keys:

Event Key	Type	Mode	Description
status	{stage, message}	All (if stream_reasoning_steps)	Current pipeline stage — analyzing/searching/reranking/generating. Drives a live "working" indicator; removes the silent pre-token window
content	string	All	A token of the streamed answer
sources	SearchResult[]	All	Retrieved source documents with scores
graph_context	object	All	Knowledge graph entities, relationships, and community data
thinking	string	Deep Research	Status message describing the current reasoning step
sub_questions	string[]	Deep Research	The decomposed research sub-questions
retrieval	string	Deep Research	Per-sub-question retrieval progress
retrieval_stats	object	Deep Research	Summary: total_sources, unique_sources, communities_used
done	boolean	All	true when the answer is complete. On memory-carrying turns it includes pending_memory: true — a memory_update event still follows before the stream closes
pending_memory	boolean	On the done event	Present (true) when a memory_update will follow the done event — keep reading the stream
error	string	All	Error message if something went wrong
communities_used	number[]	Deep Research	Community IDs used, included in the done event
memory_update	object	When conversation_memory sent	Updated memory blob to store and replay next turn — emitted after done by default (see Conversation Memory)

Each source object in sources also carries a stable sid (string) for cross-turn citation continuity.

Event order note: done fires as soon as the last answer token is streamed (finalize your UI there), but it is not necessarily the final frame: with EMIT_DONE_BEFORE_MEMORY=true (the default) the post-answer memory compaction runs afterwards and memory_update arrives 1–4 s later, then the stream closes. Consume the stream to its actual end — a client that stops reading at done loses memory continuity. Set EMIT_DONE_BEFORE_MEMORY=false to restore the legacy order (memory_update → done).

During silent windows (≥ 8 s with no event), the stream also emits SSE comment keep-alives (: ping) — ignored by the SSE spec and clients, present only to prevent proxy idle-timeouts.

Source Object Shape

Code
 
{
  "document_id": "abc123",
  "chunk_id": "chunk_456",
  "content": "The relevant text from the document...",
  "score": 0.94,
  "metadata": {
    "document_title": "Report.pdf",
    "collection_id": "research"
  }
}

Conversation History

Maintain context across multiple questions by passing previous messages:

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Can you elaborate on the third point?",
    "conversation_history": [
      {"role": "user", "content": "What are the benefits of knowledge graphs?"},
      {"role": "assistant", "content": "Knowledge graphs provide: 1) Structured relationships, 2) Semantic understanding, 3) Discovery of hidden connections."}
    ]
  }' --no-buffer

The backend automatically trims conversation history to the most recent messages (default: 6, configurable via MAX_CONVERSATION_HISTORY).

Conversation Memory

For long conversations, raw history trimming silently forgets older turns. Conversation Memory is an opt-in, client-carried alternative: the backend stays stateless, the client round-trips an opaque conversation_memory blob, and the agent curates a bounded, knowledge-grounded context from it each turn. Omit the field for today's stateless behavior — it's fully backward-compatible.

How to use it:

1. Start with "conversation_memory": {} (or omit on turn 1).
2. Read the memory_update SSE event emitted at the end of each turn — by default it arrives after the done event (which carries pending_memory: true), so keep consuming the stream until it actually closes.
3. Send that blob back as conversation_memory on the next turn (and keep sending the full conversation_history).

Code
 
// memory_update event payload — store it client-side, send it back next turn
{
  "memory_update": {
    "version": 3,
    "transcript": { "summary": "…rolling summary of older turns…", "summarized_count": 8 },
    "facts": ["Project codename: BlueFalcon", "Deadline: Friday"],
    "open_questions": [],
    "intent": "The user wants concise answers in German.",
    "source_ledger": [{ "sid": "s_7fd8dcf10785", "filename": "doc.pdf", "gist": "…" }],
    "kg_context": { "entities": [], "communities": [] }
  }
}

What it gives you:

• No amnesia — older turns fold into a rolling summary plus durable facts / open_questions / intent, rebuilt into a small fixed context each turn (lower cost and latency than ever-growing history fed twice).
• Citation continuity — every source in the sources event carries a conversation-stable sid, accumulated in source_ledger, so a source cited in turn 2 keeps its identity in turn 5.
• Memory fast-path — follow-ups answerable from memory alone ("summarize that", "why?", "in German") skip retrieval entirely for a fast, cheap response (toggle with ENABLE_MEMORY_FAST_PATH).

Compaction runs after the answer streams (a cheap fast-model call), so it adds no user-visible latency. Treat the blob as opaque — its shape may grow; only ever store and replay what the server returns.

Collection Scope

By default, both Chat and Deep Research search across all collections in your knowledge base. This means every document you've ingested is available for retrieval regardless of which collection it belongs to.

To narrow results to a specific collection, pass collection_id in your API request:

Code
 
curl -X POST "http://localhost:8000/api/ask/stream" \
  -H "X-API-Key: your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What were the Q4 results?",
    "collection_id": "financial-reports"
  }' --no-buffer

When collection_id is omitted (or null), all collections are searched. When provided, all retrieval steps (vector search, keyword search, and graph traversal) are filtered to that collection's data. Collection scoping works identically in both Chat and Deep Research modes.

In the UI: Click the Settings (cog) icon next to the send button and use the Collection Scope dropdown. The dropdown defaults to All Collections and shows a persistent indicator below the input confirming the active scope. Select a specific collection to filter, or select "All Collections" to search everything.

Frontend Integration (JavaScript/TypeScript)

Consume the SSE stream using fetch and ReadableStream:

Code
 
async function* askStream(
  baseUrl: string,
  apiKey: string,
  question: string,
  options: {
    topK?: number;
    useReranking?: boolean;
    useGraph?: boolean;
    useAgentic?: boolean;
    useFastSearch?: boolean;
    conversationHistory?: { role: string; content: string }[];
    collectionId?: string;
  } = {}
) {
  const res = await fetch(`${baseUrl}/api/ask/stream`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-API-Key": apiKey,
    },
    body: JSON.stringify({
      question,
      top_k: options.topK ?? 5,
      use_reranking: options.useReranking ?? true,
      use_graph: options.useGraph ?? true,
      use_agentic: options.useAgentic ?? false,
      use_fast_search: options.useFastSearch ?? false,
      conversation_history: options.conversationHistory,
      ...(options.collectionId
        ? { collection_id: options.collectionId }
        : {}),
    }),
  });

  if (!res.ok) {
    const error = await res.json().catch(() => ({ detail: "Stream failed" }));
    throw new Error(error.detail || `HTTP ${res.status}`);
  }

  const reader = res.body?.getReader();
  if (!reader) throw new Error("No response body");

  const decoder = new TextDecoder();
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split("\n");
    buffer = lines.pop() || "";

    for (const line of lines) {
      if (line.startsWith("data: ")) {
        try {
          yield JSON.parse(line.slice(6));
        } catch {
          // skip malformed events
        }
      }
    }
  }
}

// Usage
for await (const event of askStream("http://localhost:8000", "your-api-key", "What is GraphRAG?")) {
  if (event.sources) console.log("Sources:", event.sources);
  if (event.graph_context) console.log("Graph:", event.graph_context);
  if (event.content) process.stdout.write(event.content);
  if (event.thinking) console.log("[Thinking]", event.thinking);
  if (event.done) console.log("\n--- Done ---");
  if (event.error) console.error("Error:", event.error);
}

Python Integration

Using httpx (recommended)

Code
 
import httpx
import json

def ask_stream(base_url: str, api_key: str, question: str, **kwargs):
    """Stream answers from Cortex's Ask AI endpoint."""
    payload = {"question": question, **kwargs}

    with httpx.stream(
        "POST",
        f"{base_url}/api/ask/stream",
        headers={"X-API-Key": api_key, "Content-Type": "application/json"},
        json=payload,
        timeout=60.0,
    ) as response:
        response.raise_for_status()
        buffer = ""
        for chunk in response.iter_text():
            buffer += chunk
            while "\n" in buffer:
                line, buffer = buffer.split("\n", 1)
                if line.startswith("data: "):
                    event = json.loads(line[6:])
                    yield event

# Standard mode
for event in ask_stream("http://localhost:8000", "your-api-key", "What is GraphRAG?"):
    if "content" in event:
        print(event["content"], end="", flush=True)
    elif "sources" in event:
        print(f"\n[{len(event['sources'])} sources retrieved]")
    elif "done" in event:
        print("\n--- Complete ---")

# Deep research mode
for event in ask_stream(
    "http://localhost:8000",
    "your-api-key",
    "Compare the approaches in these papers",
    use_agentic=True,
    top_k=10,
):
    if "thinking" in event:
        print(f"  [{event['thinking']}]")
    elif "sub_questions" in event:
        for i, q in enumerate(event["sub_questions"], 1):
            print(f"  Sub-Q {i}: {q}")
    elif "retrieval" in event:
        print(f"  {event['retrieval']}")
    elif "content" in event:
        print(event["content"], end="", flush=True)
    elif "done" in event:
        print("\n--- Complete ---")

Using requests

Code
 
import requests
import json

def ask_stream(base_url: str, api_key: str, question: str, **kwargs):
    payload = {"question": question, **kwargs}

    with requests.post(
        f"{base_url}/api/ask/stream",
        headers={"X-API-Key": api_key, "Content-Type": "application/json"},
        json=payload,
        stream=True,
        timeout=60,
    ) as response:
        response.raise_for_status()
        for line in response.iter_lines(decode_unicode=True):
            if line and line.startswith("data: "):
                yield json.loads(line[6:])

Multi-Turn Conversation

Code
 
history = []

def chat(question: str):
    """Ask a follow-up question with conversation context."""
    answer_parts = []
    for event in ask_stream(
        "http://localhost:8000",
        "your-api-key",
        question,
        conversation_history=history,
    ):
        if "content" in event:
            answer_parts.append(event["content"])
            print(event["content"], end="", flush=True)
        elif "done" in event:
            print()

    # Update history for next turn
    answer = "".join(answer_parts)
    history.append({"role": "user", "content": question})
    history.append({"role": "assistant", "content": answer})

chat("What are the main findings?")
chat("Can you elaborate on the second point?")
chat("How does that compare to the introduction?")

Configuration

Code
 
# Agentic RAG (Deep Research)
ENABLE_AGENTIC_RAG=true          # Enable deep research mode

# Agent-Based Research Pipeline
ENABLE_AGENT_RESEARCH=true       # Use agent pipeline for deep research (set false for legacy)
ENABLE_AGENT_CHAT=true           # Use agent pipeline for standard chat (required for skills)
# RESEARCHER_MAX_ITERATIONS_SPEED=3    # Agent iterations for chat mode
# RESEARCHER_MAX_ITERATIONS_QUALITY=8  # Agent iterations for deep research
# WRITER_MAX_TOKENS_SPEED=1200         # Max output tokens for chat
# WRITER_MAX_TOKENS_QUALITY=4000       # Max output tokens for deep research

# Deep research: reflection & convergence (all default on)
# RESEARCHER_FORCE_REFLECTION=true       # Force a reasoning step after unreflected search rounds
# RESEARCHER_NOVELTY_MIN_NEW_RATIO=0.35  # Search round below this share of new sources counts as stale
# RESEARCHER_NOVELTY_STALE_ROUNDS=2      # Consecutive stale rounds before the answer is written
# RESEARCHER_WALL_CLOCK_SECONDS=120      # Research time budget; on expiry the writer starts (0 = unlimited)

# Conversation
MAX_CONVERSATION_HISTORY=6       # Messages to keep for context

# Visibility
STREAM_REASONING_STEPS=true      # Show thinking steps in deep research mode
SHOW_RETRIEVAL_STATS=true        # Include retrieval_stats events

# Prompt security
PROMPT_SECURITY=true             # Sanitize input and filter harmful output

Agent vs Legacy Pipeline

The agent pipeline (ENABLE_AGENT_RESEARCH=true, default) provides adaptive research depth and reasoning transparency but has specific requirements and trade-offs:

	Agent Pipeline	Legacy Pipeline
LLM requirement	Must support function calling / tool use (OpenAI tools parameter)	Any OpenAI-compatible chat endpoint
Compatible models	GPT-4o, GPT-4o-mini, Claude, Mistral Large, Command R+	Any model (including local Ollama/vLLM)
Token usage	3-5x higher (multiple researcher iterations)	Lower (2 LLM calls: decompose + synthesize)
Latency	15-30s typical (4-8 LLM round-trips)	5-10s typical (2 LLM calls)
Research depth	Adaptive — agent decides when to dig deeper	Fixed — always decomposes into N sub-questions
Behavior	Non-deterministic — agent chooses search queries dynamically	Deterministic — fixed decompose → search → synthesize path
Transparency	Reasoning tool streams agent's thought process	Hard-coded status messages

Set ENABLE_AGENT_RESEARCH=false if your model doesn't support function calling, or if you prefer lower cost/latency with predictable behavior.

Prompt Security

Cortex includes built-in protection against prompt injection attacks:

Code
 
PROMPT_SECURITY=true

When enabled:

• Validates and sanitizes user input
• Injects anti-manipulation instructions into the system prompt
• Filters potentially harmful outputs
• Returns safe refusal messages when attacks are detected