SegOps AIDocs

Copilot

The SegOps dashboard copilot is a ReAct agent powered by Claude. It can query your data, render inline charts and tables, and help you create and analyze segments — all from a conversational interface on the dashboard.


What You Can Ask#

Data questions#

"Show me my top 5 events by volume this week" "How many users are in the High LTV segment?" "What's the membership trend for Churn Risk over the last 30 days?"

Segment work#

"Create a segment for users who purchased in the last 7 days but never added to a wishlist" "Which segments are the largest?" "Show me the conditions for the VIP Buyers segment"

Platform overview#

"What connectors are active?" "List my event schemas" "What events have been ingested today?"


Inline UI Components#

The copilot emits structured UI components alongside its text responses:

ComponentWhen it appears
MetricCardSingle KPI values (member count, event volume)
SegmentTableLists of segments with member counts and status
EventVolumeChartBar chart of daily event or membership counts
CreateSegmentCTAOne-click button to open a pre-loaded segment builder

Creating Segments from the Copilot#

When you ask the copilot to suggest or create a segment:

  1. It generates the segment DSL and shows a preview
  2. Click Create segment — the definition is passed to /segments/new via sessionStorage
  3. The segment builder opens pre-populated with the AI-generated conditions
  4. Give it a name and click Save

Tools Available to the Copilot#

The copilot has 12 tools it can call:

ToolData SourceWhat It Returns
list_segmentsPostgresAll segments with name, type, member count
get_segment_detailPostgresFull segment definition + conditions
list_activationsPostgresAll active connectors with status
list_event_schemasPostgresRegistered event schemas with property counts
list_recent_eventsPostgresLast N ingested events
query_top_eventsClickHouseEvent type frequency ranking
query_event_volumeClickHouseDaily event counts over a time range
query_segment_membership_trendClickHouseDaily membership counts for a segment
display_segment_tableEmit SegmentTable inline component
display_metric_cardEmit MetricCard inline component
display_event_chartEmit EventVolumeChart inline component
display_create_segment_ctaEmit CreateSegmentCTA inline component

SSE Streaming Protocol#

For integrations, the copilot endpoint streams Server-Sent Events:

http
POST /api/ai/copilot/
Authorization: Bearer <token>
Content-Type: application/json

{ "message": "show me my top events" }

Event types:

data: {"type": "thinking",  "content": "Looking up event data..."}
data: {"type": "computing", "content": "Querying ClickHouse..."}
data: {"type": "text",      "content": "Here are your top events"}
data: {"type": "ui",        "component": "EventVolumeChart", "data": {...}}
data: {"type": "done"}

Quality & Evals#

The copilot is evaluated using an LLM-as-judge pipeline across 4 dimensions: helpfulness, accuracy, tool use, and conciseness. Current score: 4.47 / 5.0 across 15 golden test cases.

Run evals locally:

bash
docker compose exec api python manage.py run_copilot_evals --tenant-slug retailcorp