# Honeyhive — how to use (mcp.ai)

Connect your Honeyhive account and use 42 tools for artificial intelligence straight from your AI agent. Connect with your own API key. HoneyHive is a modern AI observability and evaluation platform that enables developers and domain experts to collaboratively build reliable AI applications faster.

## Option A — via MCP (recommended)
Remote MCP endpoint (HTTP, streamable): `https://api.mcp.ai/p_honeyhive?ms=1787291400000`
Add it as a custom/remote MCP connector in your client (Claude, Cursor, VS Code…), then authenticate when prompted. Once connected, ask the agent to use the server's tools (e.g. `honeyhive_add_datapoints_to_dataset`).

## Option B — via direct REST API
Base URL: `https://api.mcp.ai/api/honeyhive`
Auth: `Authorization: Bearer sk_live_…` — create a workspace API key at https://mcp.ai/settings/api-keys
Discover endpoints: `GET https://api.mcp.ai/api/honeyhive/_endpoints`

### Endpoints
- `POST https://api.mcp.ai/api/honeyhive/add/datapoints/to/dataset` — Tool to add datapoints to a dataset. Use when you need to append multiple entries with specified input, ground truth, and history mappings.
  - body: { data: object[], mapping: object, project: string, dataset_id: string }
- `POST https://api.mcp.ai/api/honeyhive/compare/runs` — Tool to retrieve experiment comparison between two evaluation runs. Use when you need to analyze the differences in metrics, datapoints, and events between two runs.
  - body: { filters?: string, new_run_id: string, old_run_id: string, project_id: string, aggregate_function?: string }
- `POST https://api.mcp.ai/api/honeyhive/compare/runs/events` — Tool to compare events between two experiment runs side-by-side. Use when analyzing differences in model behavior, performance metrics, or outputs between evaluation runs. Returns matched event pairs 
  - body: { page?: integer, limit?: integer, filter?: string, run_id_1: string, run_id_2: string, event_name?: string, event_type?: string }
- `POST https://api.mcp.ai/api/honeyhive/create/batch/datapoints` — Tool to create multiple datapoints in a single batch operation. Use when you need to bulk-import events into a dataset or create many datapoints at once. Supports filtering by date range, event IDs, o
  - body: { events?: string[], filters?: object|object[], mapping?: object, dateRange?: object, selectAll?: boolean, checkState?: object, dataset_id: string }
- `POST https://api.mcp.ai/api/honeyhive/create/batch/model/events` — Tool to create multiple model events in a single request. Use when you need to log a batch of event interactions to HoneyHive.
  - body: { model_events: object[] }
- `POST https://api.mcp.ai/api/honeyhive/create/batch/tool/events` — Tool to log a batch of external API calls as tool events. Use when you need to record multiple tool events in one request—use after gathering all event data.
  - body: { events: object[], is_single_session?: boolean }
- `POST https://api.mcp.ai/api/honeyhive/create/configuration` — Creates a new configuration in HoneyHive for managing LLM or pipeline settings. Use this to define reusable configurations with specific models, prompts, and parameters that can be deployed across dif
  - body: { env?: string[], name: string, tags?: string[], type?: string, provider: string, parameters: object, user_properties?: object }
- `POST https://api.mcp.ai/api/honeyhive/create/datapoint` — Tool to create a new datapoint with input-output pairs. Use when you need to add a single datapoint with inputs, ground truth, conversation history, and metadata.
  - body: { inputs?: object, history?: object[], metadata?: object, ground_truth?: object, linked_event?: string, linked_datasets?: string[] }
- `POST https://api.mcp.ai/api/honeyhive/create/dataset` — Tool to create a dataset. Use when you need to initialize a new dataset within a project.
  - body: { name: string, type?: string, saved?: boolean, project: string, metadata?: object, datapoints?: string[], description?: string, linked_evals?: string[], pipeline_type?: string }
- `POST https://api.mcp.ai/api/honeyhive/create/event` — Tool to create a new event in HoneyHive to track execution of different parts of your application. Use when you need to log a model call, tool execution, or chain step. Events can be grouped into sess
  - body: { event: object }
- `POST https://api.mcp.ai/api/honeyhive/create/metric` — Tool to create a new metric in HoneyHive. Use when you need to define how to evaluate model outputs, whether through code (PYTHON), AI evaluation (LLM), human review (HUMAN), or combining multiple met
  - body: { name: string, type: string, scale?: integer, filters?: object, criteria: string, threshold?: object, categories?: object[], model_name?: string, description?: string, return_type?: string, child_metrics?: object[], model_provider?: string, enabled_in_prod?: boolean, needs_ground_truth?: boolean, sampling_percentage?: number }
- `POST https://api.mcp.ai/api/honeyhive/create/model/event` — Tool to create a new model event to log LLM call data. Use when you need to track a single model interaction including messages, responses, usage, and metadata.
  - body: { model_event: object }
- `POST https://api.mcp.ai/api/honeyhive/create/tool` — Creates a new tool definition in a HoneyHive project. Use this to register functions or plugins that can be invoked and tracked within HoneyHive. Tools are defined with a JSON Schema for their paramet
  - body: { name: string, task: string, type: string, parameters: object, description?: string }
- `POST https://api.mcp.ai/api/honeyhive/delete/datapoint` — Tool to delete a specific datapoint by its ID. Use when you need to remove a datapoint from HoneyHive after confirming its identifier.
  - body: { id: string }
- `POST https://api.mcp.ai/api/honeyhive/delete/dataset` — Tool to delete a dataset by ID. Use when you need to remove a dataset after confirming its ID.
  - body: { dataset_id: string }
- `POST https://api.mcp.ai/api/honeyhive/end/evaluation/run` — Tool to update an evaluation run's status and metadata. Use to mark a run as completed after finishing evaluations, or update run properties like name, metadata, configuration, and associated event/da
  - body: { name?: string, run_id: string, status: string, metadata?: object, event_ids?: string[], dataset_id?: string, configuration?: object, datapoint_ids?: string[] }
- `POST https://api.mcp.ai/api/honeyhive/get/configurations` — Tool to retrieve a list of configurations. Use when you need to fetch all configurations for a specific project before making changes.
  - body: { env?: string, name?: string, project: string }
- `POST https://api.mcp.ai/api/honeyhive/get/datasets` — Retrieve datasets from HoneyHive for a specified project. Use this tool when you need to: - List all datasets within a project - Find datasets by type (evaluation or fine-tuning) - Retrieve a specific
  - body: { type?: string, project: string, dataset_id?: string }
- `POST https://api.mcp.ai/api/honeyhive/get/events` — Tool to query events with filters and projections from HoneyHive. Use this action when you need to retrieve events with lightweight filtering (limit 1000 results). For bulk exports or more complex que
  - body: { page?: integer, limit?: integer, filters?: string, dateRange?: string, ignoreOrder?: string, projections?: string, evaluationId?: string }
- `POST https://api.mcp.ai/api/honeyhive/get/events/by/session/id` — Tool to retrieve the complete tree of nested events for a specific session. Use when you need to analyze all events (model calls, tool calls, chains) that occurred within a session, including their hi
  - body: { id: string }
- `POST https://api.mcp.ai/api/honeyhive/get/events/chart` — Tool to retrieve charting and analytics data for events over time. Use when you need aggregated metrics (duration, cost, token usage) grouped by time buckets or fields. Supports percentile analysis (p
  - body: { bucket?: string, metric?: string, filters?: string, group_by?: string, date_range?: string, aggregation?: string, evaluation_id?: string, only_experiments?: string }
- `POST https://api.mcp.ai/api/honeyhive/get/metrics` — Retrieves all metrics associated with a HoneyHive project. Returns a list of metrics including their configuration (name, type, description, thresholds, evaluator details) and metadata (creation/updat
  - body: { project_name: string }
- `POST https://api.mcp.ai/api/honeyhive/get/projects` — Tool to retrieve all projects in the HoneyHive account. Use when you need to list available projects, get project IDs for use in other API calls, or search for a specific project by name.
  - body: { name?: string }
- `POST https://api.mcp.ai/api/honeyhive/get/run` — Tool to get details of an evaluation run by its UUID. Use when you need to check the status, configuration, results, or metadata of a specific evaluation run.
  - body: { run_id: string }
- `POST https://api.mcp.ai/api/honeyhive/get/run/metrics` — Tool to get event metrics for an experiment run. Use when you need to retrieve metrics computed on events within a specific experiment run. Returns an array of event objects with their associated metr
  - body: { run_id: string, filters?: string, dateRange?: string }
- `POST https://api.mcp.ai/api/honeyhive/get/runs` — Tool to retrieve a list of evaluation runs from HoneyHive. Use when you need to: - List all evaluation runs for analysis - Find runs by status, name, or dataset - Get specific runs by their IDs - Pagi
  - body: { name?: string, page?: integer, limit?: integer, status?: string, run_ids?: string[], sort_by?: string, dataset_id?: string, date_range?: string, sort_order?: string }
- `POST https://api.mcp.ai/api/honeyhive/get/runs/schema` — Tool to retrieve the schema for experiment runs in HoneyHive. Use when you need to understand available fields, datasets, and mappings for experiment runs.
  - body: { dateRange?: string, evaluation_id?: string }
- `POST https://api.mcp.ai/api/honeyhive/get/session` — Retrieve a complete session tree by session ID from HoneyHive. Use this tool to fetch the full session hierarchy including all nested events (model calls, tool calls, chains) with their inputs, output
  - body: { session_id: string }
- `POST https://api.mcp.ai/api/honeyhive/list/tools` — Tool to list all available Honeyhive tools. Use when you need to discover which functions or plugins are registered for use.
  - body: { page?: integer, limit?: integer, project?: string }
- `POST https://api.mcp.ai/api/honeyhive/retrieve/datapoint` — Retrieve a specific datapoint by its ID from HoneyHive. Use this tool when you need the full details of a single datapoint, including its inputs, ground truth, conversation history, linked datasets, a
  - body: { id: string }
- `POST https://api.mcp.ai/api/honeyhive/retrieve/datapoints` — Retrieve datapoints from a HoneyHive project. Use this tool to fetch evaluation datapoints containing inputs, ground truth, and metadata. Supports filtering by specific datapoint IDs or dataset name. 
  - body: { project: string, dataset_name?: string, datapoint_ids?: string[] }
- `POST https://api.mcp.ai/api/honeyhive/retrieve/events` — Retrieve and export events from a HoneyHive project. Use this tool to query traced events (model calls, tool calls, sessions, chains) with optional filters by event_type, metadata, feedback scores, or
  - body: { page?: integer, limit?: integer, filters?: object[], project: string, dateRange?: object, projections?: string[] }
- `POST https://api.mcp.ai/api/honeyhive/retrieve/experiment/result` — Tool to retrieve the result of a specific experiment run. Use when you need the status, metrics, and datapoint-level details of a completed experiment.
  - body: { run_id: string, project_id: string, aggregate_function?: string }
- `POST https://api.mcp.ai/api/honeyhive/start/evaluation/run` — Creates a new evaluation run to group and track multiple session events for analysis. Use this action when you want to: - Compare model performance across multiple sessions - Create evaluation batches
  - body: { name: string, status?: string, project: string, metadata?: object, event_ids: string[], dataset_id?: string, configuration?: object, datapoint_ids?: string[] }
- `POST https://api.mcp.ai/api/honeyhive/start/session` — Start a new HoneyHive session for tracing and observability. Use this tool to initiate a tracking session that groups together related model, tool, and chain events. Returns a session_id that should b
  - body: { session: object }
- `POST https://api.mcp.ai/api/honeyhive/update/configuration` — Tool to update an existing HoneyHive configuration. Use when you need to modify a configuration's name, provider, model parameters, environments, or other settings. You must provide the configuration 
  - body: { id: string, env?: string[], name: string, tags?: string[], type?: string, provider?: string, parameters?: object, user_properties?: object }
- `POST https://api.mcp.ai/api/honeyhive/update/datapoint` — Update an existing datapoint by ID. Use this to modify any combination of inputs, ground_truth, history, metadata, linked_datasets, or linked_evals for a datapoint. Requires a valid datapoint ID obtai
  - body: { id: string, inputs?: object, history?: object[], metadata?: object, ground_truth?: object, linked_evals?: string[], linked_datasets?: string[] }
- `POST https://api.mcp.ai/api/honeyhive/update/dataset` — Tool to update an existing dataset. Use when you need to modify a dataset's details (name, description, datapoints, linked evaluations, or metadata) after confirming its ID.
  - body: { name?: string, metadata?: object, datapoints?: string[], dataset_id: string, description?: string, linked_evals?: string[] }
- `POST https://api.mcp.ai/api/honeyhive/update/event` — Update an existing HoneyHive event by ID. Use to attach feedback, metrics, metadata, outputs, config, user properties, or update duration on events created via start_session or batch event creation. A
  - body: { config?: object, metrics?: object, outputs?: object, duration?: number, event_id: string, feedback?: object, metadata?: object, user_properties?: object }
- `POST https://api.mcp.ai/api/honeyhive/update/metric` — Tool to update an existing metric. Use when you need to modify a metric’s properties after creation. Ensure you retrieve the metric first to verify its current state.
  - body: { name?: string, type?: string, prompt?: string, criteria?: string, metric_id: string, pass_when?: boolean, threshold?: object, event_name?: string, event_type?: string, description?: string, return_type?: string, code_snippet?: string, enabled_in_prod?: boolean, needs_ground_truth?: boolean }
- `POST https://api.mcp.ai/api/honeyhive/update/project` — Updates an existing HoneyHive project's name or description. Use this action to modify project metadata after creation. You must provide the project_id and at least one field to update (name or descri
  - body: { name?: string, project_id: string, description?: string }
- `POST https://api.mcp.ai/api/honeyhive/update/tool` — Tool to update an existing tool in HoneyHive. Use when you need to modify a tool's name, description, parameters, or type after confirming its ID. At least one optional field must be provided alongsid
  - body: { id: string, name?: string, tool_type?: string, parameters?: object, description?: string }

## Example prompts
- "What can I do in Honeyhive?"
- "Show me a summary of my Honeyhive account"

## More
- Page: https://mcp.ai/honeyhive
- Agent spec (llms.txt): https://mcp.ai/honeyhive/llms.txt
- Postman collection: https://mcp.ai/honeyhive/postman.json