Add Honeyhive to your AI agent

Tool to update an evaluation run's status and metadata. Use to mark a run as completed after finishing evaluations, or update run properties like name, metadata, configuration, and associated event/datapoint IDs.

Tool to retrieve a list of configurations. Use when you need to fetch all configurations for a specific project before making changes.

Retrieve datasets from HoneyHive for a specified project. Use this tool when you need to: - List all datasets within a project - Find datasets by type (evaluation or fine-tuning) - Retrieve a specific dataset by its ID Returns dataset details including name, description, datapoints count, type, and timestamps.

Tool to retrieve the complete tree of nested events for a specific session. Use when you need to analyze all events (model calls, tool calls, chains) that occurred within a session, including their hierarchical relationships, inputs, outputs, metrics, and costs. Returns a tree structure with recursive children.

Retrieves all metrics associated with a HoneyHive project. Returns a list of metrics including their configuration (name, type, description, thresholds, evaluator details) and metadata (creation/update timestamps, sampling settings). Use this tool when you need to: - List all metrics configured for a project - Get metric IDs for updating metrics via HONEYHIVE_UPDATE_METRIC - Understand what evaluations are set up for a project Prerequisites: Obtain a valid project_name using HONEYHIVE_GET_PROJECTS first.

Tool to retrieve all projects in the HoneyHive account. Use when you need to list available projects, get project IDs for use in other API calls, or search for a specific project by name.

Tool to get details of an evaluation run by its UUID. Use when you need to check the status, configuration, results, or metadata of a specific evaluation run.

Tool to get event metrics for an experiment run. Use when you need to retrieve metrics computed on events within a specific experiment run. Returns an array of event objects with their associated metrics, which can be filtered by date range or custom filters.

Tool to retrieve a list of evaluation runs from HoneyHive. Use when you need to: - List all evaluation runs for analysis - Find runs by status, name, or dataset - Get specific runs by their IDs - Paginate through large sets of evaluation runs Returns evaluation details including status, results, configuration, and timestamps.

Retrieve a complete session tree by session ID from HoneyHive. Use this tool to fetch the full session hierarchy including all nested events (model calls, tool calls, chains) with their inputs, outputs, durations, and metadata. Returns a recursive tree structure with aggregated metrics. Prerequisites: You need a valid session ID from HONEYHIVE_START_SESSION or HONEYHIVE_RETRIEVE_EVENTS.

Tool to list all available Honeyhive tools. Use when you need to discover which functions or plugins are registered for use.

Retrieve a specific datapoint by its ID from HoneyHive. Use this tool when you need the full details of a single datapoint, including its inputs, ground truth, conversation history, linked datasets, and metadata. Prerequisites: You need a valid datapoint ID. Get datapoint IDs from: - HONEYHIVE_RETRIEVE_DATAPOINTS (list datapoints by project/dataset) - HONEYHIVE_ADD_DATAPOINTS_TO_DATASET (returns IDs of newly created datapoints)

Retrieve datapoints from a HoneyHive project. Use this tool to fetch evaluation datapoints containing inputs, ground truth, and metadata. Supports filtering by specific datapoint IDs or dataset name. Commonly used to: - Review existing test cases before running evaluations - Export datapoints for analysis - Verify datapoint contents after adding them to a dataset

Retrieve and export events from a HoneyHive project. Use this tool to query traced events (model calls, tool calls, sessions, chains) with optional filters by event_type, metadata, feedback scores, or date range. Returns events with their inputs, outputs, duration, and metrics. Supports pagination for large result sets (max 7500 per page).

Tool to retrieve the result of a specific experiment run. Use when you need the status, metrics, and datapoint-level details of a completed experiment.

Creates a new evaluation run to group and track multiple session events for analysis. Use this action when you want to: - Compare model performance across multiple sessions - Create evaluation batches for quality assurance - Link existing events to datasets for structured evaluation Prerequisites: - Get project ID using Get Projects action - Get event IDs from Start Session or Retrieve Events actions - (Optional) Get dataset ID from Get Datasets action

Start a new HoneyHive session for tracing and observability. Use this tool to initiate a tracking session that groups together related model, tool, and chain events. Returns a session_id that should be used to link subsequent events to this session. Common use cases: - Start tracing a user conversation - Begin logging an LLM pipeline execution - Initialize observability for a batch processing job

Tool to update an existing HoneyHive configuration. Use when you need to modify a configuration's name, provider, model parameters, environments, or other settings. You must provide the configuration ID (obtainable via Get Configurations action) and the name field. All other fields are optional and will only update if provided.

Update an existing datapoint by ID. Use this to modify any combination of inputs, ground_truth, history, metadata, linked_datasets, or linked_evals for a datapoint. Requires a valid datapoint ID obtained from retrieve_datapoints or add_datapoints_to_dataset.

Tool to update an existing dataset. Use when you need to modify a dataset's details (name, description, datapoints, linked evaluations, or metadata) after confirming its ID.

Update an existing HoneyHive event by ID. Use to attach feedback, metrics, metadata, outputs, config, user properties, or update duration on events created via start_session or batch event creation. At least one optional field must be provided alongside the event_id.

Tool to update an existing metric. Use when you need to modify a metric’s properties after creation. Ensure you retrieve the metric first to verify its current state.

Updates an existing HoneyHive project's name or description. Use this action to modify project metadata after creation. You must provide the project_id and at least one field to update (name or description). To find project IDs, use the Get Projects action first.

Tool to update an existing tool in HoneyHive. Use when you need to modify a tool's name, description, parameters, or type after confirming its ID. At least one optional field must be provided alongside the required tool ID.

Tool to create multiple model events in a single request. Use when you need to log a batch of event interactions to HoneyHive.

Tool to log a batch of external API calls as tool events. Use when you need to record multiple tool events in one request—use after gathering all event data.

Tool to create a new model event to log LLM call data. Use when you need to track a single model interaction including messages, responses, usage, and metadata.

Tool to query events with filters and projections from HoneyHive. Use this action when you need to retrieve events with lightweight filtering (limit 1000 results). For bulk exports or more complex queries, use the Retrieve Events action instead. Supports filtering by date range, event properties, and field projections.

Tool to retrieve charting and analytics data for events over time. Use when you need aggregated metrics (duration, cost, token usage) grouped by time buckets or fields. Supports percentile analysis (p50, p95, p99) for latency monitoring and custom filters for targeted analytics.

Tool to retrieve the schema for experiment runs in HoneyHive. Use when you need to understand available fields, datasets, and mappings for experiment runs.