Docs/Llmops/Datasets

Datasets

A dataset is a named collection of test cases. Each item has an input, an optional expected output and optional metadata. You run experiments against a dataset to compare prompts, models or versions of your application on the same inputs.

ℹ

Open it from Gauge AI → Datasets. The sidebar shows Datasets to members who manage every dataset in the project (Owner, Governance Owner, or a custom role with Admin) and to members who own at least one AI agent in the project. Other members with dataset access can still open the page at /project/<projectId>/datasets. See Who can do what.

Who can do what

RoleDatasetsExperiments
Owner, Governance OwnerEvery dataset in the projectStart prompt and remote experiments
Security & Compliance, Cost Analyst, End User (and the discontinued Project Admin and Manager Viewer)Create datasets. Edit, duplicate and delete their own datasets, add and change items, upload CSV and delete runs on them.View runs. Cannot start experiments.
Governance Viewer, AuditorNo accessNo access
  • A dataset belongs to the member who created it in the app. Other members with these roles cannot see it.
  • Datasets with no recorded creator, such as datasets created before creators were recorded or through the public API, are visible to these roles but read-only for them.
  • Starting an experiment calls a model with the project's LLM connection, so among the built-in roles only Owner and Governance Owner can do it.
  • Datasets are part of AI governance. A member with only a Budget AI role has no dataset access.
  • Custom roles use the Datasets and experiments module: Read & Write gives the own-dataset access above, and Admin gives every dataset and starting prompt and remote experiments. See Custom Roles.

Create a dataset

Create it

On the Datasets page select New dataset. Enter a Name, an optional Description and optional Metadata (JSON). A / in the name groups datasets into folders, for example support/regression.

From code:

python
from ants_platform import get_client ants = get_client() ants.create_dataset( name="support/regression", description="Questions that broke the support agent", )

Add items

Add items in any of these ways:

  • By hand. On the dataset, select Create new dataset item and enter Input, Expected output and Metadata as JSON.
  • From CSV. Select Upload CSV, map each column to input, expected output, metadata or not mapped, check the preview and import. Files can be up to 10 MB.
  • From a trace. On a trace or observation, select Add to datasets (or Add to more datasets when it is already in one), pick one or more datasets from the list, which opens at the width of the field, and select Add to dataset or Add to datasets. The input and output are copied into a new item, which keeps a link to its source trace. To add to a dataset that does not exist yet, select Create new dataset in the form: the new dataset is selected for you and you stay on the trace.
  • From code or the API. Use the SDK or POST /api/public/dataset-items.
python
ants.create_dataset_item( dataset_name="support/regression", input={"question": "How do I reset my password?"}, expected_output={"answer": "Use Forgot password on the sign-in page."}, metadata={"category": "account"}, )

Run an experiment

Select New experiment on the dataset. See Experiments.

Item fields

FieldRequiredNotes
inputYesAny JSON. For prompt experiments, its keys must match the prompt's variables.
expected_outputNoThe ideal answer, used by evaluators and your own checks.
metadataNoAny JSON, such as category or difficulty.
source_trace_id, source_observation_idNoSet when the item was created from a trace.
statusNoACTIVE or ARCHIVED. Prompt experiments skip archived items. In your own code, check item.status.

Pass id= to create_dataset_item to update an existing item instead of creating a new one.

The item form in the app warns when Input is empty, because such an item cannot run in an experiment. Items added from chat or Budget AI traces often have a list of messages as input, which prompt experiments cannot use. For those, the form offers Wrap as {"messages": [...]}: use it with a chat prompt that has a message placeholder named messages.

Read a dataset in code

python
dataset = ants.get_dataset("support/regression") for item in dataset.items: print(item.id, item.input, item.expected_output)

To run your own application over the items and record the results as a run, see Custom experiments.

Runs and comparison

Each experiment creates a run on the dataset. Open the dataset's Runs tab to see them. The runs table shows, per run, the number of run items, average latency, average total cost, aggregated run item scores and run-level scores. Select runs and choose Compare to see outputs and scores for each item side by side. Each run item links to its trace.

Other actions

Row actions in the datasets, items and runs tables, and the page actions on a dataset, item or run, open from the horizontal ellipsis (⋯) button.

  • Duplicate dataset copies a dataset and its items.
  • Edit changes the name, description and metadata.
  • Delete a dataset, an item or a run. Deleting cannot be undone.
  • Export on the items table exports the items as CSV, JSON or JSONL. It needs data export access (Owner, Governance Owner, Cost Analyst among the built-in roles). Only members with the content permission (Owner) get the input, expected output and metadata; for everyone else the menu shows Content excluded and those fields are left empty. Roles without data export access see "Export is not available for your role. Ask an admin to export this dataset for you." See Data Exports.
  • Datasets, items and runs are also available through the public API. See API Reference.

Next steps

© 2026 ANTS Platform, Inc.Docs v1.0 · Last updated June 2026