For AI agents: a documentation index is available at /docs/llms.txt. Append .md to any page URL for markdown, or send Accept: text/markdown.

Create an Agent Analytics evaluator

Creates an evaluator in the inactive state. Activation is a separate
permission and is not exposed by this API — activate in the Agent Analytics
UI once the evaluator has been reviewed.

tags constrains what a result may record. Supplying them for a binary or
categorical evaluator is what keeps results to a known set of values
rather than free-form model output.

Listing and reading evaluators is served by the Agent Analytics MCP read
surface, not by this API.

post/v1/projects/{project_id}/agent-analytics/evaluators

Authorizations

Authorizationstringrequired

http

Path parameters

project_idstringrequired

Amplitude project identifier, backed by the canonical app ID.

Example:12345
Constraints:pattern: ^[0-9]+$

Header parameters

Idempotency-Keystringrequired

Unique idempotency key used to safely retry write operations.

Constraints:min length: 8max length: 255

Body

application/json
topicstringrequired

Human-readable evaluator name.

Constraints:min length: 1max length: 255
output_typestringrequired
Allowed values:binarycategoricalscore
descriptionstringrequired

Evaluator prompt or rubric.

Constraints:min length: 1
tagsobject[]

Required for binary and categorical evaluators.

Show child attributes
idstringrequired
labelstringrequired

Value stored on a result for this tag, such as RESP_DATA_RESPONSE_MISMATCH.

descriptionstringrequired
suite_idstring
Constraints:nullable

Response

application/json
201· Evaluator created, inactive.
dataobjectrequired
Show child attributes
idstringrequired

Identifier for this evaluator version. Changes when the evaluator is updated.

objectstringrequired
project_idstringrequired
stable_idstringrequired

Identifier that survives updates. Use this to address an evaluator across versions.

topicstringrequired

Human-readable evaluator name. Called topic to match the field already returned by the Agent Analytics read surface.

Constraints:min length: 1
output_typestringrequired
Allowed values:binarycategoricalscore
descriptionstringrequired

Evaluator prompt or rubric.

suite_idstring
Constraints:nullable
suite_namestring
Constraints:nullable
tagsobject[]required

Permitted output values. Required for binary and categorical evaluators; constrains what a result may record.

idstringrequired
labelstringrequired

Value stored on a result for this tag, such as RESP_DATA_RESPONSE_MISMATCH.

descriptionstringrequired
is_activebooleanrequired

Read-only. Evaluators created or updated through this API are inactive. Activation is a separate permission and is not exposed here.

Was this helpful?