Why it’s safe to send agent traces

Agent traces hold the rawest data in your stack. The exact words a user typed, the model's reply, the arguments passed to every tool, the state that moved between spans. That’s precisely why teams hesitate to send them anywhere, and why “let’s not pipe trace data into another system” is the most common objection we hear about Agent Analytics.

So we built Agent Analytics so you can control what leaves your app, who can see it, and how long it lives. And the part most people miss: you keep the signals, evaluators, and cohorts even when the sensitive text never crosses the wire. Governance should not cost you the insight.

Who can see a trace: role-based access control

Access to Agent Analytics runs through Amplitude’s role-based access control. Three permissions gate the product:

• View Agent Analytics Objects lets a role open sessions, evaluators, and related objects. A role needs this to use the product at all.
• Manage Inactive Evals and Runs lets a role create evaluators and launch dry runs.
• Activate Evals lets a role turn an evaluator live or archive one.

You decide which roles carry which permission, so a support analyst can read sessions while only your ML leads can push an evaluator into production.

That same model governs our own AI features. They respect the permissions and data access controls already set up in your account. No new data access is granted to a user when an AI feature runs. An agent acting on your behalf can only touch data you are already authorized to see, and it cannot escalate past your permissions. Administrators can audit user inputs and agent outputs, so there is a record of what was asked and what was returned. You set access. It isn’t inherited.

What leaves your app: PII redaction and privacy modes

The real worry about traces is the free text. A user drops a phone number into a support chat, and now it’s sitting in your analytics. Agent Analytics handles this at the SDK level, before data leaves your infrastructure, through a privacy mode you set.

There are three modes:

• full is the default, and it sends prompt and response text with Personally Identifiable Information (PII) redaction on by default. This is completely configurable and extensible by customers. You can redact as needed beyond the out of the box PII scrubbing rules that the SDK ships with.
• metadata_only sends tokens, latency, model, and cost, and no text at all, which is the setting for regulated or sensitive workloads.
• customer_enriched sends pre-scored summaries from your own evaluation stack and nothing else.

Some examples of fields we redact by default are Email, Phone, Credit Card, SSN, IP Address. When these values are sent into Amplitude Agent Analytics, the resulting traces will show those as [email], [phone], [credit_card], [ssn], [ip_address] so the actual values are obfuscated.

The text-bearing fields: the message text, tool inputs and outputs, span input and output state, the system prompt, reasoning content, only ship when the privacy mode permits. Attachments come through as metadata only, never file content. Even tool definitions can stay private: we send a stable hash of your tool schema so you can detect a toolset change, without exposing the schema itself.

Agent Analytics doesn’t need PII to work. Redactions of PII values do not impact the actual out of the box signals or custom evaluations (evals). We run its always-on signals on every closed session: task completion, response quality, user friction, user intent, session safety, and a code-based data quality check. Session safety alone classifies each session as Normal, Off Topic, Prompt Injection, Abuse, Probing, Unsafe Output, or Data Leak. Your own evaluators, whether rule-based or LLM-as-judge, run on top. All of that enrichment reads structure and permitted fields, so you see that a billing session failed task completion and tripped a friction signal without ever seeing the customer’s number. You mask the sensitive value but keep the quality picture.

The trust layer underneath all of it

More than 4,900 companies and 26% of the Fortune 100 run on Amplitude, and over 2,000 trust it with their data. We run GDPR, CCPA, and HIPAA programs, will sign a Business Associate Agreement for covered entities, and self-certify under the EU-US Data Privacy Framework. Data lives in AWS, in the US or in the EU Frankfurt region if you need EU residency, on infrastructure certified to ISO 27001 and SOC 2 Type II. The full certification set is in the Customer Trust Portal.

The AI side is just as strict. We contractually prohibit our AI partners from training on your data. Workflows on AWS Bedrock keep your data inside Amplitude’s AWS environment, where the model providers never receive it. Workflows that use OpenAI or Google run through Zero Data Retention endpoints. If you provision in the EU data center, your data never leaves the EU when you use Amplitude AI.

The proof is in who already sends sensitive data here. ClearScore, which handles credit data for millions of people, is the case study Amplitude features on its own security page. Salt Bank built a digital bank on Amplitude and beat its launch targets by 2x. ACKO, a digital insurer, drove 13% more conversions and cut support calls in half with GenAI measured in Amplitude. The Economist Group put a hard number on its AI quality using the same approach Agent Analytics runs on.

Connecting agent performance to business outcomes

Keeping traces locked in a standalone observability tool feels safer, but it strands the one thing that matters most: the connection between agent quality and business outcomes. Because agent sessions share Amplitude user identity with your product events, you can build a cohort of users whose sessions failed task completion and chart their 30-day retention against users whose sessions succeeded. Pure observability tools cannot make that join. That is the reason to send the data, and the controls above are what make it safe.

We've open sourced the privacy/scrubbing patterns here and can expand on that over time to automatically package them in the SDK.