# Building CLI Agents: It’s What You Don’t Give Them That Counts In the last month of developing Amplitude Wizard CLI, we cut more than we added. Learn less is more when it comes to building CLI agents. Source: https://amplitude.com/en-us/blog/building-cli-agents --- [Kelson Warner](/blog/author/kelson-warner) [Engineering Manager, Amplitude](/blog/author/kelson-warner) [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Famplitude.com%2Fblog%2F%2Fblog%2Fbuilding-cli-agents)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Famplitude.com%2Fblog%2F%2Fblog%2Fbuilding-cli-agents)[](https://twitter.com/intent/tweet?url=https%3A%2F%2Famplitude.com%2Fblog%2F%2Fblog%2Fbuilding-cli-agents\&text=Building%20CLI%20Agents%3A%20It%E2%80%99s%20What%20You%20Don%E2%80%99t%20Give%20Them%20That%20Counts)[](mailto:?subject=Checkout%20this%20Amplitude%20Article\&body=Check%20this%20out%3A%20https%3A%2F%2Famplitude.com%2Fblog%2F%2Fblog%2Fbuilding-cli-agents) *A version of this post originally appeared on [Kelson's Twitter](https://x.com/kelsonpw/status/2049948090233491912).* My team shipped [@amplitude/wizard](https://www.npmjs.com/package/@amplitude/wizard) on April 30. One simple command, run from your terminal, and you get a fully instrumented Amplitude project: framework detection, SDK install, custom event tracking written into your code, ingestion verified before the agent says done. Because the drive was for simplicity, my engineering team actually spent most of our time removing things from the Wizard CLI. Tools the agent shouldn’t have. Context that made it worse. A mega-prompt that felt comprehensive and performed terribly. At the end of it, the Wizard we shipped is actually fourteen subagents pretending to be one tool. The architecture makes it feel like one tool instead of fourteen. We only got there by rigorously setting constraints. Here are some of the things we thought about as we worked to make our CLI agent simpler in case you’re building one of your own. ```typescript npx @amplitude/wizard ``` ## A tracking plan as simple as an SDK install Our [launch post](https://amplitude.com/blog/amplitude-wizard-cli) does a good job explaining the problem we were trying to solve with the Wizard CLI and what we built. But to summarize. Getting started with analytics is hard. The problem isn’t the install; SDKs have been a one-line install for years. It’s everything after that, where you figure out which events and properties you actually need to track. We’ve watched thousands of teams get stuck here. But if you take a step back, what’s hard about a tracking plan? The whole point of analytics is to answer questions about your product. The question of what to track is a question about your product. Your code is your product, written down. It describes exactly what’s worth tracking, in language more accurate than any docs page or onboarding wizard. So we built something that reads your code. *npx @amplitude/wizard* runs an agent in your terminal that walks your repo, proposes events that match how your product actually works, shows you the plan, and writes the calls inline once you approve. Not generic page\_viewed. Actual events, named in your product’s vocabulary, in the files where they belong. End-to-end, it does auth, framework detection, SDK install, tracking plan, Session Replay + Experiment + Guides and Surveys, API key, verification, and the MCP server. Nothing lands in your tracked code until you approve it, though. Every change is shown as a diff. Revert anytime. ## Big things that didn’t make the cut Letting the agent pick its own tools. It picked Bash for everything. Always. Including things that obviously needed a structured file edit. We narrowed each subagent’s tool set explicitly. Hallucinated git calls dropped to zero. We started with one model doing it all. That didn’t work. The latest Sonnet models ended up being the right supervisor. Narrow subagents handle classification, file walks, and plan generation. Each one runs hot, dies fast, and returns a structured result. The supervisor never sees the subagent’s reasoning trace, just the validated output. The Wizard installs an MCP server so users can ask their editor to do tasks with Amplitude, like “Add tracking to the new checkout flow,” or “What events fired on the sign-up page yesterday?” MCP is the only connector standard we tried that didn’t fall apart in testing. We’re going forward with it. ## The detail that took weeks: Polling Oh, the polling loop. This took four rewrites. When the agent finishes writing track() calls, we don’t trust that the install worked. We hit the Amplitude ingestion API and wait until the first event lands. If it doesn’t, we surface what’s wrong instead of saying “Done” and leaving you to discover at 11 p.m. three days later that nothing was flowing. The current version is uglier than we’d like. It works. It’s the kind of thing nobody notices when it works and would be furious about if it didn’t. Most setup tools skip it. We didn’t. ## Three things the Wizard taught us Building good agents is mostly about deciding what NOT to give them. Tools, files, instructions, history. Less is the entire trick. For instructions, context is a budget, not a buffer. Every token you put into a prompt is a token of attention you’re spending. We removed more instructions than we added in the last month of development. Controlling context also means constraining scope. Parallelism is great, but subagents really need isolation to be effective. One job, one context, one thing to be good at. When something fails, the supervisor’s context is still clean. Blast radius matters more than throughput. Finally, don’t give agents too many choices with how to do their job. A little flexibility is fine to reduce instruction, but if you’re too open-ended, they’ll flounder. Give guidance on tools. ## Standing on spiky shoulders Hat tip to PostHog. Their [engineering blog](https://posthog.com/blog/correct-llm-code-generation) showed that writing honestly about what you built, what broke, and what you’d do differently is a better growth channel than any landing page. We took notes. Their open-source CLI was an early inspiration for the Wizard, too. If you’re building agents, their work is worth reading alongside ours. ## Where this is heading (headless-ing?) Today’s Wizard is iteration 1. The obvious flows are great, but the hard taxonomy work isn’t done yet. We shipped because it’s already better than what was there, and real feedback beats waiting for more polish. The Wizard is one command. What’s next depends on an API-first mindset across the whole product. The CLI is the surface. Everything underneath has to be reachable, scriptable, and agent-driveable. That’s where my team is going. Try the Amplitude Wizard CLI: *npx [@amplitude/wizard](https://x.com/@amplitude/wizard)*. Read more at [amplitude.com/builders](https://amplitude.com/builders). File issues on [GitHub](https://github.com/amplitude/wizard). Use /feedback inside the wizard. Or email wizard@amplitude.com. That goes to the team, and we read everything. Tell us what could be better, and we’ll keep working to fix it. About the author Kelson Warner Engineering Manager, Amplitude [More from ](/blog/author/kelson-warner) [Kelson](/blog/author/kelson-warner) Kelson Warner is an Engineering Manager at Amplitude, where he heads up the Growth Engineering team. He focuses on making the product easier to get started with and faster to deliver value, helping users quickly connect with Amplitude’s insights. Outside of work, Kelson enjoys spending time with his family and his cats, and loves outdoor activities like camping, hiking, and exploring nature. Topics [AI](/blog/tag/artificial-intelligence) [Amplitude AI](/blog/tag/amplitude-ai) [Engineering](/blog/tag/engineering) [Getting Started](/blog/tag/getting-started) #### Recommended Reading [Read ](/blog/three-prompting-tips-AI) [Insights](/blog/three-prompting-tips-AI) ###### [Three Tips for Better Prompts in Amplitude Global Agent](/blog/three-prompting-tips-AI) [May 26, 2026](/blog/three-prompting-tips-AI) [9 min read](/blog/three-prompting-tips-AI) [Read ](/blog/ai-took-data-analyst-job-better) [Customers](/blog/ai-took-data-analyst-job-better) ###### [How AI Took the Data Analyst’s Job, and Created a Better One](/blog/ai-took-data-analyst-job-better) [May 22, 2026](/blog/ai-took-data-analyst-job-better) [8 min read](/blog/ai-took-data-analyst-job-better) [Read ](/blog/default-prompts-agent-retention) [Insights](/blog/default-prompts-agent-retention) ###### [Default Prompts Are Tanking Your Agent’s Retention](/blog/default-prompts-agent-retention) [May 21, 2026](/blog/default-prompts-agent-retention) [6 min read](/blog/default-prompts-agent-retention) [Read ](/blog/web-vitals-agents) [Product](/blog/web-vitals-agents) ###### [Optimizing Core Web Vitals with Amplitude’s Global Agent](/blog/web-vitals-agents) [May 20, 2026](/blog/web-vitals-agents) [4 min read](/blog/web-vitals-agents)