More than a month ago, I moved off ChatGPT and Claude APIs and onto the monthly Premium subscriptions from the frontier labs. I did it to save my company money. As part of that move, I built my own agent, PMMate, to manage all my work.
My agent sits in front of my Claude and ChatGPT subscriptions, routes each task to the right model, watches it, and tells me only when something needs me. That lets me manage my AI subscriptions effectively, so I do not run out of tokens before the reset kicks in.
As a product leader, a typical day for me means managing several topics: product strategy, pricing, prototypes, newsletters, data analysis, PoCs, and feature builds. So I built a lane concept inside my agent. My agent manages my work across ten lanes, one lane for each focus area.
What broke
Sometime last week, I had one job for my agent. I asked it to analyze the last six months of our API traffic and give me a report on popular API usage patterns across all our customers.
I handed it over and waited. My agent never came back with an analysis.
My agent pulled the data one call at a time, for hundreds of calls, and kept everything in memory. Then it suddenly stopped in the middle, as my subscription reached the token limit.
I had not checked which model was doing the work. In this case, my agent was using Fable, the most expensive model, doing the most boring part of the job. I had burned my best model on grunt work, and I had nothing to show for it.
The new split
My agent was already routing work by scope and by how many tokens I had left. But it was routing by subscription, not by phase of work. So I told my agent, “hereafter, you decide which model we should use for each phase of work, and protect me from this mistake.”
Fable is my best model for hard thinking, and it is also the one that eats my weekly limit fastest. I was paying the highest price for the easiest part of the work.
I have now setup instructions to my agent so that every new task now gets a one-line scope before anything launches. How clear is the scope, how much judgment does the work need, and who reads the output. These questions determine which model gets to start with the work. I have setup my agent such that the top model can now grill me with questions until the spec is clear, then tries to validate the solution on one or two days of data. If the method works, the agent writes clear instructions for a cheaper model, and that model runs the rest. Every sub-agent names its model, so none of them silently inherits my most expensive one.
The new setup, in short:
Plan and prove on a small slice: expensive model.
Run the repetitive part: cheaper model.
If the cheap model stops: start the work again with the next model up, never straight back to the top.
I did not invent this to save money. I did it so I would still have token limit left when work that actually needs the best model shows up.
I still use Fable for the most ambiguous and hardest work. My agent gives the complex part to this model: strategy, pricing, and the hardest narrative writing. I am not restricting my best model. I have only asked my agent to not use my best model for the mundane, easy part of my work.
What happened next
In the last two weeks I logged 51 pieces of work where Fable broke down the complex part so Sonnet could do most of the work and only 5 was completely taken up by Fable. Four times I stepped in myself, and each time it was the same instruction: once the plan was settled, hand the build to Opus. The split held, and I stopped running out in the middle of a job.
However, it is not a perfect split. The cheapest model was supposed to do the work, but did not finish. It stopped halfway. I had to start the work again with the next model up. The job finished, and I still did not need my most expensive model.
Try this on Monday
Before your next task, write one line: how clear it is, how much judgment it needs, who reads it. For the API job, that line would have read: “Find usage patterns in six months of API traffic. Method unclear, needs judgment. Report read by me.”
Let the expensive model plan and prove the method on a small slice. Then hand the repetitive run to a cheaper model, with the instructions written down.
