I built this system because I wanted to see how much of my normal AI workload I could realistically move off third-party providers and onto infrastructure I control.

So far, the answer is: a lot.

My ledger is currently sitting at:

50.6M total tokens tracked

48.9M processed locally

That number is a little misleading if you read it as “I generated 49 million tokens,” though. I didn’t.

For example, over a recent 24-hour period the system processed about 2.25M tokens, but only around 457k were generated output. Roughly 1.79M were input/context — code, research, logs, prompts, test results, and everything else the models were reading.

My generation speed is only around 12–14 tokens/sec.

That’s not fast. There’s also a very real physical throughput ceiling. Owning the hardware doesn’t magically remove physics.

What it does remove is the cloud-style quota.

There’s no monthly token allowance I’m trying to stay under and no meter charging me every time an agent needs more context. I can let the system work as much as the hardware allows and pay the electricity bill.

And that has changed how I’m using AI more than the raw performance has.

But as I keep testing this, my guess is that eventually only around 20% of my overall AI usage will need to go to third-party providers.

The other ~80% can probably be the boring, persistent stuff my local system is already good at: research, agents, code analysis, testing, automation, background jobs, and workloads where I simply don’t care if the answer takes longer.

That last part has actually been one of my favorite things about this experiment.

Local inference is slow enough that if I need something immediately, sometimes it’s faster for me to just open the project and code it myself.

And I like that.

AI handles the stuff I can throw into the background and let grind.

I get pulled back into actually building things when I want fast iteration.

I’m deliberately leaving the hardware and model names out because I’m not trying to make this a benchmark post. I’m still experimenting, changing things, breaking things, and figuring out what this setup is actually good at.