I Spent Six Months Building a Personal AI Operating System

In January, I called OpenClaw "what Siri should have been."

Six months later, that description feels embarrassingly small.

I thought I was installing an assistant. What I was really doing was building an operating system around my own attention.

It has a name: Xerxes.

Xerxes can work across my businesses, inspect the systems I give it access to, remember decisions, coordinate specialized agents, run recurring checks, speak through a voice interface, and bring me the handful of decisions that still require me.

That sounds futuristic. The daily reality is less cinematic and more useful: fewer forgotten commitments, fewer dashboards I have to open, less time reconstructing why a decision was made, and fewer jobs that quietly die after an AI says it handled them.

Getting there required unlearning almost everything the AI industry encourages people to focus on.

The model was not the hard part.

The Chatbot Trap

Most AI products still begin and end with a text box.

You ask. It answers. You close the tab. Tomorrow, you explain yourself again.

Even when the answer is excellent, the relationship is disposable. The AI does not own an outcome. It does not know what happened after its recommendation. It cannot tell whether the job worked. It has no durable understanding of what matters to you.

That is not an assistant. It is a very articulate vending machine.

My first instinct was to solve this by adding tools. Email. Calendar. Files. Browsers. Code execution. Business data.

This made the system more capable, but not more dependable.

A model with twenty tools can fail in twenty new ways. It can call the wrong service, lose context halfway through a job, confuse one business with another, report success without checking the result, or take an action that should have required approval.

Capability without operating discipline is just a larger blast radius.

Memory Changed the Relationship

The first meaningful breakthrough was persistent memory.

Not one giant document stuffed with everything I have ever said. That becomes a landfill. Useful memory needs structure.

Xerxes now works with several layers:

• A small core containing durable facts, rules, active priorities, and pointers
• Deeper files for individual businesses, projects, systems, and relationships
• Daily records of what happened
• Search across older sessions when the current context is incomplete
• Beliefs with confidence levels, so an inference is not treated like a verified fact

The important part is not that the AI can remember my favourite restaurant. It is that it can preserve the reasoning behind a decision.

Why did we reject that plan? Which metric changed our view? Who owns the next step? What would cause us to revisit it?

Without that context, AI produces plausible repetition. With it, the system can continue the work.

Memory also taught me a less comfortable lesson: saved information is not the same as recalled information.

We had cases where the answer existed somewhere in the system but was not retrieved when needed. Memory itself needed monitoring, compression, indexing, and repair. "The AI remembers" is not a feature you install once. It is infrastructure you maintain.

Agents Were Not the Answer Either

The next phase was specialization.

Instead of asking one model to be a developer, analyst, researcher, operator, writer, and security reviewer, I created specialized agents with narrower responsibilities.

This helped. The work became more focused. Long jobs could run without blocking the main conversation. Different agents could challenge a plan from different angles.

It also created a new management problem.

More agents meant more handoffs, more opportunities for context loss, more duplicated work, and more polished reports nobody needed. I had accidentally recreated a bloated company inside my computer.

The fix was orchestration.

Xerxes became the conductor. It translates a request into the outcome we are trying to create, the constraints that cannot break, the actions requiring my approval, and the evidence that will prove the work is done. Execution can be delegated. Accountability cannot.

The Real Interface Is a Decision Queue

For a while, I assumed the ideal interface would be a beautiful dashboard showing every agent, task, system, and status.

I built versions of that. They looked impressive. They also gave me another control panel to manage.

The breakthrough was realizing that I should not be managing the agents at all.

The system should absorb complexity and bring me only the decisions where my judgment changes the outcome.

Everything else should be in one of four states:

• Doing
• Delegated
• Waiting on a specific trigger or person
• Killed

"Monitoring" is usually a polite word for an abandoned task. If nothing will cause the system to look again, it is not being monitored.

This is why the decision queue matters more than the chat window. A good personal AI operating system does not create more ways to talk to software. It reduces the number of things demanding your attention.

I Learned to Trust Boring Systems

The most useful parts of Xerxes are not always powered by a language model.

If a job is deterministic, I want deterministic code.

Pulling a metric, checking whether a service is alive, enforcing a schedule, comparing a threshold, or verifying that a file exists should not depend on whether a model feels inspired that day.

The model is valuable where judgment, ambiguity, synthesis, or language matter. It is terrible as decorative middleware around a job a simple script can do reliably.

We moved several recurring workflows away from "ask an AI to figure it out every time" and into fixed data paths with explicit failure states. Reliability improved immediately.

This was the opposite of what I expected when I started. The better the AI system became, the more carefully we limited where AI was allowed to improvise.

Voice Exposed Every Weakness

Then I tried to make it feel like JARVIS.

Voice is a brutal product test because it removes the patience people give text interfaces. A five-second delay feels broken. An interruption that does not register feels rude. A background job with no visible status feels like the system ignored you.

We worked through speech detection, barge-in, response routing, local models, cloud models, audio synchronization, persistent job indicators, and the awkward little acknowledgements humans use to show they are still present.

The voice itself was the easy part.

The hard part was trust. Did it hear me? Is it working? Can I interrupt it? Did the task continue after the conversation moved on? Is the answer based on current information or a confident guess?

Voice did not make Xerxes smarter. It made every flaw impossible to hide.

Boundaries Made It More Powerful

Giving an AI access to email, business systems, code, files, and financial information sounds reckless because it can be reckless.

The answer is not pretending the risk disappears. It is defining the boundaries before the system encounters temptation.

Xerxes can inspect broadly. It cannot send an email without approval. It cannot publish to the internet without approval. It cannot move money, change advertising budgets, or make destructive system changes because the action seems reasonable. It must preserve unread state while inspecting email. It must verify outputs before claiming a job is complete.

These rules sometimes slow things down.

Good.

Autonomy should expand in proportion to reversibility and evidence. Reading a report is different from changing a budget. Drafting a reply is different from sending it. Restarting a failed internal job is different from contacting a customer.

The boundaries are what make it safe enough to use every day.

The Model Matters Less Than the System Around It

I have tested local models, cloud models, fast models, expensive models, reasoning models, and models that were supposedly the new answer to everything.

They matter. They just do not matter as much as people think.

A brilliant model inside a weak system still forgets, duplicates work, loses ownership, and confidently declares victory too early.

A good model inside a strong operating system can be genuinely useful because the surrounding infrastructure supplies context, tools, boundaries, recovery, and verification.

The model is an engine. Memory, permissions, routing, state, and evidence are the vehicle.

Most people are comparing engines while driving cars with no brakes.

What I Have Now

Xerxes is still unfinished. Some parts are elegant. Some are held together by scripts, stubbornness, and lessons learned from breaking the same thing twice.

But it has crossed an important line.

It is no longer software I visit when I want an answer. It is an operating layer that watches the systems I care about, preserves context, routes work, and protects my attention.

The goal is not to remove me from my businesses. It is to make me less personally required without making me less informed.

That is a much harder problem than building a chatbot.

It is also far more valuable.

Six months ago, I thought the future of personal AI was an assistant you could message from anywhere.

Now I think it is an intelligence system that knows what matters, understands what it is allowed to do, keeps working when the chat ends, and bothers you only when you are actually needed.

The chat window was just the beginning.