From PoC to MVP: When Is It Done?

Implementation is moving forward. It’s slow in some places, but I think it will pay off in the long run.

I recently wrote a small LinkedIn post about the definition of done and how AI changes the way we should think about it.

Moving From PoC to MVP

About 10 days ago, the desired cibot architecture moved from a PoC much closer to an MVP.

We now have three main parts:

• a control plane daemon, with the web app, gRPC gateway, and WebSocket server
• an orchestration daemon, responsible for creating runners that execute your code and agents
• the runners themselves, which can run locally or inside a VM

The architecture change added about 48,000 lines of production Go, 23,000 lines of Go tests, 16,500 lines of E2E code, and 9,700 lines of generated Go. The E2E suite grew from 25 cases to 55.

With this architecture, I also intentionally made the implementation more complicated by adding a proper security layer.

Each control plane installation generates its own root CA certificate. From that CA, it generates short-lived mTLS certificates for server services, the orchestration daemon, and individual runners.

Those certificates have short expiration times and are renewed early enough that there should always be enough time in reserve.

And it worked.

Testing Certificate Rotation

But does “it works” mean it is done? Not really.

There are many edge cases around something like certificate rotation.

• What happens to an active connection when a certificate is renewed?
• What happens if the root CA itself is rotated?
• What happens if a certificate expires while a runner is communicating with the control plane?
• What happens if a rotation happens in the middle of sending data?

Testing all of this manually is basically impossible. E2E tests are the only reasonable way to cover it.

Fortunately, AI is quite happy to write those tests. Unlike humans.

The full test suite for this part now takes around 30 minutes to run.

Long? Yes. Worth it? Also yes.

Growing the Test Matrix

The minimum renewal timeout alone is around seven minutes, so some of that time is simply spent waiting for real certificate lifecycle events to happen.

And the test matrix grows quickly because cibot can now run in multiple modes. For example:

• without additional protection
• with a passphrase that encrypts sensitive data stored in the database

In passphrase mode, the control plane will not accept traffic until it has been unlocked.

So our 30-minute test suite becomes roughly 30 minutes × 2 modes. Once we add MariaDB support, it will become roughly 30 minutes × 4 combinations.

Refactoring the New Architecture

And testing is only one part of the work.

Some of it was messy. Some parts still carried assumptions from the original PoC. There was duplicated logic, duplicated strings, raw SQL that should no longer be raw SQL.

So several days were spent mostly reading code and doing refactoring rounds: removing duplication, replacing repeated strings with shared definitions, moving raw SQL to Jet, cleaning up old PoC assumptions, and finding places where the same problem had been solved multiple times in slightly different ways.

It is less exciting than adding a completely new feature, but the result is code that feels much more maintainable than before. Even though there is now significantly more of it. And even more to improve…

Across the refactoring, about 17,600 Go lines were added and 5,700 removed, leaving about 11,900 more Go lines than before. The additions include around 7,200 production lines, 5,500 test lines, 330 E2E lines, and 4,600 generated lines. The documentation is about 760 lines shorter. In total, 64 commits separate the new orchestration from this refactoring.

Moving Toward the UI We Envisioned

And we did not stop there. We have also started working on the UI and moving away from the quick MVP interface toward the UI we originally envisioned. Current PoC interface:

The new UI is built around the idea that cibot should give you one place to understand what is happening across all of your terminal and agent sessions.

A few important parts of that:

• the SPA will track what is happening across all terminal and agent sessions
• each session can contain multiple windows
• you will get notifications when something is waiting for your response and you are currently somewhere else

We have also created the cibot companion, which helps you to see what is currently running and what needs your attention.

Deciding When It Is Done

There is still a lot of work ahead, but this is also the part of development I don’t want to rush.

AI makes it incredibly easy to get from an idea to something that works.

The harder question is deciding when that something is actually done.