Testing AI-Generated Code in a Shared Kubernetes Environment

Research on AI productivity have been finding out that while AI coding tools measurably increase code output, they don’t produce a proportional increase in code merged to production. The gap is the feedback loop, specifically in how long it takes to find out whether generated code actually works in a real environment. In our previous post, we looked at how remocal testing — running your service locally while connected to all of its dependencies in a real Kubernetes cluster — closes that loop for individual developers. But closing it for a team is a different problem entirely: if multiple developers and/or AI agents are all running local services against the same staging cluster, what prevents them from interfering with each other?

The shared environment problem #

When an AI agent tests generated code against staging, it interacts with other services in the cluster. It sends requests, consumes messages from queues, and reads and writes to databases. In a shared cluster, those interactions can collide.

A few specific scenarios:

• Traffic collisions: Two developers both have a locally running instance of the same service to test out code generated by their AI agents. A request comes into the cluster, which they both want redirected to their locally running process to test their code. But which local instance gets it?
• Queue interference: An AI agent generates a queue consumer and starts processing messages from the shared queue. Those messages were meant for other consumers.
• Database corruption: AI-generated code runs a migration or executes writes against the shared staging database. Everyone’s test data is now affected.

The knee-jerk response is to give each developer or AI agent their own isolated environment. But as discussed in the previous post, that trades one problem for another: high infrastructure costs, and environment drift.

mirrord takes a different approach. Rather than isolating at the environment level, it isolates at the interaction level, filtering which traffic reaches your local process, splitting queues so each developer gets their own copy of messages, and branching databases so changes don’t affect shared data. This lets the shared staging cluster stay fully operational for everyone, while each developer’s local testing stays isolated to their own interactions.

Let’s now look at some mirrord features that make testing AI-generated code in a shared Kubernetes cluster safe.

HTTP traffic filtering #

By default, when mirrord steals incoming traffic from the cluster to your local process, it steals all of it. Every request that is meant to hit the target service in the cluster gets forwarded to your local instance. Without a way to filter traffic, stealing all incoming traffic to a local process means the service in the cluster stops receiving requests, breaking the environment for everyone else who depends on it.

mirrord’s HTTP traffic filtering lets you route only requests that match specific criteria (a header, a path, or other filters) to your local process. For example, you can configure mirrord to only forward requests containing a header like baggage: mirrord-session=arsh to your locally running instance. Every other request continues to go to the live service in the cluster.

The result is that multiple developers and their AI agents can simultaneously test different changes against the same cluster without worrying about any traffic getting routed to the wrong place.

For the AI agent, this is particularly useful since it can now send requests repeatedly, observe responses, adjust the generated code, and iterate. All of this while interacting with real dependencies but without touching traffic that belongs to anyone else.

Queue splitting #

Most modern applications include a message queue like Kafka, SQS, RabbitMQ, or something similar. When an AI agent generates code for a consumer service, testing it against the real cluster creates an immediate conflict because the agent’s locally running consumer competes for messages with the existing consumer in the cluster.

If the generated code is wrong, which it often is on the first iteration, it processes or drops messages other services were expecting to receive. Depending on the system, this can cause cascading failures across your staging environment.

mirrord’s queue splitting handles this by creating an isolated temporary queue for the local process. The original consumer in the cluster continues processing from the main queue as normal. The locally running service receives its own copy of those messages in a separate queue that only it sees.

This means the AI agent can run, fail, adjust, and re-run without affecting the shared queue or any other consumers. It gets real messages from the cluster to test against, but those messages are exclusively its own.

Database branching #

Database interactions are where AI-generated code most commonly breaks in ways that are hard to catch from reading the code alone. An AI agent might generate a query that doesn’t match the actual schema, a migration that conflicts with existing indexes, or write logic that violates data constraints. These only surface when the code runs against a real database.

But testing against the shared staging database directly is risky. A bad migration, an accidental mass update, or a schema change can break the environment for everyone.

mirrord’s database branching creates an isolated branch of the database for the local process. The branch mirrors the schema and data of the real staging database, so tests run against realistic conditions. But writes, migrations, and schema changes only affect that branch, while the shared database is untouched. This lets AI agents test the full range of database interactions without any risk to the rest of the team.

Additional safeguards #

HTTP filtering, queue splitting, and database branching cover the most common collision points. But mirrord includes a few more capabilities that help make testing AI-generated code safer:

• mirrord Policies let platform teams define guardrails at the cluster level. For example, you can enforce that locally running AI-generated code can never write to shared databases by default, or restrict the namespaces where services can be targeted for stealing. Policies apply across the team so you’re not relying on each developer to configure isolation correctly every time.
• Preview environments allow AI agents to deploy code generated for a service as an isolated pod inside the shared staging cluster. This pod runs alongside the stable version of the service and only traffic explicitly routed to it reaches the new pod and everything else continues hitting the live service as normal. This lets you see and share a working version of the changes with stakeholders like product managers, QA, or sales before anything is merged, without needing a separate environment or disrupting anyone else.
• mirrord for CI extends the remocal testing model to CI pipelines. Instead of provisioning new environments or relying on mocks for CI tests, your pipeline can run tests directly against the same shared staging cluster. This further reduces your cloud bill and the number of environments you have to manage for testing code.

“It’ll never work” #

Sharing a staging cluster across a full team (and AI agents on top of that) makes people itch with anxiety. The most common objection we hear to the shared environment approach is: there’s no way an entire team can safely work on the same cluster at the same time. What if one AI agent’s generated code does something that breaks the cluster for everyone else? As a response to that personal dev environments for each AI agent seem like the safest bet.

But in practice, these personal environments don’t hold up because they drift from production, data goes stale, and infrastructure costs compound. And when AI agents are generating and iterating on code faster than any individual developer, the operational overhead of maintaining an environment per agent becomes completely impractical.

The only model that scales in this case is a shared staging cluster where isolation happens at the interaction level, not the infrastructure level. One cluster, equipped with traffic filtering, queue splitting, and database branching, can support an entire team of developers and AI agents simultaneously, without interference. Each person’s testing stays contained to their own interactions. And the environment remains operational for everyone else to keep working.

This is what mirrord is built for. Large organizations like monday.com, zooplus, and SurveyMonkey already have hundreds of developers using mirrord concurrently, sharing a Kubernetes environment in their organization every day. And as AI agents start running at higher velocity, generating complete features in minutes and iterating on tests in real time, having this infrastructure in place is what will allow you to truly see the benefits from using AI coding tools.