> For the complete documentation index, see [llms.txt](https://metalbear.com/mirrord/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://metalbear.com/mirrord/docs/using-mirrord/multiple-concurrent-sessions.md). # Multiple concurrent sessions (mirrord up) Use mirrord up to run several mirrord sessions together from one config file. ## Introduction `mirrord up` allows creating and running multiple mirrord sessions based on configuration defined in a single file — think `docker compose` but for mirrord. This can be useful for cases when you need to debug multiple related microservices and would like to manage their lifecycle together. Each service in the file is typically a *different* application with its own target, command, and configuration; this is for debugging several distinct applications together and managing their lifecycle as one unit, not for targeting multiple pods of the same application. For that, see [Targeting Pods by Label](/mirrord/docs/using-mirrord/targeting-pods-by-label.md). ## Getting started The fastest way to get a valid `mirrord-up.yaml` is the interactive wizard: ```sh mirrord up init ``` It prompts for common settings and walks you through one or more services, then writes the file (default: `./mirrord-up.yaml`). The generated file contains only the values you set; everything left at its default is omitted. See [`mirrord up init`](#mirrord-up-init) below for details. To write the file by hand instead, start with: ```yaml services: user-auth-service: run: command: ["python", "-m", "http.server"] stage-user-dashboard-app: target: path: pod/nginx run: command: ["node", "app.js"] ``` This file is the single source of configuration for all running sessions. Each entry in `services` defines a `mirrord` process that will run as part of the `mirrord up` session. You can leave out `target.path` (or the whole `target`) and `mirrord up` will infer it from the service id — see the `services.*.target` section below. Now, in the same directory of the `mirrord-up.yaml` file, run ```sh mirrord up ``` This will start all defined services, and they will run in parallel. The `mirrord up` session will be stopped once it's interrupted (`ctrl-c`) or one of the running mirrord sessions shuts down. Services default to `split` mode, which steals incoming traffic matching an `http_filter`. When no filter is provided, mirrord generates one based on the session key: `baggage: .*mirrord-session={key}.*`. For requests to reach every service in the session, each service has to forward the `baggage` header to the next one. The [`mirrord-header-propagation`](https://github.com/metalbear-co/skills/tree/main/skills/mirrord-header-propagation) skill sets this up across your services. If you'd rather have your local process take over a service completely, use `replace` mode — see [Service modes](#service-modes). ## Configuration ### Service modes Service modes are set per service with `default_mode` in the config file, or for a whole run with the `--mode` flag. The following config file sets `replace` mode for `user-auth-service`, while leaving `stage-user-dashboard-app` in the default `split` mode. ```yaml services: user-auth-service: default_mode: replace run: command: ["python", "-m", "http.server"] stage-user-dashboard-app: target: path: pod/nginx run: command: ["node", "app.js"] ``` To override the mode for every service in a run, use `mirrord up --mode replace`. #### `split` If none is given, every service uses `split` mode by default. Your local process and the deployed service both keep serving traffic, and only requests matching the service's `http_filter` are stolen to your machine. When no filter is provided, mirrord generates one from the session key: `baggage: .*mirrord-session={key}.*`. #### `replace` Your local process takes over the service entirely. mirrord creates a copy of the target workload and scales the original down to zero for the duration of the session, so every request that would have reached the deployed service reaches your machine instead. {% hint style="warning" %} `replace` scales the deployed workload down to zero while your session runs, so *everyone* hitting that service reaches your local process — not just you. On a shared cluster, prefer `split`. The original workload is restored when the session ends. {% endhint %} `replace` requires the targeted workload to be a deployment, statefulset, or replicaset. It can't be used with a [serverless target](#serverless-targets). Any `http_filter` set on a service in `replace` mode is ignored. #### `mirror` {% hint style="info" %} This feature requires at least mirrord version 3.258.0. {% endhint %} Traffic is mirrored to your local process, and the deployed service runs uninterrupted. Only requests matching the service's `http_filter` are mirrored to your machine. When no filter is provided, mirrord generates one from the session key: `baggage: .*mirrord-session={key}.*`. ### Queue Splitting `mirrord up` supports queue splitting automatically for every service in `split` and `mirror` mode. You don't need to add any special configuration. Services in `replace` mode don't use queue splitting: the deployed workload is scaled down, so your local process consumes every message from its queues. Before starting the session, set up queue splitting for the target and enable the relevant queue-splitting feature in the mirrord operator. Follow the [Queue Splitting guide](/mirrord/docs/sharing-the-cluster/queue-splitting.md) for the target's `MirrordSplitConfig` and broker-specific prerequisites. Start the services with a session key, for example: ```sh mirrord up --key checkout-debug ``` Messages intended for this session must contain `mirrord-session=checkout-debug`. The marker is matched in broker-specific message metadata: Kafka headers, RabbitMQ headers, SQS message attributes, Google Cloud Pub/Sub attributes, Azure Service Bus application properties, NATS JetStream headers, or Temporal headers. For Redis Pub/Sub and BullMQ, it is matched in the message payload. Queue splitting is available in `mirrord up` for Kafka, Amazon SQS, RabbitMQ, Google Cloud Pub/Sub, Azure Service Bus, NATS JetStream, Redis Pub/Sub, Temporal, and BullMQ. Core NATS pub/sub is split only when a service's `config_patch` sets it in `feature.split_queues`, which then replaces the automatic splits. {% hint style="info" %} RabbitMQ splitting in `mirrord up` requires an operator that supports it. Against an older operator the session still runs, with RabbitMQ splitting disabled and a warning printed for the affected service. {% endhint %} {% hint style="info" %} A service in `mirror` mode asks for `mirror` on every one of its Kafka queues. With the Java Kafka client ([`mirrord.client_implementation: java`](/mirrord/docs/sharing-the-cluster/queue-splitting/kafka.md#configure-the-operators-kafka-client), required for Kafka Streams) this needs mirrord operator `3.232.0` or later, and with Kafka Streams the deployed service can still get its copy of a message whose copy to your machine the broker rejected, and then processes it a second time when it is redelivered. See [`queue_mode`](/mirrord/docs/sharing-the-cluster/queue-splitting.md#setting-a-filter-for-a-mirrord-run) for the operator versions that fail or steal instead. {% endhint %} {% hint style="info" %} Only messages containing the session key (`checkout-debug` in this case) are routed to your local session. All other messages continue to the deployed target. {% endhint %} ### Serverless targets {% hint style="info" %} This feature requires at least mirrord version 3.288.0. {% endhint %} A service can target a [workload outside Kubernetes](/mirrord/docs/using-mirrord/beyond-kubernetes.md), such as an Amazon ECS service, alongside services that target Kubernetes workloads. Set `target.path` to `serverless/` followed by the service name, and `target.namespace` to the environment it runs in: ```yaml services: orders: target: path: serverless/orders namespace: staging run: command: ["node", "app.js"] user-auth-service: target: path: deployment/user-auth run: command: ["python", "-m", "http.server"] ``` For a serverless target, `target.namespace` is the environment the service is registered under in sessions-manager, not a Kubernetes namespace. When it's not set, `default` is used. Set up the workload and your deployment mode first, as described in [Beyond Kubernetes](/mirrord/docs/using-mirrord/beyond-kubernetes.md#setting-up). Serverless targets work with these limitations: * **`split` and `mirror` only.** A serverless target can't be copied, so a service in `replace` mode fails before any session starts, with an `incompatible mode and target` error that names the service. * **No queue splitting or DB branching.** In `split` and `mirror` mode the service runs without either, and `mirrord up` prints a warning for it. A mode that asks for queue splitting or DB branching gets the same warning instead of an error, so one mode can still cover both serverless and Kubernetes services. * **No `split_queues`, `kafka_branch`, `db_branches` or `copy_target` in `config_patch`.** A service whose `config_patch` sets any of them fails before any session starts, with an error naming the setting, for example `` `feature.db_branches` is not supported with serverless targets. `` * **Set the target explicitly.** `mirrord up init` and target inference only find Kubernetes workloads, so write the `serverless/` path into `mirrord-up.yaml` yourself. All the [limitations of workloads outside Kubernetes](/mirrord/docs/using-mirrord/beyond-kubernetes.md#limitations) also apply. ### Context `mirrord up` allows you to specify which Kubernetes context to run services in. You can do this either with the `--context` flag, or by setting it in `context` in the config file. The following config file sets the `kind` context for all services via `common` field, and overrides it to `minikube` for `user-auth-service`. ```yaml common: context: kind services: user-auth-service: context: minikube run: command: ["python", "-m", "http.server"] stage-user-dashboard-app: target: path: pod/nginx run: command: ["node", "app.js"] ``` To override the context for every service in a run, use `mirrord up --context minikube`. #### Context precedence | common context | service context | `--context` | context used | | -------------- | --------------- | ----------- | ------------------------- | | any | any | set | `--context` | | any | set | unset | service context | | set | unset | unset | common context | | unset | unset | unset | default (current context) | ### Config file (`mirrord-up.yaml`) #### `common` Common configuration options, applied to all defined services. Currently the following options are supported: * [`accept_invalid_certificates`](https://metalbear.com/mirrord/docs/config/options#root-accept_invalid_certificates) * [`operator`](https://metalbear.com/mirrord/docs/config/options#root-operator) * [`telemetry`](https://metalbear.com/mirrord/docs/config/options#root-telemetry) * [`context`](https://metalbear.com/mirrord/docs/config/options#root-kube_context) All fields map directly to their `mirrord.json` counterparts. #### `services` A map from service ids to a `ServiceConfig`. Each entry in this map defines and configures a mirrord process that will be run as part of the session. **`services.*.target`** Specifies the target of the session. Has 2 fields: `path` and `namespace`, which map directly to their `mirrord.json` counterparts. When `path` is omitted, `mirrord up` infers it from the service id (the key in the `services` map) by searching the cluster for a deployment, statefulset, rollout, or pod with that name. If a match is found, it's used automatically, otherwise `mirrord up` prompts you to pick a namespace and workload, and offers to save the choice back to `mirrord-up.yaml` so future runs skip the prompt. To run a service without a target (outgoing traffic only), set `target: none`. For a [serverless target](#serverless-targets), `namespace` is the environment of the service in sessions-manager, `default` when not set. Serverless targets aren't inferred, so set `path` explicitly. Examples: ```yaml # Specify both namespace and path target: path: deployment/test-app namespace: test-namespace ``` ```yaml # Path only, will use default namespace target: path: deployment/test-app ``` ```yaml # Path omitted: inferred from the service id, searching `test-namespace` target: namespace: test-namespace ``` ```yaml # Targetless — outgoing traffic only target: none ``` Omitting the `target` field entirely is equivalent to an empty mapping: the path is inferred from the service id in the default namespace. **`services.*.env`** Specifies the environment variable configuration for the given service. Maps directly (1:1) to [`feature.env`](https://metalbear.com/mirrord/docs/config/options#feature-env) **`services.*.default_mode`** Either `split` (the default), `replace` or `mirror`. See [Service modes](#service-modes) for what each one does and when to use it. The `--mode` flag overrides this for every service being launched. **`services.*.http_filter`** Specifies the HTTP filtering configuration for the given service. Maps directly to [`feature.network.incoming.http_filter`](https://metalbear.com/mirrord/docs/config/options#feature-network-incoming) Only applies in `split` and `mirror` modes. A service in `replace` mode receives all incoming traffic, so any filter set on it is ignored. **`services.*.ignore_ports`** List of ports that should be ignored in incoming traffic. Maps directly to [`feature.network.incoming.ignore_ports`](https://metalbear.com/mirrord/docs/config/options#feature-network-incoming) **`services.*.config_patch`** Some mirrord configuration options are not available in the `mirrord-up.yaml` format. With `config_patch` you're able to expand what's supported and use regular `mirrord.json` options for a specific service. Prefer using the dedicated `mirrord-up.yaml` fields whenever possible, the `config_patch` is an escape hatch while other settings are being ported over to `mirrord up`. ```yaml config_patch: feature: split_queues: "*": queue_type: SQS jq_filter: '.Body | fromjson | .headers["x-meow-id"] == "{{ key }}"' ``` **`services.*.run`** Specifies the command that should be run with mirrord. Has the following fields: * `command`: Array of strings containing the command to be run and its CLI arguments. * `type`: can be either `exec` or `container`, defaults to `exec`. Specifies how mirrord should be run (i.e. with `mirrord exec` or `mirrord container`) * `directory`: Working directory for `command`. Relative paths are resolved against the directory that contains `mirrord-up.yaml`, and the directory must exist. Only supported with `type: exec`. When not set, `command` runs in the directory `mirrord up` was started from. * `pre_command`: Array of strings containing a command to run to completion before the service's mirrord session starts, such as a build step. It runs locally without mirrord, in the same directory as `command`. The session starts only if it exits successfully. If it fails, or can't be started, its output and error are shown, this service is not started, and the other services keep running. Examples: ```yaml run: type: container command: ["docker", "run", "my-app"] ``` ```yaml run: # `type` defaults to `exec`, no need to specify explicitly command: ["node", "app.js"] ``` ```yaml run: directory: services/api # Built locally without mirrord, before the session starts pre_command: ["npm", "run", "build"] command: ["node", "dist/app.js"] ``` **`services.*.sync`** Syncs your local code into the service's preview with `mirrord preview sync`. See [Sync Your Local Code to a Preview Environment](/mirrord/docs/use-cases/preview-environments/preview-environments-code-sync.md) for how it works. `mirrord up` and `mirrord preview up` ignore it. Requires mirrord `3.286.0` or later. Has the following fields: * `files`: The local directories to sync. Each entry has these fields: * `local`: The local directory. Relative paths are resolved against the directory that contains `mirrord-up.yaml`. * `remote`: The absolute path in the preview's container that the directory is synced to. A sync makes it match `local`, so it deletes the image's files there, except at ignored paths. It can't be `/`, and can't be under `/mirrord`, `/mirrord-state`, `/mirrord-code`, or `/mirrord-sync-tls`. It can't equal, contain, or be inside another `remote` or `prepare.persist` path. * `ignore`: More paths to leave out, written as lines of a `.gitignore` in `local`, such as `coverage/`. They take precedence over git's rules, so `!.env.preview` syncs a file that a `.gitignore` excludes. * `prepare`: A command that prepares the synced code, such as an install of dependencies. Requires `run`. Has the following fields: * `command`: Array of strings containing the program and its arguments, such as `["npm", "ci"]`. It runs before the app first starts, and after each sync that changes a `triggered_by` file. The app restarts after it succeeds. * `directory`: The absolute working directory of `command` in the preview's container, such as `/app`. Defaults to the container's working directory. * `triggered_by`: Globs of the files whose changes run `command` again, such as `frontend/package-lock.json`. They're relative to the directory that contains `mirrord-up.yaml`, and each must start with the `local` of a `files` entry. Name specific files rather than a `**` glob over a large directory, because the pod reads every matching file after each sync. * `persist`: The absolute paths outside every `remote` that `command` writes to, such as `/usr/local/lib/python3.12/site-packages`. The preview keeps them on the code volume, so they survive a restart of the pod. List every such path, because the pod doesn't run `prepare` again after a restart when nothing changed. * `run`: How the preview pod runs the app. Without it, the container runs its own command, such as nginx serving the synced files. Has the following fields: * `command`: Array of strings containing the program and its arguments, such as `["npm", "run", "dev"]`. It replaces the container's own command. * `directory`: The absolute working directory of `command` in the preview's container, such as `/app`. Defaults to the container's working directory. * `restart`: When the app restarts. `on_prepare` restarts it only after `prepare` runs, and `on_sync` restarts it after every sync. A successful `prepare` restarts it either way. Defaults to `on_prepare`. ```yaml sync: files: - local: frontend remote: /app ignore: ["coverage/"] prepare: command: ["npm", "ci"] directory: /app triggered_by: ["frontend/package-lock.json"] run: command: ["npm", "run", "dev"] directory: /app ``` **`services.*.skip`** Don't launch this service unless it's named on the `mirrord up` command line. `mirrord preview sync` also leaves it out. If the service has `sync`, an image passed with `--image` or `MIRRORD__IMAGE` previews it anyway. `mirrord preview up` ignores `skip`, because it previews exactly the services that have an image. Defaults to `false`. **`services.*.context`** The name of the Kubernetes context to run this service in. See [Context](#context) for precedence rules when used with `common.context`. The `--context` argument overrides this for every service being launched. ### Templating The `mirrord-up.yaml` configuration file is rendered with [Tera](https://keats.github.io/tera/docs/) (Jinja2-style syntax) before it is parsed. This lets you populate configuration values dynamically from the session key and from your shell environment. #### Available variables * `{{ key }}` — The session key (specified via `--key` flag or defaults to OS username) #### Available functions * `{{ get_env(name="VAR") }}` — The value of the environment variable `VAR` from the environment `mirrord up` was started in. Rendering fails if the variable isn't set, unless you pass a fallback: `{{ get_env(name="VAR", default="fallback") }}`. #### Examples Use the session key in environment variable overrides: ```yaml services: my-service: target: path: deployment/my-app env: override: SESSION_ID: "{{ key }}" DEBUG_TAG: "debug-{{ key }}" run: command: ["node", "app.js"] ``` Use the session key in commands: ```yaml services: logger: run: command: ["python", "logger.py", "--session", "{{ key }}"] ``` When you run `mirrord up --key my-session`, the above examples will render as: * `SESSION_ID: "my-session"` * `DEBUG_TAG: "debug-my-session"` * Command: `["python", "logger.py", "--session", "my-session"]` Pull values from the environment instead of hardcoding them per developer: ```yaml services: my-service: target: namespace: "{{ get_env(name='DEV_NAMESPACE', default='default') }}" env: override: API_TOKEN: "{{ get_env(name='API_TOKEN') }}" run: command: ["node", "app.js"] ``` Here the namespace falls back to `default` when `DEV_NAMESPACE` isn't set, while a missing `API_TOKEN` fails the run with a templating error rather than starting the session with an empty value. ## CLI args ### `-f`, `--config-file` Allows specifying a different config file, e.g. `mirrord up -f mirrord-up-custom.yaml` ### `-m`, `--mode` Runs every service in the given mode, ignoring the `default_mode` set in the config file. Either `split`, `replace` or `mirror` — see [Service modes](#service-modes). When omitted, each service uses its own `default_mode`. ### `--key` Allows specifying a custom session key. When not supplied, the OS username is used. ### `--context` Runs every service in the specified context, ignoring the `context` field(s) set in the config file. When omitted, each service uses the `context` in the config file. See [Context](#context) for precedence rules. ## `mirrord up init` Interactive wizard that generates a skeleton `mirrord-up.yaml`. It does not query the cluster; workload inference and prompting happen later, when you run `mirrord up`. ```sh mirrord up init [-o path/to/mirrord-up.yaml] ``` Flow: 1. **Common settings** — prompts for `operator`, `accept_invalid_certificates`, and `telemetry`. Only values you change from the default are written. 2. **Services** — loops one service at a time, prompting for name, mode, target, HTTP filter, ignore ports (with presets for Istio/Linkerd sidecars), env overrides, run type (`exec`/`container`), and the local command. For the target you choose to infer it from the service name (looked up when you run `mirrord up`), specify one explicitly, or run without a target. Choosing `replace` mode skips the HTTP filter prompt and drops the targetless option, since neither applies to it. Repeats until you answer "no" to *Add another service?*. 3. **Preview and save** — prints the generated YAML, asks whether to save, then for a filename (re-asking if you decline to overwrite an existing file). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://metalbear.com/mirrord/docs/using-mirrord/multiple-concurrent-sessions.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.