> For the complete documentation index, see [llms.txt](https://metalbear.com/mirrord/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://metalbear.com/mirrord/docs/use-cases/chaos-testing.md).

# Chaos Testing

mirrord lets you inject artificial failures and disruptions into your app's outgoing traffic, so you can test how it behaves under unexpected conditions. Create rules that add latency to connections to specific hosts, or turn them into errors, and scope each rule to a single mirrord session.

## How chaos rules work

A chaos rule pairs a selector, which picks the traffic to disrupt, with an effect, which defines the disruption. Rules are attached to a single mirrord session and never affect other sessions.

### Selectors

A selector matches outgoing connections by their destination:

* `upstream`: the destination host, or `host:port` to match a specific port.
* `percentage`: roughly how often a matched connection gets the effect (0 to 100).

{% hint style="info" %}
Currently selectors can only match outgoing TCP connections. Selectors for file operations and HTTP requests are planned.
{% endhint %}

### Effects

An effect defines what happens to a matched connection. Two effects are supported:

* `latency`: delays the connection's read and/or write operations.

```json
{
  "name": "latency for api.example.com",
  "priority": 0,
  "selector": {
    "upstream": "api.example.com:443",
    "percentage": 100
  },
  "effect": {
    "latency": {
      "read_ms": 100,
      "write_ms": 200,
      "jitter_ms": 25
    }
  }
}
```

* `connection_error`: fails the connection. `type` can be one of: `reset` (can be applied to ongoing connections), `timed_out`, `refused`.

```json
{
  "name": "error for api.example.com",
  "priority": 0,
  "selector": {
    "upstream": "api.example.com:443",
    "percentage": 100
  },
  "effect": {
    "connection_error": {
      "type": "reset",
      "after_ms": 0
    }
  }
}
```

### Priority

When multiple rules match the same connection, only one is applied: the rule with the highest `priority` value. If not set, `priority` defaults to 0, the lowest.

## Prerequisites

1. Minimum mirrord CLI version `3.232.0`.

## Starting the UI server and a session

Chaos rules are managed through the mirrord UI server, so start it first:

```sh
mirrord ui
```

{% hint style="info" %}
For more information on the UI server, see [Local UI](/mirrord/docs/using-mirrord/local-ui.md).
{% endhint %}

Grab the `API token` and the server address from the output:

```sh
* Web UI:
 -> http://127.0.0.1:59281/auth?token=60733e94f1799442e2f5af9048fac1a0efd6051715a2131992784f9c11876349&redirect=/
* API token:
 -> x-auth-token: 60733e94f1799442e2f5af9048fac1a0efd6051715a2131992784f9c11876349
```

Next, start the mirrord session you want to apply chaos rules to. In this example we run it from the terminal, but you can start it from the IDE plugins too:

```sh
mirrord exec -f .mirrord/mirrord.json -- node app.js
```

Grab the `session ID` from the session output (logs shortened for clarity):

```sh
* session ID: c425f391-e9cc-4199-8de9-7bdbb3e7dfcc
* Running command: node app.js
```

The chaos API needs three values: the server address, the `API token` that authenticates each request, and the `session ID` that scopes your rules to this session only. Export them once with your own values, and every example below can be copied as-is:

```sh
export UI_ADDRESS='http://127.0.0.1:59281'
export CHAOS_TOKEN='60733e94f1799442e2f5af9048fac1a0efd6051715a2131992784f9c11876349'
export SESSION_ID='c425f391-e9cc-4199-8de9-7bdbb3e7dfcc'
export CHAOS_URL="$UI_ADDRESS/api/chaos/rules/$SESSION_ID"
```

## Managing rules via the API

### Creating a rule

To create a chaos rule, write it to a JSON file, for example `latency-rule.json`:

```json
{
  "name": "latency for database interactions",
  "priority": 10,
  "effect": {
    "latency": {
      "read_ms": 750
    }
  },
  "selector": {
    "upstream": "sonic.database.svc.cluster.local",
    "percentage": 35
  }
}
```

Then send it in an HTTP POST request to the UI server with the `x-auth-token` header:

```sh
curl --request POST \
  --header 'Content-Type: application/json' \
  --header "x-auth-token: $CHAOS_TOKEN" \
  --data @latency-rule.json \
  "$CHAOS_URL"
```

The response returns the created rule, including the `id` you'll need to modify or delete it later. Note that in responses the `effect` is nested inside the `selector`, and `upstream` comes back with an explicit port, where `0` means any port:

```json
{
  "id": "6b8f1c4e-2a73-4d9b-8e56-c3f0a7d1b924",
  "name": "latency for database interactions",
  "priority": 10,
  "selector": {
    "type": "tcp",
    "upstream": "sonic.database.svc.cluster.local:0",
    "percentage": 35,
    "effect": {
      "latency": {
        "read_ms": 750
      }
    }
  },
  "hit_count": 0
}
```

The outgoing traffic (roughly 35% of it) from the service we are debugging with mirrord will now be affected by latency on read operations, when this traffic is destined to a host that matches the `selector`. In this example, outgoing traffic on this database connection would be impacted by this latency rule, so we can test how the `app.js` service behaves when there's latency requesting data from this database.

### Listing rules

To list all the chaos rules that are active for a session, send an HTTP GET request:

```sh
curl --request GET \
  --header "x-auth-token: $CHAOS_TOKEN" \
  "$CHAOS_URL"
```

The response is an array of rules in the same shape as the creation response, each with its `id`.

### Modifying a rule

Modifying or deleting a rule requires its ID, which is returned when the rule is created and included in the list response. To modify a rule, edit the rule's JSON file:

```json
{
  "name": "latency for database interactions more often",
  "priority": 10,
  "effect": {
    "latency": {
      "read_ms": 750
    }
  },
  "selector": {
    "upstream": "sonic.database.svc.cluster.local",
    "percentage": 75
  }
}
```

Then send it in an HTTP PUT request with the rule ID at the end of the URL:

```sh
export RULE_ID="6b8f1c4e-2a73-4d9b-8e56-c3f0a7d1b924"
curl --request PUT \
  --header 'Content-Type: application/json' \
  --header "x-auth-token: $CHAOS_TOKEN" \
  --data @latency-rule.json \
  "$CHAOS_URL/$RULE_ID"
```

Here we changed both the `name` of the chaos rule and its `percentage`.

### Deleting a rule

To delete a chaos rule, send an HTTP DELETE request with the ID of the rule you want:

```sh
curl --request DELETE \
  --header "x-auth-token: $CHAOS_TOKEN" \
  "$CHAOS_URL/$RULE_ID"
```

The chaos rule (in this example `6b8f1c4e-2a73-4d9b-8e56-c3f0a7d1b924`) will be deleted, and will stop affecting the session immediately.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://metalbear.com/mirrord/docs/use-cases/chaos-testing.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.