> For the complete documentation index, see [llms.txt](https://metalbear.com/mirrord/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://metalbear.com/mirrord/docs/using-mirrord/incoming-traffic/steal-https.md). # Stealing HTTPS Requests How to steal HTTPS traffic with mirrord when your app terminates TLS itself. mirrord supports HTTPS stealing across all tiers. However, when the deployed application receives encrypted traffic directly - for example, when HTTPS is not decrypted by the mesh or load balancer and the app itself handles TLS termination - this functionality is available only in mirrord for Teams. With mirrord for Teams, you can steal a subset of HTTP requests coming to your target, even if the deployed application receives the traffic encrypted with TLS. {% hint style="info" %} This feature is available to users on the Team and Enterprise pricing plans. {% endhint %} **Important:** stealing HTTPS with a filter requires mirrord-operator version at least `3.106.0` and mirrord-agent version at least `1.134.0`. `agentAsClient.verification.serverName` requires mirrord-operator version at least `3.210.0` and mirrord-agent version at least `3.262.0`. `agentAsClient.identities` and `feature.network.incoming.tls_delivery.client_identities` require mirrord and mirrord-agent version at least `3.272.0`. ## Configuring HTTPS stealing in the cluster To enable mirrord users to steal HTTPS requests with a filter, you must provide the mirrord Operator with some insight into your TLS configuration. This can be done with dedicated custom resources: `MirrordTlsStealConfig` and `MirrordClusterTlsStealConfig`. These two resources look and work almost the same. The only exception is that `MirrordTlsStealConfig` is scoped to the namespace in which you create it, while `MirrordClusterTlsStealConfig` scopes the whole Kubernetes cluster. {% hint style="warning" %} `targetPath` and `selector` are what limit a configuration to a workload. A resource that sets neither applies to every mirrord target in its namespace (or, for `MirrordClusterTlsStealConfig`, in the cluster) that listens on one of its ports, with that configuration's certificates and verification rules, whether or not they fit that workload. {% endhint %} An example `MirrordTlsStealConfig` resource that configures HTTPS stealing from an `example-deploy` deployment living in namespace `example-deploy-namespace`: ```yaml apiVersion: mirrord.metalbear.co/v1alpha kind: MirrordTlsStealConfig metadata: # The name indicates that this configuration is for the `example-deploy` deployment, # but it does not really matter. The mirrord Operator does not inspect config resources' names. name: tls-steal-config-for-example-deploy # This is the namespace-scoped variant of the configuration resource, # so it must live in the same namespace as the `example-deploy` deployment. namespace: example-deploy-namespace spec: # A wildcard pattern that will be matched against session target's path. # # This pattern can contain `*` and `?` characters, where: # 1. `*` will match any amount of any characters; # 2. `?` will any character once. # # E.g `deploy/*/container/container-?` will match both `deploy/name/container/container-1` and `deploy/another-name/container/container-2`. # # mirrord session target path is produced from: # 1. Target resource type (e.g deploy, pod, rollout, statefulset, etc.); # 2. Target resource name; # 3. `container` literal (if the user selected an exact container as the target); # 4. Target container name (if the user selected an exact container as the target). # # Note that the user can target pods of the `example-deploy` deployment either indirectly, by targeting the deployment, or directly. # They can also specify an exact target container or not. # # Optional. Defaults to a pattern that matches everything. targetPath: "*/example-deploy*" # A label selector that will be matched against session target's labels. # # Optional. Defaults to a selector that matches everything. selector: matchLabels: app: example-deploy # Each port on the target can be configured separately. ports: # This entry configures HTTPS stealing from port 443. - port: 443 # Configures how the mirrord-agent authenticates itself and verifies the clients (original request senders) when acting as a TLS server. agentAsServer: # Configures how the server authenticates itself. authentication: # Path to a PEM file containing a certificate chain to use. # # This file must contain at least one certificate. # It can contain entries of other types, e.g private keys, which are ignored. # Certificates are expected to be listed from the end-entity to the root. certPem: /path/to/server/cert.pem # Path to a PEM file containing a private key matching the certificate chain from `certPem`. # # This file must contain exactly one private key. # It can contain entries of other types, e.g certificates, which are ignored. keyPem: /path/to/server/key.pem # ALPN protocols supported by the server, in order of preference. # # If empty, ALPN is disabled. # # Optional. Defaults to an empty list. alpnProtocols: - h2 - http/1.1 - http/1.0 # Configures how the server verifies the clients. # # Optional. If not present, the server will not offer TLS client authentication at all. verification: # Whether anonymous clients should be accepted. # # Optional. Defaults to false. allowAnonymous: false # Whether the server should accept any certificate, regardless of its validity and who signed it. # # Note that this setting does not affect whether anonymous clients are accepted or not. # If `allowAnonymous` is not set, some certificate will still be required. # # Optional. Defaults to false. acceptAnyCert: false # Paths to PEM files and directories PEM files containing allowed root certificates. # # Directories are not traversed recursively. # # Each certificate found in the files is treated as an allowed root. # The files can contain entries of other types, e.g private keys, which are ignored. # # Optional. Defaults to an empty list. trustRoots: - /path/to/trusted/client/root/cert.pem # Configures how the mirrord-agent authenticates itself and verifies the server (original request destination) when acting as a TLS client. agentAsClient: # Configures how the client authenticates itself. # # Optional. If not present, the client will make the connections anonymously. authentication: # Path to a PEM file containing a certificate chain to use. # # This file must contain at least one certificate. # It can contain entries of other types, e.g private keys, which are ignored. # Certificates are expected to be listed from the end-entity to the root. certPem: /path/to/client/cert.pem # Path to a PEM file containing a private key matching the certificate chain from `certPem`. # # This file must contain exactly one private key. # It can contain entries of other types, e.g certificates, which are ignored. keyPem: /path/to/client/key.pem # Additional client certificates, for servers that authorize requests based on the client's identity. # # When passing through a request from a client that presented a certificate, # the client uses the first of these with the same identity as the original client's certificate. # The identity is the set of subject alternative names, or the subject if the certificate has no SANs. # If none matches, or the original client presented no certificate, `authentication` is used. # # Requires `agentAsServer.verification` with `acceptAnyCert` disabled, # because the identity of an unverified client cannot be trusted. # # Optional. Defaults to an empty list. identities: - # Path to a PEM file containing a certificate chain to use. # # This file must contain at least one certificate. # It can contain entries of other types, e.g private keys, which are ignored. cert: /path/to/client-a/cert.pem # Path to a PEM file containing a private key matching the certificate chain from `cert`. # # This file must contain exactly one private key. # It can contain entries of other types, e.g certificates, which are ignored. key: /path/to/client-a/key.pem - cert: /path/to/client-b/cert.pem key: /path/to/client-b/key.pem # Configures how the client verifies the server. verification: # Whether to accept any certificate, regardless of its validity and who signed it. # # Optional. Defaults to false. acceptAnyCert: false # Name the server's certificate is verified against, and sent in the SNI extension, when # the original client sent no SNI. # # Set it when the certificate is issued for a DNS name and the original clients do not # send SNI (for example an nginx upstream): without it, mirrord-agent verifies the # certificate against the pod's IP address, which such a certificate does not cover. # # Optional. mirrord-agent uses the SNI from the original client, then this name, then # the request's host, then the pod's IP address. serverName: example-deploy # Paths to PEM files and directories PEM files containing allowed root certificates. # # Directories are not traversed recursively. # # Each certificate found in the files is treated as an allowed root. # The files can contain entries of other types, e.g private keys, which are ignored. # # Optional. Defaults to an empty list. trustRoots: - /path/to/trusted/server/root/cert.pem ``` Each `MirrordTlsStealConfig`/`MirrordClusterTlsStealConfig` resource configures HTTPS stealing for some set of available mirrord targets. With the use of `spec.targetPath` and `spec.selector`, you can link one configuration resource to multiple pods, deployments, rollouts, etc. When the mirrord Operator finds multiple configuration resources matching the session target path and labels, it merges their `ports` lists. The same port cannot be configured multiple times (extra entries are discarded). **Important:** mirrord-agent will search for all files and directories referenced by the config resources in the target container filesystem. ### Which certificate the application sees Requests that are not stolen are not forwarded byte for byte. mirrord-agent terminates the caller's TLS connection to read the request, then opens its own TLS connection to the application using `agentAsClient`. When `agentAsClient.authentication` is configured, the application sees that certificate; otherwise, the agent connects without a client certificate. The application never sees the original caller's certificate: the agent does not hold the caller's private key, so it cannot present that certificate. If the application authorizes callers by certificate name, give `agentAsClient` a certificate the application accepts. Pointing it at the application's own certificate makes the application see itself as the caller, which such an allowlist typically rejects with a 403. If the application treats callers differently based on who they are, a single certificate is not enough. List a certificate for each caller identity in `agentAsClient.identities`, and the agent presents the one with the same identity as the original caller's certificate: ```yaml agentAsServer: authentication: certPem: /path/to/server/cert.pem keyPem: /path/to/server/key.pem verification: trustRoots: - /path/to/trusted/client/root/cert.pem agentAsClient: # Used for callers that presented no certificate, or one with an identity not listed below. authentication: certPem: /path/to/default/cert.pem keyPem: /path/to/default/key.pem identities: - cert: /path/to/orders-service/cert.pem key: /path/to/orders-service/key.pem - cert: /path/to/billing-service/cert.pem key: /path/to/billing-service/key.pem verification: trustRoots: - /path/to/trusted/server/root/cert.pem ``` A certificate's identity is its set of subject alternative names (SANs), or its subject if it has no SANs. The certificates in `identities` do not have to be the callers' own certificates, only to have the same identity, so they can be issued separately for mirrord. The agent uses the first matching entry, and falls back to `authentication` when none matches. The agent learns the caller's identity only when it verifies the caller's certificate, so `identities` requires `agentAsServer.verification` with `acceptAnyCert` disabled. Without it, anyone could make the agent present any of the configured certificates on their behalf, and the agent refuses the configuration. mirrord-agent reads the configuration when it starts, and the Operator reuses a running agent for new sessions on the same target. After changing `agentAsClient`, stop the sessions on that target and wait for its `mirrord-agent` pod to exit before starting new ones, otherwise they keep the old certificate. ## Configuring delivery of stolen HTTPS to your local application By default, when delivering stolen HTTPS requests to your local application, mirrord uses the original protocol - TLS. The connection is made from your local machine by an anonymous TLS client that **does not** verify the server certificate. This behavior can be configured in your mirrord config with [`feature.network.incoming.tls_delivery`](https://metalbear.com/mirrord/docs/config#feature.network.incoming.tls_delivery). The older `feature.network.incoming.https_delivery` option is deprecated, use `tls_delivery` instead. If your local application requires a client certificate (mutual TLS), set `tls_delivery.client_cert` and `tls_delivery.client_key` to a certificate it accepts. Without them the handshake fails with a `BadCertificate` alert and every stolen request is answered with a 502. ```json { "feature": { "network": { "incoming": { "tls_delivery": { "protocol": "tls", "client_cert": "/path/to/client/cert.pem", "client_key": "/path/to/client/key.pem" } } } } } ``` If your local application treats callers differently based on who they are, list a certificate for each caller identity in `tls_delivery.client_identities`. For each stolen or mirrored connection, mirrord presents the first of these with the same identity as the original caller's certificate, and falls back to `client_cert` and `client_key` when none matches or the caller presented no certificate. Identities are compared the same way as for [`agentAsClient.identities`](#which-certificate-the-application-sees). ```json { "feature": { "network": { "incoming": { "tls_delivery": { "protocol": "tls", "client_cert": "/path/to/default/cert.pem", "client_key": "/path/to/default/key.pem", "client_identities": [ { "cert": "/path/to/orders-service/cert.pem", "key": "/path/to/orders-service/key.pem" }, { "cert": "/path/to/billing-service/cert.pem", "key": "/path/to/billing-service/key.pem" } ] } } } } } ``` The original caller's identity is known only when the cluster's `MirrordTlsStealConfig`/`MirrordClusterTlsStealConfig` verifies callers against trust roots (`agentAsServer.verification` without `acceptAnyCert`). Otherwise, mirrord always uses `client_cert` and `client_key`. Certificates can be presented only during the TLS handshake, so local applications that request a client certificate later (TLS 1.2 renegotiation, TLS 1.3 post-handshake authentication) are not supported. The `client_cert` and `client_key` settings also apply to [preview environments](/mirrord/docs/use-cases/preview-environments.md). There the mirrord Operator makes the TLS connection to the preview pod, so `mirrord preview start` reads both files and stores them in the session's Secret, next to `secret_mounts`, and the Operator presents the certificate when it delivers stolen requests. `server_name` is honored too. The other `tls_delivery` settings, including `client_identities`, do not apply to previews: delivery is always TLS, and the preview pod's certificate is not verified. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://metalbear.com/mirrord/docs/using-mirrord/incoming-traffic/steal-https.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.