Monitoring

Monitoring with mirrord for Teams

The mirrord Operator can produce logs in JSON format that can be digested by most popular logging solutions (DataDog, Dynatrace, etc). To enable JSON logging, set operator.jsonLog to true in the Operator Helm chart values. The log level is INFO by default, and can be changed using the RUST_LOG environment variable in the Operator container, which takes values in the following format: mirrord={log_level} (e.g. mirrord=debug).

This feature is only relevant for users on the Team and Enterprise pricing plans.

Functional Logs

The following logs are written with log level INFO, and can be used for dashboards within monitoring solutions in order to monitor mirrord usage within your organization:

Log messages:

• Target Copied

• Port Stolen

• Port Mirrored

• Port Released

• Session Started

• Session Ended

Fields:

field	description	events
target	the session's target	All
client_hostname	whoami::hostname of client	All
client_name	whoami::realname of client	All
client_user	Kubernetes user of client (via k8s RBAC)	All
client_id	unique client id produced from client's certificate	All
session_id	unique id for individual mirrord sessions	Port Steal Port Mirrored Port Released Session Started Session Ended
session_duration	the session's duration in seconds	Session Ended
port	port number	Port Stolen Port Mirrored Port Released
http_filter	the client's configured HTTP Filter	Port Stolen
scale_down	whether the session's target was scaled down	Target Copied

Prometheus

mirrord Operator can expose prometheus metrics if enabled. (default endpoint is :9000/metrics)

Helm

# values.yaml for mirrord-operator helm chart
operator:
  ...
  metrics: true
  ...

Manual

env	description	type	default
OPERATOR_METRICS_ENABLED	enable metrics endpoint	"true" | "false"	"false"
OPERATOR_METRICS_ADDR	metrics http server addr	SocketAddr	"0.0.0.0:9000"

Exposed metrics

metric	description	labels
mirrord_license_valid_seconds	Seconds left for current license validity
mirrord_sessions_create_total	Count of created sessions	client_hostname client_name client_user user_id
mirrord_sessions_duration	Histogram for session durations after they are ended	client_hostname client_name client_user user_id

DataDog Dashboard

We offer a DataDog dashboard you can import to track statistics.

Download it here

Grafana Dashboard

Alternatively there is a Grafana dashboard you can import to track statistics.

Download it here

fluentd

If you are using fluentd you can add a filter to unpack some values from the "log" message

<filter kubernetes.var.log.containers.**_mirrord_mirrord-operator-**>
  @type parser
  key_name log
  reserve_data true
  remove_key_name_field true
  <parse>
    @type json
  </parse>
</filter>

This will expand all the extra fields stored in "log" field.

fluentd + Elasticsearch

Assuming you are using logstash_format true and the connected mapping will store the extra fields in a keyword type, we have a ready made dashboard you can simply import.

Download it here (use Saved Objects to import).