New to KubeDB? Please start here.

Redis Alerting with Prometheus

This tutorial shows you how to configure Prometheus-based alerting for a KubeDB-managed Redis instance using the redis-alerts Helm chart.

Before You Begin

  • Ensure you have a Kubernetes cluster and that kubectl is configured to communicate with it. If you do not already have a cluster, you can create one using kind.

  • Install the KubeDB operator by following the steps here.

  • Deploy the database in the demo namespace:

    $ kubectl create ns demo
    namespace/demo created
    
  • This tutorial assumes you already have a kube-prometheus-stack running in your cluster, with Prometheus configured so that both serviceMonitorSelector and ruleSelector match the label release: prometheus. See the Grafana Dashboard guide for how to deploy kube-prometheus-stack if you don’t have it yet.

    To verify the selectors:

    $ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.ruleSelector}'
    {"matchLabels":{"release":"prometheus"}}
    
    $ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}'
    {"matchLabels":{"release":"prometheus"}}
    
  • To learn more about how Prometheus monitoring works with KubeDB, see the overview here.

  • For dashboards and visualisation, see Grafana Dashboard for Redis.

Note: YAML files used in this tutorial are stored in docs/examples/redis folder in GitHub repository kubedb/docs.

Overview

  • KubeDB deploys Redis with a built-in redis_exporter sidecar that exposes metrics on port 56790.
  • ServiceMonitor (named {redis-name}-stats) is created automatically by KubeDB and tells Prometheus to scrape the exporter every 10 seconds.
  • PrometheusRule is created by the redis-alerts chart and contains all Redis alert definitions grouped by concern: database health, provisioner, ops-manager, and backups (Stash and KubeStash).
  • Prometheus Operator evaluates every rule expression every 30 seconds and fires matching alerts to AlertManager.
  • AlertManager groups, inhibits, and silences alerts, then routes them to configured receivers (Slack, email, PagerDuty, webhook, etc.).

Deploy Redis with Monitoring Enabled

At first, let’s deploy a Redis database with monitoring enabled. Below is the Redis object we are going to create.

apiVersion: kubedb.com/v1
kind: Redis
metadata:
  name: rd-alert-demo
  namespace: alert-redis
spec:
  version: "6.0.20"
  deletionPolicy: WipeOut
  storage:
    storageClassName: "local-path"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 1Gi
  monitor:
    agent: prometheus.io/operator
    prometheus:
      serviceMonitor:
        interval: 10s
        labels:
          release: prometheus

Here,

  • spec.monitor.agent: prometheus.io/operator tells KubeDB to create a ServiceMonitor resource managed by the Prometheus operator.
  • spec.monitor.prometheus.serviceMonitor.labels.release: prometheus adds the release: prometheus label to the created ServiceMonitor, matching the Prometheus serviceMonitorSelector so the target is discovered automatically.

Let’s create the namespace and the Redis resource.

$ kubectl create ns alert-redis
namespace/alert-redis created

$ kubectl apply -f https://github.com/kubedb/docs/raw/v2026.6.19/docs/examples/redis/monitoring/rd-alert-demo.yaml
redis.kubedb.com/rd-alert-demo created

Now, wait for the database to go into Ready state.

$ kubectl get redis -n alert-redis rd-alert-demo
NAME            VERSION   STATUS   AGE
rd-alert-demo   6.0.20    Ready    62m

KubeDB creates a dedicated stats service with the -stats suffix for monitoring.

$ kubectl get svc -n alert-redis --selector="app.kubernetes.io/instance=rd-alert-demo"
NAME                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)              AGE
rd-alert-demo         ClusterIP   10.43.236.232   <none>        6379/TCP             62m
rd-alert-demo-pods    ClusterIP   None            <none>        6379/TCP,16379/TCP   62m
rd-alert-demo-stats   ClusterIP   10.43.191.236   <none>        56790/TCP            62m

KubeDB also creates a ServiceMonitor that tells Prometheus where to scrape.

$ kubectl get servicemonitor -n alert-redis
NAME                  AGE
rd-alert-demo-stats   62m

Verify that the ServiceMonitor carries the release: prometheus label so Prometheus discovers it.

$ kubectl get servicemonitor -n alert-redis rd-alert-demo-stats \
    -o jsonpath='{.metadata.labels.release}'
prometheus

Step 1 — Install redis-alerts

The redis-alerts chart creates a PrometheusRule resource containing all Redis alert definitions grouped by concern: database health, provisioner, ops-manager, and backups (Stash / KubeStash).

Why the Helm release name matters

The chart derives the PromQL job/instance scoping (and the PrometheusRule name) from the Helm release name, not from a values field — so the release name must match the Redis object’s name (rd-alert-demo) for the rules to be correctly scoped to this instance.

The chart’s default label is release: kube-prometheus-stack, so we must also override it at install time to match the Prometheus ruleSelector.

Install

$ helm upgrade -i rd-alert-demo oci://ghcr.io/appscode-charts/redis-alerts \
    -n alert-redis \
    --create-namespace \
    --version=v2026.7.14 \
    --set form.alert.labels.release=prometheus
FlagValuePurpose
rd-alert-demo (release name)Scopes every PromQL expression to this instance (job="rd-alert-demo-stats")
-n alert-redisalert-redisInstalls the PrometheusRule in the same namespace as the database
form.alert.labels.releaseprometheusMatches the Prometheus ruleSelector so the rules are loaded

Verify the PrometheusRule is created

$ kubectl get prometheusrule -n alert-redis
NAME            AGE
rd-alert-demo   60m

Confirm the release: prometheus label is present.

$ kubectl get prometheusrule -n alert-redis rd-alert-demo \
    -o jsonpath='{.metadata.labels.release}'
prometheus

Confirm Prometheus loaded the rules

Port-forward the Prometheus UI and open the Status → Rule health page.

$ kubectl port-forward -n monitoring \
    svc/prometheus-kube-prometheus-prometheus 9090:9090

Open http://localhost:9090/rules?search=rd-alert-demo.

Prometheus Rule Health

The redis.database.alert-redis.rd-alert-demo.rules group (and the accompanying redis.kubeStash.alert-redis.rd-alert-demo.rules group) is visible with all rules showing OK, confirming that Prometheus has loaded and is evaluating the Redis alert definitions every 30 seconds.


Verify End-to-End

1. Check the exporter is running

The exporter sidecar inside the Redis pod serves metrics at :56790/metrics. A value of redis_up 1 confirms the exporter can reach Redis.

$ kubectl exec -n alert-redis rd-alert-demo-0 -c exporter -- \
    wget -qO- localhost:56790/metrics | grep redis_up
redis_up 1

2. Check the Prometheus target is UP

Open http://localhost:9090/targets?search=rd-alert-demo. Prometheus discovers more than 20 scrape pools on this cluster, so it will ask you to pick one from the dropdown — select serviceMonitor/alert-redis/rd-alert-demo-stats/0.

Prometheus Target UP

The target serviceMonitor/alert-redis/rd-alert-demo-stats/0 shows UP, confirming metrics are being scraped from rd-alert-demo-0 in the alert-redis namespace.

3. Confirm all Redis alerts are inactive

Open http://localhost:9090/alerts?search=rd-alert-demo to see the Redis alert groups.

Prometheus Alerts — All Inactive

All 7 rules in the redis.database group show INACTIVE (7), meaning the database is healthy and no thresholds are breached.

4. Check AlertManager

Port-forward AlertManager to view any currently firing alerts.

$ kubectl port-forward -n monitoring \
    svc/prometheus-kube-prometheus-alertmanager 9093:9093

Open http://localhost:9093. With a healthy Redis instance, no alerts for rd-alert-demo will be listed here.


Simulating a Firing Alert

The previous section confirmed that all alerts are INACTIVE while the database is healthy. This section walks through deliberately triggering the RedisDown critical alert so you can observe the full alert lifecycle — from firing in Prometheus through to the AlertManager dashboard — and then resolve it.

Unlike some other database exporters, the Redis exporter runs as a separate sidecar container that keeps running fine even if the main redis container crashes and gets restarted by Kubernetes. Because RedisDown requires the outage to persist for for: 2m (not instant), and Kubernetes tends to restart a crashed container within a few seconds, a single kill is usually not enough to keep Redis down long enough to breach the 2-minute window. In practice, repeatedly stopping the Redis process for a couple of minutes (so the container keeps crash-looping) reliably keeps redis_up at 0 for long enough for the alert to fire.

1. Stop the Redis process repeatedly

Shut down the redis process inside the pod. This crashes the main container so the exporter sidecar can no longer reach it and reports redis_up 0 on the next scrape, while Kubernetes restarts the crashed container in the background.

$ kubectl exec -n alert-redis rd-alert-demo-0 -c redis -- redis-cli shutdown nosave

Because Kubernetes restarts the container quickly, repeat this command every few seconds for about two minutes to keep Redis down continuously long enough for the for: 2m window on RedisDown to be satisfied:

$ while true; do
    kubectl exec -n alert-redis rd-alert-demo-0 -c redis -- redis-cli shutdown nosave >/dev/null 2>&1
    sleep 1
  done

Let this loop run for about two minutes, then move on to the next step (leave the loop running while you check).

2. Watch the alert fire in Prometheus

Open http://localhost:9090/alerts?search=rd-alert-demo.

Prometheus Alerts — RedisDown Firing

RedisDown moves from INACTIVE to FIRING once the redis_up == 0 condition has held continuously for the configured for: 2m duration. RedisMissingMaster fires alongside it, since with no master node visible the cluster is also missing a master.

3. Check the AlertManager dashboard

Open http://localhost:9093/#/alerts?filter=%7Bnamespace%3D%22alert-redis%22%7D.

AlertManager — RedisDown Firing

AlertManager shows the RedisDown alert (alongside KubeDBRedisPhaseNotReady, since the KubeDB operator also observes the pod is not ready). The alert card displays:

  • Severity: critical
  • pod: rd-alert-demo-0 in the alert-redis namespace
  • job: rd-alert-demo-stats
  • Started: timestamp when the alert first fired

AlertManager routes this alert to every receiver configured in your alertmanagerConfig (Slack, email, PagerDuty, webhook, etc.) based on your routing tree. If no receiver is configured, the alert is visible here but silently dropped.

4. Restore Redis

Stop the loop from step 1, then delete the pod so KubeDB recreates it cleanly.

$ kubectl delete pod -n alert-redis rd-alert-demo-0

Once redis_up returns to 1 continuously, Prometheus marks the alert INACTIVE again and AlertManager sends a resolved notification to all receivers.


Alert Reference

All alerts are scoped to the rd-alert-demo instance in the alert-redis namespace via the PromQL label filters job="rd-alert-demo-stats" and namespace="alert-redis".

Database Group

Fired based on live metrics from the Redis exporter.

AlertSeverityForWhat It Means
RedisDowncritical2mExporter cannot reach Redis — instance is down or crashed.
RedisMissingMastercritical2mRedis cluster has fewer nodes marked as master than expected.
RedisTooManyMasterscritical2mRedis cluster has more nodes marked as master than expected — possible split-brain.
RedisDisconnectedSlaveswarning2mRedis is not replicating for all slaves — review the replication status.
RedisTooManyConnectionswarning2mMore than 80% of maxclients are in use — client pool nearing exhaustion.
DiskUsageHighwarning5mPersistent volume usage is between 80% and 95% — plan for capacity.
DiskAlmostFullcritical5mPersistent volume usage has exceeded 95% — the volume is nearly full.

Provisioner Group

Monitors the KubeDB operator’s view of the Redis resource phase.

AlertSeverityForWhat It Means
KubeDBRedisPhaseNotReadycritical1mKubeDB marked the Redis resource NotReady — operator cannot reach the database.
KubeDBRedisPhaseCriticalwarning15mThe instance is in a degraded/critical phase.

OpsManager Group

Tracks RedisOpsRequest lifecycle during upgrades, scaling, and reconfiguration.

AlertSeverityForWhat It Means
KubeDBRedisOpsRequestOnProgressinfoinstantA RedisOpsRequest is currently in progress.
KubeDBRedisOpsRequestStatusProgressingToLongcritical30mA RedisOpsRequest has been in progress for 30+ minutes — likely stuck.
KubeDBRedisOpsRequestFailedcriticalinstantA RedisOpsRequest failed — check the RedisOpsRequest object for the error.

Stash Group

Tracks Stash-driven backup/restore health for this instance. (You do not need to configure backups to see these rules; they are included in the PrometheusRule regardless.)

AlertSeverityForWhat It Means
RedisStashBackupSessionFailedcriticalinstantA Stash backup session failed.
RedisStashRestoreSessionFailedcriticalinstantA Stash restore session failed.
RedisStashNoBackupSessionForTooLongwarninginstantNo successful backup session for more than 18000s (5 hours).
RedisStashRepositoryCorruptedcritical5mThe Stash backup repository integrity check failed — repository is corrupted.
RedisStashRepositoryStorageRunningLowwarning5mBackup repository storage size has exceeded 10GB.
RedisStashBackupSessionPeriodTooLongwarninginstantA backup session took more than 1800s (30 minutes) to complete.
RedisStashRestoreSessionPeriodTooLongwarninginstantA restore session took more than 1800s (30 minutes) to complete.

KubeStash Group

Tracks KubeStash-driven backup/restore health for this instance. Same semantics as the Stash group above, sourced from KubeStash metrics instead.

AlertSeverityForWhat It Means
RedisKubeStashBackupSessionFailedcriticalinstantA KubeStash backup session failed.
RedisKubeStashRestoreSessionFailedcriticalinstantA KubeStash restore session failed.
RedisKubeStashNoBackupSessionForTooLongwarninginstantNo successful backup session for more than 18000s (5 hours).
RedisKubeStashRepositoryCorruptedcritical5mThe KubeStash backup repository integrity check failed — repository is corrupted.
RedisKubeStashRepositoryStorageRunningLowwarning5mBackup repository storage size has exceeded 10GB.
RedisKubeStashBackupSessionPeriodTooLongwarninginstantA backup session took more than 1800s (30 minutes) to complete.
RedisKubeStashRestoreSessionPeriodTooLongwarninginstantA restore session took more than 1800s (30 minutes) to complete.

Customising Alerts

To override thresholds or disable specific alert groups, create a custom values file and upgrade the chart.

# custom-alerts.yaml
form:
  alert:
    labels:
      release: prometheus
    groups:
      database:
        enabled: warning
        rules:
          redisTooManyConnections:
            enabled: true
            duration: "5m"
            val: 90        # fire at 90% of maxclients instead of the default 80%
            severity: warning
      opsManager:
        enabled: "none"    # disable all ops-manager alerts
$ helm upgrade rd-alert-demo oci://ghcr.io/appscode-charts/redis-alerts \
    -n alert-redis \
    --version=v2026.7.14 \
    -f custom-alerts.yaml

Cleaning up

To remove all resources created in this tutorial, run the following commands.

# Remove the redis-alerts release
$ helm uninstall rd-alert-demo -n alert-redis

# Remove the Redis instance
$ kubectl delete redis -n alert-redis rd-alert-demo

# Delete namespace
$ kubectl delete ns alert-redis

Next Steps