New to KubeDB? Please start here.

Neo4j Alerting with Prometheus

This tutorial shows you how to configure Prometheus-based alerting for a KubeDB-managed Neo4j cluster using the neo4j-alerts Helm chart. Unlike most other *-alerts charts, neo4j-alerts also bundles a Grafana dashboard that it imports automatically through a post-install Job — no separate dashboard chart is required.

Before You Begin

• Ensure you have a Kubernetes cluster and that kubectl is configured to communicate with it. If you do not already have a cluster, you can create one using kind.

• Install the KubeDB operator by following the steps here.

• Deploy the database in the alert-neo4j namespace:

  $ kubectl create ns alert-neo4j
  namespace/alert-neo4j created
  

• Before proceeding, complete the Configuration steps to deploy kube-prometheus-stack and Panopticon.

• This tutorial assumes you already have a kube-prometheus-stack running in your cluster, with Prometheus configured so that both serviceMonitorSelector and ruleSelector match the label release: prometheus.

  To verify the selectors:

  $ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.ruleSelector}'
  {"matchLabels":{"release":"prometheus"}}
  
  $ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}'
  {"matchLabels":{"release":"prometheus"}}
  

• To learn more about how Prometheus monitoring works with KubeDB, see the overview here.

• You will also need a Grafana API key / token with Editor permission so the chart’s dashboard-import Job can push the dashboard. See Step 2 below.

Note: YAML files used in this tutorial are stored in docs/examples/neo4j folder in GitHub repository kubedb/docs.

Overview

• KubeDB deploys Neo4j with metrics exposed directly by the neo4j container itself on port 2004 (Neo4j’s built-in Prometheus metrics endpoint) — there is no separate exporter sidecar.
• ServiceMonitor (named {neo4j-name}-stats) is created automatically by KubeDB and tells Prometheus to scrape the metrics endpoint every 10 seconds.
• PrometheusRule is created by the neo4j-alerts chart and contains all Neo4j alert definitions grouped by concern: database health/resource usage and provisioner.
• Dashboard-import Job — when grafana.enabled is true (the default), the chart also creates a one-shot Job that POSTs a bundled dashboard JSON straight to your Grafana instance’s /api/dashboards/import endpoint.
• Prometheus Operator evaluates every rule expression every 30 seconds and fires matching alerts to AlertManager.
• AlertManager groups, inhibits, and silences alerts, then routes them to configured receivers (Slack, email, PagerDuty, webhook, etc.).

Deploy Neo4j with Monitoring Enabled

At first, let’s deploy a 3-node Neo4j cluster with monitoring enabled. Below is the Neo4j object we are going to create.

apiVersion: kubedb.com/v1alpha2
kind: Neo4j
metadata:
  name: neo4j-alert-demo
  namespace: alert-neo4j
spec:
  replicas: 3
  version: "2025.12.1"
  deletionPolicy: WipeOut
  storage:
    storageClassName: "local-path"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 2Gi
  monitor:
    agent: prometheus.io/operator
    prometheus:
      serviceMonitor:
        interval: 10s
        labels:
          release: prometheus

Here,

• spec.replicas: 3 creates a 3-member Neo4j cluster. Neo4j Enterprise clustering requires a minimum of 3 core members.
• spec.monitor.agent: prometheus.io/operator tells KubeDB to create a ServiceMonitor resource managed by the Prometheus operator.
• spec.monitor.prometheus.serviceMonitor.labels.release: prometheus adds the release: prometheus label to the created ServiceMonitor, matching the Prometheus serviceMonitorSelector so the target is discovered automatically.

Let’s create the namespace and the Neo4j resource.

$ kubectl create ns alert-neo4j
namespace/alert-neo4j created

$ kubectl apply -f https://github.com/kubedb/docs/raw/v2026.6.19/docs/examples/neo4j/monitoring/neo4j-alert-demo.yaml
neo4j.kubedb.com/neo4j-alert-demo created

Now, wait for the database to go into Ready state.

$ kubectl get neo4j -n alert-neo4j neo4j-alert-demo
NAME               VERSION     STATUS   AGE
neo4j-alert-demo   2025.12.1   Ready    3m

KubeDB creates a dedicated stats service with the -stats suffix for monitoring.

$ kubectl get svc -n alert-neo4j --selector="app.kubernetes.io/instance=neo4j-alert-demo"
NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                 AGE
neo4j-alert-demo         ClusterIP   10.43.30.142    <none>        6362/TCP,7687/TCP,7474/TCP                              3m
neo4j-alert-demo-0       ClusterIP   None            <none>        6362/TCP,7687/TCP,7474/TCP,7688/TCP,7000/TCP,6000/TCP   3m
neo4j-alert-demo-1       ClusterIP   None            <none>        6362/TCP,7687/TCP,7474/TCP,7688/TCP,7000/TCP,6000/TCP   3m
neo4j-alert-demo-2       ClusterIP   None            <none>        6362/TCP,7687/TCP,7474/TCP,7688/TCP,7000/TCP,6000/TCP   3m
neo4j-alert-demo-stats   ClusterIP   10.43.63.217    <none>        2004/TCP                                                3m

KubeDB also creates a ServiceMonitor that tells Prometheus where to scrape.

$ kubectl get servicemonitor -n alert-neo4j
NAME                     AGE
neo4j-alert-demo-stats   3m

Verify that the ServiceMonitor carries the release: prometheus label so Prometheus discovers it.

$ kubectl get servicemonitor -n alert-neo4j neo4j-alert-demo-stats \
    -o jsonpath='{.metadata.labels.release}'
prometheus

Step 1 — Create a Grafana API Key

The chart’s dashboard-import Job authenticates to Grafana with a bearer token, so create one first.

• Grafana 9+: Administration → Service accounts → Add service account → role Editor → Add token. Copy the token.
• Grafana 8.x and earlier (no Service Accounts UI, e.g. the bundled kube-prometheus-stack Grafana 7.5.5 used while verifying this tutorial): use the legacy API Keys endpoint instead:

# Port-forward Grafana
$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80

# Retrieve the admin password
$ kubectl get secret -n monitoring prometheus-grafana \
   -o jsonpath='{.data.admin-password}' | base64 -d && echo

# Create a service account with Editor role
$ curl -s -X POST -H "Content-Type: application/json" \
   -u admin:<grafana_password> \
   http://localhost:3000/api/serviceaccounts \
   -d '{"name":"neo4j-alerts-demo","role":"Editor"}'
# Note the returned "id"

# Create a token for the service account (replace <id> with the returned service account ID)
$ curl -s -X POST -H "Content-Type: application/json" \
   -u admin:<grafana_password> \
   http://localhost:3000/api/serviceaccounts/<id>/tokens \
   -d '{"name":"neo4j-alerts-demo-key","secondsToLive":0}'
# Note the returned "key"

# Stop the port-forward
$ kill %1

Either way, you end up with a bearer token to use as grafana.apikey below.

Step 2 — Install neo4j-alerts

The neo4j-alerts chart creates a PrometheusRule resource containing all Neo4j alert definitions, and (by default) a Job that imports a pre-built Grafana dashboard.

Why the Helm release name matters

The chart derives the PromQL pod/container scoping (via .Release.Name/.Release.Namespace) and the PrometheusRule name from the Helm release name, not from a values field — so the release name must match the Neo4j object’s name (neo4j-alert-demo) for the rules to be correctly scoped to this instance.

The chart’s default label is release: kube-prometheus-stack, so we must also override it at install time to match the Prometheus ruleSelector.

Install

$ helm upgrade -i neo4j-alert-demo appscode/neo4j-alerts \
    -n alert-neo4j \
    --create-namespace \
    --version=v2026.7.14 \
    --set form.alert.labels.release=prometheus \
    --set grafana.enabled=true \
    --set grafana.url="http://prometheus-grafana.monitoring.svc:80" \
    --set grafana.apikey="<grafana-token-from-step-1>"

To install alerts only, without the dashboard, set --set grafana.enabled=false.

Verify the PrometheusRule is created

$ kubectl get prometheusrule -n alert-neo4j
NAME               AGE
neo4j-alert-demo   30s

Confirm the release: prometheus label is present.

$ kubectl get prometheusrule -n alert-neo4j neo4j-alert-demo \
    -o jsonpath='{.metadata.labels.release}'
prometheus

Verify the dashboard-import Job

$ kubectl get job -n alert-neo4j
NAME                        STATUS     COMPLETIONS   AGE
neo4j-alert-demo-post-job   Complete   1/1           17s

$ kubectl logs -n alert-neo4j job/neo4j-alert-demo-post-job
{"pluginId":"","title":"kubedb.com / Neo4j / alert-neo4j / neo4j-alert-demo","imported":true, ...}

A "imported":true response confirms the dashboard kubedb.com / Neo4j / alert-neo4j / neo4j-alert-demo now exists in Grafana.

Confirm Prometheus loaded the rules

Port-forward the Prometheus UI.

$ kubectl port-forward -n monitoring \
    svc/prometheus-kube-prometheus-prometheus 9090:9090

Open http://localhost:9090/rules and locate the neo4j.database and neo4j.provisioner groups.

Both groups are visible with all 10 rules showing OK, confirming that Prometheus has loaded and is evaluating the Neo4j alert definitions every 30 seconds.

Verify End-to-End

1. Check the metrics endpoint

The neo4j container serves its own Prometheus metrics at :2004/metrics — no exporter sidecar is involved.

$ kubectl exec -n alert-neo4j neo4j-alert-demo-0 -c neo4j -- \
                                      wget -qO- localhost:2004/metrics | grep neo4j_dbms_page_cache_hit_ratio
# HELP neo4j_dbms_page_cache_hit_ratio Generated from Dropwizard metric import (metric=neo4j.dbms.page_cache.hit_ratio, type=com.neo4j.metrics.source.db.PageCacheHitRatioGauge)
# TYPE neo4j_dbms_page_cache_hit_ratio gauge
neo4j_dbms_page_cache_hit_ratio 1.0

2. Check the Prometheus target is UP

Prometheus discovers more than 20 scrape pools on a shared cluster, so instead of the Target health page, query up directly for a reliable view.

Open http://localhost:9090/query?g0.expr=up%7Bnamespace%3D%22alert-neo4j%22%7D&g0.tab=1.

All three neo4j-alert-demo-{0,1,2} pods report up == 1, confirming Prometheus is scraping every pod in the cluster.

3. Confirm the Neo4j alerts are inactive

Open http://localhost:9090/alerts.

All 10 rules across the neo4j.database and neo4j.provisioner groups show INACTIVE, confirming the cluster is healthy and no alert thresholds are breached.

4. Check AlertManager

Port-forward AlertManager to view any currently firing alerts.

$ kubectl port-forward -n monitoring \
    svc/prometheus-kube-prometheus-alertmanager 9093:9093

Open http://localhost:9093.

No alerts are firing for the alert-neo4j namespace.

5. Explore the Grafana dashboard

Port-forward Grafana and log in.

$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80

Open http://localhost:3000 and navigate to the dashboard kubedb.com / Neo4j / alert-neo4j / neo4j-alert-demo that the Job imported in Step 2.

The dashboard mirrors the alert groups: Neo4j Phase & Availability (Down / Critical Phase), Neo4j Resource Usage (CPU, memory, disk), and Neo4j Page Cache Metrics (usage ratio, hit ratio, page faults). Note that Neo4j High CPU Usage shows “No data” on clusters without cAdvisor’s container_cpu_usage_seconds_total metric exposed (common on some k3s setups) — this is an environment limitation, not a chart bug.

Simulating a Firing Alert

The previous section showed that all genuine health alerts (everything except the buggy disk rules) are INACTIVE while the cluster is healthy. This section deliberately triggers the KubeDBNeo4jPhaseNotReady critical alert so you can observe the full alert lifecycle — from firing in Prometheus through to the AlertManager dashboard — and then resolve it.

Neo4j runs as a single container per pod — there is no separate exporter sidecar to keep reporting metrics once the database process dies, and Kubernetes will restart a killed process within seconds. Because KubeDBNeo4jPhaseNotReady requires the condition to persist for for: 1m, a single kill is not enough: you need to keep the pods crashing long enough for the KubeDB operator to mark the resource NotReady and hold it there past the one-minute window.

1. Crash the Neo4j process repeatedly

$ while true; do
    for i in 0 1 2; do
      kubectl exec -n alert-neo4j neo4j-alert-demo-$i -c neo4j -- kill 1 >/dev/null 2>&1
    done
    sleep 3
  done

Let this loop run for a couple of minutes (leave it running while you check the next steps), then stop it once you’ve captured the firing state.

2. Watch the alert fire in Prometheus

Open http://localhost:9090/alerts.

KubeDBNeo4jPhaseNotReady transitions from INACTIVE to FIRING once kubedb_com_neo4j_status_phase{phase="NotReady"} has read 1 continuously for the full for: 1m duration — this metric comes from the KubeDB operator’s own view of the resource (exported via Panopticon), not from the Neo4j metrics endpoint itself.

3. Check the AlertManager dashboard

Open http://localhost:9093.

AlertManager shows the KubeDBNeo4jPhaseNotReady alert. The alert card displays:

• Severity: critical
• neo4j: neo4j-alert-demo in the alert-neo4j namespace
• phase: NotReady
• Started: timestamp when the alert first fired

AlertManager routes this alert to every receiver configured in your alertmanagerConfig (Slack, email, PagerDuty, webhook, etc.) based on your routing tree. If no receiver is configured, the alert is visible here but silently dropped.

4. Restore Neo4j

Stop the loop from step 1. The pods recover on their own — KubeDB just needs a few uninterrupted scrape/reconcile cycles to mark the resource Ready again.

$ kubectl get neo4j -n alert-neo4j neo4j-alert-demo -w
NAME               VERSION     STATUS   AGE
neo4j-alert-demo   2025.12.1   Ready    24m

Once the phase returns to Ready, Prometheus marks the alert INACTIVE again and AlertManager sends a resolved notification to all receivers.

Alert Reference

All alerts are scoped to the neo4j-alert-demo instance in the alert-neo4j namespace via the PromQL label filters pod=~"neo4j-alert-demo-.+$" and namespace=~"alert-neo4j" (database group), or app="neo4j-alert-demo" and namespace="alert-neo4j" (provisioner group).

Database Group

Fired based on live metrics from the Neo4j container’s built-in metrics endpoint and node/kubelet metrics.

Provisioner Group

Monitors the KubeDB operator’s view of the Neo4j resource phase (sourced from Panopticon, not the Neo4j metrics endpoint).

Customising Alerts

To override thresholds or disable specific alert groups, create a custom values file and upgrade the chart.

# custom-alerts.yaml
form:
  alert:
    labels:
      release: prometheus
    groups:
      database:
        enabled: warning
        rules:
          neo4jHighMemoryUsage:
            enabled: true
            duration: "5m"
            val: 90        # fire at 90% instead of the default 80%
            severity: warning
      provisioner:
        enabled: "none"    # disable all provisioner alerts

$ helm upgrade neo4j-alert-demo appscode/neo4j-alerts \
    -n alert-neo4j \
    --version=v2026.7.14 \
    --set grafana.enabled=false \
    -f custom-alerts.yaml

Note: -f values files don’t merge grafana.url/grafana.apikey automatically — re-pass them (or set grafana.enabled=false) on every helm upgrade, otherwise the dashboard-import Job re-runs with an empty URL/token and fails.

Cleaning up

To remove all resources created in this tutorial, run the following commands.

# Remove the neo4j-alerts release (PrometheusRule + dashboard-import Job)
$ helm uninstall neo4j-alert-demo -n alert-neo4j

# Remove the imported Grafana dashboard (it is not removed by helm uninstall)
$ curl -s -X DELETE -H "Authorization: Bearer <grafana-token>" \
    http://localhost:3000/api/dashboards/uid/lhSzgLYDk

# Remove the Neo4j instance
$ kubectl delete neo4j -n alert-neo4j neo4j-alert-demo

# Delete namespace
$ kubectl delete ns alert-neo4j

Next Steps

• Monitor your Neo4j database with KubeDB using built-in Prometheus.
• Monitor your Neo4j database with KubeDB using Prometheus operator.
• Use private Docker registry to deploy Neo4j with KubeDB.
• Want to hack on KubeDB? Check our contribution guidelines.