Step 1: take a simple script
#!/usr/bin/env bash
workingServices=$(docker service ls --format "{{.Replicas}}" | grep '2/2|1/1' | wc -l)
allServices=$(docker service ls --format "{{.Replicas}}" | wc -l)
if [ $workingServices -lt $allServices ]
then
curl --request POST --header "Content-Type: application/json" --header "Authorization: Token token=__THIS_IS_SECRET__" --data '{"routing_key":"__THIS_IS_ALSO_SECRET__","payload": {"summary":"Docker Swarm - Service count","source":"instanceId","severity": "critical"},"event_action": "trigger"}' https://events.pagerduty.com/v2/enqueue
else
echo "All services are working at "$(date)
fiLet’s call it docker-service-count.sh.
So this is a script that’s sending PagerDuty events when something doesn’t match. In this example, we’re looking for 2/2 or 1/1 matches but this can be easily extended.
Step 2: make sure the script is able to be executed
chmod +x docker-service-count.shConfigure Cron to execute this awesome script on a regular interval. In this example, we’re running it every 5 minutes.
Simple, clean and easy to set up. This is magic!
Personal note: Pager Duty is just a personal preference.