Thanks to visit codestin.com
Credit goes to Github.com

Skip to content

πŸ“Ÿ Trigger a PagerDuty incident when Freshping detects that a service is down.

License

Notifications You must be signed in to change notification settings

Aldaviva/FreshPager

Freshping FreshPager

GitHub Actions Testspace Coveralls

When Freshping detects an outage, trigger an alert in PagerDuty (and resolve when it's up again).

    πŸ“Ÿ Convert Freshping check failures into PagerDuty incidents.
    🚨 Turn on a smart outlet when an incident is triggered.
    🍞 Show an interactive desktop toast notification when an incident is triggered.

This is helpful because Freshping's only built-in notification system is email, and the Gmail Android app is extremely slow to notify you of new messages: notifications are often hours late. PagerDuty, on the other hand, has realtime alerting with SMS messages and mobile app push notifications for Android and iOS.

This is a free, open-source, no sign-up, self-hostable alternative to the Zapier Freshping + PagerDuty integration.

Prerequisites

Freshping Checks β†’ PagerDuty Incidents

Installation

  1. Download the ZIP file for your operating system and CPU architecture from the latest release page.
  2. Extract the ZIP file to a directory of your choice, such as C:\Program Files\FreshPager\ or /opt/freshpager/.
    • When installing updates, don't overwrite appsettings.json.
  3. Install the service so it will start automatically when your computer boots.
    • Windows: & '.\Install service.ps1'
      • If this PowerShell script doesn't run, try removing the Mark of the Web by unblocking the file or calling Set-ExecutionPolicy RemoteSigned.
    • Linux with systemd:
      sudo cp freshpager.service /etc/systemd/system/
      sudo systemctl daemon-reload
      sudo systemctl enable freshpager.service
      • If the installation directory is not /opt/freshpager/, make sure to edit freshpager.service to match.

Configuration

  1. Create an Integration in PagerDuty and get its Integration Key.
    1. Sign into your PagerDuty account.
    2. Go to Services β€Ί Service Directory.
    3. Select an existing Service for which you want to publish events, or create a new Service.
    4. In the Integrations tab of the Service, add a new Integration.
    5. Under Most popular integrations, select Events API V2, then click Add.
    6. Expand the newly-created Integration and copy its Integration Key, which will be used to authorize this program to send Events to the correct Service.
  2. Edit the appsettings.json configuration file.
    Key Example Value Description
    pagerDutyIntegrationKeysByFreshpingCheckId
    { "123": "abc…" }
    Object where each key is the numeric ID of a check in Freshping (go to a Report page and copy the value of the check_id query parameter), and its value is the Integration Key you created for the matching PagerDuty Service in Step 1.
    kestrel.endpoints.https.url http://0.0.0.0:37374 URI specifying the hostname and TCP port on which to listen for HTTP requests from the Freshping webhook client. Must be publicly accessible on the WAN.
    pagerDutyWebhookSecrets
    ["u565tpiQ3TnI…"]
    Array of one or more PagerDuty webhook signing secrets, which are generated by PagerDuty when you create a new webhook subscription. Only required if you want to turn on smart outlets or show toast notifications on Windows.
    alarmLightUrl tcp://192.168.1.100/0 URL with the hostname of a Kasa smart outlet that you want to turn on when a PagerDuty incident is triggered. Only required when controlling an outlet. The path is a 0-indexed integer that identifies which socket on the outlet to control, which can be 0 or the empty string if the outlet only has one socket. The scheme doesn't matter. Requires a PagerDuty Webhook and secret.
    alarmLightPagerDutySubdomains
    ["mycompany"]
    Optional whitelist of PagerDuty account subdomains which should trigger the alarm light, useful if your server receives webhooks from multiple PagerDuty accounts but only some of them should turn on the alarm light. When omitted, events from all PagerDuty accounts will turn on the light, if it is configured.
  3. Create a Webhook integration in Freshping.
    1. Sign into your Freshworks account.
    2. Go to your Freshping Dashboard.
    3. Go to Settings β€Ί Integrations.
    4. Under Webhook, select + Create Integration.
    5. Set the Webook Name to any name you want.
    6. Set the Event Type to Up/Down.
    7. Select the Checks that should trigger the alert.
    8. Set the Callback URL to the location of your FreshPager server, such as http://myserver.example.com:37374/.
    9. Leave the request body set to Simple.
    10. Click Save.
  4. Create a Webhook integration in PagerDuty if you want to turn on a smart outlet or show toast notifications on Windows when incidents are triggered.
    1. Sign into your PagerDuty account.
    2. Go to Integrations β€Ί Generic Webhooks (v3).
    3. Click + New Webhook.
    4. Set the Webhook URL to the location of your FreshPager server, such as http://myserver.example.com:37374/.
    5. Choose whether events should be fired for all Services in your account, or just one Service.
    6. Choose which Events should be fired.
    7. Click Add Webhook.
    8. Copy the Signing Secret to the pagerDutyWebhookSecrets array in the FreshPager configuration JSON file.

Execution

  1. Start the service.
    • Windows: Restart-Service Freshpager
    • Linux with systemd: sudo systemctl restart freshpager.service

Signal Flow

  1. Freshping Freshping detects and confirms that a Check is down.
  2. Freshping Freshping sends an HTTP POST request to each Webhook integration subscribed to Up/Down events on that Check.
  3. Your Aldaviva FreshPager server receives the HTTP POST request from Freshping Freshping.
  4. Aldaviva FreshPager looks up the Integration Key in its configuration based on the Check ID from the request body.
  5. Aldaviva FreshPager sends an Events API V2 request to PagerDuty PagerDuty to trigger an alert on the Service that contains the Integration Key.
  6. PagerDuty PagerDuty creates a new incident for this alert, and returns a unique key for this alert, which Aldaviva FreshPager stores in memory.
  7. PagerDuty PagerDuty sends an HTTP POST request to Aldaviva FreshPager with the newly triggered incident.
  8. If configured to do so, Aldaviva FreshPager sends a TCP JSON object to a Kasa smart outlet, commanding it to turn on its electrical socket.
  9. Aldaviva FreshPager sends a WebSocket message containing the newly triggered incident to any connected Aldaviva FreshPager.Toast clients, which show Windows toast notifications.
  10. When Freshping Freshping detects that the Check is up again, it sends another POST request to Aldaviva FreshPager, which resolves the previously-created PagerDuty PagerDuty alert using the same unique key. If they are configured, the smart outlet will be turned off, and the toasts will be dismissed.

PagerDuty Incidents β†’ Desktop Push Notifications

PagerDuty makes it easy to receive push notifications for triggered incidents through their Android and iOS apps, SMS, email, and PSTN calls. However, it's not as easy to get an obvious, actionable notification if you're only on a computer while your phone isn't nearby. Phone Link is very unreliable and emails are easy to miss.

You can solve this with a background program that runs on Windows, connects to the FreshPager server, and receives push notifications for triggered incidents which it shows as native Windows toasts. These are rich notifications with buttons to acknowledge or resolve the incident with one click. You can also click on the body of the toast to open the incident web page in your browser.

Test Service. Example incident. #55 Triggered

Installation

  1. Download FreshPager.Toast-win-x64 from the latest release page.
  2. Extract the EXE from the ZIP file to your hard drive.
  3. Register this program to start when you log into Windows by adding its absolute path to a new string value in the registry key HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Run.

Configuration

  1. Create a configuration JSON file.
    1. Save the example configuration file to %appdata%\FreshPager\Toast.config.json.
  2. Set the hubAddress to the URL of your FreshPager server, with the path /pagerduty/toasts. This connects the Toast client to the WebSocket server.
  3. Replace the myorg key in the pagerDutyAccountsBySubdomain object with the subdomain of your PagerDuty organization (the part of the hostname before the .pagerduty.com base domain in the web interface, not including that base domain).
    • This is used to determine which organization of the following information should be used for a given incident, if you are subscribed to webhooks from multiple organizations.
  4. Set the userId of the organization object to the ID of your user, which is the last path segment of your PagerDuty profile page accessible from People β€Ί Users.
    • This is used to record which user acknowledged or resolved an incident.
  5. Set the userEmailAddress to your account's email address.
    • This prevents you from receiving toasts for incidents which are assigned to different people.
  6. Create an API access key in PagerDuty.
    1. Sign into your PagerDuty account.
    2. Go to Integrations β€Ί API Access Keys.
    3. Click + Create New API Key.
    4. Enter a descriptive name, and make sure Read-only API Key is unchecked.
    5. Click Create Key.
    6. Set the apiAccessKey in Toast.config.json to this new access key.
    • This is used to acknowledge and resolve incidents, since Events V2 Integration Keys are not able to do this because they control alerts, not incidents.

Execution

  1. Double-click the EXE file to start it. It will run in the background and not show any UI.

About

πŸ“Ÿ Trigger a PagerDuty incident when Freshping detects that a service is down.

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Sponsor this project

Contributors 2

  •  
  •  

Languages