Monitoring OpenStreetMap with Datadog
The work in progress can be seen as this URL
This script relies on those env variables :
- DD_API_KEY (mandatory)
- DD_TAGS (recommended), should a string of comma separated values (ex :
"env:staging,version:5.1") - DD_SITE (optional, defaults to "datadoghq.com")
Run this script every 15 to 30 seconds.
I have iframed the Datadog dashboard in a more readable domain name here :
Previous TODO list
- Send the 100 most recents changesets to Datadog
- Prevent sending duplicates
- log API calls are not optimized : for now doing one API call per changeset
- Need to leverage
<tag />within the changeset - Parse box to extract countries values to be able to use the Geomap Widget. Idea : https://pypi.org/project/reverse_geocoder/
- Clean editors (solution done on Datadog side, "editor family" created)
- add log creation to the utilities
- Clean imageries
- Add some try / catches with better logging
- Share Datadog pipelines
- Use case : need to be alerted when a bad changeset occurs (vandalism). First need definition of a bad changeset : having a lot of deletion?
- How to prevent counting some changeset multiple times?
- Some changeset have a discussion, so they might be pushed to the replica at each discussion update : example with changeset
118161820inhttps://planet.osm.org/replication/changesets/004/878/999.osm.gz
TODO has been moved to my personnal website for easier tracking :
If you have any questions / feedback / ideas, feel free to open an issue or ping me at [email protected]