When building this solution the Github API couldn't tell if a runner was busy or not, so we resorted to trying to delete each runner via de API. If that returned a 500 Internal Server Error, we would know if it was busy.

Just played with the API again and saw the busy flag was added for runners. See https://docs.github.com/en/rest/reference/actions#list-self-hosted-runners-for-a-repository

Instead of trying to delete a runner we should use this flag so reduce the number of API calls on Github.