Tags: twiny/wbot
Tags
feat(crawler): fixing crawl functionality (#5) ## Motive this MR to fix issue with the crawling functionallity of the bot. the issue was with default fetcher that was using the `http.Client` that and it will block if the request takes too long to respond. the fix was to use `http.Client` with a timeout and a custom transport that will limit the number of concurrent requests. ## Changes - removed `OnError` Method. - added zerolog logger. - refactor metrics monitoring and add new metrics - reverted to simple queue. - timeout to HTTP client. - update gitignore.