List of libraries, tools and APIs for web scraping and data processing.
Async Python 3.6+ web scraping micro-framework based on asyncio
Web Scan Lazy Tools - Python Package
SpideyX a multipurpose Web Penetration Testing tool with asynchronous concurrent performance with multiple mode and configurations.
Crawlzone is a fast asynchronous internet crawling framework for PHP.
Vietnamese text data crawler scripts for various sites (including Youtube, Facebook, 4rum, news, ...)
๐ High productivity semi-automatic crawler generator ๐ ๏ธ๐งฐ
Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. Itโs the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis
Serritor is an open source web crawler framework built upon Selenium and written in Java. It can be used to crawl dynamic web pages that require JavaScript to render data.
๐ OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK ๐ค
A Python framework to build polite, but tenacious crawlers / scrapers with a MariaDB backend
Easily crawl news portals or blog sites using Storm Crawler.
Web crawling & scraping framework for Node.js on top of headless Chrome browser
๐ A powerful web-crawling framework, based on aiohttp.
An intelligent proxy server. Provide durable, real-time, high-quality proxies as a middleman or datasource server.
Crawler written in TypeScript using ES6 generators.
ๅบไบpythonๅ็จๆฑ ใ็จๆณ็ตๆดป็้ซๆง่ฝ็ฌ่ซๆกๆถ
A crawler program to extract all of the data and the price for symbols in the global stock exchange.
๐ THIS WEB SCRAPING TEMPLATE PROVIDES YOU WITH A GREAT STARTING POINT WHEN CREATING WEB SCRAPING BOTS. ๐ค
Useful functions for connecting to the network in the PHP based applications.
Add a description, image, and links to the crawling-framework topic page so that developers can more easily learn about it.
To associate your repository with the crawling-framework topic, visit your repo's landing page and select "manage topics."
