Thanks to visit codestin.com
Credit goes to github.com

Skip to content
@scrapy

Scrapy project

An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

Pinned Loading

  1. scrapy scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python 64.3k 12k

  2. scrapyd scrapyd Public

    A service daemon to run Scrapy spiders

    Python 3.1k 579

  3. parsel parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    Python 1.4k 172

  4. w3lib w3lib Public

    Python library of web-related functions

    Python 417 122

  5. protego protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    DIGITAL Command Language 95 34

  6. itemadapter itemadapter Public

    Common interface for data container classes

    Python 69 16

Repositories

Showing 10 of 33 repositories
  • parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    scrapy/parsel's past year of commit activity
    Python 1,354 BSD-3-Clause 172 14 11 Updated Sep 13, 2026
  • w3lib Public

    Python library of web-related functions

    scrapy/w3lib's past year of commit activity
    Python 417 BSD-3-Clause 122 4 10 Updated Sep 13, 2026
  • scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    scrapy/scrapy's past year of commit activity
    Python 64,320 BSD-3-Clause 11,956 213 (9 issues need help) 181 Updated Sep 10, 2026
  • scrapy-agent-plugin Public

    Scrapy plugin for agents

    scrapy/scrapy-agent-plugin's past year of commit activity
    4 Apache-2.0 0 0 0 Updated Sep 10, 2026
  • protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    scrapy/protego's past year of commit activity
    DIGITAL Command Language 95 BSD-3-Clause 34 3 2 Updated Sep 8, 2026
  • itemadapter Public

    Common interface for data container classes

    scrapy/itemadapter's past year of commit activity
    Python 69 BSD-3-Clause 16 4 3 Updated Sep 8, 2026
  • scrapy-lint Public

    A linter for Scrapy projects.

    scrapy/scrapy-lint's past year of commit activity
    Python 22 MIT 7 49 (2 issues need help) 46 Updated Sep 7, 2026
  • queuelib Public

    Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

    scrapy/queuelib's past year of commit activity
    Python 300 BSD-3-Clause 56 0 0 Updated Sep 7, 2026
  • scrapy-mcp-official Public

    Scrapy MCP server

    scrapy/scrapy-mcp-official's past year of commit activity
    Python 6 Apache-2.0 1 0 0 Updated Sep 7, 2026
  • frostwork Public
    scrapy/frostwork's past year of commit activity
    Python 12 Apache-2.0 1 0 0 Updated Sep 5, 2026
View all repositories