CrawlerDetectBundle

Notes for Tempest:

This is a largely unmodified repo forked from nicolasmure's github as referenced below. That package appears to be abandoned and was stuck on an older version of Symfony, so this package is meant to just maintain the Symfony wrapper for Symfony 6+.

That means we have a few things we need to do now that we're maintaining this repo:

• Update the Symfony wiring as the Symfony core is updated in the Tempest applications (AutoTempest, SearchTempest)
• Keep the jaybizzle/crawler-detect bundle up-to-date as new versions are released. (See here: https://github.com/JayBizzle/Crawler-Detect)

A Symfony bundle for the Crawler-Detect library (detects bots/crawlers/spiders via the user agent).

Table of contents

• Introduction
• Installation
• Usage
• Testing

Introduction

This Bundle integrates the Crawler-Detect library into Symfony. It is recommended to read the lib's documentation before continuing here.

The aim of this bundle is to expose the CrawlerDetect class as a service (crawler_detect) to make it easier to use with Symfony (dependency injection, usable from a controller, etc...).

Installation

Download the bundle using composer :

$ composer require nmure/crawler-detect-bundle "^2.0.0"

For Symfony < 4.0, run :

$ composer require nmure/crawler-detect-bundle "^1.0.0"

then enable the bundle in your AppKernel :

// app/AppKernel.php
class AppKernel extends Kernel
{
    public function registerBundles()
    {
        $bundles = array(
            // ...
            new Nmure\CrawlerDetectBundle\CrawlerDetectBundle(),
            // ...
        );
    }
}

Usage

The crawler_detect service is initialized with the data from the Symfony's master request.

To use this service from a controller :

public function indexAction()
{
    if ($this->get('crawler_detect')->isCrawler()) {
        // this request is from a crawler :)
    }

    // you can also specify an user agent if you don't want
    // to use the one of the master request or if the app
    // is accessed by the CLI :
    $ua = 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)';
    if ($this->get('crawler_detect')->isCrawler($ua)) {
        // this user agent belongs to a crawler :)
    }
}

You can also inject this service as a dependency using the crawler_detect service id.

Testing

$ docker run --rm -v `pwd`:/app phpunit/phpunit -c /app