-
-
Notifications
You must be signed in to change notification settings - Fork 9.6k
[HttpKernel] Prevent search engines from indexing dev applications #30325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
src/Symfony/Component/HttpKernel/EventListener/RobotsTagListener.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/EventListener/RobotsTagListener.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/EventListener/RobotsTagListener.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/EventListener/RobotsTagListener.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/Tests/EventListener/RobotsTagListenerTest.php
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice idea.
src/Symfony/Component/HttpKernel/Tests/EventListener/RobotsTagListenerTest.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/Tests/EventListener/RobotsTagListenerTest.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/EventListener/RobotsTagListener.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/EventListener/RobotsTagListener.php
Outdated
Show resolved
Hide resolved
src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php
Outdated
Show resolved
Hide resolved
29b9d02
to
d294c4f
Compare
@GaryPEGEOT Please, don't change code that your PR does not change (even if fabbot ask for it). |
src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Configuration.php
Outdated
Show resolved
Hide resolved
src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Configuration.php
Outdated
Show resolved
Hide resolved
@javiereguiluz where should I document the new option? |
src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php
Outdated
Show resolved
Hide resolved
src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php
Outdated
Show resolved
Hide resolved
src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Configuration.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/Tests/EventListener/DisallowRobotsIndexingListenerTest.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/Tests/EventListener/DisallowRobotsIndexingListenerTest.php
Outdated
Show resolved
Hide resolved
afa4a09
to
5b7f836
Compare
src/Symfony/Component/HttpKernel/EventListener/DisallowRobotsIndexingListener.php
Outdated
Show resolved
Hide resolved
src/Symfony/Component/HttpKernel/EventListener/DisallowRobotsIndexingListener.php
Outdated
Show resolved
Hide resolved
src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php
Outdated
Show resolved
Hide resolved
67ab198
to
3dd8671
Compare
Thank you @GaryPEGEOT. |
…applications (GaryPEGEOT) This PR was squashed before being merged into the 4.3-dev branch (closes #30325). Discussion ---------- [HttpKernel] Prevent search engines from indexing dev applications | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #30318 | License | MIT | Doc PR | TODO Add the *X-Robots-Tag: noindex* to dev (and test) applications to prevent search engines to index them. Commits ------- 3dd8671 [HttpKernel] Prevent search engines from indexing dev applications
@GaryPEGEOT sorry I didn't reply to your comment. This feature is being documented in symfony/symfony-docs#11148. Thanks. |
Sorry I'm late on this one, but this feature could have been a bit more generic: the config could allow to set headers that will always be sent, including I can work on a PR. |
@dunglas can you please show an example of the HTTP header/s added to the response with your proposal and a sample of the YAML config used to do that? Thanks. |
on the other side, having rich configuration with sane defaults, per case is nice. But agree this could be aggregated into a single default header listener 👍 |
framework:
default_headers:
Content-Security-Policy : "script-src 'self' www.google-analytics.com ajax.googleapis.com"
Referrer-Policy: same-origin
Strict-Transport-Security: "max-age=31536000; includeSubDomains; preload"
X-Robot-Tag: noindex
X-XSS-Protection: "1; mode=block"
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff We could even provide sane defaults. WDYT? |
I like this idea @dunglas ! Perhaps it should be under |
@dunglas are we completely sure that 100% of existing apps are going to keep working if we apply those headers by default? Thanks. |
@javiereguiluz no! By default I mean "in the config generated by the recipe". |
Please don't call other people "idiots". |
For all these security headers, I think using https://github.com/nelmio/NelmioSecurityBundle/ is better. Most of them require more configuration than just knowing whether we are in debug mode or no. And this |
@HTMLGuyLLC I agree with @linaori's comment and I ask you to please not call other people "idiots". About your original comment, I'm afraid that those people who "run Symfony in dev because they cannot make it work in prod" will need to try again to put things in prod and report here any problem that they find in Symfony itself. Thanks! |
Even if we don't go with an idea of having config for these like @dunglas suggested, at least listener class itself could be made in more generic way, so userland can write compiler pass with own rules. Having such single purpose listener which is completely inflexible seems such a waste to me. Changing it so headers can be injected should be trivial. edit: on second thought, problem with this might be be that it needs to be re-registered in prod :/ |
one can still provide default headers per environment config, given this node merges each. A sub config might want to clear an inherited header using
Agree, but for the simple case this config spares out some boilerplate code. The flip side is we get more feature requests eventually; like expression support; better security defaults. I think that's reasonable, but so is a code solution or an external bundle. Sticking with "config per case" avoids that decision path. |
…eguiluz) This PR was merged into the master branch. Discussion ---------- Document the disallow_search_engine_index option This documents symfony/symfony#30325. Commits ------- a94b920 Document the disallow_search_engine_index option
Add the X-Robots-Tag: noindex to dev (and test) applications to prevent search engines to index them.