Nice idea.

Could be also named:

• test.AppBundle\Service\MyService
• AppBundle\Service\MyService.test

Not ideal but I avoided any pain by making all my services public thanks a compiler pass for my libraries...

I added this to the 4.0 milestone, so we will remember to do it. But a PR for this should go into 3.3 I believe.

... with a note that the config paths will need to change when merged into 4.0. We're going to be trying to remember to do that a lot :)

I'd suggest to prefix the aliases by test. as a convention we would advocate.

@nicolas-grekas I know you're not happy with it but I don't use the container as a service locator anywhere except in my tests so having this limitation makes really no sense to me.

I've got the choice between updating hundreds of service definitions and registering a compiler pass turning my library/application services into public services. I don't have much time for it, so I need to be pragmatic, the former solution is too much of a burden.

Another idea: maybe having a "non secure" container registered as a different service name than container which one could use during tests? This way you wouldn't need to change any alias but instead simply doing something like:

$kernel->getTestContainer()->get('foo');
// or
$kernel->getContainer()->get('test_container')->get('foo');

@6ecuk don't do that: if you happen to use a service in eg your controllers, your tests will be green, but your prod will fail. The worst combination of events.

@6ecuk you do as you want of course - but advocating this is giving a footgun to ppl

@6ecuk your snippet is incomplete as you are missing the aliases as well. But already mentioned I would strongly suggest against it if there is commands or controllers using the container as a service locator.

@nicolas-grekas what do you think of my second proposal? I think that would be a more acceptable compromise than the two extremes we currently see (alias everything or make everything public).

@theofidry a container is a compiled php class. The removed services are actually removed. So there is "hand" to get them. The only way to make that work is to dump another container just for tests. Which is actually what the compiler pass is doing. I don't think there is anything in between.

@theofidry The only is issue I see with your method is if I test a controller doing this:

$this->get('a.service.that.should.be.private');

It will works on your test, but not in production neither and dev env.

@soullivaneuh I'm aware of that issue which is why I advised against if you are using that. But in my case, I don't. Tests are the only place where I ever do this.

I tried to solve the problem with a service locator. I'm currently adding the following compiler pass in the test environment only:

namespace App\DependencyInjection;

use Symfony\Component\DependencyInjection\Compiler\CompilerPassInterface;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Reference;
use Symfony\Component\DependencyInjection\ServiceLocator;

class TestContainerPass implements CompilerPassInterface
{
    public function process(ContainerBuilder $container)
    {
        $services = array_keys($container->findTaggedServiceIds('test.container', true));

        foreach ($container->getDefinitions() as $id => $definition) {
            if ($definition->isPublic() && !$definition->isAbstract()) {
                $services[] = $id;
            }
        }

        foreach ($container->getAliases() as $id => $alias) {
            if ($alias->isPublic()) {
                $services[] = $id;
            }
        }

        $container->register('test.container', ServiceLocator::class)
            ->setPublic(true)
            ->addTag('container.service_locator')
            ->setArguments([
                array_combine($services, array_map(
                    function (string $id): Reference {
                        return new Reference($id);
                    },
                    $services
                ))
            ]);
    }
}

The result is a public service test.container which is a service locator that contains all services that have been tagged with test.container. So if I need to test a private service, I simply add this tag to that service.

I then use this service locator instead of the application container in my integration tests:

// old
$container = self::$kernel->getContainer();
// new
$container = self::$kernel->getContainer()->get('test.container');

The locator allows me to fetch those private services without tinkering with service visibilities or introducing prefixed fake services.

As a convenience feature, the locator also provides all public services from the application container, so I don't have to remember which service needs to be pulled from which container. If you don't want this, simply remove the foreach block.

I really do not like the solutions that require you to write extra yaml configuration. I believe Symfony has introduced the service autodiscovery to avoid it and now you are telling me I have to put extra three lines in yaml for every service I want to integration test?

I really appreciate the idea with a compiler pass though. However, tagging services still requires you to write yaml, so I ended up with something simpler (not necessarily "cleaner"):

<?php
namespace YourBundle\Tests;

use YourBundle\Services\AService;
use YourBundle\Services\BService;
use Symfony\Component\DependencyInjection\Compiler\CompilerPassInterface;
use Symfony\Component\DependencyInjection\ContainerBuilder;

/**
 * Makes enumerated private services that needs to be tested public 
 * so they can be fetched from the container without a deprecation warning.
 *
 * @see https://github.com/symfony/symfony-docs/issues/8097
 * @see https://github.com/symfony/symfony/issues/24543
 */
class TestContainerPass implements CompilerPassInterface {
    private static $PUBLIC_IN_TESTS = [
        AService::class,
        BService::class,
    ];

    public function process(ContainerBuilder $container) {
        foreach ($container->getDefinitions() as $id => $definition) {
            if (in_array($id, self::$PUBLIC_IN_TESTS, true)) {
                $definition->setPublic(true);
            }
        }
    }
}

Need to register it in your kernel:

class AppKernel extends Kernel {
    // ...
    protected function build(ContainerBuilder $container) {
        parent::build($container);
        if ($this->getEnvironment() === 'test') {
            $container->addCompilerPass(new TestContainerPass(), PassConfig::TYPE_OPTIMIZE);
        }
    }
}

@fracz I think that solution has been discussed in multiple variants in this thread already.

You will still create a situation in which services that are private in prod will be public during test execution. That is potentially dangerous. This solution might work for you, but it's not the one that should be advertised in the documentation, imho.

You are right and I'm aware of the danger here.

I still wonder why it is impossible to have tests instantiated by the container like e.g. Spring does with its custom test runner?

I mean that tests should declare their dependencies just like production classes do.

class PostControllerTest extends ContainerAutoconfiguredTestCaseOrWhatever
{
    /** @var MyPrivateService */
    private $serviceToTest;

    public function __construct(MyPrivateService $service) {
        $this->serviceToTest = $service;
    }

    public function testSomething() {
        //...
    }
}

@fracz The main reason why test cases aren't built by the container is probably that (to my knowledge) nobody has implemented that yet. 😉 If you want to start a discussion about that feature, I'd suggest to open an issue in the symfony/symfony issue tracker.

@derrabus @fracz Running PHPUnit tests within the existing Symfony container is a terrible idea. What you can do is using a dedicated container to simplify DI within PHPUnit like I did in my test tools.

In fact, there are already way too many dependencies in a typical Symfony project. Tell me if I'm wrong, but the original idea of DI containers was to simplify loose coupling. Instead it now encourages some developers to create a dependency hell powered by auto-wiring (interfaces enable changing dependencies, but they don't remove them). Imagine forms created by factories and builders do request parsing, validation and database requests via ORM. In the worst case, half of the logic is hidden in annotations you can not possibly cover with tests.

When faced with such a situation, you either have to invest a lot of time to understand what exactly is happening behind the scenes and build mocks (remember that excessive mocking is a TDD anti-pattern?) for all of that complexity or you go the easy way and fetch the service with all its many dependencies and a working configuration from the existing container (yes, it will be slow and using the app container is not a great idea). I've seen good developers getting an impostor syndrome from trying to write unit tests for a Symfony application.

You know what? I'm starting to understand DHH when he said TDD is dead. I'm at the point where I think I'm giving up and only do acceptance testing when working with full featured Symfony applications. The direction in which development is going is not doing the agile community a favor, the rest might be happy but will probably never understand what's the point about writing tests.

Instead it now encourages some developers to create a dependency hell powered by auto-wiring

Try the alternative without DI, good luck with that. Refactoring is a massive pain. The dependency hell you describe is not an issue of the DI pattern but of having too many dependencies.

Imagine forms created by factories and builders do request parsing, validation and database requests via ORM. In the worst case, half of the logic is hidden in annotations you can not possibly cover with tests

Of course it can. That "logic" is plain configuration, it makes zero difference wether it's PHP, XML, YAML or annotations.

You know what? I'm starting to understand DHH when he said TDD is dead. I'm at the point where I think I'm giving up and only do acceptance testing when working with full featured Symfony applications. The direction in which development is going is not doing the agile community a favor, the rest might be happy but will probably never understand what's the point about writing tests.

I think you are missing DDH point. The thing is e2e/acceptance tests are the most effective tests. They however also tend to:

• Be brittle because heavily depend on infrastructure concerns and the delivery mechanism
• Be slow

If you are in a position where the slowness is not bad and they are not very brittle, then go for it. It's the most reliable way to ensure things are working.

Unit tests are are here to test isolated bricks from every angle which might be painfully hard to do with an e2e tests (and horrible brittle & slow). They also have the big advantage to be a hell lot faster.

And FYI, TDD is not about unit tests, it's about testing: make sure that each line you write is tested, unit test or not it doesn't matter.

This looks like a nice approach: https://github.com/jakzal/phpunit-injector

Thank you @symm, what you found looks interesting. I'll give it a try.

Dear @theofidry, I'm a huge fan of DI and I love Symfony. My daily business for the last 20 years was to refactor slow and untested PHP applications. So let me explain:

This ticket is just a symptom of an ongoing process in the community to add more magic, more complexity and force developers to use workarounds if they want to work in a different way. I advocate for building less bloated PHP applications (and frameworks), which was the original selling point against Java, if somebody remembers.

First, it makes testing easier and causes less overhead (also developers can possibly understand what's going on). Code that is written to work in a complex environment with lots of dependencies is also hard to test as "isolated bricks", if that refers to mocking. Mocking makes tests easier to maintain because they don't break when they should, so that's generally not an option for me. The type of tests matters and performance is especially important for refactoring. It also matters if you do bottom up or top down development.

Second, PHP applications need to be bootstrapped for every single request, so complex software with lots of classes and dependencies is very slow. I'm currently working with a Symfony application that requires 16 seconds to load in dev mode (sure, docker for mac is slow, but similar PHP apps only need 500ms, even in dev mode). I'm totally clueless why some developers in the PHP community think performance is not important but it's good for my business. It's not an issue with Symfony in general, just to be clear. You can also work in a lean way, it just doesn't happen very often in practice. I'm already way off topic... sorry.

This ticket is just a symptom of an ongoing process in the community to add more magic, more complexity and force developers to use workarounds if they want to work in a different way

I think you'll have to expand here, because you lost me as for me the recent updates removes the need for workarounds.

First, it makes testing easier and causes less overhead (also developers can possibly understand what's going on). Code that is written to work in a complex environment with lots of dependencies is also hard to test as "isolated bricks", if that refers to mocking. Mocking makes tests easier to maintain because they don't break when they should, so that's generally not an option for me. The type of tests matters and performance is especially important for refactoring. It also matters if you do bottom up or top down development.

By "it" I assume you mean acceptance testing? Then yes I fully agree with what you've just said that's my point. Unit tests are a compromise: they are cheaper but they provide less guarantee. If you can cover everything with acceptance tests which are cheap enough (i.e. not too hard to write, not too brittle & not too slow) then you should always favour acceptance tests IMO.

I'm currently working with a Symfony application that requires 16 seconds to load in dev mode (sure, docker for mac is slow, but similar PHP apps only need 500ms, even in dev mode)

Get rid of docker :) That may sound like a troll but really docker for mac is just not cut for it. I've seen projects of various size (and different docker optimisation strategies) taking well over 15s where on linux the slowest was 5s. What I personally do is use docker for the DB and non-PHP related things which requires little FS and leave the PHP config to phpbrew and likes which make it relatively easy. Not ideal but at least that works better than docker at least in my experience.

The only thing that could make it better here is maybe switch to a React based web server for dev which will be a hell lot faster.

Running PHPUnit tests within the existing Symfony container is a terrible idea

Let's not put such claims out there without backing them up. Anything can be a right idea in the right context, or a terrible idea in a wrong context.

In fact, there are already way too many dependencies in a typical Symfony project. Tell me if I'm wrong, but the original idea of DI containers was to simplify loose coupling. Instead it now encourages some developers to create a dependency hell [...]

The aim of a service container is to simplify creating objects since we're making the process more complex by following dependency injection. Not sure what are you trying to accomplish with this comment? With Flex number of dependencies can be really minimal. The rest is up to you.

When faced with such a situation, you either have to invest a lot of time to understand what exactly is happening behind the scenes and build mocks [..] I've seen good developers getting an impostor syndrome from trying to write unit tests for a Symfony application.

Symfony doesn't prevent you from writing good unit tests in any way. If anything, it encouraged many developers to improve their testing strategies by popularizing dependency injecting with the service container in the PHP community. It only up to developers to choose the right testing strategy that will work for them. Again, what are you suggesting here?

You know what? I'm starting to understand DHH when he said TDD is dead. I'm at the point where I think I'm giving up and only do acceptance testing when working with full featured Symfony applications. The direction in which development is going is not doing the agile community a favor, the rest might be happy but will probably never understand what's the point about writing tests.

Do you have anything constructive to add here please? I've been following TDD for years, consider myself an agile programmer, and one of the reasons I choose to use Symfony for my PHP projects is that it doesn't get into my way and lets me to design my part the way I want. Hexagonal architecture? No problem! RAD? Sure! What direction are you talking about?

This ticket is just a symptom of an ongoing process in the community to add more magic, more complexity and force developers to use workarounds if they want to work in a different way. [...]

My perception is we're trying to avoid magic and complexity for the end users. Would be nice if we heard some examples of the magic you're talking about so we could fix that.

First, it makes testing easier and causes less overhead (also developers can possibly understand what's going on). Code that is written to work in a complex environment with lots of dependencies is also hard to test as "isolated bricks", if that refers to mocking. Mocking makes tests easier to maintain because they don't break when they should, so that's generally not an option for me. The type of tests matters and performance is especially important for refactoring. It also matters if you do bottom up or top down development.

No one is negating the value of unit tests or mocking. Of course the type of tests matters. Furthermore, most projects will need a wide range of different kinds of tests in appropraite proportions. This ticket is only about improving experience while writing integration tests. Unit tests are rather stratight forward, as Symfony will be hardly involved in those. Conventionally I'd write unit tests to my code, and then integration test my boundaries with the framework or libraries. Perhaps we could add an article to docs explaining unit testing.

Second, PHP applications need to be bootstrapped for every single request, so complex software with lots of classes and dependencies is very slow. [...]

Not sure if you saw the latest benchmarks, but Symfony is rather fast. Especially if you look at what it offers. Did you know that not all services are actually loaded on each request, but only those that are actually needed? PHP 7 allowed us to load a smaller container on each request. We're going a bit offtopic here, but there are ways to "fix" docker even on mac (I prefer to use sth like docker sync).

Finally, if you don't like complex software as you describe it, feel free to create smaller software. Symfony doesn't care how big your app is and behaves equally well in all sizes.

@lastzero do you have any concrete suggestion to improve your experience with Symfony?

As I wrote, I think it's getting a bit off topic. I'll still write one last reply and hope that's ok with everyone (I'm happy to continue the conversation via e-mail or you can flame me on Twitter).

Here are some examples of being forced to use workarounds:

• For me (and my clients) it is important that applications run in very similar environments on development, integration and production to avoid "works for me" situations and simplify automation. Therefore fixing performance issues by not using docker for development is a clear workaround. In fact, I like how slow it is because I helps to see performance issues early.
• I originally wanted to write classic unit tests without mocks for the somewhat slow Symfony app I mentioned earlier. However that proved to be very difficult as lots of components were involved just to test a single form (some of them required significant configuration on top). Also it proved next to impossible to use self initializing fakes as test doubles for Doctrine ORM (tests of course should not be exposed to state). Then I tried to use mocks as a workaround, like it is a common practice in the Symfony community.
• Also gave up on this, as I felt this doesn't scale for the amount of tests I needed to write and I already spent more than 2 hours for the form (still not working). So I gave up and resorted to the service in the app container to play around with single classes / services (I guess for you testing is a QA activity and not so much about bottom up development and good software design? how does acceptance testing affect your software design?).
• This is how I ended up on this GitHub issue where I learned I need to add an alias to the container as a workaround to access all services in the container from my test.

For me, the best framework is the one that doesn't get in your way. Diversity is a good thing and it is not my intention to offend anyone. There are different needs like when some people speak about high performance they mean < 10 ms response time while others have 100 to 200 ms in their mind. All I want is to highlight some context and look at the big picture when I stumble upon an issue, like in this case.

Public aliases are the way to go in 3.4.
In 4.1, the new self::$container property gives access to used services. To access unused services (rare I suppose), public aliases are still the way to go in 4.1.

Anyone willing to send a doc PR?

See https://symfony.com/blog/new-in-symfony-4-1-simpler-service-testing for some intro.

Out of the options available.. creating a service locator is probably the easiest to maintain.

services:

    app.test.container:
        class: 'AppBundle\Tests\TestServiceContainer'
        autowire: true
        public: true # require on SF4, where everything is private by default

imports:
    - { resource: config_dev.yml }
    - { resource: services_test.yml }
    - { resource: parameters.test.yml }

<?php 

namespace AppBundle\Tests;

use Symfony\Component\DependencyInjection\ServiceSubscriberInterface;
use Psr\Log\LoggerInterface;
use Symfony\Bridge\Doctrine\RegistryInterface;
use Predis\ClientInterface;
use Symfony\Component\EventDispatcher\EventDispatcherInterface;
use Psr\Container\ContainerInterface;

class TestServiceContainer implements ServiceSubscriberInterface
{
    private $locator;

    public static function getSubscribedServices()
    {
        return [
            'logger' => LoggerInterface::class,
            'doctrine' => RegistryInterface::class,
            'snc_redis.default' => ClientInterface::class,
            'event_dispatcher' => EventDispatcherInterface::class,
            // more services here
        ];
    }
    
    /**
     * @required
     * @param ContainerInterface $locator
     */
    public function setLocator(ContainerInterface $locator)
    {
        $this->locator = $locator;
    }
    
    /**
     * 
     * @return \Psr\Container\ContainerInterface
     */
    public function getLocator()
    {
        return $this->locator;
    }
}

class WebTestCase extends BaseWebTestCase
{
    private $locator;

    /**
     *
     * {@inheritDoc}
     * @see \PHPUnit\Framework\TestCase::setUp()
     */
    protected function setUp()
    {
        $this->locator = $this->getServiceLocator();
    }
    
    /**
     * 
     * @return ContainerInterface
     */
    protected function getServiceLocator()
    {
        return $this->getContainer()->get('app.test.container')->getLocator();
    }
    
    /**
     * 
     * @return LoggerInterface
     */
    public function getLogger()
    {
    	return $this->locator->get('logger');
    }
	
    /**
     * 
     * @return RegistryInterface
     */
    public function getDoctrine()
    {
    	return $this->locator->get('doctrine');
    }
	
    /**
     * 
     * @return ClientInterface
     */
    public function getRedis()
    {
    	return $this->locator->get('snc_redis.default');
    }

    /**
     *
     * @return EventDispatcherInterface
     */
    public function getEventDispatcher()
    {
        return $this->locator->get('event_dispatcher');
    }
}

I tried to solve the problem with a service locator. I'm currently adding the following compiler pass in the test environment only:

namespace App\DependencyInjection;

use Symfony\Component\DependencyInjection\Compiler\CompilerPassInterface;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Reference;
use Symfony\Component\DependencyInjection\ServiceLocator;

class TestContainerPass implements CompilerPassInterface
{
    public function process(ContainerBuilder $container)
    {
        $services = array_keys($container->findTaggedServiceIds('test.container', true));

        foreach ($container->getDefinitions() as $id => $definition) {
            if ($definition->isPublic() && !$definition->isAbstract()) {
                $services[] = $id;
            }
        }

        foreach ($container->getAliases() as $id => $alias) {
            if ($alias->isPublic()) {
                $services[] = $id;
            }
        }

        $container->register('test.container', ServiceLocator::class)
            ->setPublic(true)
            ->addTag('container.service_locator')
            ->setArguments([
                array_combine($services, array_map(
                    function (string $id): Reference {
                        return new Reference($id);
                    },
                    $services
                ))
            ]);
    }
}

The result is a public service test.container which is a service locator that contains all services that have been tagged with test.container. So if I need to test a private service, I simply add this tag to that service.

I then use this service locator instead of the application container in my integration tests:

// old
$container = self::$kernel->getContainer();
// new
$container = self::$kernel->getContainer()->get('test.container');

The locator allows me to fetch those private services without tinkering with service visibilities or introducing prefixed fake services.

As a convenience feature, the locator also provides all public services from the application container, so I don't have to remember which service needs to be pulled from which container. If you don't want this, simply remove the foreach block.

Use static::$kernel->getContainer()->get() or static::$container->get()

https://symfony.com/doc/4.4/testing.html#accessing-the-container
Doesn't this section resolve this issue?

I posted that 6months ago. Of course, it is handled in Symfony 4.4. In 3.x, services were mostly public by default.

friendly @weaverryan or @wouterj, this one could be close with #8097 (comment)

What about replacing built in stuff?
For example, I need to mock InMemoryUserProvider which implements UserProviderInterface.
$container->set(InMemoryUserProvider::class, new MockedStuff());
OR
$container->set(UserProviderInterface::class, new MockedStuff());

I know that I can define my custom version of this provider, make it public and then replacing it. But if I need to do it only for tests - it's not really convenient...

As said in #8097 (comment), this issue should be closed now that 4.4 (the lower maintained branch in the docs) can use the special test container.

For the record, see #17314.