I think this method call should be wrapped with try/catch like

        try {
            $filesystem->remove($oldCacheDir);
        } catch (IOException $e) {
            $this->logger->error($e);
        }

since \Symfony\Component\Filesystem\Filesystem::remove() really throws lot's of exceptions.
And corresponding unitTest should be added.

How to reproduce:

cd pimcore && COMPOSER_MEMORY_LIMIT=-1 ./vendor/bin/pimcore-install --admin-username=admin --admin-password=a
dmin --mysql-host-socket=mysql --mysql-username=pimcore --mysql-password=pimcore --mysql-database=pimcore --no-interaction

Booting new kernel...
5/11 [============>---------------] 45%
[ERROR] The following errors were encountered during installation

Failed to remove directory "/var/www/html/pimcore/var/cach~/prod/ContainerMw7xMln": rmdir(/var/www/html/pimcore/var/cach~/prod/ContainerMw7
xMln): Directory not empty.
[2020-10-06 17:30:48] app.ERROR: Symfony\Component\Filesystem\Exception\IOException: Failed to remove directory "/var/www/html/pimcore/var/cach~/prod/ContainerMw7xMln": rmdir(/var/www/html/pimcore/var/cach~/prod/ContainerMw7xMln): Directory not empty. in /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Component/Filesystem/Filesystem.php:181

Stack trace:
#0 /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Component/Filesystem/Filesystem.php(178): Symfony\Component\Filesystem\Filesystem->remove(Array)
#1 /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Component/Filesystem/Filesystem.php(178): Symfony\Component\Filesystem\Filesystem->remove(Array)
#2 /var/www/html/pimcore/vendor/pimcore/pimcore/bundles/InstallBundle/Installer.php(558): Symfony\Component\Filesystem\Filesystem->remove(Array)
#3 /var/www/html/pimcore/vendor/pimcore/pimcore/bundles/InstallBundle/Installer.php(352): Pimcore\Bundle\InstallBundle\Installer->clearKernelCacheDir(Object(AppKernel))
#4 /var/www/html/pimcore/vendor/pimcore/pimcore/bundles/InstallBundle/Installer.php(276): Pimcore\Bundle\InstallBundle\Installer->runInstall(Array, Array)
#5 /var/www/html/pimcore/vendor/pimcore/pimcore/bundles/InstallBundle/Command/InstallCommand.php(367): Pimcore\Bundle\InstallBundle\Installer->install(Array)
#6 /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php(255): Pimcore\Bundle\InstallBundle\Command\InstallCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(933): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(87): Symfony\Component\Console\Application->doRunCommand(Object(Pimcore\Bundle\InstallBundle\Command\InstallCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(272): Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand(Object(Pimcore\Bundle\InstallBundle\Command\InstallCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#10 /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(73): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /var/www/html/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(148): Symfony\Bundle\FrameworkBundle\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /var/www/html/pimcore/vendor/pimcore/pimcore/bin/pimcore-install(78): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput))
#13 {main} [] []