Hi all!
I've updated my application to Doctrine ORM 2.8 some days ago.
Before that, my Repository code to iterate through a large result set looked like the following and worked perfectly even on >500.000 rows without leaking any memory.
public function iterateAll(): Generator
{
$iterator = $this->createQueryBuilder('e')->getQuery()->iterate(null, Query::HYDRATE_SIMPLEOBJECT);
foreach ($iterator as $equipment) {
// index 0 is always the object
yield $equipment[0];
}
}
Since Query::iterate() is deprecated now, I tried to use Query::toIterable() as suggested.
public function iterateAll(): Generator
{
$iterator = $this->createQueryBuilder('e')->getQuery()->toIterable([], Query::HYDRATE_SIMPLEOBJECT);
foreach ($iterator as $equipment) {
yield $equipment;
}
}
With this implementation I stumbled over a massive memory leak in my application. I debugged this and found out that the AbstractHydrator is never releasing the Objects in AbstractHydrator::toIterable(). The called method AbstractHydrator::hydrateRowData() (or in my case SimpleObjectHydrator::hydrateRowData) just adds new entries to $result.
|
$this->hydrateRowData($row, $result); |
Is that the intended behaviour? Wouldn't it be better to clear $result in every loop cycle to free memory?
If that's intended, is there a better way to loop through large result sets?
Hi all!
I've updated my application to Doctrine ORM 2.8 some days ago.
Before that, my Repository code to iterate through a large result set looked like the following and worked perfectly even on >500.000 rows without leaking any memory.
Since
Query::iterate()is deprecated now, I tried to useQuery::toIterable()as suggested.With this implementation I stumbled over a massive memory leak in my application. I debugged this and found out that the
AbstractHydratoris never releasing the Objects inAbstractHydrator::toIterable(). The called methodAbstractHydrator::hydrateRowData()(or in my caseSimpleObjectHydrator::hydrateRowData) just adds new entries to$result.orm/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php
Line 177 in 378944d
orm/lib/Doctrine/ORM/Internal/Hydration/SimpleObjectHydrator.php
Line 157 in 378944d
Is that the intended behaviour? Wouldn't it be better to clear
$resultin every loop cycle to free memory?If that's intended, is there a better way to loop through large result sets?