Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Use file_get_contents instead of SplFileObject::fpassthru() #6224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Use file_get_contents instead of SplFileObject::fpassthru() #6224

wants to merge 1 commit into from

Conversation

armetiz
Copy link
Contributor

@armetiz armetiz commented Dec 7, 2012

Hi there.
For a local operation, I'm parsing some Apache Log Files.
It's a simple code using the Finder, the Symfony\Finder\SplFileInfo::getContents function & preg_match_all function : Sample

The biggest file size is 500M, and I have around ten files.
Without the PR, I have to set the "memory_limit" value to 1G to use the above script. And with my PR the "memory_limit" is really near the biggest file, it's working with memory_limit set to 550M.

Also,
When the PHP Fatal error: Allowed memory size comes, without the PR, the standard output is spammed by the loaded content.. Which is really annoying.

I'm using PHP 5.3.17 on a MacOS.

@armetiz
Copy link
Contributor Author

armetiz commented Dec 7, 2012

From symfony/finder#2

@fabpot
Copy link
Member

fabpot commented Dec 7, 2012

The usage of fpassthru was introduced in #4751... but the reason is not really clear to me anymore. ping @gajdaw, @stealth35

@gajdaw
Copy link
Contributor

gajdaw commented Dec 7, 2012

The reason for fpassthru() was to use internal exceptions handling provided by SplFileObject.

I didn't compare fpassthru() to file_get_contents(). I assumed that they both work in the same way.

If file_get_contents() is better, then I think the best solution is to revert changes introduced in PR:

https://github.com/symfony/symfony/pull/4751/files

@fabpot fabpot closed this in a1734dd Dec 10, 2012
@armetiz
Copy link
Contributor Author

armetiz commented Dec 10, 2012

An other problem appears.

If I open a big file (e.g 500M) with a memory_limit of 300M, the process shutdown without any error and any other information...

    public function getContents()
    {
        $level = error_reporting(0);
        $content = file_get_contents($this->getRealpath());
        error_reporting($level);
        if (false === $content) {
            $error = error_get_last();
            throw new \RuntimeException($error['message']);
        }

        return $content;
    }

The normal process of the getContents function is to throw a RuntimeException when there is a problem on file_get_contents.. Which is not the case when a Fatal Error occurs...

Maybe we could add a register through : register_shutdown_function and check the error_get_last function.. But there is no unregister_shutdown_function .. Any idea ?

I'm running a CLI script with PHP 5.3.17.

@stof
Copy link
Member

stof commented Dec 10, 2012

@armetiz catching the fatal error in a register_shutdown_function will not allow you to handle the error inside this function as the shutdown is too late. Other part of your app may have been destructed already.
And if it is only about providing a better error page for developers, this is done in 2.2

@armetiz
Copy link
Contributor Author

armetiz commented Dec 10, 2012

@stof, yes you right about the register_shutdown_function

Like I said, I'm using a short script via CLI. I'm using the Finder as a standalone component.

ini_set('memory_limit', '100M');

use Symfony\Component\Finder\Finder;

$finder = new Finder();
$finder->files()
        ->in(__DIR__ . "/../")
        ->name("access_log*");

echo "begin\n";

foreach ($finder as $file) {
        $contents = $file->getContents();

        echo count($contents) . "\n";

        unset($contents);
}

echo "end\n";

If I'm running this script with some files sized up to 200M, the process will display "begin" and that's all.. No error.

IMHO, what I want to say is that disable error reporting, when there is no error manager is not a good thing. If I remove from my example the three "echo", I can think that it's working like a charm, and it's not.

Regards,
Armetiz.

@mvrhov
Copy link

mvrhov commented Dec 10, 2012

You could handle OOM case yourself, by not opening files larger than memory_limit - memory_get_usage()

@armetiz
Copy link
Contributor Author

armetiz commented Dec 11, 2012

@mvrhov of course I can. I'm using a script which is working very well, there is no problem for me.

My last comment is just to improve the Finder component.. Because One day, someone will use the Finder with a file bigger than the memory_limit and there will have no error message.

Thomas.

fabpot added a commit that referenced this pull request Dec 11, 2012
* 2.1:
  fixed CS
  fixed CS
  [Security] fixed path info encoding (closes #6040, closes #5695)
  [HttpFoundation] added some tests for the previous merge and removed dead code (closes #6037)
  Improved Cache-Control header when no-cache is sent
  removed unneeded comment
  Fix to allow null values in labels array
  fix date in changelog
  removed the Travis icon (as this is not stable enough -- many false positive, closes #6186)
  Revert "merged branch gajdaw/finder_splfileinfo_fpassthu (PR #4751)" (closes #6224)
  Fixed a typo
  Fixed: HeaderBag::parseCacheControl() not parsing quoted zero correctly
  [Form] Fix const inside an anonymous function
  [Config] Loader::import must return imported data
  [DoctrineBridge] Fixed caching in DoctrineType when "choices" or "preferred_choices" is passed
  [Form] Fixed the default value of "format" in DateType to DateType::DEFAULT_FORMAT if "widget" is not "single_text"
  [HttpFoundation] fixed a small regression

Conflicts:
	src/Symfony/Component/HttpFoundation/Tests/Session/Storage/Handler/MongoDbSessionHandlerTest.php
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants