SitePoint PHP Blog: How to Read Big Files with PHP (Without Killing Your Server)

On the SitePoint PHP blog, there's a tutorial posted showing you how to deal with large files without "killing your server". In this case, it's not about the upload process but about the handling of large files on the server side.

It’s not often that we, as PHP developers, need to worry about memory management. The PHP engine does a stellar job of cleaning up after us, and the web server model of short-lived execution contexts means even the sloppiest code has no long-lasting effects.

There are rare times when we may need to step outside of this comfortable boundary — like when we’re trying to run Composer for a large project on the smallest VPS we can create, or when we need to read large files on an equally small server. It’s the latter problem we’ll look at in this tutorial.

They start off by describing how they plan to measure the success of the improved file handling, mostly around the memory usage required to work with the file. It then gets into some of the options available including:

• reading files line by line
• piping between files
• using filters

The last option, the filters, seems to be the best one. He uses this one and customizes the handling with different configurations and custom protocols. All related code is included in the post and is avaialble on GitHub.