Fix zlib support for large files #17775
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
gzread() and gzwrite() have effectively a 4GiB limit at the moment because the APIs of the zlib library use unsigned ints. For example, this means that the count argument of gzread() and gzwrite() & co effectively are modulo 2**32.
Fix this by adding a loop to handle all bytes.
As for automated testing, I didn't find an easy way to write a phpt for this that wouldn't use a lot of memory or requires a large file. For instance, the gzread() test that I manually ran requires a 4MiB input file (and I can't shrink it because zlib has a max window size).
Here are the testing instructions, run on 64-bit:
To test for gzwrite():
Then use
zcat out.txt.gz|wc -c
to check that all bytes were written (should be 4294967360).To test for gzread():
Create a file containing all a's for example that is 4GiB + 64 bytes. Then compress it into out.txt.gz using the gzip command. Then run:
For automated testing we would first need #14895 and then generate a file dynamically using gzwrite() and then check it with gzread() I suppose.