Skip to content

Commit 35d8ac7

Browse files
authored
GH-120754: Disable buffering in Path.read_bytes (#122111)
`Path.read_bytes()` is used to read a whole file. buffering / BufferedIO is focused around making small, possibly interleaved, read/write efficient which doesn't add value in this case. On my Mac, running the benchmark: ```python import pyperf from pathlib import Path def read_all(all_paths): for p in all_paths: p.read_bytes() def read_file(path_obj): path_obj.read_bytes() all_rst = list(Path("Doc").glob("**/*.rst")) all_py = list(Path(".").glob("**/*.py")) assert all_rst, "Should have found rst files" assert all_py, "Should have found python source files" runner = pyperf.Runner() runner.bench_func("read_file_small", read_file, Path("Doc/howto/clinic.rst")) runner.bench_func("read_file_large", read_file, Path("Doc/c-api/typeobj.rst")) ``` before: ```python ..................... read_file_small: Mean +- std dev: 6.80 us +- 0.07 us ..................... read_file_large: Mean +- std dev: 10.8 us +- 0.2 us ```` after: ```python ..................... read_file_small: Mean +- std dev: 5.67 us +- 0.05 us ..................... read_file_large: Mean +- std dev: 9.77 us +- 0.52 us ```
1 parent 8ef358d commit 35d8ac7

File tree

3 files changed

+3
-2
lines changed

3 files changed

+3
-2
lines changed

Lib/pathlib/_abc.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -585,7 +585,7 @@ def read_bytes(self):
585585
"""
586586
Open the file in bytes mode, read it, and close the file.
587587
"""
588-
with self.open(mode='rb') as f:
588+
with self.open(mode='rb', buffering=0) as f:
589589
return f.read()
590590

591591
def read_text(self, encoding=None, errors=None, newline=None):

Lib/test/test_pathlib/test_pathlib_abc.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1464,7 +1464,7 @@ def stat(self, *, follow_symlinks=True):
14641464

14651465
def open(self, mode='r', buffering=-1, encoding=None,
14661466
errors=None, newline=None):
1467-
if buffering != -1:
1467+
if buffering != -1 and not (buffering == 0 and 'b' in mode):
14681468
raise NotImplementedError
14691469
path_obj = self.resolve()
14701470
path = str(path_obj)
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
``Pathlib.read_bytes`` no longer opens the file in Python's buffered I/O mode. This reduces overheads as the code reads a file in whole leading to a modest speedup.

0 commit comments

Comments
 (0)