Description
Bug report
Description
When setting up a virtual environment, the venv package copies its scripts to the created venv/bin directory. In Python 3.12, the source file is read and the contents are written to the destination file. However, in Python 3.13, the script also attempts to copy the source file's extended file attributes to the destination file. This behaviour can be seen observed using strace.
When CPython is installed on a filesystem which contains large extended attributes (such as CVMFS, more below), and the venv is created on a filesystem which only supports xattr values one block big (usually 4KiB, such as ext3/4 or btrfs), an error arises when Python fails to write the extended attributes:
$ python3.13 -m venv venv --without-pip
Error: [Errno 28] No space left on device: '/data/venv/bin/Activate.ps1'
This error message is misleading, as the filesystem behind the quoted path has lots of free space. The error message actually refers to the extended attribute value being longer than the maximum allowed. It is clear in strace that this error is returned by the setxattr system call:
getxattr("{install path}/lib/python3.13/venv/scripts/common/Activate.ps1", "user.proxy_list", "{cvmfs url}"..., 65536) = 3136
setxattr("/data/venv/bin/Activate.ps1", "user.proxy_list", "{cvmfs url}"..., 3136, 0) = -1 ENOSPC (No space left on device)
Cause
This change in behaviour is due to the changes in this commit, from line 461 onwards in Lib/venv/__init__.py. The procedure for writing the destination files previously wrote the file directly and copied the file mode using shutil.copymode, which does not copy extended attributes. After the commit, if no replacements were made in the source file, shutil.copy2 is called, which in turn calls shutil.copystat which does copy extended attributes.
This commit was merged as part of #113129.
Impact on CVMFS users
CernVM-FS (CVMFS) is a fuse-based filesystem used to distribute software in the scientific computing community and HPC environments. It uses extended attributes to provide information about the filesystem. Its extended attributes apply to every file - they do not belong to the file but are generated by the filesystem when requested. For example, the user.pubkeys attribute contains the public keys used to sign the filesystem contents, which can easily be tens of kilobytes - way more than can fit in an ext4 extended attribute. There is no need for these attributes to be copied to the venv.
This bug means that Python 3.13 venvs are broken for the those using CVMFS to distribute their software and using ext/btrfs locally, which I expect will cause problems adopting Python 3.13 for the scientific computing and HPC community.
Proposed fix
It is not necessary for the extended attributes to be copied from Python's installation directory to the venv directory - Python does not and should not rely on them to function properly. Therefore, my proposed fix is simply to eliminate this if statement in the venv package. The code currently in the else branch should always run, as was the case in Python 3.12.
This change can be hacked into effect using sed -i 's/new_data == data:/False and new_data == data:/' /venv/__init__.py, and with this patch the extended attributes are no longer copied and the venv is created successfully.
CPython versions tested on:
3.13
Operating systems tested on:
Linux
(cc @vsajip as author of stdlib venv)