Detect QEMU linux-user emulation in subprocess._use_posix_spawn()

[andreas-schwab]

The qemu linux-user emulation does not support CLONE_VFORK and emulates it
with a true fork. That breaks posix_spawn because it cannot report errors
back to the parent process. The subprocess modules should detect that is
it being executed in qemu linux-user emulation and return False in
_use_posix_spawn. One way to do that is to look for "uarch *: qemu" in
/proc/cpuinfo.

[vstinner]

How is it an issue to use vfork() if it's emulated with fork()? Is it a performance issue? Do you have examples of benchmark showing the performance problem?

[andreas-schwab]

Without a true vfork, the posix_spawn implementation in glibc cannot
report a proper errno value for errors that happen in the child before
execve.

See https://lists.gnu.org/archive/html/qemu-devel/2025-03/msg04706.html
for more information.

[vstinner]

cc @gpshead

[vstinner]

Without a true vfork, the posix_spawn implementation in glibc cannot report a proper errno value for errors that happen in the child before execve.

Do you have an example to reproduce the issue?

[andreas-schwab]

It causes a failure while building the MozillaFirefox package:

4:55.15 Traceback (most recent call last):
4:55.16 File "/home/abuild/rpmbuild/BUILD/MozillaFirefox-136.0.1-build/firefox-136.0.1/security/nss/./coreconf/werror.py", line 80, in
4:55.16 main()
4:55.16 ~~~~^^
4:55.16 File "/home/abuild/rpmbuild/BUILD/MozillaFirefox-136.0.1-build/firefox-136.0.1/security/nss/./coreconf/werror.py", line 10, in main
4:55.16 cc_is_clang = 'clang' in subprocess.check_output(
4:55.16 ~~~~~~~~~~~~~~~~~~~~~~~^
4:55.16 [cc, '--version'], universal_newlines=True, stderr=sink)
4:55.16 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4:55.16 File "/usr/lib64/python3.13/subprocess.py", line 474, in check_output
4:55.16 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
4:55.16 ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4:55.17 **kwargs).stdout
4:55.17 ^^^^^^^^^
4:55.17 File "/usr/lib64/python3.13/subprocess.py", line 579, in run
4:55.17 raise CalledProcessError(retcode, process.args,
4:55.17 output=stdout, stderr=stderr)
4:55.17 subprocess.CalledProcessError: Command '['/usr/bin/ccache /usr/bin/gcc', '--version']' returned non-zero exit status 127.

The issue here is that posix_spawn should have set errno to ENOENT.

Also, the python testsuite tests this in test_subprocess.py.

[andreas-schwab]

For exmaple:

FAIL: test_exception_bad_args_0 (test.test_subprocess.POSIXProcessTestCase.test_exception_bad_args_0)
Test error in the child raised in the parent for a bad args[0].

Traceback (most recent call last):
File "/home/abuild/rpmbuild/BUILD/python313-3.13.2-build/Python-3.13.2/Lib/test/test_subprocess.py", line 1903, in test_exception_bad_args_0
self.fail("Expected OSError: %s" % desired_exception)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected OSError: [Errno 2] No such file or directory: '/_this/pa.th/does/not/exist'

[gpshead]

FWIW I agreee with the approach of detecting this unusual/incomplete environment and having _use_posix_spawn() return False there.

[vfazio]

Seems somewhat related: #129204. I ran into a similar scenario until i bumped QEMU to v7.2+

[vfazio]

Might also be worthwhile to not use posix_spawn if proc isn't mounted. As mentioned in the issue I linked, QEMU is sometimes run without proc or sys mounted due to limitations of the container environment so you may not even be able to query if you're running under emulation, in which case it may make sense to just assume we are.

For now, there's at least a knob (#132184) that can be exported into the environment to force the code to not use posix_spawn.

[gpshead]

Saving some clicks, the undocumented knob added for CPython >=3.13.3 and >=3.14 is to set _PYTHON_SUBPROCESS_USE_POSIX_SPAWN=0 in your environment when you have an unusual environment that must not use it.