Skip to content

Multiprocessing not propagating -Xfrozen_modules=off #106723

Closed
@ergoithz

Description

@ergoithz

Bug report

After python3.11 changes around frozen imports, when using multiprocessing contexts other than fork, the newly added -Xfrozen_modules=off isn't passed to spawned process interpreters.

Simple snippet demonstrating the issue:

"""

$ python -Xfrozen_modules=off test.py
main: {'frozen_modules': 'off'}
forkserver: {}
spawn: {}
fork: {'frozen_modules': 'off'}

"""

import sys
import multiprocessing

def xoptions():
    return sys._xoptions

def main():
    print('main:', xoptions())
    for ctx in ('forkserver', 'spawn', 'fork'):
        with multiprocessing.get_context(ctx).Pool(1) as pool:
            print(f'{ctx}:', pool.apply(xoptions))

if __name__ == '__main__':
    main()

The issue seems to be subprocess._args_from_interpreter_flags not honoring frozen_modules key from sys._xoptions.

$ python -Xfrozen_modules=off -c 'import subprocess;print(subprocess._args_from_interpreter_flags())'
[]

Your environment

python 3.11.4

Linked PRs

Metadata

Metadata

Assignees

Labels

3.11only security fixes3.12only security fixestopic-multiprocessingtype-bugAn unexpected behavior, bug, or error

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions