Skip to content

[mypyc] Enable partial, unsafe support for free-threading #19167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 29, 2025

Conversation

JukkaL
Copy link
Collaborator

@JukkaL JukkaL commented May 29, 2025

Enable multi-phase init when using a free-threaded (no-GIL) CPython build so we can enable proper multihreading.

Work on mypyc/mypyc#1104. Work on mypyc/mypyc#1038.

The implementation is still quite incomplete. We are missing synchronization in various places, so race conditions can cause segfaults. Only single-module compilation units are supported for now.

Here's a toy benchmark I used to check that free threading works and can improve performance:

import sys
import threading
import time

def fib(n: int) -> int:
    if n <= 1:
        return n
    else:
        return fib(n - 1) + fib(n - 2)

NTHREADS = 6
print(f"Using {NTHREADS} threads")
print(f"{sys._is_gil_enabled()=}")

t0 = time.time()

threads = []
for i in range(NTHREADS):
    t = threading.Thread(target=lambda: fib(36))
    t.start()
    threads.append(t)

for t in threads:
    t.join()

print()
print('elapsed time:', time.time() - t0)

@JukkaL JukkaL merged commit 6551bce into master May 29, 2025
12 checks passed
@JukkaL JukkaL deleted the mypyc-free-threading branch May 29, 2025 16:37
cdce8p pushed a commit to cdce8p/mypy that referenced this pull request May 31, 2025
Enable multi-phase init when using a free-threaded (no-GIL) CPython
build so we can enable proper multihreading.

Work on mypyc/mypyc#1104. Work on mypyc/mypyc#1038.

The implementation is still quite incomplete. We are missing
synchronization in various places, so race conditions can cause
segfaults. Only single-module compilation units are supported for now.

Here's a toy benchmark I used to check that free threading works and can
improve performance:
```
import sys
import threading
import time

def fib(n: int) -> int:
    if n <= 1:
        return n
    else:
        return fib(n - 1) + fib(n - 2)

NTHREADS = 6
print(f"Using {NTHREADS} threads")
print(f"{sys._is_gil_enabled()=}")

t0 = time.time()

threads = []
for i in range(NTHREADS):
    t = threading.Thread(target=lambda: fib(36))
    t.start()
    threads.append(t)

for t in threads:
    t.join()

print()
print('elapsed time:', time.time() - t0)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants