Potential SegFault with multithreading garbage collection. #101975
Labels
interpreter-core
(Objects, Python, Grammar, and Parser dirs)
type-crash
A hard crash of the interpreter, possibly with a core dump
Comments
gaogaotiantian
added
the
type-crash
arhadthedev
added
the
interpreter-core
|
I believe in main branch GC now runs only at eval breaker? So one possible approach here would be to update stacktop on eval breaker, not sure how expensive that would be. |
|
Some extra context here. For my specific case, adding Adding |
|
Okay I think I finally got this. This problem only happens when the profile/trace function is enabled, and is related to garbage collection. I believe it's because in This is done in the very similar code in @markshannon could you take a look at this because you wrote the original code and I think it would be pretty obvious to you if this fix is correct. |
|
Thanks for taking the time to find and fix this. |
markshannon
pushed a commit
that referenced
this issue
Mar 18, 2023
markshannon
pushed a commit
to faster-cpython/cpython
that referenced
this issue
Mar 18, 2023
markshannon
added a commit
that referenced
this issue
Mar 20, 2023
…H-102803) (GH-102807) Authored-by: gaogaotiantian <[email protected]>
|
Is there more work left here, or can we close this? |
Fidget-Spinner
pushed a commit
to Fidget-Spinner/cpython
that referenced
this issue
Mar 27, 2023
warsaw
pushed a commit
to warsaw/cpython
that referenced
this issue
Apr 11, 2023
This comment was marked as spam.
This comment was marked as spam.
|
Pretty sure I just stumbled across this as well. Python 3.10.8. Fedora 35. What version was this fixed in? Edited 2024-01-141. Footnotes
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
For now, I can only occationally observe the segfault on github actions. This is an issue that's not easy to reproduce, but I tried to understand the cause of it.
The direct cause would be in
deduce_unreachableingcmodule.c. In that function,gctries to find cycles by traversing objects, including frame, which uses_PyFrame_Traversefor all its objects. In_PyFrame_Traverse, it usesframe->stacktopas the index range for all the locals and temporary data on stack(not sure if that's on purpose). However,frame->stacktopis not updated in real-time, which means the object it traverses might not be valid.For example, in
FOR_ITERdispatch code, there's aPy_DECREF(iter); STACK_SHRINK(1);when the iterator is exhausted. However,STACK_SHIRNKonly increasesstack_pointer, notframe->stacktop. At this point, theiterthat's just freed will be traversed during garbage collection.There might be something I missed because it's not trivial to reproduce this, but I got a demo that could reproduce this issue occasionally.
It might have something to do with the profile function, I think I can only reproduce this with it. You need to enable
--with-address-sanitizerto find an error ofERROR: AddressSanitizer: heap-use-after-free on address. Normally inPy_TYPE Include/object.h:135, where the code dereferencedob, which could be freed already.The memory it points to is often benign so I'm not able to reliably generate SegFaults, but in theory, this is a memory violation.
Python Version: cpython/main
OS Version: Ubuntu 20 on WSL
Linked PRs
The text was updated successfully, but these errors were encountered: