bpo-36139: release GIL around munmap() #12073
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hello, and thanks for your contribution! I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA). Unfortunately we couldn't find an account corresponding to your GitHub username on bugs.python.org (b.p.o) to verify you have signed the CLA (this might be simply due to a missing "GitHub Name" entry in your b.p.o account settings). This is necessary for legal reasons before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue. You can check yourself to see if the CLA has been received. Thanks again for your contribution, we look forward to reviewing it! |
|
Should we also be doing this around the |
It would make sense but I wasn't sure if we should do it separately or in one block, and if it should go inside the |
|
I think it makes sense to stick them both in the same block. We' certainly release the GIL when closing files. |
|
@benjaminp I extended the GIL release to include the fd close, both on dealloc and |
benjaminp
reviewed
Mar 3, 2019
Modules/mmapmodule.c
Outdated
| @@ -180,6 +183,7 @@ mmap_close_method(mmap_object *self, PyObject *unused) | |||
| self->data = NULL; | |||
There was a problem hiding this comment.
I don't think it's safe to not be holding the GIL here. Someone could be trying to read or write the mmap object while calling close.
|
@benjaminp may I ask for another look? If this is still not ok for |
benjaminp
reviewed
Mar 6, 2019
Modules/mmapmodule.c
Outdated
| self->data = NULL; | ||
| HANDLE map_handle = self->map_handle; | ||
| HANDLE file_handle = self->file_handle; | ||
| char * data = self->data; |
Modules/mmapmodule.c
Outdated
| if (0 <= self->fd) | ||
| (void) close(self->fd); | ||
| int fd = self->fd; | ||
| char * data = self->data; |
|
vstinner
reviewed
Mar 6, 2019
| @@ -117,6 +117,7 @@ typedef struct { | |||
| static void | |||
| mmap_object_dealloc(mmap_object *m_obj) | |||
| { | |||
| Py_BEGIN_ALLOW_THREADS | |||
| #ifdef MS_WINDOWS | |||
| if (m_obj->data != NULL) | |||
| UnmapViewOfFile (m_obj->data); | |||
There was a problem hiding this comment.
PyMem_Free(m_obj->tagname) is called below without holding the GIL: that's illegal. Other move the call to free when the GIL is hold again, or use PyMem_RawFree().
"Warning: The GIL must be held when using these functions. "
https://docs.python.org/dev/c-api/memory.html#memory-interface
Note: You can use PYTHONMALLOC=debug or -X dev to reproduce the issue on a Python compiled in release mode.
|
|
hauntsaninja
added a commit
to hauntsaninja/cpython
that referenced
this pull request
Oct 10, 2022
This seems pretty straightforward. The issue mentions other calls in mmapmodule that we could release the GIL on, but those are in methods where we'd need to be careful to ensure that something sensible happens if those are called concurrently. In prior art, note that python#12073 released the GIL for munmap. In a toy benchmark, I see the speed up you'd expect from doing this.
miss-islington
pushed a commit
that referenced
this pull request
Oct 10, 2022
This seems pretty straightforward. The issue mentions other calls in mmapmodule that we could release the GIL on, but those are in methods where we'd need to be careful to ensure that something sensible happens if those are called concurrently. In prior art, note that #12073 released the GIL for munmap. In a toy benchmark, I see the speedup you'd expect from doing this. Automerge-Triggered-By: GH:gvanrossum
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://bugs.python.org/issue36139