Skip to content

Add more tests for pickle and fix error messages #122311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
serhiy-storchaka opened this issue Jul 26, 2024 · 0 comments
Closed

Add more tests for pickle and fix error messages #122311

serhiy-storchaka opened this issue Jul 26, 2024 · 0 comments
Assignees
@serhiy-storchaka
Labels
3.12 only security fixes 3.13 bugs and security fixes 3.14 bugs and security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Jul 26, 2024 •
edited by bedevere-app bot
Loading

Bug report

Much error generating code in the pickle module is not tested at all. As result, not only exception types and messages can differ between Python and C implementations, but other bugs are left unnoticed:

  • NameError in formatting one of error messages.
  • "Can't pickle" in the error message during unpickling.
  • Error message including the repr of wrong object.
  • Some errors are only detected in one of implementations (Python or C).
  • Crash in untested corner case (Crash after error in PickleBuffer #122306).

I am going to add more tests and fix the most odious errors. This will be backported to 3.12 and 3.13, but the backport to 3.12 will have more lenient variants of tests. Late, I will unify and improve other error messages (this is for main only).

Linked PRs
@serhiy-storchaka serhiy-storchaka added type-bug An unexpected behavior, bug, or error tests Tests in the Lib/test dir 3.12 only security fixes 3.13 bugs and security fixes 3.14 bugs and security fixes labels Jul 26, 2024
@serhiy-storchaka serhiy-storchaka self-assigned this Jul 26, 2024
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 26, 2024
@serhiy-storchaka
pythongh-122311: Fix typo in the pickle error formatting code
9a68845
@bedevere-app bedevere-app bot mentioned this issue Jul 26, 2024
Merged
serhiy-storchaka added a commit that referenced this issue Jul 26, 2024
@serhiy-storchaka
gh-122311: Fix typo in the pickle error formatting code (GH-122312)
7c29218
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 26, 2024
@serhiy-storchaka @miss-islington
pythongh-122311: Fix typo in the pickle error formatting code (python…
f1bafd4 
…GH-122312)

(cherry picked from commit 7c29218)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 26, 2024
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 26, 2024
@serhiy-storchaka @miss-islington
pythongh-122311: Fix typo in the pickle error formatting code (python…
1ee84b5 
…GH-122312)

(cherry picked from commit 7c29218)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 26, 2024
Merged
serhiy-storchaka added a commit that referenced this issue Jul 26, 2024
@miss-islington @serhiy-storchaka
[3.12] gh-122311: Fix typo in the pickle error formatting code (GH-12…
7353106 
…2312) (GH-122315)

(cherry picked from commit 7c29218)

Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit that referenced this issue Jul 26, 2024
@miss-islington @serhiy-storchaka
[3.13] gh-122311: Fix typo in the pickle error formatting code (GH-12…
816a157 
…2312) (GH-122314)

(cherry picked from commit 7c29218)

Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 28, 2024
@serhiy-storchaka
pythongh-122311: Add more tests for pickle
8e31ad0
@bedevere-app bedevere-app bot mentioned this issue Jul 28, 2024
Merged
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 28, 2024
@serhiy-storchaka
pythongh-122311: Add more tests for pickle
5233d92
@bedevere-app bedevere-app bot mentioned this issue Jul 28, 2024
Merged
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 28, 2024
@serhiy-storchaka
pythongh-122311: Add more tests for pickle
1b50df9
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 28, 2024
@serhiy-storchaka
pythongh-122311: Add more tests for pickle
77d035f
serhiy-storchaka added a commit that referenced this issue Jul 28, 2024
@serhiy-storchaka
gh-122311: Add more tests for pickle (GH-122376)
bc93923
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 28, 2024
@serhiy-storchaka @miss-islington
pythongh-122311: Add more tests for pickle (pythonGH-122376)
a44f1a3 
(cherry picked from commit bc93923)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 28, 2024
Merged
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 28, 2024
@serhiy-storchaka
[3.12] pythongh-122311: Add more tests for pickle (pythonGH-122376)
98eb168 
(cherry picked from commit bc93923)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 28, 2024
Merged
serhiy-storchaka added a commit that referenced this issue Jul 28, 2024
@miss-islington @serhiy-storchaka
[3.13] gh-122311: Add more tests for pickle (GH-122376) (GH-122377)
d113359 
(cherry picked from commit bc93923)

Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 28, 2024
@serhiy-storchaka
pythongh-122311: Add more details in tests for pickle
b7e043e
serhiy-storchaka added a commit that referenced this issue Jul 28, 2024
@serhiy-storchaka
[3.12] gh-122311: Add more tests for pickle (GH-122376) (GH-122378)
ecc97cb 
(cherry picked from commit bc93923)
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 29, 2024
@serhiy-storchaka
pythongh-122311: Fix some error messages in pickle
b064335
@bedevere-app bedevere-app bot mentioned this issue Jul 29, 2024
Merged
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 29, 2024
@serhiy-storchaka
pythongh-122311: Fix some error messages in pickle
d7d8ba3
serhiy-storchaka added a commit that referenced this issue Jul 29, 2024
@serhiy-storchaka
gh-122311: Fix some error messages in pickle (GH-122386)
3b034d2
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 29, 2024
@serhiy-storchaka @miss-islington
pythongh-122311: Fix some error messages in pickle (pythonGH-122386)
2814935 
(cherry picked from commit 3b034d2)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 29, 2024
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 29, 2024
@serhiy-storchaka @miss-islington
pythongh-122311: Fix some error messages in pickle (pythonGH-122386)
acc97bb 
(cherry picked from commit 3b034d2)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 29, 2024
Merged
serhiy-storchaka added a commit that referenced this issue Jul 29, 2024
@miss-islington @serhiy-storchaka
[3.13] gh-122311: Fix some error messages in pickle (GH-122386) (GH-1…
c26dd27 
…22387)

(cherry picked from commit 3b034d2)

Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit that referenced this issue Jul 29, 2024
@miss-islington @serhiy-storchaka
[3.12] gh-122311: Fix some error messages in pickle (GH-122386) (GH-1…
a3076c7 
…22388)

(cherry picked from commit 3b034d2)

Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit that referenced this issue Jul 29, 2024
@serhiy-storchaka
gh-122311: Add more tests for error messages in pickle (GH-122373)
0697188
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jul 29, 2024
@serhiy-storchaka
pythongh-122311: Fix a refleak in pickle
885e945
@bedevere-app bedevere-app bot mentioned this issue Jul 29, 2024
Merged
serhiy-storchaka added a commit that referenced this issue Jul 29, 2024
@serhiy-storchaka
gh-122311: Fix a refleak in pickle (GH-122411)
68840e9
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 29, 2024
@serhiy-storchaka @miss-islington
pythongh-122311: Fix a refleak in pickle (pythonGH-122411)
c7bc40f 
(cherry picked from commit 68840e9)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 29, 2024
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 29, 2024
@serhiy-storchaka @miss-islington
pythongh-122311: Fix a refleak in pickle (pythonGH-122411)
4993303 
(cherry picked from commit 68840e9)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 29, 2024
Merged
serhiy-storchaka added a commit that referenced this issue Jul 29, 2024
@miss-islington @serhiy-storchaka
[3.12] gh-122311: Fix a refleak in pickle (GH-122411) (GH-122416)
bad8497 
(cherry picked from commit 68840e9)

Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit that referenced this issue Jul 29, 2024
@miss-islington @serhiy-storchaka
[3.13] gh-122311: Fix a refleak in pickle (GH-122411) (GH-122415)
d8701e2 
(cherry picked from commit 68840e9)

Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Aug 7, 2024
@serhiy-storchaka
pythongh-122311: Improve and unify pickle errors
554abbc 
* Raise PicklingError instead of UnicodeEncodeError, ValueError
  and AttributeError in both implementations.
* Chain the original exception to the pickle-specific one as __context__.
* Include the error message of ImportError and some AttributeError in
  the PicklingError error message.
* Unify error messages between Python and C implementations.
* Refer to documented __reduce__ and __newobj__ callables instead of
  internal methods (e.g. save_reduce()) or pickle opcodes (e.g. NEWOBJ).
* Include more details in error messages (what expected, what got).
* Avoid including a potentially long repr of an arbitrary object in
  error messages.
@bedevere-app bedevere-app bot mentioned this issue Aug 7, 2024
Merged
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Aug 7, 2024
@serhiy-storchaka
pythongh-122311: Improve and unify pickle errors
43d94f2 
* Raise PicklingError instead of UnicodeEncodeError, ValueError
  and AttributeError in both implementations.
* Chain the original exception to the pickle-specific one as __context__.
* Include the error message of ImportError and some AttributeError in
  the PicklingError error message.
* Unify error messages between Python and C implementations.
* Refer to documented __reduce__ and __newobj__ callables instead of
  internal methods (e.g. save_reduce()) or pickle opcodes (e.g. NEWOBJ).
* Include more details in error messages (what expected, what got).
* Avoid including a potentially long repr of an arbitrary object in
  error messages.
serhiy-storchaka added a commit that referenced this issue Sep 9, 2024
@serhiy-storchaka
gh-122311: Improve and unify pickle errors (GH-122771)
b2a8c38 
* Raise PicklingError instead of UnicodeEncodeError, ValueError
  and AttributeError in both implementations.
* Chain the original exception to the pickle-specific one as __context__.
* Include the error message of ImportError and some AttributeError in
  the PicklingError error message.
* Unify error messages between Python and C implementations.
* Refer to documented __reduce__ and __newobj__ callables instead of
  internal methods (e.g. save_reduce()) or pickle opcodes (e.g. NEWOBJ).
* Include more details in error messages (what expected, what got).
* Avoid including a potentially long repr of an arbitrary object in
  error messages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@serhiy-storchaka serhiy-storchaka

Labels
3.12 only security fixes 3.13 bugs and security fixes 3.14 bugs and security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
Projects
Pickle and copy issues 🥒
Status: Done
Milestone
No milestone
Development

No branches or pull requests
1 participant
@serhiy-storchaka