gh-103489: Add get/set config methods to sqlite3.Connection

[erlend-aasland]

• Issue: Expose sqlite3_db_config and verbs (or equivalent) #103489

[erlend-aasland]

cc. @simonw / @numist: proof of concept; feel free to test.

I deliberately did not add support for SQLITE_DBCONFIG_MAINDBNAME and SQLITE_DBCONFIG_LOOKASIDE.

[numist]

Good call on SQLITE_DBCONFIG_LOOKASIDE. Is the decision wrt SQLITE_DBCONFIG_MAINDBNAME due more to the risk of breaking existing code relying on the magic constant MAIN. or the memory lifetime requirements of its parameter?

I'm surprised to see SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER made the cut given it gates operation of fts3_tokenizer(), which takes a C struct of function pointers.

[erlend-aasland]

Good call on SQLITE_DBCONFIG_LOOKASIDE. Is the decision wrt SQLITE_DBCONFIG_MAINDBNAME due more to the risk of breaking existing code relying on the magic constant MAIN. or the memory lifetime requirements of its parameter?

Well, those two are the only ones with non-standard arg spec, so by keeping them out we get a cleaner API. They are also pretty esoteric features IMO, so I'm not sure they're worth the added complexity.

I'm surprised to see SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER made the cut given it gates operation of fts3_tokenizer(), which takes a C struct of function pointers.

Thanks, you're quite correct; there's no reason to expose that constant.

[numist]

Well, those two are the only ones with non-standard arg spec, so by keeping them out we get a cleaner API.

Fair enough, though it seems like this runs a risk the SQLite project adds a desirable verb with non-int parameters in the future.

I don't see an API analogue for sqlite3_config in Python to act as precedent¹; would it make sense/be more idiomatic for the verbs to be subproperties of a config property on the database object? Something like:

db.config.defensive = False

There's still a type risk here—SQLite has a history of adding third options to formerly boolean configurations (like PRAGMA auto_vacuum) but it's not nearly as rigid

Footnotes

1. just as well, global configurations wreak havoc on unsuspecting code modules that expect default behaviours ↩

[erlend-aasland]

Fair enough, though it seems like this runs a risk the SQLite project adds a desirable verb with non-int parameters in the future.

Yep, that's a risk, but we can expand the scope of the new Python API later, so it's not too big of a risk. Another option is of course the namespace design you hinted at.

I don't see an API analogue for sqlite3_config in Python to act as precedent [...]

The problem with that SQLite C API, is that it must be called prior to SQLite initialisation. Currently, that happens during sqlite3 initialisation. We initialise SQLite explicitly with a call to sqlite3_initialize, and we kind of need it to stay like that, because the runtime library may have been compiled with SQLITE_OMIT_AUTOINIT. I see four possible paths:

1. Expose sqlite3_config and initialize SQLite lazily (for example during sqlite3.connect / sqlite3.Connection.__init__); raise an exception if the user tries to alter the configuration post init.
2. Expose sqlite3_config and require explicit SQLite init (backwards incompatible, so not a real alternative)
3. Expose sqlite3_config and wrap it with a sqlite3_shutdown/sqlite3_initialize dance; fail if there are open connections.
4. Don't expose sqlite3_config

Anyway, this discussion is off-topic for this PR :) If you want to follow this up, please create a topic on Discourse; we can continue there.

[erlend-aasland]

@numist:

[...] would it make sense/be more idiomatic for the verbs to be subproperties of a config property on the database object? Something like:

db.config.defensive = False

A namespace could of course be an option, but I think a simple get/set pair is a cleaner API for this feature (both programmatically and regarding to documentation). (Also, we've already got setlimit() and getlimit() for connection limits.)

[erlend-aasland]

@AlexWaygood: The added docs are quite sparse, but I'm hesitant to explode this PR by documenting every single constant. I've added a link to the SQLite docs, maybe that is enough. What do you think?

[AlexWaygood]

@AlexWaygood: The added docs are quite sparse, but I'm hesitant to explode this PR by documenting every single constant. I've added a link to the SQLite docs, maybe that is enough. What do you think?

This looks like a reasonable solution to me, but I'm not an SQLite expert :)

[AlexWaygood]

Docs approval!

[erlend-aasland]

Thanks everybody! I'll land this later today.