MVP Memo Table (DO NOT MERGE)

[connortsui20]

Very incomplete

Less incomplete

TODO:

• We should stabilize the on-disk representation of expressions (kind + serializable and deserializable data, where each operator kind needs to know how to retrieve their children from this data field). Once this is stabilized, it should not change.
• Potentially make APIs clearer. There are a bunch of Result<Result<_, _>> which can be unclear, but I have no idea how to make this better.
• Potentially should make a distinction at the type level between a GroupId and a RootGroupId, where impl From<RootGroupId> for GroupId is implemented but not the other way around...
• Since fingerprints are in a separate table, it might be good to do updates instead of additional inserts because the old fingerprints will all be stale and there shouldn't be that much contention on them?
• Should the union-find pointer data structure be separated out into its own table? Right now it is embedded into the group table records, but if we separate it out into a separate table we will have a level of abstraction that allows us to change the implementation when we need to.
• Add union by size optimization after deciding if the union find data structure should be separated from the main group records.
• Logical expression semantics should be formalized and a trait should be made that represents these semantics.
• All methods on the memo table should be serializable transactions, and the implementations should reflect that. This might mean that the transaction object needs to be passed around to different function calls, as the nested transaction API SeaORM might not be ideal (but maybe it is, we'll have to see).
• Add much more tests
• Add support for running tests in CI (should test both in-memory and disk based DBMS)
• Add code coverage in CI
• Add tracing and log support

[connortsui20]

To run tests, use:

cargo t -- --test-threads 1 --ignored

They are ignored because tests don't work in CI. I need to figure out how to run migrations in CI before those will work.

Also, each of the tests assumes they are isolated. I need to convert all of the implementation methods to use transaction instead in order to make this thread safe, which won't be hard but will be a bit tedious.

[skyzh]

You can create a sqlite in memory database and run the migrate function on it for every test case?

[connortsui20]

Yes itll be easy I just haven't had time to do it yet and it's not that important