feat: introduce rtdb server-client caching (WIP)

[patrickmichalina]

Before going too far down the rabbit hole, I wanted to get feedback on the approach. So far this only accounts for READING from the cache (if it exists, once angular loads and takes over)

[davideast]

Hey @patrickmichalina! I'll let @jamesdaniels do more explaining, but we have an implementation for this planned that will take use of an internal SDK method. It will make it easier for the SDK to take over from transferred state.

[patrickmichalina]

@jamesdaniels @davideast Ok so there are a lot of concerns with this feature to address. After spending a great deal of time trying to work it out, it seems the most efficient way to handle this is to have a separate server side module/service that handles each database (rtdb/firestore) differently.

Instead of using the client libraries (which have auth limitations, assumes localstorage of auth, sometimes hangs server reloads), we should use HTTP (or possible RPC) to perform the requests. In general I think only a subset of the features need to be supported on the server. Users are likely to only need to get/list their documents with the ability to query. This is actually pretty easy for RTDB, but more challenging for firestore since their libraries do not expose a method to convert queries to "structured queries" for the http POST.

RTDB mostly complete implementation (even includes auth from current user via cookies):

• client
• server

Firestore is not done, since I need to figure out the StructuredQuery situation :(

Thoughts?

[hiepxanh]

hello, any update on this?

[patrickmichalina]

Anyone want to comment on the approach we can take to authorize the current user on the server for firestore calls? RTDB is straight forward, but the complexity of firestore make me not want to right the REST wrapper just so I can set an auth token.

[hiepxanh]

can you explain current situation, as I know, RTDB is working with auth but firestore doesn't.
We need a way to implement authentication for firestore ? or we need a way (without library) to set an auth token?
why current firebasejs library cannot resolve it?

[patrickmichalina]

@hiepxanh sure.

RTDB
In my current test repo, I have created a server side wrapper that converts the client query into an HTTP GET /w query params to the RTDB rest endpoint, with the user's auth token via cookies passed to the server. This is fairly easy since the RTDB client library exposes a method toRestQueryStringParameters() which allows for the easy creation of the correct url to send for the request.

The working code can be found here. I believe this to be a great solution for transferring state to the client + authorized requests. It would be nice if the initial client request was not made, but its handled with a distinctUntilChanged operator call. Simple.

Firestore
This is an entire different beast. There is no such method toRestQueryStringParameters() to help create a structured query to send over HTTP. I have begun mapping the orderBy, limit, and filter commands to create enough surface area to get the job done until this is all solved.

It would be nice to simple use the client library on both server and client, but the auth situation is the issue with this. I can find no way to force a token to be used. It also wants to save the session in localStorage I believe? This is kind of an issue on the server. @jamesdaniels might have some knowledge here?

[patrickmichalina]

Both working here: https://fusing-angular-pr-64.herokuapp.com/firebase

But firestore is not using any queries and no auth is enforced in this example database (so you don't need to login to see the example)

[hiepxanh]

first:
I already login but still cannot add note.

second:
Can we make any implementation on firebasejs sdk?

[patrickmichalina]

@hiepxanh you won't be able to login to that PR because it is a heroku review app and I have not enabled that domain on Auth0 :/, but I can assure you that works locally.

[hiepxanh]

Ok I see

[jamesdaniels]

Going to close this PR, as I'm planning a different approach here.
Namely making the delta hash rehydratable in the JS SDK. That paired with startWith and a cache will mean once the connection is reestablished client-side you will only get changes since the server fetched the value last.

We don't want to go down the rabbit hole of mapping to http calls and having conditional behavior server/client.

If you feel strongly about this being the correct approach for you, you may want to check out angularfire-lite.

Thanks for the ideation and the work you put in here. I may source some of this change-set to bootstrap the state transfer work, if I do I'll make sure to credit you ;)