This patch updates `chunk_large_query_over`. This function works great
when it's required to chunk some data over a query. However, if the
data don't need to be chunked, it is possible to get a quicker path that
doesn't involve 2 `Vec` allocations, nor a `split_off` of one `Vec`, nor
a `Vec::extend` (which move data in memory). The quicker path, which is
also the most likely to be hit, simply does a single `Vec` allocation,
and that's it.
* chore(oidc): put impl of OIDC server behind a trait and add tests for the OIDC flow
chore(oidc): add first tests for login/AuthorizationResponse
chore(oidc): add tests for finish_authorization/finish_login/refresh_access_token
chore(oidc): add test for logout
chore(🤷): clippy/fmt
* chore(oidc): move account management test to the new `tests` module
* chore(oidc): address review comments
Prior to this patch, we were using 2 constants to define the
sync indicator delays: `SYNC_INDICATOR_DELAY_BEFORE_SHOWING` and
`SYNC_INDICATOR_DELAY_BEFORE_HIDING`. After some discussions with
some users, it appears that it's desirable to make these values
parameterizable.
Thus, this patch updates `RoomListService::sync_indicator` to accept 2
parameters: `delay_before_showing` and `delay_before_hiding`. The patch
also updates the FFI bindings.
In `RoomList::entries_with_dynamic_adapters`, we were using the
`Limit::dynamic` constructor. It builds a `Limit` stream adapter where
the limits come from a `Stream`. The immediate impact is that this
constructor does only return a new `Stream`, but it cannot return
the initial items of the vector. Thus, it wasn't possible to reset
the final `Stream` with a `VectorDiff::Reset { values }`. Instead, we
were emitting a `Clear` (from this code) then a `Append` (from the
`Limit` stream). Sadly, for some client apps, like Element X iOS, these
2 diffs (`Clear` and `Append`) result in a “rendering blink”.
Hopefully for us, now there is new constructors for `Limit`! One of them
is `Limit::dynamic_with_initial_limit`, which returns the initial items
along with the new `Stream`. That's perfect, we can reset the final
`Stream` with `Reset { values }` again, thus removing the “UI blinking”
effect in Element X iOS.
This patch switches `Limit::dynamic` to
`Limit::dynamic_with_initial_limit`, and updates/simplifies the tests
accordingly.
`RoomListService::new_internal` were creating the `invites` sliding
sync list, with a cache. It is not a good idea. It should not be cached.
Let's remove that :-).
`RoomListLoadingState` was initialized with the `NotLoaded` state. This
is always true… except when there is a cache! If the cache contains
N rooms, the loading state should be `Loaded { … }`. Then the next
sync (if any) will update the loading state to `Loaded { … }` with
an adjusted number of rooms or something like that, but semantically
speaking, the presence of the cache should result in a `Loaded` state.
This patch fixes this behavior, and also adds the tests.
For some room lists, the number of entries can be gigantic. For example,
some accounts have 800, 2500, or even 4000 rooms! It's not necessary for
the client/app to display all the 4000 rooms. First, it can create some
performance issues, second, nobody will scroll 4000 rooms to search for
a particular room :-). Such users are more likely to use a search bar or
something equivalent. The idea is that `RoomListService` will continue
to sync all the data, but only a _limited_ version of it will be shared
to the client/app.
This patch takes `RoomList::entries_with_dynamic_filter`, and improves
it to include this (dynamic) limit.
This patch renames `RoomList::entries_with_dynamic_filter`
to `::entries_with_dynamic_adapters`. It now returns a
`RoomListDynamicEntriesController`, which is a renaming of
`DynamicRoomListFilter`. Basically, the “dynamic filter” becomes a
“dynamic controller” because `RoomList::entries_with_dynamic_adapters`
manages more than a filter. It now uses
`eyeball_im_util::vector::DynamicLimit` to dynamically limit the size
of entries. And that's the major idea behind this patch.
`RoomListDynamicEntriesController::set` is renamed `::set_filter`, and 2
new methods are introduced: `add_one_page` and `reset_to_one_page`.
A _page_ is like a chunk of room entries we want to view or add. When
doing `next_page`, the limit increases to `old_limit + page_size`. The
`reset_pages` method resets the `limit` to `page_size` only.
This adds a cross-process lock for refresh to work correctly.
We want to coordinate token refresh across multiple processes. For that, we're using a cross-process lock, and a value in the database identifying the latest session tokens that are valid (a hash of the actual tokens, for security reasons).
Whenever we run into an HTTP error indicating that the tokens have been invalidated, we try to refresh the access tokens; that's already existing prior to this PR. The novelty introduced is that we take a cross-process lock before doing so, now. Taking this lock will also load a session hash from the database, and we'll compare it against the latest "known" session hash (that the current process saved into its memory).
If there's no mismatch (i.e. the database and the currently known are the same), then we're all good and can keep going with the refresh, synchronize the hashes everywhere (in-memory and database), make sure the client is notified about it (through a new user-provided callback `SaveSessionCallback`; on iOS this will save it into the device's keychain).
Otherwise, that means another process has done a refresh under our feet. In that case, we ask an authoritative source for trusted session tokens. On iOS, they're reloaded from the device keychain; that happens through a new user-provided callback `ReloadSessionCallback`. Then, we make sure that the DB and the in-memory value recall this latest value.
An embedder who would like to make use of the cross-process locking mechanism should call `client.oidc().set_session_callbacks` and `client.oidc().enable_cross_process_refresh`. If only interested with the pings for new sessions, the client may only call `client.oidc().set_session_callbacks`.
Fixes https://github.com/matrix-org/matrix-rust-sdk/issues/2418.
Fixes https://github.com/matrix-org/matrix-rust-sdk/issues/2476
## Future improvements
- More testing of the whole flow. Not sure if mocking will be quite fit for OIDC, as this may require setting up an HTTPS server for the authentication code exchange and other OIDC-specific flows.
- Get rid of `SessionChange`, which duplicates in some way how a client can be notified about session changes.
---
* chore: replace manual StateMemoryStore::new with derived Default
* feat: add store backing for cross-process locking in state store
* chore: rename CryptoStoreLock to CrossProcessStoreLock
* chore: generalize cross-process lock
* feat: move the cross-process locking mechanism to the main crate
* feat: add support for cross-process store lock in the state store 🥳
* feat: implement a cross-process lock for OIDC token refresh
* chore: tweak comment + function name
* feat: make restore_session safe wrt cross-process lock
* feat: add FFI method + add mechanism to reload from keychain
* fix rename
* feat: return early when there was another process refreshed tokens
* fix FFI compile error + tweak some comments
* fix: put the reload_session callback and cross-process locks behind Arc to share them across clients
* feat: Add session retrieval to FFI.
* HACKY; KIDS DON'T DO THIS AT HOME
* chore: log if the hash from db isn't the same from the one from the returned session
* make it simpler to test OIDC token refresh
* some work, that includes fixes and a first test
* feat: require that the reload_session_callback be set at the same time as the cross-process lock
* chore: traces, traces everywhere
* fix: inherit session_change_sender when creating the notification client
* Some FFI improvements to help with tokio problems
* feat: resilient mode when DB/callback disagree about session (callback wins!)
* chore: move sender.send to the finish_refreshing function
* feat: add a save_session callback in the FFI and use it to save the session in keychain while holding XP lock
* fix test expectation after adding the check 🤷
* feat: split the ClientDelegate into two parts, including brand new ClientSessionDelegate
* chore: get rid of lease lock impl in the state store, as it's now unused
* a mix of fmt + clippy
* feat: add ctor for the crossprocessrefreshlockctx
* Include user ID when retrieving session.
Necessary as this isn't known when creating the AuthenticationService.
* yo dawg, you can't block while you block
* share auth data between parent and child client, add lock, AAAAAA this is messy
* tweaks
* feat: make the cross-process store locks generic
And move the implementation to the common crate.
* chore: upgrade some code comments to doc comments in `OngoingMigration`
* feat: implement `CryptoStore::remove_custom_value`
As it's going to be used for the OIDC PR, so as to remove a remembered hash of session tokens.
* remove unneeded remnants
* correctly wait for current request to finish
* feat: make it possible to setup session delegates on android too(?)
* put the cross process stuff in its own file
* typos 🤷
* fix: detach before sending token refresh request, to make sure the response tokens are always properly saved
* kleepee
* First round of review, thanks jonas!
* review round 2. FIGHT
* remove useless logs + avoid using deref explicitly
* more specialized error when cross-process lock is enabled without session callbacks
* fix: avoid cyclic reference between the session callback and client
---------
Co-authored-by: Doug <douglase@element.io>
Co-authored-by: Jonas Platte <jplatte@matrix.org>
Benjamin Bouvier