chore: Release matrix-sdk version 0.9.0

doc(crypto): Add changelog entry for #4424
chore: Make Clippy happy.
2024-12-18 15:12:23 +01:00 · 2024-12-18 13:51:00 +00:00 · 2024-12-18 13:24:55 +01:00 · 2024-12-18 11:25:53 +00:00 · 2024-12-18 11:22:53 +00:00 · 2024-12-18 11:22:53 +00:00
226 changed files with 14340 additions and 4071 deletions
@@ -24,6 +24,7 @@ allow = [
    "ISC",
    "MIT",
    "MPL-2.0",
+    "Unicode-3.0",
    "Zlib",
 ]
 exceptions = [
@@ -54,6 +55,8 @@ allow-git = [
    "https://github.com/element-hq/tracing.git",
    # Sam as for the tracing dependency.
    "https://github.com/element-hq/paranoid-android.git",
+    # Well, it's Ruma.
+    "https://github.com/ruma/ruma",
    # A patch override for the bindings: https://github.com/rodrimati1992/const_panic/pull/10
    "https://github.com/jplatte/const_panic",
    # A patch override for the bindings: https://github.com/smol-rs/async-compat/pull/22
@@ -131,7 +131,7 @@ jobs:
  test-apple:
    name: matrix-rust-components-swift
    needs: xtask
-    runs-on: macos-14
+    runs-on: macos-15
    if: github.event_name == 'push' || !github.event.pull_request.draft

    steps:
@@ -175,7 +175,7 @@ jobs:
        run: swift test

      - name: Build Framework
-        run: target/debug/xtask swift build-framework --target=aarch64-apple-ios --profile=dev
+        run: target/debug/xtask swift build-framework --target=aarch64-apple-ios --profile=reldbg

  complement-crypto:
    name: "Run Complement Crypto tests"
@@ -186,7 +186,7 @@ jobs:

  test-crypto-apple-framework-generation:
    name: Generate Crypto FFI Apple XCFramework
-    runs-on: macos-14
+    runs-on: macos-15
    if: github.event_name == 'push' || !github.event.pull_request.draft

    steps:
@@ -304,7 +304,7 @@ jobs:
        uses: actions/checkout@v4

      - name: Check the spelling of the files in our repo
-        uses: crate-ci/typos@v1.27.3
+        uses: crate-ci/typos@v1.28.3

  clippy:
    name: Run clippy
@@ -35,7 +35,7 @@ jobs:
            os-name: 🐧
            cachekey-id: linux

-          - os: macos-14
+          - os: macos-15
            os-name: 🍏
            cachekey-id: macos

@@ -45,9 +45,46 @@ that is, just the branch name.)

 # Writing changelog entries

-We aim to maintain clear and informative changelogs that accurately reflect the
-changes in our project. This guide will help you write useful changelog entries
-using git-cliff, which fetches changelog entries from commit messages.
+Our goal is to maintain clear, concise, and informative changelogs that
+accurately document changes in the project. Changelog entries should be written
+manually for each crate in the `/crates/$CRATE_NAME/Changelog.md` file.
+
+Be sure to include a link to the pull request for additional context. A
+well-written changelog entry should be understandable even to those who may not
+be deeply familiar with the project. Provide enough context to ensure clarity
+and ease of understanding.
+
+A couple of examples of bad changelog entry would look like:
+
+```markdown
+- Fixed a panic.
+```
+
+```markdown
+- Added the Bar function to Foo.
+```
+
+A good example of a changelog entry could look like the following:
+
+```markdown
+- Use the inviter's server name and the server name from the room alias as
+  fallback values for the via parameter when requesting the room summary from
+  the homeserver. This ensures requests succeed even when the room being
+  previewed is hosted on a federated server.
+  ([#4357](https://github.com/matrix-org/matrix-rust-sdk/pull/4357))
+```
+
+For security-related changelog entries, please include the following additional
+details alongside the pull request number:
+
+* Impact: Clearly describe the issue's potential impact on users or systems.
+* CVE Number: If available, include the CVE (Common Vulnerabilities and Exposures) identifier.
+* GitHub Advisory Link: Provide a link to the corresponding GitHub security advisory for further context.
+
+```markdown
+- Use a constant-time Base64 encoder for secret key material to mitigate
+  side-channel attacks leaking secret key material ([#156](https://github.com/matrix-org/vodozemac/pull/156)) (Low, [CVE-2024-40640](https://www.cve.org/CVERecord?id=CVE-2024-40640), [GHSA-j8cm-g7r6-hfpq](https://github.com/matrix-org/vodozemac/security/advisories/GHSA-j8cm-g7r6-hfpq)).
+```

 ## Commit message format

@@ -74,45 +111,20 @@ The type of changes which will be included in changelogs is one of the following
 The scope is optional and can specify the area of the codebase affected (e.g.,
 olm, cipher).

-### Changelog trailer
-
-In addition to the Conventional Commit format, you can use the `Changelog` git
-trailer to specify the changelog message explicitly. When that trailer is
-present, its value will be used as the changelog entry instead of the commit's
-leading line. The `Breaking-Change` git trailer can be used in a similar manner
-if the changelog entry should be marked as a breaking change.
-
-
-#### Example commit message
-
-```
-feat: Add a method to encode Ed25519 public keys to Base64
-
-This patch adds the `Ed25519PublicKey::to_base64()` method, which allows us to
-stringify Ed25519 and thus present them to users. It's also commonly used when
-Ed25519 keys need to be inserted into JSON.
-
-Changelog: Add the `Ed25519PublicKey::to_base64()` method which can be used to
- stringify the Ed25519 public key.
-```
-
-In this commit message, the content specified in the `Changelog` trailer will be
-used for the changelog entry.
-
-Be careful to add at least one whitespace after new lines to create a paragraph.
-
 ### Security fixes

 Commits addressing security vulnerabilities must include specific trailers for
-vulnerability metadata. These commits are required to include at least the
-`Security-Impact` trailer to indicate that the commit is a security fix.
+vulnerability metadata, which should also be reflected in the corresponding
+changelog entry.

-Security issues have some additional git-trailers:
+The metadata must be included in the following git-trailers:

 * `Security-Impact`: The magnitude of harm that can be expected, i.e. low/moderate/high/critical.
 * `CVE`: The CVE that was assigned to this issue.
 * `GitHub-Advisory`: The GitHub advisory identifier.

+Please include all of the fields that are available.
+
 Example:

 ```
@@ -131,9 +143,6 @@ material.
 Security-Impact: Low
 CVE: CVE-2024-40640
 GitHub-Advisory: GHSA-j8cm-g7r6-hfpq
-
-Changelog: Use a constant-time Base64 encoder for secret key material
-to mitigate side-channel attacks leaking secret key material.
 ```

 ## Review process
@@ -18,35 +18,45 @@ default-members = ["benchmarks", "crates/*", "labs/*"]
 resolver = "2"

 [workspace.package]
-rust-version = "1.76"
+rust-version = "1.82"

 [workspace.dependencies]
-anyhow = "1.0.68"
+anyhow = "1.0.93"
 aquamarine = "0.6.0"
-assert-json-diff = "2"
+assert-json-diff = "2.0.2"
 assert_matches = "1.5.0"
-assert_matches2 = "0.1.1"
+assert_matches2 = "0.1.2"
 async-rx = "0.1.3"
-async-stream = "0.3.3"
-async-trait = "0.1.60"
+async-stream = "0.3.5"
+async-trait = "0.1.83"
 as_variant = "1.2.0"
-base64 = "0.22.0"
-byteorder = "1.4.3"
+base64 = "0.22.1"
+byteorder = "1.5.0"
+chrono = "0.4.38"
 eyeball = { version = "0.8.8", features = ["tracing"] }
 eyeball-im = { version = "0.5.1", features = ["tracing"] }
 eyeball-im-util = "0.7.0"
-futures-core = "0.3.28"
+futures-core = "0.3.31"
 futures-executor = "0.3.21"
-futures-util = "0.3.26"
-growable-bloom-filter = "2.1.0"
+futures-util = "0.3.31"
+gloo-timers = "0.3.0"
+growable-bloom-filter = "2.1.1"
+hkdf = "0.12.4"
+hmac = "0.12.1"
 http = "1.1.0"
 imbl = "3.0.0"
-itertools = "0.12.0"
-once_cell = "1.16.0"
-pin-project-lite = "0.2.9"
+indexmap = "2.6.0"
+itertools = "0.13.0"
+js-sys = "0.3.69"
+mime = "0.3.17"
+once_cell = "1.20.2"
+pbkdf2 = { version = "0.12.2" }
+pin-project-lite = "0.2.15"
+proptest = { version = "1.5.0", default-features = false, features = ["std"] }
 rand = "0.8.5"
 reqwest = { version = "0.12.4", default-features = false }
-ruma = { version = "0.11.1", features = [
+rmp-serde = "1.3.0"
+ruma = { version = "0.12.0", features = [
    "client-api-c",
    "compat-upload-signatures",
    "compat-user-id",
@@ -59,38 +69,45 @@ ruma = { version = "0.11.1", features = [
    "unstable-msc3489",
    "unstable-msc4075",
    "unstable-msc4140",
+    "unstable-msc4171",
 ] }
-ruma-common = "0.14.1"
+ruma-common = "0.15.0"
 serde = "1.0.151"
 serde_html_form = "0.2.0"
 serde_json = "1.0.91"
 sha2 = "0.10.8"
-similar-asserts = "1.5.0"
+similar-asserts = "1.6.0"
 stream_assert = "0.1.1"
-thiserror = "1.0.38"
-tokio = { version = "1.39.1", default-features = false, features = ["sync"] }
+tempfile = "3.9.0"
+thiserror = "2.0.3"
+tokio = { version = "1.41.1", default-features = false, features = ["sync"] }
 tokio-stream = "0.1.14"
 tracing = { version = "0.1.40", default-features = false, features = ["std"] }
 tracing-core = "0.1.32"
 tracing-subscriber = "0.3.18"
+unicode-normalization = "0.1.24"
 uniffi = { version = "0.28.0" }
 uniffi_bindgen = { version = "0.28.0" }
-url = "2.5.0"
-vodozemac = { version = "0.8.0", features = ["insecure-pk-encryption"] }
-wiremock = "0.6.0"
-zeroize = "1.6.0"
+url = "2.5.4"
+uuid = "1.11.0"
+vodozemac = { version = "0.8.1", features = ["insecure-pk-encryption"] }
+wasm-bindgen = "0.2.84"
+wasm-bindgen-test = "0.3.33"
+web-sys = "0.3.69"
+wiremock = "0.6.2"
+zeroize = "1.8.1"

-matrix-sdk = { path = "crates/matrix-sdk", version = "0.8.0", default-features = false }
-matrix-sdk-base = { path = "crates/matrix-sdk-base", version = "0.8.0" }
-matrix-sdk-common = { path = "crates/matrix-sdk-common", version = "0.8.0" }
-matrix-sdk-crypto = { path = "crates/matrix-sdk-crypto", version = "0.8.0" }
+matrix-sdk = { path = "crates/matrix-sdk", version = "0.9.0", default-features = false }
+matrix-sdk-base = { path = "crates/matrix-sdk-base", version = "0.9.0" }
+matrix-sdk-common = { path = "crates/matrix-sdk-common", version = "0.9.0" }
+matrix-sdk-crypto = { path = "crates/matrix-sdk-crypto", version = "0.9.0" }
 matrix-sdk-ffi-macros = { path = "bindings/matrix-sdk-ffi-macros", version = "0.7.0" }
-matrix-sdk-indexeddb = { path = "crates/matrix-sdk-indexeddb", version = "0.8.0", default-features = false }
-matrix-sdk-qrcode = { path = "crates/matrix-sdk-qrcode", version = "0.8.0" }
-matrix-sdk-sqlite = { path = "crates/matrix-sdk-sqlite", version = "0.8.0", default-features = false }
-matrix-sdk-store-encryption = { path = "crates/matrix-sdk-store-encryption", version = "0.8.0" }
+matrix-sdk-indexeddb = { path = "crates/matrix-sdk-indexeddb", version = "0.9.0", default-features = false }
+matrix-sdk-qrcode = { path = "crates/matrix-sdk-qrcode", version = "0.9.0" }
+matrix-sdk-sqlite = { path = "crates/matrix-sdk-sqlite", version = "0.9.0", default-features = false }
+matrix-sdk-store-encryption = { path = "crates/matrix-sdk-store-encryption", version = "0.9.0" }
 matrix-sdk-test = { path = "testing/matrix-sdk-test", version = "0.7.0" }
-matrix-sdk-ui = { path = "crates/matrix-sdk-ui", version = "0.8.0", default-features = false }
+matrix-sdk-ui = { path = "crates/matrix-sdk-ui", version = "0.9.0", default-features = false }

 # Default release profile, select with `--release`
 [profile.release]
@@ -132,7 +149,10 @@ paranoid-android = { git = "https://github.com/element-hq/paranoid-android.git",
 [workspace.lints.rust]
 rust_2018_idioms = "warn"
 semicolon_in_expressions_from_macros = "warn"
-unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tarpaulin_include)'] }
+unexpected_cfgs = { level = "warn", check-cfg = [
+    'cfg(tarpaulin_include)', # Used by tarpaulin (code coverage)
+    'cfg(ruma_unstable_exhaustive_types)', # Used by Ruma's EventContent derive macro
+] }
 unused_extern_crates = "warn"
 unused_import_braces = "warn"
 unused_qualifications = "warn"
@@ -24,10 +24,6 @@ The rust-sdk consists of multiple crates that can be picked at your convenience:
 - **matrix-sdk-crypto** - No (network) IO encryption state machine that can be
  used to add Matrix E2EE support to your client or client library.

-## Minimum Supported Rust Version (MSRV)
-
-These crates are built with the Rust language version 2021 and require a minimum compiler version of `1.70`.
-
 ## Status

 The library is in an alpha state, things that are implemented generally work but
@@ -17,8 +17,8 @@ The procedure is as follows:
  git switch -c release-x.y.z
   ```

-2. Prepare the release. This will update the `README.md`, prepend the `CHANGELOG.md`
-   file using `git cliff`, and bump the version in the `Cargo.toml` file.
+2. Prepare the release. This will update the `README.md`, set the versions in
+   the `CHANGELOG.md` file, and bump the version in the `Cargo.toml` file.

 ```bash
 cargo xtask release prepare --execute minor|patch|rc
@@ -23,7 +23,7 @@ tokio = { workspace = true, default-features = false, features = ["rt-multi-thre
 wiremock = { workspace = true }

 [target.'cfg(target_os = "linux")'.dependencies]
-pprof = { version = "0.13.0", features = ["flamegraph", "criterion"] }
+pprof = { version = "0.14.0", features = ["flamegraph", "criterion"] }

 [[bench]]
 name = "crypto_bench"
@@ -71,10 +71,6 @@ $ cp ../../target/aarch64-linux-android/debug/libmatrix_crypto.so \
     /home/example/matrix-sdk-android/src/main/jniLibs/aarch64/libuniffi_olm.so
 ```

-## Minimum Supported Rust Version (MSRV)
-
-These crates are built with the Rust language version 2021 and require a minimum compiler version of `1.62`.
-
 ## License

 [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)
@@ -1,10 +1,10 @@
-use std::{env, error::Error};
+use std::{env, error::Error, path::PathBuf, process::Command};

 use vergen::EmitBuilder;

 /// Adds a temporary workaround for an issue with the Rust compiler and Android
 /// in x86_64 devices: https://github.com/rust-lang/rust/issues/109717.
-/// The workaround comes from: https://github.com/mozilla/application-services/pull/5442
+/// The workaround is based on: https://github.com/mozilla/application-services/pull/5442
 ///
 /// IMPORTANT: if you modify this, make sure to modify
 /// [../matrix-sdk-ffi/build.rs] too!
@@ -12,26 +12,45 @@ fn setup_x86_64_android_workaround() {
    let target_os = env::var("CARGO_CFG_TARGET_OS").expect("CARGO_CFG_TARGET_OS not set");
    let target_arch = env::var("CARGO_CFG_TARGET_ARCH").expect("CARGO_CFG_TARGET_ARCH not set");
    if target_arch == "x86_64" && target_os == "android" {
-        let android_ndk_home = env::var("ANDROID_NDK_HOME").expect("ANDROID_NDK_HOME not set");
-        let build_os = match env::consts::OS {
-            "linux" => "linux",
-            "macos" => "darwin",
-            "windows" => "windows",
-            _ => panic!(
-                "Unsupported OS. You must use either Linux, MacOS or Windows to build the crate."
-            ),
-        };
-        const DEFAULT_CLANG_VERSION: &str = "18";
-        let clang_version =
-            env::var("NDK_CLANG_VERSION").unwrap_or_else(|_| DEFAULT_CLANG_VERSION.to_owned());
-        let linux_x86_64_lib_dir = format!(
-            "toolchains/llvm/prebuilt/{build_os}-x86_64/lib/clang/{clang_version}/lib/linux/"
+        // Configure rust to statically link against the `libclang_rt.builtins` supplied
+        // with clang.
+
+        // cargo-ndk sets CC_x86_64-linux-android to the path to `clang`, within the
+        // Android NDK.
+        let clang_path = PathBuf::from(
+            env::var("CC_x86_64-linux-android").expect("CC_x86_64-linux-android not set"),
        );
-        println!("cargo:rustc-link-search={android_ndk_home}/{linux_x86_64_lib_dir}");
+
+        // clang_path should now look something like
+        // `.../sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/bin/clang`.
+        // We strip `/bin/clang` from the end to get the toolchain path.
+        let toolchain_path = clang_path
+            .ancestors()
+            .nth(2)
+            .expect("could not find NDK toolchain path")
+            .to_str()
+            .expect("NDK toolchain path is not valid UTF-8");
+
+        let clang_version = get_clang_major_version(&clang_path);
+
+        println!("cargo:rustc-link-search={toolchain_path}/lib/clang/{clang_version}/lib/linux/");
        println!("cargo:rustc-link-lib=static=clang_rt.builtins-x86_64-android");
    }
 }

+/// Run the clang binary at `clang_path`, and return its major version number
+fn get_clang_major_version(clang_path: &PathBuf) -> String {
+    let clang_output =
+        Command::new(clang_path).arg("-dumpversion").output().expect("failed to start clang");
+
+    if !clang_output.status.success() {
+        panic!("failed to run clang: {}", String::from_utf8_lossy(&clang_output.stderr));
+    }
+
+    let clang_version = String::from_utf8(clang_output.stdout).expect("clang output is not utf8");
+    clang_version.split('.').next().expect("could not parse clang output").to_owned()
+}
+
 fn main() -> Result<(), Box<dyn Error>> {
    setup_x86_64_android_workaround();

@@ -1,13 +1,17 @@
 use std::{mem::ManuallyDrop, sync::Arc};

-use matrix_sdk_crypto::dehydrated_devices::{
-    DehydratedDevice as InnerDehydratedDevice, DehydratedDevices as InnerDehydratedDevices,
-    RehydratedDevice as InnerRehydratedDevice,
+use matrix_sdk_crypto::{
+    dehydrated_devices::{
+        DehydratedDevice as InnerDehydratedDevice, DehydratedDevices as InnerDehydratedDevices,
+        RehydratedDevice as InnerRehydratedDevice,
+    },
+    store::DehydratedDeviceKey as InnerDehydratedDeviceKey,
 };
 use ruma::{api::client::dehydrated_device, events::AnyToDeviceEvent, serde::Raw, OwnedDeviceId};
 use serde_json::json;
 use tokio::runtime::Handle;
-use zeroize::Zeroize;
+
+use crate::{CryptoStoreError, DehydratedDeviceKey};

 #[derive(Debug, thiserror::Error, uniffi::Error)]
 #[uniffi(flat_error)]
@@ -22,6 +26,8 @@ pub enum DehydrationError {
    Store(#[from] matrix_sdk_crypto::CryptoStoreError),
    #[error("The pickle key has an invalid length, expected 32 bytes, got {0}")]
    PickleKeyLength(usize),
+    #[error(transparent)]
+    Rand(#[from] rand::Error),
 }

 impl From<matrix_sdk_crypto::dehydrated_devices::DehydrationError> for DehydrationError {
@@ -33,6 +39,9 @@ impl From<matrix_sdk_crypto::dehydrated_devices::DehydrationError> for Dehydrati
                Self::MissingSigningKey(e)
            }
            matrix_sdk_crypto::dehydrated_devices::DehydrationError::Store(e) => Self::Store(e),
+            matrix_sdk_crypto::dehydrated_devices::DehydrationError::PickleKeyLength(l) => {
+                Self::PickleKeyLength(l)
+            }
        }
    }
 }
@@ -66,14 +75,14 @@ impl DehydratedDevices {

    pub fn rehydrate(
        &self,
-        pickle_key: Vec<u8>,
+        pickle_key: &DehydratedDeviceKey,
        device_id: String,
        device_data: String,
    ) -> Result<Arc<RehydratedDevice>, DehydrationError> {
        let device_data: Raw<_> = serde_json::from_str(&device_data)?;
        let device_id: OwnedDeviceId = device_id.into();

-        let mut key = get_pickle_key(&pickle_key)?;
+        let key = InnerDehydratedDeviceKey::from_slice(&pickle_key.inner)?;

        let ret = RehydratedDevice {
            runtime: self.runtime.to_owned(),
@@ -85,10 +94,41 @@ impl DehydratedDevices {
        }
        .into();

-        key.zeroize();
-
        Ok(ret)
    }
+
+    /// Get the cached dehydrated device pickle key if any.
+    ///
+    /// None if the key was not previously cached (via
+    /// [`Self::save_dehydrated_device_pickle_key`]).
+    ///
+    /// Should be used to periodically rotate the dehydrated device to avoid
+    /// OTK exhaustion and accumulation of to_device messages.
+    pub fn get_dehydrated_device_key(
+        &self,
+    ) -> Result<Option<crate::DehydratedDeviceKey>, CryptoStoreError> {
+        Ok(self
+            .runtime
+            .block_on(self.inner.get_dehydrated_device_pickle_key())?
+            .map(crate::DehydratedDeviceKey::from))
+    }
+
+    /// Store the dehydrated device pickle key in the crypto store.
+    ///
+    /// This is useful if the client wants to periodically rotate dehydrated
+    /// devices to avoid OTK exhaustion and accumulated to_device problems.
+    pub fn save_dehydrated_device_key(
+        &self,
+        pickle_key: &crate::DehydratedDeviceKey,
+    ) -> Result<(), CryptoStoreError> {
+        let pickle_key = InnerDehydratedDeviceKey::from_slice(&pickle_key.inner)?;
+        Ok(self.runtime.block_on(self.inner.save_dehydrated_device_pickle_key(&pickle_key))?)
+    }
+
+    /// Deletes the previously stored dehydrated device pickle key.
+    pub fn delete_dehydrated_device_key(&self) -> Result<(), CryptoStoreError> {
+        Ok(self.runtime.block_on(self.inner.delete_dehydrated_device_pickle_key())?)
+    }
 }

 #[derive(uniffi::Object)]
@@ -138,15 +178,13 @@ impl DehydratedDevice {
    pub fn keys_for_upload(
        &self,
        device_display_name: String,
-        pickle_key: Vec<u8>,
+        pickle_key: &DehydratedDeviceKey,
    ) -> Result<UploadDehydratedDeviceRequest, DehydrationError> {
-        let mut key = get_pickle_key(&pickle_key)?;
+        let key = InnerDehydratedDeviceKey::from_slice(&pickle_key.inner)?;

        let request =
            self.runtime.block_on(self.inner.keys_for_upload(device_display_name, &key))?;

-        key.zeroize();
-
        Ok(request.into())
    }
 }
@@ -177,15 +215,36 @@ impl From<dehydrated_device::put_dehydrated_device::unstable::Request>
    }
 }

-fn get_pickle_key(pickle_key: &[u8]) -> Result<Box<[u8; 32]>, DehydrationError> {
-    let pickle_key_length = pickle_key.len();
+#[cfg(test)]
+mod tests {
+    use crate::{dehydrated_devices::DehydrationError, DehydratedDeviceKey};

-    if pickle_key_length == 32 {
-        let mut key = Box::new([0u8; 32]);
-        key.copy_from_slice(pickle_key);
+    #[test]
+    fn test_creating_dehydrated_key() {
+        let result = DehydratedDeviceKey::new();
+        assert!(result.is_ok());
+        let dehydrated_device_key = result.unwrap();
+        let base_64 = dehydrated_device_key.to_base64();
+        let inner_bytes = dehydrated_device_key.inner;

-        Ok(key)
-    } else {
-        Err(DehydrationError::PickleKeyLength(pickle_key_length))
+        let copy = DehydratedDeviceKey::from_slice(&inner_bytes).unwrap();
+
+        assert_eq!(base_64, copy.to_base64());
+    }
+
+    #[test]
+    fn test_creating_dehydrated_key_failure() {
+        let bytes = [0u8; 24];
+
+        let pickle_key = DehydratedDeviceKey::from_slice(&bytes);
+
+        assert!(pickle_key.is_err());
+
+        match pickle_key {
+            Err(DehydrationError::PickleKeyLength(pickle_key_length)) => {
+                assert_eq!(bytes.len(), pickle_key_length);
+            }
+            _ => panic!("Should have failed!"),
+        }
    }
 }
@@ -1,8 +1,9 @@
 #![allow(missing_docs)]

 use matrix_sdk_crypto::{
-    store::CryptoStoreError as InnerStoreError, KeyExportError, MegolmError, OlmError,
-    SecretImportError as RustSecretImportError, SignatureError as InnerSignatureError,
+    store::{CryptoStoreError as InnerStoreError, DehydrationError as InnerDehydrationError},
+    KeyExportError, MegolmError, OlmError, SecretImportError as RustSecretImportError,
+    SignatureError as InnerSignatureError,
 };
 use matrix_sdk_sqlite::OpenStoreError;
 use ruma::{IdParseError, OwnedUserId};
@@ -57,6 +58,8 @@ pub enum CryptoStoreError {
    InvalidUserId(String, IdParseError),
    #[error(transparent)]
    Identifier(#[from] IdParseError),
+    #[error(transparent)]
+    DehydrationError(#[from] InnerDehydrationError),
 }

 #[derive(Debug, thiserror::Error, uniffi::Error)]
@@ -112,7 +115,7 @@ mod tests {

    #[test]
    fn test_withheld_error_mapping() {
-        use matrix_sdk_crypto::types::events::room_key_withheld::WithheldCode;
+        use matrix_sdk_common::deserialized_responses::WithheldCode;

        let inner_error = MegolmError::MissingRoomKey(Some(WithheldCode::Unverified));

@@ -36,7 +36,10 @@ pub use machine::{KeyRequestPair, OlmMachine, SignatureVerification};
 use matrix_sdk_common::deserialized_responses::{ShieldState as RustShieldState, ShieldStateCode};
 use matrix_sdk_crypto::{
    olm::{IdentityKeys, InboundGroupSession, SenderData, Session},
-    store::{Changes, CryptoStore, PendingChanges, RoomSettings as RustRoomSettings},
+    store::{
+        Changes, CryptoStore, DehydratedDeviceKey as InnerDehydratedDeviceKey, PendingChanges,
+        RoomSettings as RustRoomSettings,
+    },
    types::{
        DeviceKey, DeviceKeys, EventEncryptionAlgorithm as RustEventEncryptionAlgorithm, SigningKey,
    },
@@ -62,6 +65,8 @@ pub use verification::{
 };
 use vodozemac::{Curve25519PublicKey, Ed25519PublicKey};

+use crate::dehydrated_devices::DehydrationError;
+
 /// Struct collecting data that is important to migrate to the rust-sdk
 #[derive(Deserialize, Serialize, uniffi::Record)]
 pub struct MigrationData {
@@ -822,6 +827,39 @@ impl TryFrom<matrix_sdk_crypto::store::BackupKeys> for BackupKeys {
    }
 }

+/// Dehydrated device key
+#[derive(uniffi::Record, Clone)]
+pub struct DehydratedDeviceKey {
+    pub(crate) inner: Vec<u8>,
+}
+
+impl DehydratedDeviceKey {
+    /// Generates a new random pickle key.
+    pub fn new() -> Result<Self, DehydrationError> {
+        let inner = InnerDehydratedDeviceKey::new()?;
+        Ok(inner.into())
+    }
+
+    /// Creates a new dehydration pickle key from the given slice.
+    ///
+    /// Fail if the slice length is not 32.
+    pub fn from_slice(slice: &[u8]) -> Result<Self, DehydrationError> {
+        let inner = InnerDehydratedDeviceKey::from_slice(slice)?;
+        Ok(inner.into())
+    }
+
+    /// Export the [`DehydratedDeviceKey`] as a base64 encoded string.
+    pub fn to_base64(&self) -> String {
+        let inner = InnerDehydratedDeviceKey::from_slice(&self.inner).unwrap();
+        inner.to_base64()
+    }
+}
+impl From<InnerDehydratedDeviceKey> for DehydratedDeviceKey {
+    fn from(pickle_key: InnerDehydratedDeviceKey) -> Self {
+        DehydratedDeviceKey { inner: pickle_key.into() }
+    }
+}
+
 impl From<matrix_sdk_crypto::store::RoomKeyCounts> for RoomKeyCounts {
    fn from(count: matrix_sdk_crypto::store::RoomKeyCounts) -> Self {
        Self { total: count.total as i64, backed_up: count.backed_up as i64 }
@@ -17,8 +17,8 @@ use matrix_sdk_crypto::{
    decrypt_room_key_export, encrypt_room_key_export,
    olm::ExportedRoomKey,
    store::{BackupDecryptionKey, Changes},
-    DecryptionSettings, LocalTrust, OlmMachine as InnerMachine, ToDeviceRequest,
-    UserIdentity as SdkUserIdentity,
+    types::requests::ToDeviceRequest,
+    DecryptionSettings, LocalTrust, OlmMachine as InnerMachine, UserIdentity as SdkUserIdentity,
 };
 use ruma::{
    api::{
@@ -4,9 +4,12 @@ use std::collections::HashMap;

 use http::Response;
 use matrix_sdk_crypto::{
-    CrossSigningBootstrapRequests, IncomingResponse, KeysBackupRequest, OutgoingRequest,
-    OutgoingVerificationRequest as SdkVerificationRequest, RoomMessageRequest, ToDeviceRequest,
-    UploadSigningKeysRequest as RustUploadSigningKeysRequest,
+    types::requests::{
+        AnyIncomingResponse, KeysBackupRequest, OutgoingRequest,
+        OutgoingVerificationRequest as SdkVerificationRequest, RoomMessageRequest, ToDeviceRequest,
+        UploadSigningKeysRequest as RustUploadSigningKeysRequest,
+    },
+    CrossSigningBootstrapRequests,
 };
 use ruma::{
    api::client::{
@@ -136,7 +139,7 @@ pub enum Request {

 impl From<OutgoingRequest> for Request {
    fn from(r: OutgoingRequest) -> Self {
-        use matrix_sdk_crypto::OutgoingRequests::*;
+        use matrix_sdk_crypto::types::requests::AnyOutgoingRequest::*;

        match r.request() {
            KeysUpload(u) => {
@@ -338,16 +341,16 @@ impl From<RoomMessageResponse> for OwnedResponse {
    }
 }

-impl<'a> From<&'a OwnedResponse> for IncomingResponse<'a> {
+impl<'a> From<&'a OwnedResponse> for AnyIncomingResponse<'a> {
    fn from(r: &'a OwnedResponse) -> Self {
        match r {
-            OwnedResponse::KeysClaim(r) => IncomingResponse::KeysClaim(r),
-            OwnedResponse::KeysQuery(r) => IncomingResponse::KeysQuery(r),
-            OwnedResponse::KeysUpload(r) => IncomingResponse::KeysUpload(r),
-            OwnedResponse::ToDevice(r) => IncomingResponse::ToDevice(r),
-            OwnedResponse::SignatureUpload(r) => IncomingResponse::SignatureUpload(r),
-            OwnedResponse::KeysBackup(r) => IncomingResponse::KeysBackup(r),
-            OwnedResponse::RoomMessage(r) => IncomingResponse::RoomMessage(r),
+            OwnedResponse::KeysClaim(r) => AnyIncomingResponse::KeysClaim(r),
+            OwnedResponse::KeysQuery(r) => AnyIncomingResponse::KeysQuery(r),
+            OwnedResponse::KeysUpload(r) => AnyIncomingResponse::KeysUpload(r),
+            OwnedResponse::ToDevice(r) => AnyIncomingResponse::ToDevice(r),
+            OwnedResponse::SignatureUpload(r) => AnyIncomingResponse::SignatureUpload(r),
+            OwnedResponse::KeysBackup(r) => AnyIncomingResponse::KeysBackup(r),
+            OwnedResponse::RoomMessage(r) => AnyIncomingResponse::RoomMessage(r),
        }
    }
 }
@@ -33,3 +33,4 @@ Additions:

 - Add `Encryption::get_user_identity` which returns `UserIdentity`
 - Add `ClientBuilder::room_key_recipient_strategy`
+- Add `Room::send_raw`
@@ -1,10 +1,10 @@
-use std::{env, error::Error};
+use std::{env, error::Error, path::PathBuf, process::Command};

 use vergen::EmitBuilder;

 /// Adds a temporary workaround for an issue with the Rust compiler and Android
 /// in x86_64 devices: https://github.com/rust-lang/rust/issues/109717.
-/// The workaround comes from: https://github.com/mozilla/application-services/pull/5442
+/// The workaround is based on: https://github.com/mozilla/application-services/pull/5442
 ///
 /// IMPORTANT: if you modify this, make sure to modify
 /// [../matrix-sdk-crypto-ffi/build.rs] too!
@@ -12,26 +12,45 @@ fn setup_x86_64_android_workaround() {
    let target_os = env::var("CARGO_CFG_TARGET_OS").expect("CARGO_CFG_TARGET_OS not set");
    let target_arch = env::var("CARGO_CFG_TARGET_ARCH").expect("CARGO_CFG_TARGET_ARCH not set");
    if target_arch == "x86_64" && target_os == "android" {
-        let android_ndk_home = env::var("ANDROID_NDK_HOME").expect("ANDROID_NDK_HOME not set");
-        let build_os = match env::consts::OS {
-            "linux" => "linux",
-            "macos" => "darwin",
-            "windows" => "windows",
-            _ => panic!(
-                "Unsupported OS. You must use either Linux, MacOS or Windows to build the crate."
-            ),
-        };
-        const DEFAULT_CLANG_VERSION: &str = "18";
-        let clang_version =
-            env::var("NDK_CLANG_VERSION").unwrap_or_else(|_| DEFAULT_CLANG_VERSION.to_owned());
-        let linux_x86_64_lib_dir = format!(
-            "toolchains/llvm/prebuilt/{build_os}-x86_64/lib/clang/{clang_version}/lib/linux/"
+        // Configure rust to statically link against the `libclang_rt.builtins` supplied
+        // with clang.
+
+        // cargo-ndk sets CC_x86_64-linux-android to the path to `clang`, within the
+        // Android NDK.
+        let clang_path = PathBuf::from(
+            env::var("CC_x86_64-linux-android").expect("CC_x86_64-linux-android not set"),
        );
-        println!("cargo:rustc-link-search={android_ndk_home}/{linux_x86_64_lib_dir}");
+
+        // clang_path should now look something like
+        // `.../sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/bin/clang`.
+        // We strip `/bin/clang` from the end to get the toolchain path.
+        let toolchain_path = clang_path
+            .ancestors()
+            .nth(2)
+            .expect("could not find NDK toolchain path")
+            .to_str()
+            .expect("NDK toolchain path is not valid UTF-8");
+
+        let clang_version = get_clang_major_version(&clang_path);
+
+        println!("cargo:rustc-link-search={toolchain_path}/lib/clang/{clang_version}/lib/linux/");
        println!("cargo:rustc-link-lib=static=clang_rt.builtins-x86_64-android");
    }
 }

+/// Run the clang binary at `clang_path`, and return its major version number
+fn get_clang_major_version(clang_path: &PathBuf) -> String {
+    let clang_output =
+        Command::new(clang_path).arg("-dumpversion").output().expect("failed to start clang");
+
+    if !clang_output.status.success() {
+        panic!("failed to run clang: {}", String::from_utf8_lossy(&clang_output.stderr));
+    }
+
+    let clang_version = String::from_utf8(clang_output.stdout).expect("clang output is not utf8");
+    clang_version.split('.').next().expect("could not parse clang output").to_owned()
+}
+
 fn main() -> Result<(), Box<dyn Error>> {
    setup_x86_64_android_workaround();
    uniffi::generate_scaffolding("./src/api.udl").expect("Building the UDL file failed");
@@ -120,7 +120,7 @@ impl TryFrom<PusherKind> for RumaPusherKind {
                let mut ruma_data = RumaHttpPusherData::new(data.url);
                if let Some(payload) = data.default_payload {
                    let json: Value = serde_json::from_str(&payload)?;
-                    ruma_data.default_payload = json;
+                    ruma_data.data.insert("default_payload".to_owned(), json);
                }
                ruma_data.format = data.format.map(Into::into);
                Ok(Self::Http(ruma_data))
@@ -8,6 +8,7 @@ use matrix_sdk::{
        CollectStrategy, TrustRequirement,
    },
    encryption::{BackupDownloadStrategy, EncryptionSettings},
+    event_cache::EventCacheError,
    reqwest::Certificate,
    ruma::{ServerName, UserId},
    sliding_sync::{
@@ -202,6 +203,8 @@ pub enum ClientBuildError {
    SlidingSyncVersion(VersionBuilderError),
    #[error(transparent)]
    Sdk(MatrixClientBuildError),
+    #[error(transparent)]
+    EventCache(#[from] EventCacheError),
    #[error("Failed to build the client: {message}")]
    Generic { message: String },
 }
@@ -269,6 +272,10 @@ pub struct ClientBuilder {
    room_key_recipient_strategy: CollectStrategy,
    decryption_trust_requirement: TrustRequirement,
    request_config: Option<RequestConfig>,
+
+    /// Whether to enable use of the event cache store, for reloading events
+    /// when building timelines et al.
+    use_event_cache_persistent_storage: bool,
 }

 #[matrix_sdk_ffi_macros::export]
@@ -299,9 +306,27 @@ impl ClientBuilder {
            room_key_recipient_strategy: Default::default(),
            decryption_trust_requirement: TrustRequirement::Untrusted,
            request_config: Default::default(),
+            use_event_cache_persistent_storage: false,
        })
    }

+    /// Whether to use the event cache persistent storage or not.
+    ///
+    /// This is a temporary feature flag, for testing the event cache's
+    /// persistent storage. Follow new developments in https://github.com/matrix-org/matrix-rust-sdk/issues/3280.
+    ///
+    /// This is disabled by default. When disabled, a one-time cleanup is
+    /// performed when creating the client, and it will clear all the events
+    /// previously stored in the event cache.
+    ///
+    /// When enabled, it will attempt to store events in the event cache as
+    /// they're received, and reuse them when reconstructing timelines.
+    pub fn use_event_cache_persistent_storage(self: Arc<Self>, value: bool) -> Arc<Self> {
+        let mut builder = unwrap_or_clone_arc(self);
+        builder.use_event_cache_persistent_storage = value;
+        Arc::new(builder)
+    }
+
    pub fn cross_process_store_locks_holder_name(
        self: Arc<Self>,
        holder_name: String,
@@ -624,6 +649,19 @@ impl ClientBuilder {

        let sdk_client = inner_builder.build().await?;

+        if builder.use_event_cache_persistent_storage {
+            // Enable the persistent storage \o/
+            sdk_client.event_cache().enable_storage()?;
+        } else {
+            // Get rid of all the previous events, if any.
+            let store = sdk_client
+                .event_cache_store()
+                .lock()
+                .await
+                .map_err(EventCacheError::LockingStorage)?;
+            store.clear_all_rooms_chunks().await.map_err(EventCacheError::Storage)?;
+        }
+
        Ok(Arc::new(
            Client::new(sdk_client, builder.enable_oidc_refresh_lock, builder.session_delegate)
                .await?,
@@ -254,7 +254,7 @@ impl Encryption {
    /// Therefore it is necessary to poll the server for an answer every time
    /// you want to differentiate between those two states.
    pub async fn backup_exists_on_server(&self) -> Result<bool, ClientError> {
-        Ok(self.inner.backups().exists_on_server().await?)
+        Ok(self.inner.backups().fetch_exists_on_server().await?)
    }

    pub fn recovery_state(&self) -> RecoveryState {
@@ -3,7 +3,10 @@ use matrix_sdk::IdParseError;
 use matrix_sdk_ui::timeline::TimelineEventItemId;
 use ruma::{
    events::{
-        room::{message::Relation, redaction::SyncRoomRedactionEvent},
+        room::{
+            message::{MessageType as RumaMessageType, Relation},
+            redaction::SyncRoomRedactionEvent,
+        },
        AnySyncMessageLikeEvent, AnySyncStateEvent, AnySyncTimelineEvent, AnyTimelineEvent,
        MessageLikeEventContent as RumaMessageLikeEventContent, RedactContent,
        RedactedStateEventContent, StaticStateEventContent, SyncMessageLikeEvent, SyncStateEvent,
@@ -356,6 +359,39 @@ impl From<MessageLikeEventType> for ruma::events::MessageLikeEventType {
    }
 }

+#[derive(Debug, PartialEq, Clone, uniffi::Enum)]
+pub enum RoomMessageEventMessageType {
+    Audio,
+    Emote,
+    File,
+    Image,
+    Location,
+    Notice,
+    ServerNotice,
+    Text,
+    Video,
+    VerificationRequest,
+    Other,
+}
+
+impl From<RumaMessageType> for RoomMessageEventMessageType {
+    fn from(val: ruma::events::room::message::MessageType) -> Self {
+        match val {
+            RumaMessageType::Audio { .. } => Self::Audio,
+            RumaMessageType::Emote { .. } => Self::Emote,
+            RumaMessageType::File { .. } => Self::File,
+            RumaMessageType::Image { .. } => Self::Image,
+            RumaMessageType::Location { .. } => Self::Location,
+            RumaMessageType::Notice { .. } => Self::Notice,
+            RumaMessageType::ServerNotice { .. } => Self::ServerNotice,
+            RumaMessageType::Text { .. } => Self::Text,
+            RumaMessageType::Video { .. } => Self::Video,
+            RumaMessageType::VerificationRequest { .. } => Self::VerificationRequest,
+            _ => Self::Other,
+        }
+    }
+}
+
 /// Contains the 2 possible identifiers of an event, either it has a remote
 /// event id or a local transaction id, never both or none.
 #[derive(Clone, uniffi::Enum)]
@@ -1,7 +1,7 @@
 use std::{collections::HashMap, pin::pin, sync::Arc};

 use anyhow::{Context, Result};
-use futures_util::StreamExt;
+use futures_util::{pin_mut, StreamExt};
 use matrix_sdk::{
    crypto::LocalTrust,
    event_cache::paginator::PaginatorError,
@@ -11,7 +11,7 @@ use matrix_sdk::{
    ComposerDraft as SdkComposerDraft, ComposerDraftType as SdkComposerDraftType,
    RoomHero as SdkRoomHero, RoomMemberships, RoomState,
 };
-use matrix_sdk_ui::timeline::{PaginationError, RoomExt, TimelineFocus};
+use matrix_sdk_ui::timeline::{default_event_filter, PaginationError, RoomExt, TimelineFocus};
 use mime::Mime;
 use ruma::{
    api::client::room::report_content,
@@ -23,7 +23,7 @@ use ruma::{
            message::RoomMessageEventContentWithoutRelation,
            power_levels::RoomPowerLevels as RumaPowerLevels, MediaSource,
        },
-        TimelineEventType,
+        AnyMessageLikeEventContent, AnySyncTimelineEvent, TimelineEventType,
    },
    EventId, Int, OwnedDeviceId, OwnedUserId, RoomAliasId, UserId,
 };
@@ -34,12 +34,12 @@ use super::RUNTIME;
 use crate::{
    chunk_iterator::ChunkIterator,
    error::{ClientError, MediaInfoError, RoomError},
-    event::{MessageLikeEventType, StateEventType},
+    event::{MessageLikeEventType, RoomMessageEventMessageType, StateEventType},
    identity_status_change::IdentityStatusChange,
    room_info::RoomInfo,
    room_member::RoomMember,
    ruma::{ImageInfo, Mentions, NotifyType},
-    timeline::{FocusEventError, ReceiptType, SendHandle, Timeline},
+    timeline::{DateDividerMode, FocusEventError, ReceiptType, SendHandle, Timeline},
    utils::u64_to_uint,
    TaskHandle,
 };
@@ -50,6 +50,7 @@ pub enum Membership {
    Joined,
    Left,
    Knocked,
+    Banned,
 }

 impl From<RoomState> for Membership {
@@ -59,6 +60,7 @@ impl From<RoomState> for Membership {
            RoomState::Joined => Membership::Joined,
            RoomState::Left => Membership::Left,
            RoomState::Knocked => Membership::Knocked,
+            RoomState::Banned => Membership::Banned,
        }
    }
 }
@@ -260,6 +262,51 @@ impl Room {
        Ok(Timeline::new(timeline))
    }

+    /// A timeline instance that can be configured to only include RoomMessage
+    /// type events and filter those further based on their message type.
+    ///
+    /// Virtual timeline items will still be provided and the
+    /// `default_event_filter` will be applied before everything else.
+    ///
+    /// # Arguments
+    ///
+    /// * `internal_id_prefix` - An optional String that will be prepended to
+    ///   all the timeline item's internal IDs, making it possible to
+    ///   distinguish different timeline instances from each other.
+    ///
+    /// * `allowed_message_types` - A list of `RoomMessageEventMessageType` that
+    ///   will be allowed to appear in the timeline
+    pub async fn message_filtered_timeline(
+        &self,
+        internal_id_prefix: Option<String>,
+        allowed_message_types: Vec<RoomMessageEventMessageType>,
+        date_divider_mode: DateDividerMode,
+    ) -> Result<Arc<Timeline>, ClientError> {
+        let mut builder = matrix_sdk_ui::timeline::Timeline::builder(&self.inner);
+
+        if let Some(internal_id_prefix) = internal_id_prefix {
+            builder = builder.with_internal_id_prefix(internal_id_prefix);
+        }
+
+        builder = builder.with_date_divider_mode(date_divider_mode.into());
+
+        builder = builder.event_filter(move |event, room_version_id| {
+            default_event_filter(event, room_version_id)
+                && match event {
+                    AnySyncTimelineEvent::MessageLike(msg) => match msg.original_content() {
+                        Some(AnyMessageLikeEventContent::RoomMessage(content)) => {
+                            allowed_message_types.contains(&content.msgtype.into())
+                        }
+                        _ => false,
+                    },
+                    _ => false,
+                }
+        });
+
+        let timeline = builder.build().await?;
+        Ok(Timeline::new(timeline))
+    }
+
    pub fn is_encrypted(&self) -> Result<bool, ClientError> {
        Ok(RUNTIME.block_on(self.inner.is_encrypted())?)
    }
@@ -336,6 +383,22 @@ impl Room {
        Ok(())
    }

+    /// Send a raw event to the room.
+    ///
+    /// # Arguments
+    ///
+    /// * `event_type` - The type of the event to send.
+    ///
+    /// * `content` - The content of the event to send encoded as JSON string.
+    pub async fn send_raw(&self, event_type: String, content: String) -> Result<(), ClientError> {
+        let content_json: serde_json::Value = serde_json::from_str(&content)
+            .map_err(|e| ClientError::Generic { msg: format!("Failed to parse JSON: {e}") })?;
+
+        self.inner.send_raw(&event_type, content_json).await?;
+
+        Ok(())
+    }
+
    /// Redacts an event from the room.
    ///
    /// # Arguments
@@ -840,6 +903,125 @@ impl Room {

        Ok(())
    }
+
+    /// Clear the event cache storage for the current room.
+    ///
+    /// This will remove all the information related to the event cache, in
+    /// memory and in the persisted storage, if enabled.
+    pub async fn clear_event_cache_storage(&self) -> Result<(), ClientError> {
+        let (room_event_cache, _drop_handles) = self.inner.event_cache().await?;
+        room_event_cache.clear().await?;
+        Ok(())
+    }
+
+    /// Subscribes to requests to join this room (knock member events), using a
+    /// `listener` to be notified of the changes.
+    ///
+    /// The current requests to join the room will be emitted immediately
+    /// when subscribing, along with a [`TaskHandle`] to cancel the
+    /// subscription.
+    pub async fn subscribe_to_knock_requests(
+        self: Arc<Self>,
+        listener: Box<dyn KnockRequestsListener>,
+    ) -> Result<Arc<TaskHandle>, ClientError> {
+        let stream = self.inner.subscribe_to_knock_requests().await?;
+
+        let handle = Arc::new(TaskHandle::new(RUNTIME.spawn(async move {
+            pin_mut!(stream);
+            while let Some(requests) = stream.next().await {
+                listener.call(requests.into_iter().map(Into::into).collect());
+            }
+        })));
+
+        Ok(handle)
+    }
+
+    /// Return a debug representation for the internal room events data
+    /// structure, one line per entry in the resulting vector.
+    pub async fn room_events_debug_string(&self) -> Result<Vec<String>, ClientError> {
+        let (cache, _drop_guards) = self.inner.event_cache().await?;
+        Ok(cache.debug_string().await)
+    }
+}
+
+impl From<matrix_sdk::room::knock_requests::KnockRequest> for KnockRequest {
+    fn from(request: matrix_sdk::room::knock_requests::KnockRequest) -> Self {
+        Self {
+            event_id: request.event_id.to_string(),
+            user_id: request.member_info.user_id.to_string(),
+            room_id: request.room_id().to_string(),
+            display_name: request.member_info.display_name.clone(),
+            avatar_url: request.member_info.avatar_url.as_ref().map(|url| url.to_string()),
+            reason: request.member_info.reason.clone(),
+            timestamp: request.timestamp.map(|ts| ts.into()),
+            is_seen: request.is_seen,
+            actions: Arc::new(KnockRequestActions { inner: request }),
+        }
+    }
+}
+
+/// A listener for receiving new requests to a join a room.
+#[matrix_sdk_ffi_macros::export(callback_interface)]
+pub trait KnockRequestsListener: Send + Sync {
+    fn call(&self, join_requests: Vec<KnockRequest>);
+}
+
+/// An FFI representation of a request to join a room.
+#[derive(Debug, Clone, uniffi::Record)]
+pub struct KnockRequest {
+    /// The event id of the event that contains the `knock` membership change.
+    pub event_id: String,
+    /// The user id of the user who's requesting to join the room.
+    pub user_id: String,
+    /// The room id of the room whose access was requested.
+    pub room_id: String,
+    /// The optional display name of the user who's requesting to join the room.
+    pub display_name: Option<String>,
+    /// The optional avatar url of the user who's requesting to join the room.
+    pub avatar_url: Option<String>,
+    /// An optional reason why the user wants join the room.
+    pub reason: Option<String>,
+    /// The timestamp when this request was created.
+    pub timestamp: Option<u64>,
+    /// Whether the knock request has been marked as `seen` so it can be
+    /// filtered by the client.
+    pub is_seen: bool,
+    /// A set of actions to perform for this knock request.
+    pub actions: Arc<KnockRequestActions>,
+}
+
+/// A set of actions to perform for a knock request.
+#[derive(Debug, Clone, uniffi::Object)]
+pub struct KnockRequestActions {
+    inner: matrix_sdk::room::knock_requests::KnockRequest,
+}
+
+#[matrix_sdk_ffi_macros::export]
+impl KnockRequestActions {
+    /// Accepts the knock request by inviting the user to the room.
+    pub async fn accept(&self) -> Result<(), ClientError> {
+        self.inner.accept().await.map_err(Into::into)
+    }
+
+    /// Declines the knock request by kicking the user from the room with an
+    /// optional reason.
+    pub async fn decline(&self, reason: Option<String>) -> Result<(), ClientError> {
+        self.inner.decline(reason.as_deref()).await.map_err(Into::into)
+    }
+
+    /// Declines the knock request by banning the user from the room with an
+    /// optional reason.
+    pub async fn decline_and_ban(&self, reason: Option<String>) -> Result<(), ClientError> {
+        self.inner.decline_and_ban(reason.as_deref()).await.map_err(Into::into)
+    }
+
+    /// Marks the knock request as 'seen'.
+    ///
+    /// **IMPORTANT**: this won't update the current reference to this request,
+    /// a new one with the updated value should be emitted instead.
+    pub async fn mark_as_seen(&self) -> Result<(), ClientError> {
+        self.inner.mark_as_seen().await.map_err(Into::into)
+    }
 }

 /// Generates a `matrix.to` permalink to the given room alias.
@@ -616,7 +616,8 @@ impl RoomListItem {

        // Do the thing.
        let client = self.inner.client();
-        let (room_or_alias_id, server_names) = if let Some(alias) = self.inner.canonical_alias() {
+        let (room_or_alias_id, mut server_names) = if let Some(alias) = self.inner.canonical_alias()
+        {
            let room_or_alias_id: OwnedRoomOrAliasId = alias.into();
            (room_or_alias_id, Vec::new())
        } else {
@@ -624,6 +625,16 @@ impl RoomListItem {
            (room_or_alias_id, server_names)
        };

+        // If no server names are provided and the room's membership is invited,
+        // add the server name from the sender's user id as a fallback value
+        if server_names.is_empty() {
+            if let Ok(invite_details) = self.inner.invite_details().await {
+                if let Some(inviter) = invite_details.inviter {
+                    server_names.push(inviter.user_id().server_name().to_owned());
+                }
+            }
+        }
+
        let room_preview = client.get_room_preview(&room_or_alias_id, server_names).await?;

        Ok(Arc::new(RoomPreview::new(AsyncRuntimeDropped::new(client), room_preview)))
@@ -201,6 +201,22 @@ pub struct UnableToDecryptInfo {
    /// What we know about what caused this UTD. E.g. was this event sent when
    /// we were not a member of this room?
    pub cause: UtdCause,
+
+    /// The difference between the event creation time (`origin_server_ts`) and
+    /// the time our device was created. If negative, this event was sent
+    /// *before* our device was created.
+    pub event_local_age_millis: i64,
+
+    /// Whether the user had verified their own identity at the point they
+    /// received the UTD event.
+    pub user_trusts_own_identity: bool,
+
+    /// The homeserver of the user that sent the undecryptable event.
+    pub sender_homeserver: String,
+
+    /// Our local user's own homeserver, or `None` if the client is not logged
+    /// in.
+    pub own_homeserver: Option<String>,
 }

 impl From<SdkUnableToDecryptInfo> for UnableToDecryptInfo {
@@ -209,6 +225,10 @@ impl From<SdkUnableToDecryptInfo> for UnableToDecryptInfo {
            event_id: value.event_id.to_string(),
            time_to_decrypt_ms: value.time_to_decrypt.map(|ttd| ttd.as_millis() as u64),
            cause: value.cause,
+            event_local_age_millis: value.event_local_age_millis,
+            user_trusts_own_identity: value.user_trusts_own_identity,
+            sender_homeserver: value.sender_homeserver.to_string(),
+            own_homeserver: value.own_homeserver.map(String::from),
        }
    }
 }
@@ -986,7 +986,7 @@ impl TimelineItem {
    pub fn as_virtual(self: Arc<Self>) -> Option<VirtualTimelineItem> {
        use matrix_sdk_ui::timeline::VirtualTimelineItem as VItem;
        match self.0.as_virtual()? {
-            VItem::DayDivider(ts) => Some(VirtualTimelineItem::DayDivider { ts: ts.0.into() }),
+            VItem::DateDivider(ts) => Some(VirtualTimelineItem::DateDivider { ts: ts.0.into() }),
            VItem::ReadMarker => Some(VirtualTimelineItem::ReadMarker),
        }
    }
@@ -1255,8 +1255,9 @@ impl SendAttachmentJoinHandle {
 /// A [`TimelineItem`](super::TimelineItem) that doesn't correspond to an event.
 #[derive(uniffi::Enum)]
 pub enum VirtualTimelineItem {
-    /// A divider between messages of two days.
-    DayDivider {
+    /// A divider between messages of different day or month depending on
+    /// timeline settings.
+    DateDivider {
        /// A timestamp in milliseconds since Unix Epoch on that day in local
        /// time.
        ts: u64,
@@ -1358,3 +1359,20 @@ impl LazyTimelineItemProvider {
        self.0.local_echo_send_handle().map(|handle| Arc::new(SendHandle::new(handle)))
    }
 }
+
+/// Changes how date dividers get inserted, either in between each day or in
+/// between each month
+#[derive(Debug, Clone, uniffi::Enum)]
+pub enum DateDividerMode {
+    Daily,
+    Monthly,
+}
+
+impl From<DateDividerMode> for matrix_sdk_ui::timeline::DateDividerMode {
+    fn from(value: DateDividerMode) -> Self {
+        match value {
+            DateDividerMode::Daily => Self::Daily,
+            DateDividerMode::Monthly => Self::Monthly,
+        }
+    }
+}
@@ -1,47 +0,0 @@
-# This git-cliff configuration file is used to generate weekly reports for This
-# Week in Matrix amongst others.
-
-[changelog]
-header = """
-# This Week in the Matrix Rust SDK ({{ now() | date(format="%Y-%m-%d") }})
-"""
-body = """
-{% for commit in commits %}
-    {% set_global commit_message = commit.message -%}
-    {% for footer in commit.footers -%}
-        {% if footer.token | lower  == "changelog" -%}
-            {% set_global commit_message = footer.value -%}
-        {% elif footer.token | lower  == "breaking-change" -%}
-            {% set_global commit_message = footer.value -%}
-        {% endif -%}
-    {% endfor -%}
-    - {{ commit_message | upper_first }}
-{% endfor %}
-"""
-trim = true
-footer = ""
-
-[git]
-conventional_commits = true
-filter_unconventional = true
-commit_preprocessors = [
-    { pattern = '\((\w+\s)?#([0-9]+)\)', replace = "([#${2}](https://github.com/matrix-org/matrix-rust-sdk/pull/${2}))"},
-]
-commit_parsers = [
-    { message = "^feat", group = "Features" },
-    { message = "^fix", group = "Bug Fixes" },
-    { message = "^doc", group = "Documentation" },
-    { message = "^perf", group = "Performance" },
-    { message = "^refactor", group = "Refactor", skip = true },
-    { message = "^chore\\(release\\): prepare for", skip = true },
-    { message = "^chore", skip = true },
-    { message = "^style", group = "Styling", skip = true },
-    { message = "^test", skip = true },
-    { message = "^ci", skip = true },
-]
-filter_commits = true
-tag_pattern = "[0-9]*"
-skip_tags = ""
-ignore_tags = ""
-date_order = false
-sort_commits = "newest"
@@ -1,91 +0,0 @@
-# This git-cliff configuration file is used to generate release reports.
-
-[changelog]
-# changelog header
-header = """
-# Changelog\n
-All notable changes to this project will be documented in this file.\n
-"""
-# template for the changelog body
-# https://keats.github.io/tera/docs/
-body = """
-{% if version %}\
-    ## [{{ version | trim_start_matches(pat="v") }}] - {{ timestamp | date(format="%Y-%m-%d") }}
-{% else %}\
-    ## [unreleased]
-{% endif %}\
-{% for group, commits in commits | group_by(attribute="group") %}
-    ### {{ group | upper_first }}
-    {% for commit in commits %}
-        {% set_global commit_message = commit.message -%}
-        {% set_global breaking = commit.breaking -%}
-        {% for footer in commit.footers -%}
-            {% if footer.token | lower  == "changelog" -%}
-                {% set_global commit_message = footer.value -%}
-            {% elif footer.token | lower  == "breaking-change" -%}
-                {% set_global commit_message = footer.value -%}
-            {% elif footer.token | lower == "security-impact" -%}
-                {% set_global security_impact = footer.value -%}
-            {% elif footer.token | lower == "cve" -%}
-                {% set_global cve = footer.value -%}
-            {% elif footer.token | lower == "github-advisory" -%}
-                {% set_global github_advisory = footer.value -%}
-            {% endif -%}
-        {% endfor -%}
-        - {% if breaking %}[**breaking**] {% endif %}{{ commit_message | upper_first }}
-          {% if security_impact -%}
-              (\
-              *{{ security_impact | upper_first }}*\
-              {% if cve -%}, [{{ cve | upper }}](https://www.cve.org/CVERecord?id={{ cve }}){% endif -%}\
-              {% if github_advisory -%}, [{{ github_advisory | upper }}](https://github.com/matrix-org/matrix-rust-sdk/security/advisories/{{ github_advisory }}){% endif -%}
-              )
-          {% endif -%}
-    {% endfor %}
-{% endfor %}\n
-"""
-# remove the leading and trailing whitespace from the template
-trim = true
-# changelog footer
-footer = """
-<!-- generated by git-cliff -->
-"""
-
-[git]
-# parse the commits based on https://www.conventionalcommits.org
-conventional_commits = true
-# filter out the commits that are not conventional
-filter_unconventional = true
-# regex for preprocessing the commit messages
-commit_preprocessors = [
-    { pattern = '\((\w+\s)?#([0-9]+)\)', replace = "([#${2}](https://github.com/matrix-org/matrix-rust-sdk/pull/${2}))"},
-]
-# regex for parsing and grouping commits
-commit_parsers = [
-    { footer = "Security-Impact:", group = "Security" },
-    { footer = "CVE:", group = "Security" },
-    { footer = "GitHub-Advisory:", group = "Security" },
-    { message = "^feat", group = "Features" },
-    { message = "^fix", group = "Bug Fixes" },
-    { message = "^doc", group = "Documentation" },
-    { message = "^perf", group = "Performance" },
-    { message = "^refactor", group = "Refactor" },
-    { message = "^chore\\(release\\): prepare for", skip = true },
-    { message = "^chore", skip = true },
-    { message = "^style", group = "Styling", skip = true },
-    { message = "^test", skip = true },
-    { message = "^ci", skip = true },
-]
-# forbid parsers from skipping breaking changes
-protect_breaking_commits = true
-# filter out the commits that are not matched by commit parsers
-filter_commits = true
-# glob pattern for matching git tags
-tag_pattern = "[0-9]*"
-# regex for skipping tags
-skip_tags = ""
-# regex for ignoring tags
-ignore_tags = ""
-# sort the tags chronologically
-date_order = false
-# sort the commits inside sections by oldest/newest order
-sort_commits = "oldest"
@@ -2,6 +2,27 @@

 All notable changes to this project will be documented in this file.

+<!-- next-header -->
+
+## [Unreleased] - ReleaseDate
+
+## [0.9.0] - 2024-12-18
+
+### Features
+
+- Introduced support for
+  [MSC4171](https://github.com/matrix-org/matrix-rust-sdk/pull/4335), enabling
+  the designation of certain users as service members. These flagged users are
+  excluded from the room display name calculation.
+  ([#4335](https://github.com/matrix-org/matrix-rust-sdk/pull/4335))
+
+### Bug Fixes
+
+- Fix an off-by-one error in the `ObservableMap` when the `remove()` method is
+  called. Previously, items following the removed item were not shifted left by
+  one position, leaving them at incorrect indices.
+  ([#4346](https://github.com/matrix-org/matrix-rust-sdk/pull/4346))
+
 ## [0.8.0] - 2024-11-19

 ### Bug Fixes
@@ -9,7 +9,7 @@ name = "matrix-sdk-base"
 readme = "README.md"
 repository = "https://github.com/matrix-org/matrix-rust-sdk"
 rust-version = { workspace = true }
-version = "0.8.0"
+version = "0.9.0"

 [package.metadata.docs.rs]
 all-features = true
@@ -49,9 +49,9 @@ as_variant = { workspace = true }
 assert_matches = { workspace = true, optional = true }
 assert_matches2 = { workspace = true, optional = true }
 async-trait = { workspace = true }
-bitflags = { version = "2.4.0", features = ["serde"] }
-decancer = "3.2.4"
-eyeball = { workspace = true }
+bitflags = { version = "2.6.0", features = ["serde"] }
+decancer = "3.2.8"
+eyeball = { workspace = true, features = ["async-lock"] }
 eyeball-im = { workspace = true }
 futures-util = { workspace = true }
 growable-bloom-filter = { workspace = true }
@@ -61,9 +61,9 @@ matrix-sdk-crypto = { workspace = true, optional = true }
 matrix-sdk-store-encryption = { workspace = true }
 matrix-sdk-test = { workspace = true, optional = true }
 once_cell = { workspace = true }
-regex = "1.11.0"
+regex = "1.11.1"
 ruma = { workspace = true, features = ["canonical-json", "unstable-msc3381", "unstable-msc2867", "rand"] }
-unicode-normalization = "0.1.24"
+unicode-normalization = { workspace = true }
 serde = { workspace = true, features = ["rc"] }
 serde_json = { workspace = true }
 tokio = { workspace = true }
@@ -85,7 +85,7 @@ similar-asserts = { workspace = true }
 tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }

 [target.'cfg(target_arch = "wasm32")'.dev-dependencies]
-wasm-bindgen-test = "0.3.33"
+wasm-bindgen-test = { workspace = true }

 [lints]
 workspace = true
@@ -26,8 +26,8 @@ use eyeball_im::{Vector, VectorDiff};
 use futures_util::Stream;
 #[cfg(feature = "e2e-encryption")]
 use matrix_sdk_crypto::{
-    store::DynCryptoStore, CollectStrategy, DecryptionSettings, EncryptionSettings,
-    EncryptionSyncChanges, OlmError, OlmMachine, RoomEventDecryptionResult, ToDeviceRequest,
+    store::DynCryptoStore, types::requests::ToDeviceRequest, CollectStrategy, DecryptionSettings,
+    EncryptionSettings, EncryptionSyncChanges, OlmError, OlmMachine, RoomEventDecryptionResult,
    TrustRequirement,
 };
 #[cfg(feature = "e2e-encryption")]
@@ -30,7 +30,7 @@ use ruma::{
        StateEventContent, StaticStateEventContent, StrippedStateEvent, SyncStateEvent,
    },
    serde::Raw,
-    EventId, MilliSecondsSinceUnixEpoch, OwnedEventId, OwnedRoomId, OwnedUserId, UserId,
+    EventId, MilliSecondsSinceUnixEpoch, OwnedEventId, OwnedRoomId, OwnedUserId, UInt, UserId,
 };
 use serde::Serialize;
 use unicode_normalization::UnicodeNormalization;
@@ -160,12 +160,12 @@ impl PartialEq for DisplayName {

 impl DisplayName {
    /// Regex pattern matching an MXID.
-    const MXID_PATTERN: &str = "@.+[:.].+";
+    const MXID_PATTERN: &'static str = "@.+[:.].+";

    /// Regex pattern matching some left-to-right formatting marks:
    ///     * LTR and RTL marks U+200E and U+200F
    ///     * LTR/RTL and other directional formatting marks U+202A - U+202F
-    const LEFT_TO_RIGHT_PATTERN: &str = "[\u{202a}-\u{202f}\u{200e}\u{200f}]";
+    const LEFT_TO_RIGHT_PATTERN: &'static str = "[\u{202a}-\u{202f}\u{200e}\u{200f}]";

    /// Regex pattern matching bunch of unicode control characters and otherwise
    /// misleading/invisible characters.
@@ -176,7 +176,7 @@ impl DisplayName {
    ///     * Blank/invisible characters (U2800, U2062-U2063)
    ///     * Arabic Letter RTL mark U+061C
    ///     * Zero width no-break space (BOM) U+FEFF
-    const HIDDEN_CHARACTERS_PATTERN: &str =
+    const HIDDEN_CHARACTERS_PATTERN: &'static str =
        "[\u{2000}-\u{200D}\u{300}-\u{036f}\u{2062}-\u{2063}\u{2800}\u{061c}\u{feff}]";

    /// Creates a new [`DisplayName`] from the given raw string.
@@ -476,6 +476,23 @@ impl MemberEvent {
                .unwrap_or_else(|| self.user_id().localpart()),
        )
    }
+
+    /// The optional reason why the membership changed.
+    pub fn reason(&self) -> Option<&str> {
+        match self {
+            MemberEvent::Sync(SyncStateEvent::Original(c)) => c.content.reason.as_deref(),
+            MemberEvent::Stripped(e) => e.content.reason.as_deref(),
+            _ => None,
+        }
+    }
+
+    /// The optional timestamp for this member event.
+    pub fn timestamp(&self) -> Option<UInt> {
+        match self {
+            MemberEvent::Sync(SyncStateEvent::Original(c)) => Some(c.origin_server_ts.0),
+            _ => None,
+        }
+    }
 }

 impl SyncOrStrippedState<RoomPowerLevelsEventContent> {
@@ -14,13 +14,85 @@

 //! Trait and macro of integration tests for `EventCacheStore` implementations.

+use assert_matches::assert_matches;
 use async_trait::async_trait;
+use matrix_sdk_common::{
+    deserialized_responses::{
+        AlgorithmInfo, DecryptedRoomEvent, EncryptionInfo, SyncTimelineEvent, TimelineEventKind,
+        VerificationState,
+    },
+    linked_chunk::{ChunkContent, LinkedChunk, LinkedChunkBuilder, Position, RawChunk, Update},
+};
+use matrix_sdk_test::{event_factory::EventFactory, ALICE, DEFAULT_TEST_ROOM_ID};
 use ruma::{
-    api::client::media::get_content_thumbnail::v3::Method, events::room::MediaSource, mxc_uri, uint,
+    api::client::media::get_content_thumbnail::v3::Method, events::room::MediaSource, mxc_uri,
+    push::Action, room_id, uint, RoomId,
 };

 use super::DynEventCacheStore;
-use crate::media::{MediaFormat, MediaRequestParameters, MediaThumbnailSettings};
+use crate::{
+    event_cache::{Event, Gap},
+    media::{MediaFormat, MediaRequestParameters, MediaThumbnailSettings},
+};
+
+/// Create a test event with all data filled, for testing that linked chunk
+/// correctly stores event data.
+///
+/// Keep in sync with [`check_test_event`].
+pub fn make_test_event(room_id: &RoomId, content: &str) -> SyncTimelineEvent {
+    let encryption_info = EncryptionInfo {
+        sender: (*ALICE).into(),
+        sender_device: None,
+        algorithm_info: AlgorithmInfo::MegolmV1AesSha2 {
+            curve25519_key: "1337".to_owned(),
+            sender_claimed_keys: Default::default(),
+        },
+        verification_state: VerificationState::Verified,
+    };
+
+    let event = EventFactory::new()
+        .text_msg(content)
+        .room(room_id)
+        .sender(*ALICE)
+        .into_raw_timeline()
+        .cast();
+
+    SyncTimelineEvent {
+        kind: TimelineEventKind::Decrypted(DecryptedRoomEvent {
+            event,
+            encryption_info,
+            unsigned_encryption_info: None,
+        }),
+        push_actions: vec![Action::Notify],
+    }
+}
+
+/// Check that an event created with [`make_test_event`] contains the expected
+/// data.
+///
+/// Keep in sync with [`make_test_event`].
+#[track_caller]
+pub fn check_test_event(event: &SyncTimelineEvent, text: &str) {
+    // Check push actions.
+    let actions = &event.push_actions;
+    assert_eq!(actions.len(), 1);
+    assert_matches!(&actions[0], Action::Notify);
+
+    // Check content.
+    assert_matches!(&event.kind, TimelineEventKind::Decrypted(d) => {
+        // Check encryption fields.
+        assert_eq!(d.encryption_info.sender, *ALICE);
+        assert_matches!(&d.encryption_info.algorithm_info, AlgorithmInfo::MegolmV1AesSha2 { curve25519_key, .. } => {
+            assert_eq!(curve25519_key, "1337");
+        });
+
+        // Check event.
+        let deserialized = d.event.deserialize().unwrap();
+        assert_matches!(deserialized, ruma::events::AnyMessageLikeEvent::RoomMessage(msg) => {
+            assert_eq!(msg.as_original().unwrap().content.body(), text);
+        });
+    });
+}

 /// `EventCacheStore` integration tests.
 ///
@@ -34,6 +106,21 @@ pub trait EventCacheStoreIntegrationTests {

    /// Test replacing a MXID.
    async fn test_replace_media_key(&self);
+
+    /// Test handling updates to a linked chunk and reloading these updates from
+    /// the store.
+    async fn test_handle_updates_and_rebuild_linked_chunk(&self);
+
+    /// Test that rebuilding a linked chunk from an empty store doesn't return
+    /// anything.
+    async fn test_rebuild_empty_linked_chunk(&self);
+
+    /// Test that clear all the rooms' linked chunks works.
+    async fn test_clear_all_rooms_chunks(&self);
+}
+
+fn rebuild_linked_chunk(raws: Vec<RawChunk<Event, Gap>>) -> Option<LinkedChunk<3, Event, Gap>> {
+    LinkedChunkBuilder::from_raw_parts(raws).build().unwrap()
 }

 #[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
@@ -83,6 +170,11 @@ impl EventCacheStoreIntegrationTests for DynEventCacheStore {
            Some(&content),
            "media not found though added"
        );
+        assert_eq!(
+            self.get_media_content_for_uri(uri).await.unwrap().as_ref(),
+            Some(&content),
+            "media not found by URI though added"
+        );

        // Let's remove the media.
        self.remove_media_content(&request_file).await.expect("removing media failed");
@@ -92,6 +184,10 @@ impl EventCacheStoreIntegrationTests for DynEventCacheStore {
            self.get_media_content(&request_file).await.unwrap().is_none(),
            "media still there after removing"
        );
+        assert!(
+            self.get_media_content_for_uri(uri).await.unwrap().is_none(),
+            "media still found by URI after removing"
+        );

        // Let's add the media again.
        self.add_media_content(&request_file, content.clone())
@@ -116,6 +212,12 @@ impl EventCacheStoreIntegrationTests for DynEventCacheStore {
            "thumbnail not found"
        );

+        // We get a file with the URI, we don't know which one.
+        assert!(
+            self.get_media_content_for_uri(uri).await.unwrap().is_some(),
+            "media not found by URI though two where added"
+        );
+
        // Let's add another media with a different URI.
        self.add_media_content(&request_other_file, other_content.clone())
            .await
@@ -127,6 +229,11 @@ impl EventCacheStoreIntegrationTests for DynEventCacheStore {
            Some(&other_content),
            "other file not found"
        );
+        assert_eq!(
+            self.get_media_content_for_uri(other_uri).await.unwrap().as_ref(),
+            Some(&other_content),
+            "other file not found by URI"
+        );

        // Let's remove media based on URI.
        self.remove_media_content_for_uri(uri).await.expect("removing all media for uri failed");
@@ -143,6 +250,14 @@ impl EventCacheStoreIntegrationTests for DynEventCacheStore {
            self.get_media_content(&request_other_file).await.unwrap().is_some(),
            "other media was removed"
        );
+        assert!(
+            self.get_media_content_for_uri(uri).await.unwrap().is_none(),
+            "media found by URI wasn't removed"
+        );
+        assert!(
+            self.get_media_content_for_uri(other_uri).await.unwrap().is_some(),
+            "other media found by URI was removed"
+        );
    }

    async fn test_replace_media_key(&self) {
@@ -182,6 +297,149 @@ impl EventCacheStoreIntegrationTests for DynEventCacheStore {
        // Finding with the new request does work.
        assert_eq!(self.get_media_content(&new_req).await.unwrap().unwrap(), b"hello");
    }
+
+    async fn test_handle_updates_and_rebuild_linked_chunk(&self) {
+        use matrix_sdk_common::linked_chunk::ChunkIdentifier as CId;
+
+        let room_id = room_id!("!r0:matrix.org");
+
+        self.handle_linked_chunk_updates(
+            room_id,
+            vec![
+                // new chunk
+                Update::NewItemsChunk { previous: None, new: CId::new(0), next: None },
+                // new items on 0
+                Update::PushItems {
+                    at: Position::new(CId::new(0), 0),
+                    items: vec![
+                        make_test_event(room_id, "hello"),
+                        make_test_event(room_id, "world"),
+                    ],
+                },
+                // a gap chunk
+                Update::NewGapChunk {
+                    previous: Some(CId::new(0)),
+                    new: CId::new(1),
+                    next: None,
+                    gap: Gap { prev_token: "parmesan".to_owned() },
+                },
+                // another items chunk
+                Update::NewItemsChunk { previous: Some(CId::new(1)), new: CId::new(2), next: None },
+                // new items on 0
+                Update::PushItems {
+                    at: Position::new(CId::new(2), 0),
+                    items: vec![make_test_event(room_id, "sup")],
+                },
+            ],
+        )
+        .await
+        .unwrap();
+
+        // The linked chunk is correctly reloaded.
+        let raws = self.reload_linked_chunk(room_id).await.unwrap();
+        let lc = rebuild_linked_chunk(raws).expect("linked chunk not empty");
+
+        let mut chunks = lc.chunks();
+
+        {
+            let first = chunks.next().unwrap();
+            // Note: we can't assert the previous/next chunks, as these fields and their
+            // getters are private.
+            assert_eq!(first.identifier(), CId::new(0));
+
+            assert_matches!(first.content(), ChunkContent::Items(events) => {
+                assert_eq!(events.len(), 2);
+                check_test_event(&events[0], "hello");
+                check_test_event(&events[1], "world");
+            });
+        }
+
+        {
+            let second = chunks.next().unwrap();
+            assert_eq!(second.identifier(), CId::new(1));
+
+            assert_matches!(second.content(), ChunkContent::Gap(gap) => {
+                assert_eq!(gap.prev_token, "parmesan");
+            });
+        }
+
+        {
+            let third = chunks.next().unwrap();
+            assert_eq!(third.identifier(), CId::new(2));
+
+            assert_matches!(third.content(), ChunkContent::Items(events) => {
+                assert_eq!(events.len(), 1);
+                check_test_event(&events[0], "sup");
+            });
+        }
+
+        assert!(chunks.next().is_none());
+    }
+
+    async fn test_rebuild_empty_linked_chunk(&self) {
+        // When I rebuild a linked chunk from an empty store, it's empty.
+        let raw_parts = self.reload_linked_chunk(&DEFAULT_TEST_ROOM_ID).await.unwrap();
+        assert!(rebuild_linked_chunk(raw_parts).is_none());
+    }
+
+    async fn test_clear_all_rooms_chunks(&self) {
+        use matrix_sdk_common::linked_chunk::ChunkIdentifier as CId;
+
+        let r0 = room_id!("!r0:matrix.org");
+        let r1 = room_id!("!r1:matrix.org");
+
+        // Add updates for the first room.
+        self.handle_linked_chunk_updates(
+            r0,
+            vec![
+                // new chunk
+                Update::NewItemsChunk { previous: None, new: CId::new(0), next: None },
+                // new items on 0
+                Update::PushItems {
+                    at: Position::new(CId::new(0), 0),
+                    items: vec![make_test_event(r0, "hello"), make_test_event(r0, "world")],
+                },
+            ],
+        )
+        .await
+        .unwrap();
+
+        // Add updates for the second room.
+        self.handle_linked_chunk_updates(
+            r1,
+            vec![
+                // Empty items chunk.
+                Update::NewItemsChunk { previous: None, new: CId::new(0), next: None },
+                // a gap chunk
+                Update::NewGapChunk {
+                    previous: Some(CId::new(0)),
+                    new: CId::new(1),
+                    next: None,
+                    gap: Gap { prev_token: "bleu d'auvergne".to_owned() },
+                },
+                // another items chunk
+                Update::NewItemsChunk { previous: Some(CId::new(1)), new: CId::new(2), next: None },
+                // new items on 0
+                Update::PushItems {
+                    at: Position::new(CId::new(2), 0),
+                    items: vec![make_test_event(r0, "yummy")],
+                },
+            ],
+        )
+        .await
+        .unwrap();
+
+        // Sanity check: both linked chunks can be reloaded.
+        assert!(rebuild_linked_chunk(self.reload_linked_chunk(r0).await.unwrap()).is_some());
+        assert!(rebuild_linked_chunk(self.reload_linked_chunk(r1).await.unwrap()).is_some());
+
+        // Clear the chunks.
+        self.clear_all_rooms_chunks().await.unwrap();
+
+        // Both rooms now have no linked chunk.
+        assert!(rebuild_linked_chunk(self.reload_linked_chunk(r0).await.unwrap()).is_none());
+        assert!(rebuild_linked_chunk(self.reload_linked_chunk(r1).await.unwrap()).is_none());
+    }
 }

 /// Macro building to allow your `EventCacheStore` implementation to run the
@@ -236,6 +494,27 @@ macro_rules! event_cache_store_integration_tests {
                    get_event_cache_store().await.unwrap().into_event_cache_store();
                event_cache_store.test_replace_media_key().await;
            }
+
+            #[async_test]
+            async fn test_handle_updates_and_rebuild_linked_chunk() {
+                let event_cache_store =
+                    get_event_cache_store().await.unwrap().into_event_cache_store();
+                event_cache_store.test_handle_updates_and_rebuild_linked_chunk().await;
+            }
+
+            #[async_test]
+            async fn test_rebuild_empty_linked_chunk() {
+                let event_cache_store =
+                    get_event_cache_store().await.unwrap().into_event_cache_store();
+                event_cache_store.test_rebuild_empty_linked_chunk().await;
+            }
+
+            #[async_test]
+            async fn test_clear_all_rooms_chunks() {
+                let event_cache_store =
+                    get_event_cache_store().await.unwrap().into_event_cache_store();
+                event_cache_store.test_clear_all_rooms_chunks().await;
+            }
        }
    };
 }
@@ -16,7 +16,7 @@ use std::{collections::HashMap, num::NonZeroUsize, sync::RwLock as StdRwLock, ti

 use async_trait::async_trait;
 use matrix_sdk_common::{
-    linked_chunk::{relational::RelationalLinkedChunk, Update},
+    linked_chunk::{relational::RelationalLinkedChunk, RawChunk, Update},
    ring_buffer::RingBuffer,
    store_locks::memory_store_helper::try_take_leased_lock,
 };
@@ -93,6 +93,22 @@ impl EventCacheStore for MemoryStore {
        Ok(())
    }

+    async fn reload_linked_chunk(
+        &self,
+        room_id: &RoomId,
+    ) -> Result<Vec<RawChunk<Event, Gap>>, Self::Error> {
+        let inner = self.inner.read().unwrap();
+        inner
+            .events
+            .reload_chunks(room_id)
+            .map_err(|err| EventCacheStoreError::InvalidData { details: err })
+    }
+
+    async fn clear_all_rooms_chunks(&self) -> Result<(), Self::Error> {
+        self.inner.write().unwrap().events.clear();
+        Ok(())
+    }
+
    async fn add_media_content(
        &self,
        request: &MediaRequestParameters,
@@ -154,6 +170,17 @@ impl EventCacheStore for MemoryStore {
        Ok(())
    }

+    async fn get_media_content_for_uri(
+        &self,
+        uri: &MxcUri,
+    ) -> Result<Option<Vec<u8>>, Self::Error> {
+        let inner = self.inner.read().unwrap();
+
+        Ok(inner.media.iter().find_map(|(media_uri, _media_key, media_content)| {
+            (media_uri == uri).then(|| media_content.to_owned())
+        }))
+    }
+
    async fn remove_media_content_for_uri(&self, uri: &MxcUri) -> Result<()> {
        let mut inner = self.inner.write().unwrap();

@@ -36,7 +36,7 @@ pub use matrix_sdk_store_encryption::Error as StoreEncryptionError;
 pub use self::integration_tests::EventCacheStoreIntegrationTests;
 pub use self::{
    memory_store::MemoryStore,
-    traits::{DynEventCacheStore, EventCacheStore, IntoEventCacheStore},
+    traits::{DynEventCacheStore, EventCacheStore, IntoEventCacheStore, DEFAULT_CHUNK_CAPACITY},
 };

 /// The high-level public type to represent an `EventCacheStore` lock.
@@ -138,12 +138,23 @@ pub enum EventCacheStoreError {
    #[error("Error encoding or decoding data from the event cache store: {0}")]
    Codec(#[from] Utf8Error),

+    /// The store failed to serialize or deserialize some data.
+    #[error("Error serializing or deserializing data from the event cache store: {0}")]
+    Serialization(#[from] serde_json::Error),
+
    /// The database format has changed in a backwards incompatible way.
    #[error(
        "The database format of the event cache store changed in an incompatible way, \
         current version: {0}, latest version: {1}"
    )]
    UnsupportedDatabaseVersion(usize, usize),
+
+    /// The store contains invalid data.
+    #[error("The store contains invalid data: {details}")]
+    InvalidData {
+        /// Details why the data contained in the store was invalid.
+        details: String,
+    },
 }

 impl EventCacheStoreError {
@@ -15,7 +15,10 @@
 use std::{fmt, sync::Arc};

 use async_trait::async_trait;
-use matrix_sdk_common::{linked_chunk::Update, AsyncTraitDeps};
+use matrix_sdk_common::{
+    linked_chunk::{RawChunk, Update},
+    AsyncTraitDeps,
+};
 use ruma::{MxcUri, RoomId};

 use super::EventCacheStoreError;
@@ -24,6 +27,11 @@ use crate::{
    media::MediaRequestParameters,
 };

+/// A default capacity for linked chunks, when manipulating in conjunction with
+/// an `EventCacheStore` implementation.
+// TODO: move back?
+pub const DEFAULT_CHUNK_CAPACITY: usize = 128;
+
 /// An abstract trait that can be used to implement different store backends
 /// for the event cache of the SDK.
 #[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
@@ -49,6 +57,19 @@ pub trait EventCacheStore: AsyncTraitDeps {
        updates: Vec<Update<Event, Gap>>,
    ) -> Result<(), Self::Error>;

+    /// Return all the raw components of a linked chunk, so the caller may
+    /// reconstruct the linked chunk later.
+    async fn reload_linked_chunk(
+        &self,
+        room_id: &RoomId,
+    ) -> Result<Vec<RawChunk<Event, Gap>>, Self::Error>;
+
+    /// Clear persisted events for all the rooms.
+    ///
+    /// This will empty and remove all the linked chunks stored previously,
+    /// using the above [`Self::handle_linked_chunk_updates`] methods.
+    async fn clear_all_rooms_chunks(&self) -> Result<(), Self::Error>;
+
    /// Add a media file's content in the media store.
    ///
    /// # Arguments
@@ -107,6 +128,23 @@ pub trait EventCacheStore: AsyncTraitDeps {
        request: &MediaRequestParameters,
    ) -> Result<(), Self::Error>;

+    /// Get a media file's content associated to an `MxcUri` from the
+    /// media store.
+    ///
+    /// In theory, there could be several files stored using the same URI and a
+    /// different `MediaFormat`. This API is meant to be used with a media file
+    /// that has only been stored with a single format.
+    ///
+    /// If there are several media files for a given URI in different formats,
+    /// this API will only return one of them. Which one is left as an
+    /// implementation detail.
+    ///
+    /// # Arguments
+    ///
+    /// * `uri` - The `MxcUri` of the media file.
+    async fn get_media_content_for_uri(&self, uri: &MxcUri)
+        -> Result<Option<Vec<u8>>, Self::Error>;
+
    /// Remove all the media files' content associated to an `MxcUri` from the
    /// media store.
    ///
@@ -151,6 +189,17 @@ impl<T: EventCacheStore> EventCacheStore for EraseEventCacheStoreError<T> {
        self.0.handle_linked_chunk_updates(room_id, updates).await.map_err(Into::into)
    }

+    async fn reload_linked_chunk(
+        &self,
+        room_id: &RoomId,
+    ) -> Result<Vec<RawChunk<Event, Gap>>, Self::Error> {
+        self.0.reload_linked_chunk(room_id).await.map_err(Into::into)
+    }
+
+    async fn clear_all_rooms_chunks(&self) -> Result<(), Self::Error> {
+        self.0.clear_all_rooms_chunks().await.map_err(Into::into)
+    }
+
    async fn add_media_content(
        &self,
        request: &MediaRequestParameters,
@@ -181,6 +230,13 @@ impl<T: EventCacheStore> EventCacheStore for EraseEventCacheStoreError<T> {
        self.0.remove_media_content(request).await.map_err(Into::into)
    }

+    async fn get_media_content_for_uri(
+        &self,
+        uri: &MxcUri,
+    ) -> Result<Option<Vec<u8>>, Self::Error> {
+        self.0.get_media_content_for_uri(uri).await.map_err(Into::into)
+    }
+
    async fn remove_media_content_for_uri(&self, uri: &MxcUri) -> Result<(), Self::Error> {
        self.0.remove_media_content_for_uri(uri).await.map_err(Into::into)
    }
@@ -16,7 +16,6 @@
 #![doc = include_str!("../README.md")]
 #![cfg_attr(docsrs, feature(doc_auto_cfg))]
 #![cfg_attr(target_arch = "wasm32", allow(clippy::arc_with_non_send_sync))]
-#![cfg_attr(test, allow(unexpected_cfgs))] // Triggered by the init_tracing_for_tests!() invocation.
 #![warn(missing_docs, missing_debug_implementations)]

 pub use matrix_sdk_common::*;
@@ -18,9 +18,11 @@ use std::{
 };

 use ruma::{
-    events::{AnyGlobalAccountDataEvent, GlobalAccountDataEventType},
+    events::{
+        direct::OwnedDirectUserIdentifier, AnyGlobalAccountDataEvent, GlobalAccountDataEventType,
+    },
    serde::Raw,
-    OwnedUserId, RoomId,
+    RoomId,
 };
 use tracing::{debug, instrument, trace, warn};

@@ -94,10 +96,10 @@ impl AccountDataProcessor {
        for event in events {
            let AnyGlobalAccountDataEvent::Direct(direct_event) = event else { continue };

-            let mut new_dms = HashMap::<&RoomId, HashSet<OwnedUserId>>::new();
-            for (user_id, rooms) in direct_event.content.iter() {
+            let mut new_dms = HashMap::<&RoomId, HashSet<OwnedDirectUserIdentifier>>::new();
+            for (user_identifier, rooms) in direct_event.content.iter() {
                for room_id in rooms {
-                    new_dms.entry(room_id).or_default().insert(user_id.clone());
+                    new_dms.entry(room_id).or_default().insert(user_identifier.clone());
                }
            }

@@ -1,5 +1,4 @@
 #![allow(clippy::assign_op_pattern)] // Triggered by bitflags! usage
-#![allow(unexpected_cfgs)] // Triggered by the `EventContent` macro usage

 mod members;
 pub(crate) mod normal;
@@ -22,6 +21,7 @@ use ruma::{
    events::{
        beacon_info::BeaconInfoEventContent,
        call::member::{CallMemberEventContent, CallMemberStateKey},
+        direct::OwnedDirectUserIdentifier,
        macros::EventContent,
        room::{
            avatar::RoomAvatarEventContent,
@@ -128,7 +128,7 @@ pub struct BaseRoomInfo {
    pub(crate) create: Option<MinimalStateEvent<RoomCreateWithCreatorEventContent>>,
    /// A list of user ids this room is considered as direct message, if this
    /// room is a DM.
-    pub(crate) dm_targets: HashSet<OwnedUserId>,
+    pub(crate) dm_targets: HashSet<OwnedDirectUserIdentifier>,
    /// The `m.room.encryption` event content that enabled E2EE in this room.
    pub(crate) encryption: Option<RoomEncryptionEventContent>,
    /// The guest access policy of this room.
@@ -20,8 +20,9 @@ use std::{
    sync::{atomic::AtomicBool, Arc},
 };

+use as_variant::as_variant;
 use bitflags::bitflags;
-use eyeball::{SharedObservable, Subscriber};
+use eyeball::{AsyncLock, ObservableWriteGuard, SharedObservable, Subscriber};
 use futures_util::{Stream, StreamExt};
 #[cfg(feature = "experimental-sliding-sync")]
 use matrix_sdk_common::deserialized_responses::TimelineEventKind;
@@ -33,7 +34,9 @@ use ruma::{
    api::client::sync::sync_events::v3::RoomSummary as RumaSummary,
    events::{
        call::member::{CallMemberStateKey, MembershipData},
+        direct::OwnedDirectUserIdentifier,
        ignored_user_list::IgnoredUserListEventContent,
+        member_hints::MemberHintsEventContent,
        receipt::{Receipt, ReceiptThread, ReceiptType},
        room::{
            avatar::{self, RoomAvatarEventContent},
@@ -49,7 +52,7 @@ use ruma::{
        },
        tag::{TagEventContent, Tags},
        AnyRoomAccountDataEvent, AnyStrippedStateEvent, AnySyncStateEvent,
-        RoomAccountDataEventType,
+        RoomAccountDataEventType, StateEventType, SyncStateEvent,
    },
    room::RoomType,
    serde::Raw,
@@ -58,7 +61,7 @@ use ruma::{
 };
 use serde::{Deserialize, Serialize};
 use tokio::sync::broadcast;
-use tracing::{debug, field::debug, info, instrument, warn};
+use tracing::{debug, field::debug, info, instrument, trace, warn};

 use super::{
    members::MemberRoomInfo, BaseRoomInfo, RoomCreateWithCreatorEventContent, RoomDisplayName,
@@ -67,12 +70,15 @@ use super::{
 #[cfg(feature = "experimental-sliding-sync")]
 use crate::latest_event::LatestEvent;
 use crate::{
-    deserialized_responses::{DisplayName, MemberEvent, RawSyncOrStrippedState},
+    deserialized_responses::{
+        DisplayName, MemberEvent, RawSyncOrStrippedState, SyncOrStrippedState,
+    },
    notification_settings::RoomNotificationMode,
    read_receipts::RoomReadReceipts,
    store::{DynStateStore, Result as StoreResult, StateStoreExt},
    sync::UnreadNotificationsCount,
-    Error, MinimalStateEvent, OriginalMinimalStateEvent, RoomMemberships,
+    Error, MinimalStateEvent, OriginalMinimalStateEvent, RoomMemberships, StateStoreDataKey,
+    StateStoreDataValue, StoreError,
 };

 /// Indicates that a notable update of `RoomInfo` has been applied, and why.
@@ -114,6 +120,23 @@ bitflags! {
    }
 }

+/// The result of a room summary computation.
+///
+/// If the homeserver does not provide a room summary, we perform a best-effort
+/// computation to generate one ourselves. If the homeserver does provide the
+/// summary, we augment it with additional information about the service members
+/// in the room.
+struct ComputedSummary {
+    /// The list of display names that will be used to calculate the room
+    /// display name.
+    heroes: Vec<String>,
+    /// The number of joined service members in the room.
+    num_service_members: u64,
+    /// The number of joined and invited members, not including any service
+    /// members.
+    num_joined_invited_guess: u64,
+}
+
 impl Default for RoomInfoNotableUpdateReasons {
    fn default() -> Self {
        Self::empty()
@@ -145,6 +168,12 @@ pub struct Room {
    /// to disk but held in memory.
    #[cfg(all(feature = "e2e-encryption", feature = "experimental-sliding-sync"))]
    pub latest_encrypted_events: Arc<SyncRwLock<RingBuffer<Raw<AnySyncTimelineEvent>>>>,
+
+    /// A map for ids of room membership events in the knocking state linked to
+    /// the user id of the user affected by the member event, that the current
+    /// user has marked as seen so they can be ignored.
+    pub seen_knock_request_ids_map:
+        SharedObservable<Option<BTreeMap<OwnedEventId, OwnedUserId>>, AsyncLock>,
 }

 /// The room summary containing member counts and members that should be used to
@@ -196,14 +225,14 @@ pub enum RoomState {
    Invited,
    /// The room is in a knocked state.
    Knocked,
+    /// The room is in a banned state.
+    Banned,
 }

 impl From<&MembershipState> for RoomState {
    fn from(membership_state: &MembershipState) -> Self {
-        // We consider Ban, Knock and Leave to be Left, because they all mean we are not
-        // in the room.
        match membership_state {
-            MembershipState::Ban => Self::Left,
+            MembershipState::Ban => Self::Banned,
            MembershipState::Invite => Self::Invited,
            MembershipState::Join => Self::Joined,
            MembershipState::Knock => Self::Knocked,
@@ -221,6 +250,18 @@ impl From<&MembershipState> for RoomState {
 /// try to behave similarly here.
 const NUM_HEROES: usize = 5;

+/// A filter to remove our own user and the users specified in the member hints
+/// state event, so called service members, from the list of heroes.
+///
+/// The heroes will then be used to calculate a display name for the room if one
+/// wasn't explicitly defined.
+fn heroes_filter<'a>(
+    own_user_id: &'a UserId,
+    member_hints: &'a MemberHintsEventContent,
+) -> impl Fn(&UserId) -> bool + use<'a> {
+    move |user_id| user_id != own_user_id && !member_hints.service_members.contains(user_id)
+}
+
 impl Room {
    /// The size of the latest_encrypted_events RingBuffer
    // SAFETY: `new_unchecked` is safe because 10 is not zero.
@@ -255,6 +296,7 @@ impl Room {
                Self::MAX_ENCRYPTED_EVENTS,
            ))),
            room_info_notable_update_sender,
+            seen_knock_request_ids_map: SharedObservable::new_async(None),
        }
    }

@@ -423,7 +465,7 @@ impl Room {
    #[instrument(skip_all, fields(room_id = ?self.room_id))]
    pub async fn is_direct(&self) -> StoreResult<bool> {
        match self.state() {
-            RoomState::Joined | RoomState::Left => {
+            RoomState::Joined | RoomState::Left | RoomState::Banned => {
                Ok(!self.inner.read().base_info.dm_targets.is_empty())
            }

@@ -460,7 +502,7 @@ impl Room {
    /// only be considered as guidance. We leave members in this list to allow
    /// us to re-find a DM with a user even if they have left, since we may
    /// want to re-invite them.
-    pub fn direct_targets(&self) -> HashSet<OwnedUserId> {
+    pub fn direct_targets(&self) -> HashSet<OwnedDirectUserIdentifier> {
        self.inner.read().base_info.dm_targets.clone()
    }

@@ -629,29 +671,24 @@ impl Room {
        &self,
        summary: RoomSummary,
    ) -> StoreResult<RoomDisplayName> {
-        let summary_member_count = summary.joined_member_count + summary.invited_member_count;
-
-        let (heroes, num_joined_invited_guess) = if !summary.room_heroes.is_empty() {
-            let heroes = self.extract_heroes(&summary.room_heroes).await?;
-            (heroes, None)
+        let computed_summary = if !summary.room_heroes.is_empty() {
+            self.extract_and_augment_summary(&summary).await?
        } else {
-            let (heroes, num_joined_invited) = self.compute_summary().await?;
-            (heroes, Some(num_joined_invited))
+            self.compute_summary().await?
        };

+        let ComputedSummary { heroes, num_service_members, num_joined_invited_guess } =
+            computed_summary;
+
+        let summary_member_count = (summary.joined_member_count + summary.invited_member_count)
+            .saturating_sub(num_service_members);
+
        let num_joined_invited = if self.state() == RoomState::Invited {
            // when we were invited we don't have a proper summary, we have to do best
            // guessing
            heroes.len() as u64 + 1
        } else if summary_member_count == 0 {
-            if let Some(num_joined_invited) = num_joined_invited_guess {
-                num_joined_invited
-            } else {
-                self.store
-                    .get_user_ids(self.room_id(), RoomMemberships::JOIN | RoomMemberships::INVITE)
-                    .await?
-                    .len() as u64
-            }
+            num_joined_invited_guess
        } else {
            summary_member_count
        };
@@ -672,17 +709,39 @@ impl Room {
        Ok(display_name)
    }

-    /// Extract and collect the display names of the room heroes from a
-    /// [`RoomSummary`].
+    /// Extracts and enhances the [`RoomSummary`] provided by the homeserver.
    ///
-    /// Returns the display names as a list of strings.
-    async fn extract_heroes(&self, heroes: &[RoomHero]) -> StoreResult<Vec<String>> {
-        let own_user_id = self.own_user_id().as_str();
+    /// This method extracts the relevant data from the [`RoomSummary`] and
+    /// augments it with additional information that may not be included in
+    /// the initial response, such as details about service members in the
+    /// room.
+    ///
+    /// Returns a [`ComputedSummary`].
+    async fn extract_and_augment_summary(
+        &self,
+        summary: &RoomSummary,
+    ) -> StoreResult<ComputedSummary> {
+        let heroes = &summary.room_heroes;

        let mut names = Vec::with_capacity(heroes.len());
-        let heroes = heroes.iter().filter(|hero| hero.user_id != own_user_id);
+        let own_user_id = self.own_user_id();
+        let member_hints = self.get_member_hints().await?;

-        for hero in heroes {
+        // If we have some service members in the heroes, that means that they are also
+        // part of the joined member counts. They shouldn't be so, otherwise
+        // we'll wrongly assume that there are more members in the room than
+        // they are for the "Bob and 2 others" case.
+        let num_service_members = heroes
+            .iter()
+            .filter(|hero| member_hints.service_members.contains(&hero.user_id))
+            .count() as u64;
+
+        // Construct a filter that is specific to this own user id, set of member hints,
+        // and accepts a `RoomHero` type.
+        let heroes_filter = heroes_filter(own_user_id, &member_hints);
+        let heroes_filter = |hero: &&RoomHero| heroes_filter(&hero.user_id);
+
+        for hero in heroes.iter().filter(heroes_filter) {
            if let Some(display_name) = &hero.display_name {
                names.push(display_name.clone());
            } else {
@@ -700,21 +759,56 @@ impl Room {
            }
        }

-        Ok(names)
+        let num_joined_invited_guess = summary.joined_member_count + summary.invited_member_count;
+
+        // If the summary doesn't provide the number of joined/invited members, let's
+        // guess something.
+        let num_joined_invited_guess = if num_joined_invited_guess == 0 {
+            let guess = self
+                .store
+                .get_user_ids(self.room_id(), RoomMemberships::JOIN | RoomMemberships::INVITE)
+                .await?
+                .len() as u64;
+
+            guess.saturating_sub(num_service_members)
+        } else {
+            // Otherwise, accept the numbers provided by the summary as the guess.
+            num_joined_invited_guess
+        };
+
+        Ok(ComputedSummary { heroes: names, num_service_members, num_joined_invited_guess })
    }

    /// Compute the room summary with the data present in the store.
    ///
    /// The summary might be incorrect if the database info is outdated.
    ///
-    /// Returns a `(heroes_names, num_joined_invited)` tuple.
-    async fn compute_summary(&self) -> StoreResult<(Vec<String>, u64)> {
+    /// Returns the [`ComputedSummary`].
+    async fn compute_summary(&self) -> StoreResult<ComputedSummary> {
+        let member_hints = self.get_member_hints().await?;
+
+        // Construct a filter that is specific to this own user id, set of member hints,
+        // and accepts a `RoomMember` type.
+        let heroes_filter = heroes_filter(&self.own_user_id, &member_hints);
+        let heroes_filter = |u: &RoomMember| heroes_filter(u.user_id());
+
        let mut members = self.members(RoomMemberships::JOIN | RoomMemberships::INVITE).await?;

+        // If we have some service members, they shouldn't count to the number of
+        // joined/invited members, otherwise we'll wrongly assume that there are more
+        // members in the room than they are for the "Bob and 2 others" case.
+        let num_service_members = members
+            .iter()
+            .filter(|member| member_hints.service_members.contains(member.user_id()))
+            .count();
+
        // We can make a good prediction of the total number of joined and invited
        // members here. This might be incorrect if the database info is
        // outdated.
-        let num_joined_invited = members.len() as u64;
+        //
+        // Note: Subtracting here is fine because `num_service_members` is a subset of
+        // `members.len()` due to the above filter operation.
+        let num_joined_invited = members.len() - num_service_members;

        if num_joined_invited == 0
            || (num_joined_invited == 1 && members[0].user_id() == self.own_user_id)
@@ -728,12 +822,37 @@ impl Room {

        let heroes = members
            .into_iter()
-            .filter(|u| u.user_id() != self.own_user_id)
+            .filter(heroes_filter)
            .take(NUM_HEROES)
            .map(|u| u.name().to_owned())
            .collect();

-        Ok((heroes, num_joined_invited))
+        trace!(
+            ?heroes,
+            num_joined_invited,
+            num_service_members,
+            "Computed a room summary since we didn't receive one."
+        );
+
+        let num_service_members = num_service_members as u64;
+        let num_joined_invited_guess = num_joined_invited as u64;
+
+        Ok(ComputedSummary { heroes, num_service_members, num_joined_invited_guess })
+    }
+
+    async fn get_member_hints(&self) -> StoreResult<MemberHintsEventContent> {
+        Ok(self
+            .store
+            .get_state_event_static::<MemberHintsEventContent>(self.room_id())
+            .await?
+            .and_then(|event| {
+                event
+                    .deserialize()
+                    .inspect_err(|e| warn!("Couldn't deserialize the member hints event: {e}"))
+                    .ok()
+            })
+            .and_then(|event| as_variant!(event, SyncOrStrippedState::Sync(SyncStateEvent::Original(e)) => e.content))
+            .unwrap_or_default())
    }

    /// Returns the cached computed display name, if available.
@@ -1058,6 +1177,88 @@ impl Room {
    pub fn pinned_event_ids(&self) -> Option<Vec<OwnedEventId>> {
        self.inner.read().pinned_event_ids()
    }
+
+    /// Mark a list of requests to join the room as seen, given their state
+    /// event ids.
+    pub async fn mark_knock_requests_as_seen(&self, user_ids: &[OwnedUserId]) -> StoreResult<()> {
+        let raw_user_ids: Vec<&str> = user_ids.iter().map(|id| id.as_str()).collect();
+        let member_raw_events = self
+            .store
+            .get_state_events_for_keys(self.room_id(), StateEventType::RoomMember, &raw_user_ids)
+            .await?;
+        let mut event_to_user_ids = Vec::with_capacity(member_raw_events.len());
+
+        // Map the list of events ids to their user ids, if they are event ids for knock
+        // membership events. Log an error and continue otherwise.
+        for raw_event in member_raw_events {
+            let event = raw_event.cast::<RoomMemberEventContent>().deserialize()?;
+            match event {
+                SyncOrStrippedState::Sync(SyncStateEvent::Original(event)) => {
+                    if event.content.membership == MembershipState::Knock {
+                        event_to_user_ids.push((event.event_id, event.state_key))
+                    } else {
+                        warn!("Could not mark knock event as seen: event {} for user {} is not in Knock membership state.", event.event_id, event.state_key);
+                    }
+                }
+                _ => warn!(
+                    "Could not mark knock event as seen: event for user {} is not valid.",
+                    event.state_key()
+                ),
+            }
+        }
+
+        let mut current_seen_events_guard = self.seen_knock_request_ids_map.write().await;
+        // We're not calling `get_seen_join_request_ids` here because we need to keep
+        // the Mutex's guard until we've updated the data
+        let mut current_seen_events = if current_seen_events_guard.is_none() {
+            self.load_cached_knock_request_ids().await?
+        } else {
+            current_seen_events_guard.clone().unwrap()
+        };
+
+        current_seen_events.extend(event_to_user_ids);
+
+        ObservableWriteGuard::set(
+            &mut current_seen_events_guard,
+            Some(current_seen_events.clone()),
+        );
+
+        self.store
+            .set_kv_data(
+                StateStoreDataKey::SeenKnockRequests(self.room_id()),
+                StateStoreDataValue::SeenKnockRequests(current_seen_events),
+            )
+            .await?;
+
+        Ok(())
+    }
+
+    /// Get the list of seen knock request event ids in this room.
+    pub async fn get_seen_knock_request_ids(
+        &self,
+    ) -> Result<BTreeMap<OwnedEventId, OwnedUserId>, StoreError> {
+        let mut guard = self.seen_knock_request_ids_map.write().await;
+        if guard.is_none() {
+            ObservableWriteGuard::set(
+                &mut guard,
+                Some(self.load_cached_knock_request_ids().await?),
+            );
+        }
+        Ok(guard.clone().unwrap_or_default())
+    }
+
+    /// This loads the current list of seen knock request ids from the state
+    /// store.
+    async fn load_cached_knock_request_ids(
+        &self,
+    ) -> StoreResult<BTreeMap<OwnedEventId, OwnedUserId>> {
+        Ok(self
+            .store
+            .get_kv_data(StateStoreDataKey::SeenKnockRequests(self.room_id()))
+            .await?
+            .and_then(|v| v.into_seen_knock_requests())
+            .unwrap_or_default())
+    }
 }

 // See https://github.com/matrix-org/matrix-rust-sdk/pull/3749#issuecomment-2312939823.
@@ -1219,6 +1420,11 @@ impl RoomInfo {
        self.set_state(RoomState::Knocked);
    }

+    /// Mark this Room as banned.
+    pub fn mark_as_banned(&mut self) {
+        self.set_state(RoomState::Banned);
+    }
+
    /// Set the membership RoomState of this Room
    pub fn set_state(&mut self, room_state: RoomState) {
        if room_state != self.room_state {
@@ -1237,6 +1443,11 @@ impl RoomInfo {
        self.members_synced = false;
    }

+    /// Returns whether the room members are synced.
+    pub fn are_members_synced(&self) -> bool {
+        self.members_synced
+    }
+
    /// Mark this Room as still missing some state information.
    pub fn mark_state_partially_synced(&mut self) {
        self.sync_info = SyncInfo::PartiallySynced;
@@ -1775,6 +1986,8 @@ bitflags! {
        const LEFT     = 0b00000100;
        /// The room is in a knocked state.
        const KNOCKED  = 0b00001000;
+        /// The room is in a banned state.
+        const BANNED   = 0b00010000;
    }
 }

@@ -1790,6 +2003,7 @@ impl RoomStateFilter {
            RoomState::Left => Self::LEFT,
            RoomState::Invited => Self::INVITED,
            RoomState::Knocked => Self::KNOCKED,
+            RoomState::Banned => Self::BANNED,
        };

        self.contains(bit_state)
@@ -1808,6 +2022,12 @@ impl RoomStateFilter {
        if self.contains(Self::INVITED) {
            states.push(RoomState::Invited);
        }
+        if self.contains(Self::KNOCKED) {
+            states.push(RoomState::Knocked);
+        }
+        if self.contains(Self::BANNED) {
+            states.push(RoomState::Banned);
+        }

        states
    }
@@ -1853,6 +2073,7 @@ fn compute_display_name_from_heroes(
 #[cfg(test)]
 mod tests {
    use std::{
+        collections::BTreeSet,
        ops::{Not, Sub},
        str::FromStr,
        sync::Arc,
@@ -1864,6 +2085,7 @@ mod tests {
    use matrix_sdk_common::deserialized_responses::SyncTimelineEvent;
    use matrix_sdk_test::{
        async_test,
+        event_factory::EventFactory,
        test_json::{sync_events::PINNED_EVENTS, TAG},
        ALICE, BOB, CAROL,
    };
@@ -1879,22 +2101,20 @@ mod tests {
            room::{
                canonical_alias::RoomCanonicalAliasEventContent,
                encryption::{OriginalSyncRoomEncryptionEvent, RoomEncryptionEventContent},
-                member::{
-                    MembershipState, RoomMemberEventContent, StrippedRoomMemberEvent,
-                    SyncRoomMemberEvent,
-                },
+                member::{MembershipState, RoomMemberEventContent, StrippedRoomMemberEvent},
                name::RoomNameEventContent,
                pinned_events::RoomPinnedEventsEventContent,
            },
            AnySyncStateEvent, EmptyStateKey, StateEventType, StateUnsigned, SyncStateEvent,
        },
-        owned_event_id, owned_user_id, room_alias_id, room_id,
+        owned_event_id, owned_room_id, owned_user_id, room_alias_id, room_id,
        serde::Raw,
        time::SystemTime,
        user_id, DeviceId, EventEncryptionAlgorithm, EventId, MilliSecondsSinceUnixEpoch,
        OwnedEventId, OwnedUserId, UserId,
    };
    use serde_json::json;
+    use similar_asserts::assert_eq;
    use stream_assert::{assert_pending, assert_ready};

    use super::{compute_display_name_from_heroes, Room, RoomHero, RoomInfo, RoomState, SyncInfo};
@@ -1903,8 +2123,9 @@ mod tests {
    use crate::{
        rooms::RoomNotableTags,
        store::{IntoStateStore, MemoryStore, StateChanges, StateStore, StoreConfig},
+        test_utils::logged_in_base_client,
        BaseClient, MinimalStateEvent, OriginalMinimalStateEvent, RoomDisplayName,
-        RoomInfoNotableUpdateReasons, SessionMeta,
+        RoomInfoNotableUpdateReasons, RoomStateFilter, SessionMeta,
    };

    #[test]
@@ -2369,21 +2590,6 @@ mod tests {
        Raw::new(&ev_json).unwrap().cast()
    }

-    fn make_member_event(user_id: &UserId, name: &str) -> Raw<SyncRoomMemberEvent> {
-        let ev_json = json!({
-            "type": "m.room.member",
-            "content": assign!(RoomMemberEventContent::new(MembershipState::Join), {
-                displayname: Some(name.to_owned())
-            }),
-            "sender": user_id,
-            "state_key": user_id,
-            "event_id": "$h29iv0s1:example.com",
-            "origin_server_ts": 208,
-        });
-
-        Raw::new(&ev_json).unwrap().cast()
-    }
-
    #[async_test]
    async fn test_display_name_for_joined_room_is_empty_if_no_info() {
        let (_, room) = make_room_test_helper(RoomState::Joined);
@@ -2535,20 +2741,23 @@ mod tests {
        let room_id = room_id!("!test:localhost");
        let matthew = user_id!("@matthew:example.org");
        let me = user_id!("@me:example.org");
+
        let mut changes = StateChanges::new("".to_owned());
        let summary = assign!(RumaSummary::new(), {
            joined_member_count: Some(2u32.into()),
            heroes: vec![me.to_owned(), matthew.to_owned()],
        });

+        let f = EventFactory::new().room(room_id!("!test:localhost"));
+
        let members = changes
            .state
            .entry(room_id.to_owned())
            .or_default()
            .entry(StateEventType::RoomMember)
            .or_default();
-        members.insert(matthew.into(), make_member_event(matthew, "Matthew").cast());
-        members.insert(me.into(), make_member_event(me, "Me").cast());
+        members.insert(matthew.into(), f.member(matthew).display_name("Matthew").into_raw());
+        members.insert(me.into(), f.member(me).display_name("Me").into_raw());

        store.save_changes(&changes).await.unwrap();

@@ -2560,12 +2769,21 @@ mod tests {
    }

    #[async_test]
-    async fn test_display_name_dm_joined_no_heroes() {
+    async fn test_display_name_dm_joined_service_members() {
        let (store, room) = make_room_test_helper(RoomState::Joined);
        let room_id = room_id!("!test:localhost");
-        let matthew = user_id!("@matthew:example.org");
+
+        let matthew = user_id!("@sahasrhala:example.org");
        let me = user_id!("@me:example.org");
+        let bot = user_id!("@bot:example.org");
+
        let mut changes = StateChanges::new("".to_owned());
+        let summary = assign!(RumaSummary::new(), {
+            joined_member_count: Some(3u32.into()),
+            heroes: vec![me.to_owned(), matthew.to_owned(), bot.to_owned()],
+        });
+
+        let f = EventFactory::new().room(room_id!("!test:localhost"));

        let members = changes
            .state
@@ -2573,8 +2791,131 @@ mod tests {
            .or_default()
            .entry(StateEventType::RoomMember)
            .or_default();
-        members.insert(matthew.into(), make_member_event(matthew, "Matthew").cast());
-        members.insert(me.into(), make_member_event(me, "Me").cast());
+        members.insert(matthew.into(), f.member(matthew).display_name("Matthew").into_raw());
+        members.insert(me.into(), f.member(me).display_name("Me").into_raw());
+        members.insert(bot.into(), f.member(bot).display_name("Bot").into_raw());
+
+        let member_hints_content =
+            f.member_hints(BTreeSet::from([bot.to_owned()])).sender(me).into_raw();
+        changes
+            .state
+            .entry(room_id.to_owned())
+            .or_default()
+            .entry(StateEventType::MemberHints)
+            .or_default()
+            .insert("".to_owned(), member_hints_content);
+
+        store.save_changes(&changes).await.unwrap();
+
+        room.inner.update_if(|info| info.update_from_ruma_summary(&summary));
+        // Bot should not contribute to the display name.
+        assert_eq!(
+            room.compute_display_name().await.unwrap(),
+            RoomDisplayName::Calculated("Matthew".to_owned())
+        );
+    }
+
+    #[async_test]
+    async fn test_display_name_dm_joined_alone_with_service_members() {
+        let (store, room) = make_room_test_helper(RoomState::Joined);
+        let room_id = room_id!("!test:localhost");
+
+        let me = user_id!("@me:example.org");
+        let bot = user_id!("@bot:example.org");
+
+        let mut changes = StateChanges::new("".to_owned());
+        let summary = assign!(RumaSummary::new(), {
+            joined_member_count: Some(2u32.into()),
+            heroes: vec![me.to_owned(), bot.to_owned()],
+        });
+
+        let f = EventFactory::new().room(room_id!("!test:localhost"));
+
+        let members = changes
+            .state
+            .entry(room_id.to_owned())
+            .or_default()
+            .entry(StateEventType::RoomMember)
+            .or_default();
+        members.insert(me.into(), f.member(me).display_name("Me").into_raw());
+        members.insert(bot.into(), f.member(bot).display_name("Bot").into_raw());
+
+        let member_hints_content =
+            f.member_hints(BTreeSet::from([bot.to_owned()])).sender(me).into_raw();
+        changes
+            .state
+            .entry(room_id.to_owned())
+            .or_default()
+            .entry(StateEventType::MemberHints)
+            .or_default()
+            .insert("".to_owned(), member_hints_content);
+
+        store.save_changes(&changes).await.unwrap();
+
+        room.inner.update_if(|info| info.update_from_ruma_summary(&summary));
+        // Bot should not contribute to the display name.
+        assert_eq!(room.compute_display_name().await.unwrap(), RoomDisplayName::Empty);
+    }
+
+    #[async_test]
+    async fn test_display_name_dm_joined_no_heroes() {
+        let (store, room) = make_room_test_helper(RoomState::Joined);
+        let room_id = room_id!("!test:localhost");
+        let matthew = user_id!("@matthew:example.org");
+        let me = user_id!("@me:example.org");
+        let mut changes = StateChanges::new("".to_owned());
+
+        let f = EventFactory::new().room(room_id!("!test:localhost"));
+
+        let members = changes
+            .state
+            .entry(room_id.to_owned())
+            .or_default()
+            .entry(StateEventType::RoomMember)
+            .or_default();
+        members.insert(matthew.into(), f.member(matthew).display_name("Matthew").into_raw());
+        members.insert(me.into(), f.member(me).display_name("Me").into_raw());
+
+        store.save_changes(&changes).await.unwrap();
+
+        assert_eq!(
+            room.compute_display_name().await.unwrap(),
+            RoomDisplayName::Calculated("Matthew".to_owned())
+        );
+    }
+
+    #[async_test]
+    async fn test_display_name_dm_joined_no_heroes_service_members() {
+        let (store, room) = make_room_test_helper(RoomState::Joined);
+        let room_id = room_id!("!test:localhost");
+
+        let matthew = user_id!("@matthew:example.org");
+        let me = user_id!("@me:example.org");
+        let bot = user_id!("@bot:example.org");
+
+        let mut changes = StateChanges::new("".to_owned());
+
+        let f = EventFactory::new().room(room_id!("!test:localhost"));
+
+        let members = changes
+            .state
+            .entry(room_id.to_owned())
+            .or_default()
+            .entry(StateEventType::RoomMember)
+            .or_default();
+        members.insert(matthew.into(), f.member(matthew).display_name("Matthew").into_raw());
+        members.insert(me.into(), f.member(me).display_name("Me").into_raw());
+        members.insert(bot.into(), f.member(bot).display_name("Bot").into_raw());
+
+        let member_hints_content =
+            f.member_hints(BTreeSet::from([bot.to_owned()])).sender(me).into_raw();
+        changes
+            .state
+            .entry(room_id.to_owned())
+            .or_default()
+            .entry(StateEventType::MemberHints)
+            .or_default()
+            .insert("".to_owned(), member_hints_content);

        store.save_changes(&changes).await.unwrap();

@@ -2598,6 +2939,8 @@ mod tests {

        let mut changes = StateChanges::new("".to_owned());

+        let f = EventFactory::new().room(room_id!("!test:localhost"));
+
        // Save members in two batches, so that there's no implied ordering in the
        // store.
        {
@@ -2607,10 +2950,10 @@ mod tests {
                .or_default()
                .entry(StateEventType::RoomMember)
                .or_default();
-            members.insert(carol.into(), make_member_event(carol, "Carol").cast());
-            members.insert(bob.into(), make_member_event(bob, "Bob").cast());
-            members.insert(fred.into(), make_member_event(fred, "Fred").cast());
-            members.insert(me.into(), make_member_event(me, "Me").cast());
+            members.insert(carol.into(), f.member(carol).display_name("Carol").into_raw());
+            members.insert(bob.into(), f.member(bob).display_name("Bob").into_raw());
+            members.insert(fred.into(), f.member(fred).display_name("Fred").into_raw());
+            members.insert(me.into(), f.member(me).display_name("Me").into_raw());
            store.save_changes(&changes).await.unwrap();
        }

@@ -2621,9 +2964,9 @@ mod tests {
                .or_default()
                .entry(StateEventType::RoomMember)
                .or_default();
-            members.insert(alice.into(), make_member_event(alice, "Alice").cast());
-            members.insert(erica.into(), make_member_event(erica, "Erica").cast());
-            members.insert(denis.into(), make_member_event(denis, "Denis").cast());
+            members.insert(alice.into(), f.member(alice).display_name("Alice").into_raw());
+            members.insert(erica.into(), f.member(erica).display_name("Erica").into_raw());
+            members.insert(denis.into(), f.member(denis).display_name("Denis").into_raw());
            store.save_changes(&changes).await.unwrap();
        }

@@ -2651,6 +2994,8 @@ mod tests {
        let fred = user_id!("@fred:example.org");
        let me = user_id!("@me:example.org");

+        let f = EventFactory::new().room(room_id!("!test:localhost"));
+
        let mut changes = StateChanges::new("".to_owned());

        // Save members in two batches, so that there's no implied ordering in the
@@ -2662,10 +3007,11 @@ mod tests {
                .or_default()
                .entry(StateEventType::RoomMember)
                .or_default();
-            members.insert(carol.into(), make_member_event(carol, "Carol").cast());
-            members.insert(bob.into(), make_member_event(bob, "Bob").cast());
-            members.insert(fred.into(), make_member_event(fred, "Fred").cast());
-            members.insert(me.into(), make_member_event(me, "Me").cast());
+            members.insert(carol.into(), f.member(carol).display_name("Carol").into_raw());
+            members.insert(bob.into(), f.member(bob).display_name("Bob").into_raw());
+            members.insert(fred.into(), f.member(fred).display_name("Fred").into_raw());
+            members.insert(me.into(), f.member(me).display_name("Me").into_raw());
+
            store.save_changes(&changes).await.unwrap();
        }

@@ -2676,9 +3022,9 @@ mod tests {
                .or_default()
                .entry(StateEventType::RoomMember)
                .or_default();
-            members.insert(alice.into(), make_member_event(alice, "Alice").cast());
-            members.insert(erica.into(), make_member_event(erica, "Erica").cast());
-            members.insert(denis.into(), make_member_event(denis, "Denis").cast());
+            members.insert(alice.into(), f.member(alice).display_name("Alice").into_raw());
+            members.insert(erica.into(), f.member(erica).display_name("Erica").into_raw());
+            members.insert(denis.into(), f.member(denis).display_name("Denis").into_raw());
            store.save_changes(&changes).await.unwrap();
        }

@@ -2700,14 +3046,16 @@ mod tests {
            heroes: vec![me.to_owned(), matthew.to_owned()],
        });

+        let f = EventFactory::new().room(room_id!("!test:localhost"));
+
        let members = changes
            .state
            .entry(room_id.to_owned())
            .or_default()
            .entry(StateEventType::RoomMember)
            .or_default();
-        members.insert(matthew.into(), make_member_event(matthew, "Matthew").cast());
-        members.insert(me.into(), make_member_event(me, "Me").cast());
+        members.insert(matthew.into(), f.member(matthew).display_name("Matthew").into_raw());
+        members.insert(me.into(), f.member(me).display_name("Me").into_raw());

        store.save_changes(&changes).await.unwrap();

@@ -3296,5 +3644,78 @@ mod tests {
        room.set_room_info(room_info, RoomInfoNotableUpdateReasons::MEMBERSHIP);
        assert_eq!(room.prev_state(), Some(RoomState::Joined));
        assert_eq!(room.state(), RoomState::Left);
+
+        // Left -> Banned
+        let mut room_info = room.clone_info();
+        room_info.mark_as_banned();
+        room.set_room_info(room_info, RoomInfoNotableUpdateReasons::MEMBERSHIP);
+        assert_eq!(room.prev_state(), Some(RoomState::Left));
+        assert_eq!(room.state(), RoomState::Banned);
+    }
+
+    #[async_test]
+    async fn test_room_state_filters() {
+        let client = logged_in_base_client(None).await;
+
+        let joined_room_id = owned_room_id!("!joined:example.org");
+        client.get_or_create_room(&joined_room_id, RoomState::Joined);
+
+        let invited_room_id = owned_room_id!("!invited:example.org");
+        client.get_or_create_room(&invited_room_id, RoomState::Invited);
+
+        let left_room_id = owned_room_id!("!left:example.org");
+        client.get_or_create_room(&left_room_id, RoomState::Left);
+
+        let knocked_room_id = owned_room_id!("!knocked:example.org");
+        client.get_or_create_room(&knocked_room_id, RoomState::Knocked);
+
+        let banned_room_id = owned_room_id!("!banned:example.org");
+        client.get_or_create_room(&banned_room_id, RoomState::Banned);
+
+        let joined_rooms = client.rooms_filtered(RoomStateFilter::JOINED);
+        assert_eq!(joined_rooms.len(), 1);
+        assert_eq!(joined_rooms[0].state(), RoomState::Joined);
+        assert_eq!(joined_rooms[0].room_id, joined_room_id);
+
+        let invited_rooms = client.rooms_filtered(RoomStateFilter::INVITED);
+        assert_eq!(invited_rooms.len(), 1);
+        assert_eq!(invited_rooms[0].state(), RoomState::Invited);
+        assert_eq!(invited_rooms[0].room_id, invited_room_id);
+
+        let left_rooms = client.rooms_filtered(RoomStateFilter::LEFT);
+        assert_eq!(left_rooms.len(), 1);
+        assert_eq!(left_rooms[0].state(), RoomState::Left);
+        assert_eq!(left_rooms[0].room_id, left_room_id);
+
+        let knocked_rooms = client.rooms_filtered(RoomStateFilter::KNOCKED);
+        assert_eq!(knocked_rooms.len(), 1);
+        assert_eq!(knocked_rooms[0].state(), RoomState::Knocked);
+        assert_eq!(knocked_rooms[0].room_id, knocked_room_id);
+
+        let banned_rooms = client.rooms_filtered(RoomStateFilter::BANNED);
+        assert_eq!(banned_rooms.len(), 1);
+        assert_eq!(banned_rooms[0].state(), RoomState::Banned);
+        assert_eq!(banned_rooms[0].room_id, banned_room_id);
+    }
+
+    #[test]
+    fn test_room_state_filters_as_vec() {
+        assert_eq!(RoomStateFilter::JOINED.as_vec(), vec![RoomState::Joined]);
+        assert_eq!(RoomStateFilter::LEFT.as_vec(), vec![RoomState::Left]);
+        assert_eq!(RoomStateFilter::INVITED.as_vec(), vec![RoomState::Invited]);
+        assert_eq!(RoomStateFilter::KNOCKED.as_vec(), vec![RoomState::Knocked]);
+        assert_eq!(RoomStateFilter::BANNED.as_vec(), vec![RoomState::Banned]);
+
+        // Check all filters are taken into account
+        assert_eq!(
+            RoomStateFilter::all().as_vec(),
+            vec![
+                RoomState::Joined,
+                RoomState::Left,
+                RoomState::Invited,
+                RoomState::Knocked,
+                RoomState::Banned
+            ]
+        );
    }
 }
@@ -269,7 +269,7 @@ impl BaseClient {
                        .or_insert_with(JoinedRoomUpdate::default)
                        .account_data
                        .append(&mut raw.to_vec()),
-                    RoomState::Left => new_rooms
+                    RoomState::Left | RoomState::Banned => new_rooms
                        .leave
                        .entry(room_id.to_owned())
                        .or_insert_with(LeftRoomUpdate::default)
@@ -546,7 +546,7 @@ impl BaseClient {
                ))
            }

-            RoomState::Left => Ok((
+            RoomState::Left | RoomState::Banned => Ok((
                room_info,
                None,
                Some(LeftRoomUpdate::new(
@@ -911,7 +911,7 @@ mod tests {
        api::client::sync::sync_events::UnreadNotificationsCount,
        assign, event_id,
        events::{
-            direct::DirectEventContent,
+            direct::{DirectEventContent, DirectUserIdentifier, OwnedDirectUserIdentifier},
            room::{
                avatar::RoomAvatarEventContent,
                canonical_alias::RoomCanonicalAliasEventContent,
@@ -1247,7 +1247,7 @@ mod tests {
            room.required_state.push(make_state_event(
                user_b_id,
                user_a_id.as_str(),
-                RoomMemberEventContent::new(membership),
+                RoomMemberEventContent::new(membership.clone()),
                None,
            ));
            let response = response_with_room(room_id, room);
@@ -1256,8 +1256,17 @@ mod tests {
                .await
                .expect("Failed to process sync");

-            // The room is left.
-            assert_eq!(client.get_room(room_id).unwrap().state(), RoomState::Left);
+            match membership {
+                MembershipState::Leave => {
+                    // The room is left.
+                    assert_eq!(client.get_room(room_id).unwrap().state(), RoomState::Left);
+                }
+                MembershipState::Ban => {
+                    // The room is banned.
+                    assert_eq!(client.get_room(room_id).unwrap().state(), RoomState::Banned);
+                }
+                _ => panic!("Unexpected membership state found: {membership}"),
+            }

            // And it is added to the list of left rooms only.
            assert!(!sync_resp.rooms.join.contains_key(room_id));
@@ -1337,7 +1346,7 @@ mod tests {
        create_dm(&client, room_id, user_a_id, user_b_id, MembershipState::Join).await;

        // (Sanity: B is a direct target, and is in Join state)
-        assert!(direct_targets(&client, room_id).contains(user_b_id));
+        assert!(direct_targets(&client, room_id).contains(<&DirectUserIdentifier>::from(user_b_id)));
        assert_eq!(membership(&client, room_id, user_b_id).await, MembershipState::Join);

        // When B leaves
@@ -1346,7 +1355,7 @@ mod tests {
        // Then B is still a direct target, and is in Leave state (B is a direct target
        // because we want to return to our old DM in the UI even if the other
        // user left, so we can reinvite them. See https://github.com/matrix-org/matrix-rust-sdk/issues/2017)
-        assert!(direct_targets(&client, room_id).contains(user_b_id));
+        assert!(direct_targets(&client, room_id).contains(<&DirectUserIdentifier>::from(user_b_id)));
        assert_eq!(membership(&client, room_id, user_b_id).await, MembershipState::Leave);
    }

@@ -1362,7 +1371,7 @@ mod tests {
        create_dm(&client, room_id, user_a_id, user_b_id, MembershipState::Invite).await;

        // (Sanity: B is a direct target, and is in Invite state)
-        assert!(direct_targets(&client, room_id).contains(user_b_id));
+        assert!(direct_targets(&client, room_id).contains(<&DirectUserIdentifier>::from(user_b_id)));
        assert_eq!(membership(&client, room_id, user_b_id).await, MembershipState::Invite);

        // When B declines the invitation (i.e. leaves)
@@ -1371,7 +1380,7 @@ mod tests {
        // Then B is still a direct target, and is in Leave state (B is a direct target
        // because we want to return to our old DM in the UI even if the other
        // user left, so we can reinvite them. See https://github.com/matrix-org/matrix-rust-sdk/issues/2017)
-        assert!(direct_targets(&client, room_id).contains(user_b_id));
+        assert!(direct_targets(&client, room_id).contains(<&DirectUserIdentifier>::from(user_b_id)));
        assert_eq!(membership(&client, room_id, user_b_id).await, MembershipState::Leave);
    }

@@ -1389,7 +1398,7 @@ mod tests {
        assert_eq!(membership(&client, room_id, user_a_id).await, MembershipState::Join);

        // (Sanity: B is a direct target, and is in Join state)
-        assert!(direct_targets(&client, room_id).contains(user_b_id));
+        assert!(direct_targets(&client, room_id).contains(<&DirectUserIdentifier>::from(user_b_id)));
        assert_eq!(membership(&client, room_id, user_b_id).await, MembershipState::Join);

        let room = client.get_room(room_id).unwrap();
@@ -1413,7 +1422,7 @@ mod tests {
        assert_eq!(membership(&client, room_id, user_a_id).await, MembershipState::Join);

        // (Sanity: B is a direct target, and is in Join state)
-        assert!(direct_targets(&client, room_id).contains(user_b_id));
+        assert!(direct_targets(&client, room_id).contains(<&DirectUserIdentifier>::from(user_b_id)));
        assert_eq!(membership(&client, room_id, user_b_id).await, MembershipState::Invite);

        let room = client.get_room(room_id).unwrap();
@@ -2558,9 +2567,10 @@ mod tests {
        let mut room_response = http::response::Room::new();
        set_room_joined(&mut room_response, user_a_id);
        let mut response = response_with_room(room_id_1, room_response);
-        let mut direct_content = BTreeMap::new();
-        direct_content.insert(user_a_id.to_owned(), vec![room_id_1.to_owned()]);
-        direct_content.insert(user_b_id.to_owned(), vec![room_id_2.to_owned()]);
+        let mut direct_content: BTreeMap<OwnedDirectUserIdentifier, Vec<OwnedRoomId>> =
+            BTreeMap::new();
+        direct_content.insert(user_a_id.into(), vec![room_id_1.to_owned()]);
+        direct_content.insert(user_b_id.into(), vec![room_id_2.to_owned()]);
        response
            .extensions
            .account_data
@@ -2656,7 +2666,7 @@ mod tests {
            .unwrap(),
            UnableToDecryptInfo {
                session_id: Some("".to_owned()),
-                reason: UnableToDecryptReason::MissingMegolmSession,
+                reason: UnableToDecryptReason::MissingMegolmSession { withheld_code: None },
            },
        )
    }
@@ -2671,7 +2681,7 @@ mod tests {
        member.membership().clone()
    }

-    fn direct_targets(client: &BaseClient, room_id: &RoomId) -> HashSet<OwnedUserId> {
+    fn direct_targets(client: &BaseClient, room_id: &RoomId) -> HashSet<OwnedDirectUserIdentifier> {
        let room = client.get_room(room_id).expect("Room not found!");
        room.direct_targets()
    }
@@ -2730,8 +2740,9 @@ mod tests {
        user_id: OwnedUserId,
        room_ids: Vec<OwnedRoomId>,
    ) {
-        let mut direct_content = BTreeMap::new();
-        direct_content.insert(user_id, room_ids);
+        let mut direct_content: BTreeMap<OwnedDirectUserIdentifier, Vec<OwnedRoomId>> =
+            BTreeMap::new();
+        direct_content.insert(user_id.into(), room_ids);
        response
            .extensions
            .account_data
@@ -23,6 +23,7 @@ use std::{
 use matrix_sdk_common::deserialized_responses::SyncTimelineEvent;
 use ruma::{
    events::{
+        direct::OwnedDirectUserIdentifier,
        room::{
            avatar::RoomAvatarEventContent,
            canonical_alias::RoomCanonicalAliasEventContent,
@@ -200,12 +201,17 @@ impl BaseRoomInfoV1 {
            MinimalStateEvent::Redacted(ev) => MinimalStateEvent::Redacted(ev),
        });

+        let mut converted_dm_targets = HashSet::new();
+        for dm_target in dm_targets {
+            converted_dm_targets.insert(OwnedDirectUserIdentifier::from(dm_target));
+        }
+
        Box::new(BaseRoomInfo {
            avatar,
            beacons: BTreeMap::new(),
            canonical_alias,
            create,
-            dm_targets,
+            dm_targets: converted_dm_targets,
            encryption,
            guest_access,
            history_visibility,
@@ -138,6 +138,13 @@ where
        L: Hash + Eq + ?Sized,
    {
        let position = self.mapping.remove(key)?;
+
+        // Reindex every mapped entry that is after the position we're looking to
+        // remove.
+        for mapped_pos in self.mapping.values_mut().filter(|pos| **pos > position) {
+            *mapped_pos = mapped_pos.saturating_sub(1);
+        }
+
        Some(self.values.remove(position))
    }
 }
@@ -195,6 +202,12 @@ mod tests {
        assert_eq!(map.get(&'a'), Some(&'E'));
        assert_eq!(map.get(&'b'), Some(&'f'));
        assert_eq!(map.get(&'c'), Some(&'G'));
+
+        // remove non-last item
+        assert_eq!(map.remove(&'b'), Some('f'));
+
+        // get_or_create item after the removed one
+        assert_eq!(map.get_or_create(&'c', || 'G'), &'G');
    }

    #[test]
@@ -208,20 +221,26 @@ mod tests {
        // new items
        map.insert('a', 'e');
        map.insert('b', 'f');
+        map.insert('c', 'g');

        assert_eq!(map.get(&'a'), Some(&'e'));
        assert_eq!(map.get(&'b'), Some(&'f'));
-        assert!(map.get(&'c').is_none());
+        assert_eq!(map.get(&'c'), Some(&'g'));
+        assert!(map.get(&'d').is_none());

-        // remove one item
-        assert_eq!(map.remove(&'b'), Some('f'));
+        // remove last item
+        assert_eq!(map.remove(&'c'), Some('g'));

        assert_eq!(map.get(&'a'), Some(&'e'));
-        assert_eq!(map.get(&'b'), None);
+        assert_eq!(map.get(&'b'), Some(&'f'));
        assert_eq!(map.get(&'c'), None);

        // remove a non-existent item
        assert_eq!(map.remove(&'c'), None);
+
+        // remove a non-last item
+        assert_eq!(map.remove(&'a'), Some('e'));
+        assert_eq!(map.get(&'b'), Some(&'f'));
    }

    #[test]
@@ -248,9 +248,15 @@ pub struct FinishUploadThumbnailInfo {
    /// Transaction id for the thumbnail upload.
    pub txn: OwnedTransactionId,
    /// Thumbnail's width.
-    pub width: UInt,
+    ///
+    /// Used previously, kept for backwards compatibility.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub width: Option<UInt>,
    /// Thumbnail's height.
-    pub height: UInt,
+    ///
+    /// Used previously, kept for backwards compatibility.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub height: Option<UInt>,
 }

 /// A transaction id identifying a [`DependentQueuedRequest`] rather than its
@@ -1022,6 +1022,9 @@ pub enum StateStoreDataValue {
    ///
    /// [`ComposerDraft`]: Self::ComposerDraft
    ComposerDraft(ComposerDraft),
+
+    /// A list of knock request ids marked as seen in a room.
+    SeenKnockRequests(BTreeMap<OwnedEventId, OwnedUserId>),
 }

 /// Current draft of the composer for the room.
@@ -1088,6 +1091,11 @@ impl StateStoreDataValue {
    pub fn into_server_capabilities(self) -> Option<ServerCapabilities> {
        as_variant!(self, Self::ServerCapabilities)
    }
+
+    /// Get this value if it is the data for the ignored join requests.
+    pub fn into_seen_knock_requests(self) -> Option<BTreeMap<OwnedEventId, OwnedUserId>> {
+        as_variant!(self, Self::SeenKnockRequests)
+    }
 }

 /// A key for key-value data.
@@ -1117,6 +1125,9 @@ pub enum StateStoreDataKey<'a> {
    ///
    /// [`ComposerDraft`]: Self::ComposerDraft
    ComposerDraft(&'a RoomId),
+
+    /// A list of knock request ids marked as seen in a room.
+    SeenKnockRequests(&'a RoomId),
 }

 impl StateStoreDataKey<'_> {
@@ -1142,6 +1153,10 @@ impl StateStoreDataKey<'_> {
    /// Key prefix to use for the [`ComposerDraft`][Self::ComposerDraft]
    /// variant.
    pub const COMPOSER_DRAFT: &'static str = "composer_draft";
+
+    /// Key prefix to use for the
+    /// [`SeenKnockRequests`][Self::SeenKnockRequests] variant.
+    pub const SEEN_KNOCK_REQUESTS: &'static str = "seen_knock_requests";
 }

 #[cfg(test)]
@@ -2,6 +2,32 @@

 All notable changes to this project will be documented in this file.

+<!-- next-header -->
+
+## [Unreleased] - ReleaseDate
+
+## [0.9.0] - 2024-12-18
+
+### Bug Fixes
+
+- Change the behavior of `LinkedChunk::new_with_update_history()` to emit an
+  `Update::NewItemsChunk` when a new, initial empty, chunk is created.
+  ([#4327](https://github.com/matrix-org/matrix-rust-sdk/pull/4321))
+
+- [**breaking**] Make `Room::history_visibility()` return an Option, and
+  introduce `Room::history_visibility_or_default()` to return a better
+  sensible default, according to the spec.
+  ([#4325](https://github.com/matrix-org/matrix-rust-sdk/pull/4325))
+
+- Clear the internal state of the `AsVector` struct if an `Update::Clear`
+  state has been received.
+  ([#4321](https://github.com/matrix-org/matrix-rust-sdk/pull/4321))
+
+### Documentation
+
+- Document that a decrypted raw event always has a room id.
+  ([#728e1fd](https://github.com/matrix-org/matrix-rust-sdk/commit/728e1fda2ae9f1bfa87df162aa553040be705223))
+
 ## [0.8.0] - 2024-11-19

 ### Refactor
@@ -9,7 +9,7 @@ name = "matrix-sdk-common"
 readme = "README.md"
 repository = "https://github.com/matrix-org/matrix-rust-sdk"
 rust-version = { workspace = true }
-version = "0.8.0"
+version = "0.9.0"

 [package.metadata.docs.rs]
 default-target = "x86_64-unknown-linux-gnu"
@@ -36,16 +36,16 @@ uniffi = { workspace = true, optional = true }
 [target.'cfg(target_arch = "wasm32")'.dependencies]
 futures-util = { workspace = true, features = ["channel"] }
 wasm-bindgen-futures = { version = "0.4.33", optional = true }
-gloo-timers = { version = "0.3.0", features = ["futures"] }
-web-sys = { version = "0.3.60", features = ["console"] }
+gloo-timers = { workspace = true, features = ["futures"] }
+web-sys = { workspace = true, features = ["console"] }
 tracing-subscriber = { workspace = true, features = ["fmt", "ansi"] }
-wasm-bindgen = "0.2.84"
+wasm-bindgen = { workspace = true }

 [dev-dependencies]
 assert_matches = { workspace = true }
-proptest = { version = "1.4.0", default-features = false, features = ["std"] }
+proptest = { workspace = true }
 matrix-sdk-test-macros = { path = "../../testing/matrix-sdk-test-macros" }
-wasm-bindgen-test = "0.3.33"
+wasm-bindgen-test = { workspace = true }

 [target.'cfg(not(target_arch = "wasm32"))'.dependencies]
 # Enable the test macro.
@@ -54,7 +54,7 @@ tokio = { workspace = true, features = ["rt", "macros"] }
 [target.'cfg(target_arch = "wasm32")'.dev-dependencies]
 # Enable the JS feature for getrandom.
 getrandom = { version = "0.2.6", default-features = false, features = ["js"] }
-js-sys = "0.3.64"
+js-sys = { workspace = true }

 [lints]
 workspace = true
@@ -17,7 +17,10 @@ use std::{collections::BTreeMap, fmt};
 use ruma::{
    events::{AnyMessageLikeEvent, AnySyncTimelineEvent, AnyTimelineEvent},
    push::Action,
-    serde::{JsonObject, Raw},
+    serde::{
+        AsRefStr, AsStrAsRefStr, DebugAsRefStr, DeserializeFromCowStr, FromString, JsonObject, Raw,
+        SerializeAsRefStr,
+    },
    DeviceKeyAlgorithm, OwnedDeviceId, OwnedEventId, OwnedUserId,
 };
 use serde::{Deserialize, Serialize};
@@ -176,6 +179,7 @@ pub enum VerificationLevel {

    /// The message was sent by a user identity we have not verified, but the
    /// user was previously verified.
+    #[serde(alias = "PreviouslyVerified")]
    VerificationViolation,

    /// The message was sent by a device not linked to (signed by) any user
@@ -259,6 +263,7 @@ pub enum ShieldStateCode {
    /// An unencrypted event in an encrypted room.
    SentInClear,
    /// The sender was previously verified but changed their identity.
+    #[serde(alias = "PreviouslyVerified")]
    VerificationViolation,
 }

@@ -587,6 +592,11 @@ impl fmt::Debug for TimelineEventKind {
 /// A successfully-decrypted encrypted event.
 pub struct DecryptedRoomEvent {
    /// The decrypted event.
+    ///
+    /// Note: it's not an error that this contains an `AnyMessageLikeEvent`: an
+    /// encrypted payload *always contains* a room id, by the [spec].
+    ///
+    /// [spec]: https://spec.matrix.org/v1.12/client-server-api/#mmegolmv1aes-sha2
    pub event: Raw<AnyMessageLikeEvent>,

    /// The encryption info about the event.
@@ -661,7 +671,7 @@ pub struct UnableToDecryptInfo {
    pub session_id: Option<String>,

    /// Reason code for the decryption failure
-    #[serde(default = "unknown_utd_reason")]
+    #[serde(default = "unknown_utd_reason", deserialize_with = "deserialize_utd_reason")]
    pub reason: UnableToDecryptReason,
 }

@@ -669,6 +679,24 @@ fn unknown_utd_reason() -> UnableToDecryptReason {
    UnableToDecryptReason::Unknown
 }

+/// Provides basic backward compatibility for deserializing older serialized
+/// `UnableToDecryptReason` values.
+pub fn deserialize_utd_reason<'de, D>(d: D) -> Result<UnableToDecryptReason, D::Error>
+where
+    D: serde::Deserializer<'de>,
+{
+    // Start by deserializing as to an untyped JSON value.
+    let v: serde_json::Value = Deserialize::deserialize(d)?;
+    // Backwards compatibility: `MissingMegolmSession` used to be stored without the
+    // withheld code.
+    if v.as_str().is_some_and(|s| s == "MissingMegolmSession") {
+        return Ok(UnableToDecryptReason::MissingMegolmSession { withheld_code: None });
+    }
+    // Otherwise, use the derived deserialize impl to turn the JSON into a
+    // UnableToDecryptReason
+    serde_json::from_value::<UnableToDecryptReason>(v).map_err(serde::de::Error::custom)
+}
+
 /// Reason code for a decryption failure
 #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
 pub enum UnableToDecryptReason {
@@ -684,9 +712,11 @@ pub enum UnableToDecryptReason {

    /// Decryption failed because we're missing the megolm session that was used
    /// to encrypt the event.
-    ///
-    /// TODO: support withheld codes?
-    MissingMegolmSession,
+    MissingMegolmSession {
+        /// If the key was withheld on purpose, the associated code. `None`
+        /// means no withheld code was received.
+        withheld_code: Option<WithheldCode>,
+    },

    /// Decryption failed because, while we have the megolm session that was
    /// used to encrypt the message, it is ratcheted too far forward.
@@ -718,7 +748,86 @@ impl UnableToDecryptReason {
    /// Returns true if this UTD is due to a missing room key (and hence might
    /// resolve itself if we wait a bit.)
    pub fn is_missing_room_key(&self) -> bool {
-        matches!(self, Self::MissingMegolmSession | Self::UnknownMegolmMessageIndex)
+        // In case of MissingMegolmSession with a withheld code we return false here
+        // given that this API is used to decide if waiting a bit will help.
+        matches!(
+            self,
+            Self::MissingMegolmSession { withheld_code: None } | Self::UnknownMegolmMessageIndex
+        )
+    }
+}
+
+/// A machine-readable code for why a Megolm key was not sent.
+///
+/// Normally sent as the payload of an [`m.room_key.withheld`](https://spec.matrix.org/v1.12/client-server-api/#mroom_keywithheld) to-device message.
+#[derive(
+    Clone,
+    PartialEq,
+    Eq,
+    Hash,
+    AsStrAsRefStr,
+    AsRefStr,
+    FromString,
+    DebugAsRefStr,
+    SerializeAsRefStr,
+    DeserializeFromCowStr,
+)]
+pub enum WithheldCode {
+    /// the user/device was blacklisted.
+    #[ruma_enum(rename = "m.blacklisted")]
+    Blacklisted,
+
+    /// the user/devices is unverified.
+    #[ruma_enum(rename = "m.unverified")]
+    Unverified,
+
+    /// The user/device is not allowed have the key. For example, this would
+    /// usually be sent in response to a key request if the user was not in
+    /// the room when the message was sent.
+    #[ruma_enum(rename = "m.unauthorised")]
+    Unauthorised,
+
+    /// Sent in reply to a key request if the device that the key is requested
+    /// from does not have the requested key.
+    #[ruma_enum(rename = "m.unavailable")]
+    Unavailable,
+
+    /// An olm session could not be established.
+    /// This may happen, for example, if the sender was unable to obtain a
+    /// one-time key from the recipient.
+    #[ruma_enum(rename = "m.no_olm")]
+    NoOlm,
+
+    #[doc(hidden)]
+    _Custom(PrivOwnedStr),
+}
+
+impl fmt::Display for WithheldCode {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> Result<(), fmt::Error> {
+        let string = match self {
+            WithheldCode::Blacklisted => "The sender has blocked you.",
+            WithheldCode::Unverified => "The sender has disabled encrypting to unverified devices.",
+            WithheldCode::Unauthorised => "You are not authorised to read the message.",
+            WithheldCode::Unavailable => "The requested key was not found.",
+            WithheldCode::NoOlm => "Unable to establish a secure channel.",
+            _ => self.as_str(),
+        };
+
+        f.write_str(string)
+    }
+}
+
+// The Ruma macro expects the type to have this name.
+// The payload is counter intuitively made public in order to avoid having
+// multiple copies of this struct.
+#[doc(hidden)]
+#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
+pub struct PrivOwnedStr(pub Box<str>);
+
+#[cfg(not(tarpaulin_include))]
+impl fmt::Debug for PrivOwnedStr {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        self.0.fmt(f)
    }
 }

@@ -812,9 +921,9 @@ mod tests {
    use super::{
        AlgorithmInfo, DecryptedRoomEvent, EncryptionInfo, SyncTimelineEvent, TimelineEvent,
        TimelineEventKind, UnableToDecryptInfo, UnableToDecryptReason, UnsignedDecryptionResult,
-        UnsignedEventLocation, VerificationState,
+        UnsignedEventLocation, VerificationState, WithheldCode,
    };
-    use crate::deserialized_responses::{DeviceLinkProblem, VerificationLevel};
+    use crate::deserialized_responses::{DeviceLinkProblem, ShieldStateCode, VerificationLevel};

    fn example_event() -> serde_json::Value {
        json!({
@@ -889,6 +998,74 @@ mod tests {
        );
    }

+    #[test]
+    fn test_verification_level_deserializes() {
+        // Given a JSON VerificationLevel
+        #[derive(Deserialize)]
+        struct Container {
+            verification_level: VerificationLevel,
+        }
+        let container = json!({ "verification_level": "VerificationViolation" });
+
+        // When we deserialize it
+        let deserialized: Container = serde_json::from_value(container)
+            .expect("We can deserialize the old PreviouslyVerified value");
+
+        // Then it is populated correctly
+        assert_eq!(deserialized.verification_level, VerificationLevel::VerificationViolation);
+    }
+
+    #[test]
+    fn test_verification_level_deserializes_from_old_previously_verified_value() {
+        // Given a JSON VerificationLevel with the old value PreviouslyVerified
+        #[derive(Deserialize)]
+        struct Container {
+            verification_level: VerificationLevel,
+        }
+        let container = json!({ "verification_level": "PreviouslyVerified" });
+
+        // When we deserialize it
+        let deserialized: Container = serde_json::from_value(container)
+            .expect("We can deserialize the old PreviouslyVerified value");
+
+        // Then it is migrated to the new value
+        assert_eq!(deserialized.verification_level, VerificationLevel::VerificationViolation);
+    }
+
+    #[test]
+    fn test_shield_state_code_deserializes() {
+        // Given a JSON ShieldStateCode with value VerificationViolation
+        #[derive(Deserialize)]
+        struct Container {
+            shield_state_code: ShieldStateCode,
+        }
+        let container = json!({ "shield_state_code": "VerificationViolation" });
+
+        // When we deserialize it
+        let deserialized: Container = serde_json::from_value(container)
+            .expect("We can deserialize the old PreviouslyVerified value");
+
+        // Then it is populated correctly
+        assert_eq!(deserialized.shield_state_code, ShieldStateCode::VerificationViolation);
+    }
+
+    #[test]
+    fn test_shield_state_code_deserializes_from_old_previously_verified_value() {
+        // Given a JSON ShieldStateCode with the old value PreviouslyVerified
+        #[derive(Deserialize)]
+        struct Container {
+            shield_state_code: ShieldStateCode,
+        }
+        let container = json!({ "shield_state_code": "PreviouslyVerified" });
+
+        // When we deserialize it
+        let deserialized: Container = serde_json::from_value(container)
+            .expect("We can deserialize the old PreviouslyVerified value");
+
+        // Then it is migrated to the new value
+        assert_eq!(deserialized.shield_state_code, ShieldStateCode::VerificationViolation);
+    }
+
    #[test]
    fn sync_timeline_event_serialisation() {
        let room_event = SyncTimelineEvent {
@@ -1033,4 +1210,111 @@ mod tests {
            });
        });
    }
+
+    #[test]
+    fn sync_timeline_event_deserialisation_migration_for_withheld() {
+        // Old serialized version was
+        //    "utd_info": {
+        //         "reason": "MissingMegolmSession",
+        //         "session_id": "session000"
+        //       }
+
+        // The new version would be
+        //      "utd_info": {
+        //         "reason": {
+        //           "MissingMegolmSession": {
+        //              "withheld_code": null
+        //           }
+        //         },
+        //         "session_id": "session000"
+        //       }
+
+        let serialized = json!({
+             "kind": {
+                "UnableToDecrypt": {
+                  "event": {
+                    "content": {
+                      "algorithm": "m.megolm.v1.aes-sha2",
+                      "ciphertext": "AwgAEoABzL1JYhqhjW9jXrlT3M6H8mJ4qffYtOQOnPuAPNxsuG20oiD/Fnpv6jnQGhU6YbV9pNM+1mRnTvxW3CbWOPjLKqCWTJTc7Q0vDEVtYePg38ncXNcwMmfhgnNAoW9S7vNs8C003x3yUl6NeZ8bH+ci870BZL+kWM/lMl10tn6U7snNmSjnE3ckvRdO+11/R4//5VzFQpZdf4j036lNSls/WIiI67Fk9iFpinz9xdRVWJFVdrAiPFwb8L5xRZ8aX+e2JDMlc1eW8gk",
+                      "device_id": "SKCGPNUWAU",
+                      "sender_key": "Gim/c7uQdSXyrrUbmUOrBT6sMC0gO7QSLmOK6B7NOm0",
+                      "session_id": "hgLyeSqXfb8vc5AjQLsg6TSHVu0HJ7HZ4B6jgMvxkrs"
+                    },
+                    "event_id": "$xxxxx:example.org",
+                    "origin_server_ts": 2189,
+                    "room_id": "!someroom:example.com",
+                    "sender": "@carl:example.com",
+                    "type": "m.room.message"
+                  },
+                  "utd_info": {
+                    "reason": "MissingMegolmSession",
+                    "session_id": "session000"
+                  }
+                }
+              }
+        });
+
+        let result = serde_json::from_value(serialized);
+        assert!(result.is_ok());
+
+        // should have migrated to the new format
+        let event: SyncTimelineEvent = result.unwrap();
+        assert_matches!(
+            event.kind,
+            TimelineEventKind::UnableToDecrypt { utd_info, .. }=> {
+                assert_matches!(
+                    utd_info.reason,
+                    UnableToDecryptReason::MissingMegolmSession { withheld_code: None }
+                );
+            }
+        )
+    }
+
+    #[test]
+    fn unable_to_decrypt_info_migration_for_withheld() {
+        let old_format = json!({
+            "reason": "MissingMegolmSession",
+            "session_id": "session000"
+        });
+
+        let deserialized = serde_json::from_value::<UnableToDecryptInfo>(old_format).unwrap();
+        let session_id = Some("session000".to_owned());
+
+        assert_eq!(deserialized.session_id, session_id);
+        assert_eq!(
+            deserialized.reason,
+            UnableToDecryptReason::MissingMegolmSession { withheld_code: None },
+        );
+
+        let new_format = json!({
+             "session_id": "session000",
+              "reason": {
+                "MissingMegolmSession": {
+                  "withheld_code": null
+                }
+              }
+        });
+
+        let deserialized = serde_json::from_value::<UnableToDecryptInfo>(new_format).unwrap();
+
+        assert_eq!(
+            deserialized.reason,
+            UnableToDecryptReason::MissingMegolmSession { withheld_code: None },
+        );
+        assert_eq!(deserialized.session_id, session_id);
+    }
+
+    #[test]
+    fn unable_to_decrypt_reason_is_missing_room_key() {
+        let reason = UnableToDecryptReason::MissingMegolmSession { withheld_code: None };
+        assert!(reason.is_missing_room_key());
+
+        let reason = UnableToDecryptReason::MissingMegolmSession {
+            withheld_code: Some(WithheldCode::Blacklisted),
+        };
+        assert!(!reason.is_missing_room_key());
+
+        let reason = UnableToDecryptReason::UnknownMegolmMessageIndex;
+        assert!(reason.is_missing_room_key());
+    }
 }
@@ -306,7 +306,7 @@ impl<'a> JsFieldVisitor<'a> {
    }
 }

-impl<'a> tracing::field::Visit for JsFieldVisitor<'a> {
+impl tracing::field::Visit for JsFieldVisitor<'_> {
    fn record_debug(&mut self, field: &Field, value: &dyn Debug) {
        if self.result.is_err() {
            return;
@@ -21,7 +21,7 @@ use tracing::error;

 use super::{
    Chunk, ChunkContent, ChunkIdentifier, ChunkIdentifierGenerator, Ends, LinkedChunk,
-    ObservableUpdates,
+    ObservableUpdates, RawChunk,
 };

 /// A temporary chunk representation in the [`LinkedChunkBuilder`].
@@ -260,6 +260,22 @@ impl<const CAP: usize, Item, Gap> LinkedChunkBuilder<CAP, Item, Gap> {

        Ok(Some(LinkedChunk { links, chunk_identifier_generator, updates, marker: PhantomData }))
    }
+
+    /// Fills a linked chunk builder from all the given raw parts.
+    pub fn from_raw_parts(raws: Vec<RawChunk<Item, Gap>>) -> Self {
+        let mut this = Self::new();
+        for raw in raws {
+            match raw.content {
+                ChunkContent::Gap(gap) => {
+                    this.push_gap(raw.previous, raw.identifier, raw.next, gap);
+                }
+                ChunkContent::Items(vec) => {
+                    this.push_items(raw.previous, raw.identifier, raw.next, vec);
+                }
+            }
+        }
+        this
+    }
 }

 #[derive(thiserror::Error, Debug)]
@@ -356,7 +372,7 @@ mod tests {
        assert!(chunks.next().is_none());

        // The linked chunk had 5 items.
-        assert_eq!(lc.len(), 5);
+        assert_eq!(lc.num_items(), 5);

        // Now, if we add a new chunk, its identifier should be the previous one we used
        // + 1.
@@ -274,14 +274,23 @@ impl<const CAP: usize, Item, Gap> LinkedChunk<CAP, Item, Gap> {
    /// [`ObservableUpdates::take`] method must be called to consume and
    /// clean the updates. See [`Self::updates`].
    pub fn new_with_update_history() -> Self {
+        let first_chunk_identifier = ChunkIdentifierGenerator::FIRST_IDENTIFIER;
+
+        let mut updates = ObservableUpdates::new();
+        updates.push(Update::NewItemsChunk {
+            previous: None,
+            new: first_chunk_identifier,
+            next: None,
+        });
+
        Self {
            links: Ends {
                // INVARIANT: The first chunk must always be an Items, not a Gap.
-                first: Chunk::new_items_leaked(ChunkIdentifierGenerator::FIRST_IDENTIFIER),
+                first: Chunk::new_items_leaked(first_chunk_identifier),
                last: None,
            },
            chunk_identifier_generator: ChunkIdentifierGenerator::new_from_scratch(),
-            updates: Some(ObservableUpdates::new()),
+            updates: Some(updates),
            marker: PhantomData,
        }
    }
@@ -625,6 +634,47 @@ impl<const CAP: usize, Item, Gap> LinkedChunk<CAP, Item, Gap> {
        Ok(())
    }

+    /// Remove a gap with the given identifier.
+    ///
+    /// This returns the next insert position, viz. the start of the next
+    /// chunk, if any, or none if there was no next chunk.
+    pub fn remove_gap_at(
+        &mut self,
+        chunk_identifier: ChunkIdentifier,
+    ) -> Result<Option<Position>, Error> {
+        let chunk = self
+            .links
+            .chunk_mut(chunk_identifier)
+            .ok_or(Error::InvalidChunkIdentifier { identifier: chunk_identifier })?;
+
+        if chunk.is_items() {
+            return Err(Error::ChunkIsItems { identifier: chunk_identifier });
+        };
+
+        let next = chunk.next;
+
+        chunk.unlink(&mut self.updates);
+
+        let chunk_ptr = chunk.as_ptr();
+
+        // If this ever changes, we may need to update self.links.first too.
+        debug_assert!(chunk.is_first_chunk().not(), "A gap cannot be the first chunk");
+
+        if chunk.is_last_chunk() {
+            self.links.last = chunk.previous;
+        }
+
+        // SAFETY: `chunk` is unlinked and not borrowed anymore. `LinkedChunk` doesn't
+        // use it anymore, it's a leak. It is time to re-`Box` it and drop it.
+        let _chunk_boxed = unsafe { Box::from_raw(chunk_ptr.as_ptr()) };
+
+        // Return the first position of the next chunk, if any.
+        Ok(next.map(|next| {
+            let chunk = unsafe { next.as_ref() };
+            chunk.first_position()
+        }))
+    }
+
    /// Replace the gap identified by `chunk_identifier`, by items.
    ///
    /// Because the `chunk_identifier` can represent non-gap chunk, this method
@@ -652,27 +702,25 @@ impl<const CAP: usize, Item, Gap> LinkedChunk<CAP, Item, Gap> {
                .chunk_mut(chunk_identifier)
                .ok_or(Error::InvalidChunkIdentifier { identifier: chunk_identifier })?;

+            if chunk.is_items() {
+                return Err(Error::ChunkIsItems { identifier: chunk_identifier });
+            };
+
            debug_assert!(chunk.is_first_chunk().not(), "A gap cannot be the first chunk");

-            let maybe_last_chunk_ptr = match &mut chunk.content {
-                ChunkContent::Gap(..) => {
-                    let items = items.into_iter();
+            let maybe_last_chunk_ptr = {
+                let items = items.into_iter();

-                    let last_inserted_chunk = chunk
-                        // Insert a new items chunk…
-                        .insert_next(
-                            Chunk::new_items_leaked(self.chunk_identifier_generator.next()),
-                            &mut self.updates,
-                        )
-                        // … and insert the items.
-                        .push_items(items, &self.chunk_identifier_generator, &mut self.updates);
+                let last_inserted_chunk = chunk
+                    // Insert a new items chunk…
+                    .insert_next(
+                        Chunk::new_items_leaked(self.chunk_identifier_generator.next()),
+                        &mut self.updates,
+                    )
+                    // … and insert the items.
+                    .push_items(items, &self.chunk_identifier_generator, &mut self.updates);

-                    last_inserted_chunk.is_last_chunk().then(|| last_inserted_chunk.as_ptr())
-                }
-
-                ChunkContent::Items(..) => {
-                    return Err(Error::ChunkIsItems { identifier: chunk_identifier })
-                }
+                last_inserted_chunk.is_last_chunk().then(|| last_inserted_chunk.as_ptr())
            };

            new_chunk_ptr = chunk
@@ -807,8 +855,9 @@ impl<const CAP: usize, Item, Gap> LinkedChunk<CAP, Item, Gap> {
            .skip_while({
                let expected_index = position.index();

-                move |(Position(_chunk_identifier, item_index), _item)| {
-                    *item_index != expected_index
+                move |(Position(chunk_identifier, item_index), _item)| {
+                    *chunk_identifier == position.chunk_identifier()
+                        && *item_index != expected_index
                }
            }))
    }
@@ -870,7 +919,7 @@ impl<const CAP: usize, Item, Gap> LinkedChunk<CAP, Item, Gap> {
    }

    /// Returns the number of items of the linked chunk.
-    fn len(&self) -> usize {
+    pub fn num_items(&self) -> usize {
        self.items().count()
    }
 }
@@ -952,12 +1001,12 @@ pub struct ChunkIdentifier(u64);

 impl ChunkIdentifier {
    /// Create a new [`ChunkIdentifier`].
-    pub(super) fn new(identifier: u64) -> Self {
+    pub fn new(identifier: u64) -> Self {
        Self(identifier)
    }

    /// Get the underlying identifier.
-    fn index(&self) -> u64 {
+    pub fn index(&self) -> u64 {
        self.0
    }
 }
@@ -976,7 +1025,7 @@ pub struct Position(ChunkIdentifier, usize);

 impl Position {
    /// Create a new [`Position`].
-    pub(super) fn new(chunk_identifier: ChunkIdentifier, index: usize) -> Self {
+    pub fn new(chunk_identifier: ChunkIdentifier, index: usize) -> Self {
        Self(chunk_identifier, index)
    }

@@ -1055,7 +1104,7 @@ impl<'a, const CAP: usize, Item, Gap> Iterator for Iter<'a, CAP, Item, Gap> {
 }

 /// This enum represents the content of a [`Chunk`].
-#[derive(Debug)]
+#[derive(Clone, Debug)]
 pub enum ChunkContent<Item, Gap> {
    /// The chunk represents a gap in the linked chunk, i.e. a hole. It
    /// means that some items are missing in this location.
@@ -1410,6 +1459,26 @@ impl EmptyChunk {
    }
 }

+/// The raw representation of a linked chunk, as persisted in storage.
+///
+/// It may rebuilt into [`Chunk`] and shares the same internal representation,
+/// except that links are materialized using [`ChunkIdentifier`] instead of raw
+/// pointers to the previous and next chunks.
+#[derive(Clone, Debug)]
+pub struct RawChunk<Item, Gap> {
+    /// Content section of the linked chunk.
+    pub content: ChunkContent<Item, Gap>,
+
+    /// Link to the previous chunk, via its identifier.
+    pub previous: Option<ChunkIdentifier>,
+
+    /// Current chunk's identifier.
+    pub identifier: ChunkIdentifier,
+
+    /// Link to the next chunk, via its identifier.
+    pub next: Option<ChunkIdentifier>,
+}
+
 #[cfg(test)]
 mod tests {
    use std::{
@@ -1446,7 +1515,7 @@ mod tests {
    fn test_empty() {
        let items = LinkedChunk::<3, char, ()>::new();

-        assert_eq!(items.len(), 0);
+        assert_eq!(items.num_items(), 0);

        // This test also ensures that `Drop` for `LinkedChunk` works when
        // there is only one chunk.
@@ -1458,11 +1527,27 @@ mod tests {
        assert!(LinkedChunk::<3, char, ()>::new_with_update_history().updates().is_some());
    }

+    #[test]
+    fn test_new_with_initial_update() {
+        use super::Update::*;
+
+        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        assert_eq!(
+            linked_chunk.updates().unwrap().take(),
+            &[NewItemsChunk { previous: None, new: ChunkIdentifier(0), next: None }]
+        );
+    }
+
    #[test]
    fn test_push_items() {
        use super::Update::*;

        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        // Ignore initial update.
+        let _ = linked_chunk.updates().unwrap().take();
+
        linked_chunk.push_items_back(['a']);

        assert_items_eq!(linked_chunk, ['a']);
@@ -1513,7 +1598,7 @@ mod tests {
            ]
        );

-        assert_eq!(linked_chunk.len(), 10);
+        assert_eq!(linked_chunk.num_items(), 10);
    }

    #[test]
@@ -1521,6 +1606,10 @@ mod tests {
        use super::Update::*;

        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        // Ignore initial update.
+        let _ = linked_chunk.updates().unwrap().take();
+
        linked_chunk.push_items_back(['a']);
        assert_items_eq!(linked_chunk, ['a']);
        assert_eq!(
@@ -1601,7 +1690,7 @@ mod tests {
            ]
        );

-        assert_eq!(linked_chunk.len(), 9);
+        assert_eq!(linked_chunk.num_items(), 9);
    }

    #[test]
@@ -1764,6 +1853,26 @@ mod tests {
        assert_matches!(iterator.next(), None);
    }

+    #[test]
+    fn test_ritems_with_final_gap() -> Result<(), Error> {
+        let mut linked_chunk = LinkedChunk::<3, char, ()>::new();
+        linked_chunk.push_items_back(['a', 'b']);
+        linked_chunk.push_gap_back(());
+        linked_chunk.push_items_back(['c', 'd', 'e']);
+        linked_chunk.push_gap_back(());
+
+        let mut iterator = linked_chunk.ritems();
+
+        assert_matches!(iterator.next(), Some((Position(ChunkIdentifier(2), 2), 'e')));
+        assert_matches!(iterator.next(), Some((Position(ChunkIdentifier(2), 1), 'd')));
+        assert_matches!(iterator.next(), Some((Position(ChunkIdentifier(2), 0), 'c')));
+        assert_matches!(iterator.next(), Some((Position(ChunkIdentifier(0), 1), 'b')));
+        assert_matches!(iterator.next(), Some((Position(ChunkIdentifier(0), 0), 'a')));
+        assert_matches!(iterator.next(), None);
+
+        Ok(())
+    }
+
    #[test]
    fn test_ritems_empty() {
        let linked_chunk = LinkedChunk::<2, char, ()>::new();
@@ -1838,6 +1947,10 @@ mod tests {
        use super::Update::*;

        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        // Ignore initial update.
+        let _ = linked_chunk.updates().unwrap().take();
+
        linked_chunk.push_items_back(['a', 'b', 'c', 'd', 'e', 'f']);
        assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['d', 'e', 'f']);
        assert_eq!(
@@ -1865,7 +1978,7 @@ mod tests {
                linked_chunk,
                ['a', 'b', 'c'] ['d', 'w', 'x'] ['y', 'z', 'e'] ['f']
            );
-            assert_eq!(linked_chunk.len(), 10);
+            assert_eq!(linked_chunk.num_items(), 10);
            assert_eq!(
                linked_chunk.updates().unwrap().take(),
                &[
@@ -1899,7 +2012,7 @@ mod tests {
                linked_chunk,
                ['l', 'm', 'n'] ['o', 'a', 'b'] ['c'] ['d', 'w', 'x'] ['y', 'z', 'e'] ['f']
            );
-            assert_eq!(linked_chunk.len(), 14);
+            assert_eq!(linked_chunk.num_items(), 14);
            assert_eq!(
                linked_chunk.updates().unwrap().take(),
                &[
@@ -1933,7 +2046,7 @@ mod tests {
                linked_chunk,
                ['l', 'm', 'n'] ['o', 'a', 'b'] ['r', 's', 'c'] ['d', 'w', 'x'] ['y', 'z', 'e'] ['f']
            );
-            assert_eq!(linked_chunk.len(), 16);
+            assert_eq!(linked_chunk.num_items(), 16);
            assert_eq!(
                linked_chunk.updates().unwrap().take(),
                &[
@@ -1961,7 +2074,7 @@ mod tests {
                linked_chunk.updates().unwrap().take(),
                &[PushItems { at: Position(ChunkIdentifier(3), 1), items: vec!['p', 'q'] }]
            );
-            assert_eq!(linked_chunk.len(), 18);
+            assert_eq!(linked_chunk.num_items(), 18);
        }

        // Insert in a chunk that does not exist.
@@ -2006,7 +2119,7 @@ mod tests {
            );
        }

-        assert_eq!(linked_chunk.len(), 18);
+        assert_eq!(linked_chunk.num_items(), 18);

        Ok(())
    }
@@ -2016,9 +2129,13 @@ mod tests {
        use super::Update::*;

        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        // Ignore initial update.
+        let _ = linked_chunk.updates().unwrap().take();
+
        linked_chunk.push_items_back(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k']);
        assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['d', 'e', 'f'] ['g', 'h', 'i'] ['j', 'k']);
-        assert_eq!(linked_chunk.len(), 11);
+        assert_eq!(linked_chunk.num_items(), 11);

        // Ignore previous updates.
        let _ = linked_chunk.updates().unwrap().take();
@@ -2031,21 +2148,21 @@ mod tests {

            assert_eq!(removed_item, 'f');
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['d', 'e'] ['g', 'h', 'i'] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 10);
+            assert_eq!(linked_chunk.num_items(), 10);

            let position_of_e = linked_chunk.item_position(|item| *item == 'e').unwrap();
            let removed_item = linked_chunk.remove_item_at(position_of_e, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'e');
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['d'] ['g', 'h', 'i'] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 9);
+            assert_eq!(linked_chunk.num_items(), 9);

            let position_of_d = linked_chunk.item_position(|item| *item == 'd').unwrap();
            let removed_item = linked_chunk.remove_item_at(position_of_d, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'd');
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['g', 'h', 'i'] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 8);
+            assert_eq!(linked_chunk.num_items(), 8);

            assert_eq!(
                linked_chunk.updates().unwrap().take(),
@@ -2066,19 +2183,19 @@ mod tests {

            assert_eq!(removed_item, 'a');
            assert_items_eq!(linked_chunk, ['b', 'c'] ['g', 'h', 'i'] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 7);
+            assert_eq!(linked_chunk.num_items(), 7);

            let removed_item = linked_chunk.remove_item_at(first_position, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'b');
            assert_items_eq!(linked_chunk, ['c'] ['g', 'h', 'i'] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 6);
+            assert_eq!(linked_chunk.num_items(), 6);

            let removed_item = linked_chunk.remove_item_at(first_position, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'c');
            assert_items_eq!(linked_chunk, [] ['g', 'h', 'i'] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 5);
+            assert_eq!(linked_chunk.num_items(), 5);

            assert_eq!(
                linked_chunk.updates().unwrap().take(),
@@ -2098,19 +2215,19 @@ mod tests {

            assert_eq!(removed_item, 'g');
            assert_items_eq!(linked_chunk, [] ['h', 'i'] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 4);
+            assert_eq!(linked_chunk.num_items(), 4);

            let removed_item = linked_chunk.remove_item_at(first_position, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'h');
            assert_items_eq!(linked_chunk, [] ['i'] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 3);
+            assert_eq!(linked_chunk.num_items(), 3);

            let removed_item = linked_chunk.remove_item_at(first_position, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'i');
            assert_items_eq!(linked_chunk, [] ['j', 'k']);
-            assert_eq!(linked_chunk.len(), 2);
+            assert_eq!(linked_chunk.num_items(), 2);

            assert_eq!(
                linked_chunk.updates().unwrap().take(),
@@ -2132,14 +2249,14 @@ mod tests {
            assert_eq!(removed_item, 'k');
            #[rustfmt::skip]
            assert_items_eq!(linked_chunk, [] ['j']);
-            assert_eq!(linked_chunk.len(), 1);
+            assert_eq!(linked_chunk.num_items(), 1);

            let position_of_j = linked_chunk.item_position(|item| *item == 'j').unwrap();
            let removed_item = linked_chunk.remove_item_at(position_of_j, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'j');
            assert_items_eq!(linked_chunk, []);
-            assert_eq!(linked_chunk.len(), 0);
+            assert_eq!(linked_chunk.num_items(), 0);

            assert_eq!(
                linked_chunk.updates().unwrap().take(),
@@ -2157,13 +2274,13 @@ mod tests {

            #[rustfmt::skip]
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['d']);
-            assert_eq!(linked_chunk.len(), 4);
+            assert_eq!(linked_chunk.num_items(), 4);

            let position_of_c = linked_chunk.item_position(|item| *item == 'c').unwrap();
            linked_chunk.insert_gap_at((), position_of_c)?;

            assert_items_eq!(linked_chunk, ['a', 'b'] [-] ['c'] ['d']);
-            assert_eq!(linked_chunk.len(), 4);
+            assert_eq!(linked_chunk.num_items(), 4);

            // Ignore updates.
            let _ = linked_chunk.updates().unwrap().take();
@@ -2173,27 +2290,27 @@ mod tests {

            assert_eq!(removed_item, 'c');
            assert_items_eq!(linked_chunk, ['a', 'b'] [-] ['d']);
-            assert_eq!(linked_chunk.len(), 3);
+            assert_eq!(linked_chunk.num_items(), 3);

            let position_of_d = linked_chunk.item_position(|item| *item == 'd').unwrap();
            let removed_item = linked_chunk.remove_item_at(position_of_d, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'd');
            assert_items_eq!(linked_chunk, ['a', 'b'] [-]);
-            assert_eq!(linked_chunk.len(), 2);
+            assert_eq!(linked_chunk.num_items(), 2);

            let first_position = linked_chunk.item_position(|item| *item == 'a').unwrap();
            let removed_item = linked_chunk.remove_item_at(first_position, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'a');
            assert_items_eq!(linked_chunk, ['b'] [-]);
-            assert_eq!(linked_chunk.len(), 1);
+            assert_eq!(linked_chunk.num_items(), 1);

            let removed_item = linked_chunk.remove_item_at(first_position, EmptyChunk::Remove)?;

            assert_eq!(removed_item, 'b');
            assert_items_eq!(linked_chunk, [] [-]);
-            assert_eq!(linked_chunk.len(), 0);
+            assert_eq!(linked_chunk.num_items(), 0);

            assert_eq!(
                linked_chunk.updates().unwrap().take(),
@@ -2216,9 +2333,13 @@ mod tests {
        use super::Update::*;

        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        // Ignore initial update.
+        let _ = linked_chunk.updates().unwrap().take();
+
        linked_chunk.push_items_back(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']);
        assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['d', 'e', 'f'] ['g', 'h']);
-        assert_eq!(linked_chunk.len(), 8);
+        assert_eq!(linked_chunk.num_items(), 8);

        // Ignore previous updates.
        let _ = linked_chunk.updates().unwrap().take();
@@ -2231,19 +2352,19 @@ mod tests {

            assert_eq!(removed_item, 'd');
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['e', 'f'] ['g', 'h']);
-            assert_eq!(linked_chunk.len(), 7);
+            assert_eq!(linked_chunk.num_items(), 7);

            let removed_item = linked_chunk.remove_item_at(position, EmptyChunk::Keep)?;

            assert_eq!(removed_item, 'e');
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['f'] ['g', 'h']);
-            assert_eq!(linked_chunk.len(), 6);
+            assert_eq!(linked_chunk.num_items(), 6);

            let removed_item = linked_chunk.remove_item_at(position, EmptyChunk::Keep)?;

            assert_eq!(removed_item, 'f');
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] [] ['g', 'h']);
-            assert_eq!(linked_chunk.len(), 5);
+            assert_eq!(linked_chunk.num_items(), 5);

            assert_eq!(
                linked_chunk.updates().unwrap().take(),
@@ -2263,13 +2384,13 @@ mod tests {

            assert_eq!(removed_item, 'g');
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] [] ['h']);
-            assert_eq!(linked_chunk.len(), 4);
+            assert_eq!(linked_chunk.num_items(), 4);

            let removed_item = linked_chunk.remove_item_at(position, EmptyChunk::Keep)?;

            assert_eq!(removed_item, 'h');
            assert_items_eq!(linked_chunk, ['a', 'b', 'c'] [] []);
-            assert_eq!(linked_chunk.len(), 3);
+            assert_eq!(linked_chunk.num_items(), 3);

            assert_eq!(
                linked_chunk.updates().unwrap().take(),
@@ -2288,19 +2409,19 @@ mod tests {

            assert_eq!(removed_item, 'a');
            assert_items_eq!(linked_chunk, ['b', 'c'] [] []);
-            assert_eq!(linked_chunk.len(), 2);
+            assert_eq!(linked_chunk.num_items(), 2);

            let removed_item = linked_chunk.remove_item_at(position, EmptyChunk::Keep)?;

            assert_eq!(removed_item, 'b');
            assert_items_eq!(linked_chunk, ['c'] [] []);
-            assert_eq!(linked_chunk.len(), 1);
+            assert_eq!(linked_chunk.num_items(), 1);

            let removed_item = linked_chunk.remove_item_at(position, EmptyChunk::Keep)?;

            assert_eq!(removed_item, 'c');
            assert_items_eq!(linked_chunk, [] [] []);
-            assert_eq!(linked_chunk.len(), 0);
+            assert_eq!(linked_chunk.num_items(), 0);

            assert_eq!(
                linked_chunk.updates().unwrap().take(),
@@ -2320,6 +2441,10 @@ mod tests {
        use super::Update::*;

        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        // Ignore initial update.
+        let _ = linked_chunk.updates().unwrap().take();
+
        linked_chunk.push_items_back(['a', 'b', 'c', 'd', 'e', 'f']);
        assert_items_eq!(linked_chunk, ['a', 'b', 'c'] ['d', 'e', 'f']);
        assert_eq!(
@@ -2485,7 +2610,7 @@ mod tests {
            assert!(linked_chunk.updates().unwrap().take().is_empty());
        }

-        assert_eq!(linked_chunk.len(), 6);
+        assert_eq!(linked_chunk.num_items(), 6);

        Ok(())
    }
@@ -2495,6 +2620,10 @@ mod tests {
        use super::Update::*;

        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        // Ignore initial update.
+        let _ = linked_chunk.updates().unwrap().take();
+
        linked_chunk.push_items_back(['a', 'b']);
        linked_chunk.push_gap_back(());
        linked_chunk.push_items_back(['l', 'm']);
@@ -2596,7 +2725,73 @@ mod tests {
            );
        }

-        assert_eq!(linked_chunk.len(), 13);
+        assert_eq!(linked_chunk.num_items(), 13);
+
+        Ok(())
+    }
+
+    #[test]
+    fn test_remove_gap() -> Result<(), Error> {
+        use super::Update::*;
+
+        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        // Ignore initial update.
+        let _ = linked_chunk.updates().unwrap().take();
+
+        linked_chunk.push_items_back(['a', 'b']);
+        linked_chunk.push_gap_back(());
+        linked_chunk.push_items_back(['l', 'm']);
+        linked_chunk.push_gap_back(());
+        assert_items_eq!(linked_chunk, ['a', 'b'] [-] ['l', 'm'] [-]);
+        assert_eq!(
+            linked_chunk.updates().unwrap().take(),
+            &[
+                PushItems { at: Position(ChunkIdentifier(0), 0), items: vec!['a', 'b'] },
+                NewGapChunk {
+                    previous: Some(ChunkIdentifier(0)),
+                    new: ChunkIdentifier(1),
+                    next: None,
+                    gap: (),
+                },
+                NewItemsChunk {
+                    previous: Some(ChunkIdentifier(1)),
+                    new: ChunkIdentifier(2),
+                    next: None,
+                },
+                PushItems { at: Position(ChunkIdentifier(2), 0), items: vec!['l', 'm'] },
+                NewGapChunk {
+                    previous: Some(ChunkIdentifier(2)),
+                    new: ChunkIdentifier(3),
+                    next: None,
+                    gap: (),
+                },
+            ]
+        );
+
+        // Try to remove a gap that's not a gap.
+        let err = linked_chunk.remove_gap_at(ChunkIdentifier(0)).unwrap_err();
+        assert_matches!(err, Error::ChunkIsItems { .. });
+
+        // Try to remove an unknown gap chunk.
+        let err = linked_chunk.remove_gap_at(ChunkIdentifier(42)).unwrap_err();
+        assert_matches!(err, Error::InvalidChunkIdentifier { .. });
+
+        // Remove the gap in the middle.
+        let maybe_next = linked_chunk.remove_gap_at(ChunkIdentifier(1)).unwrap();
+        let next = maybe_next.unwrap();
+        // The next insert position at the start of the next chunk.
+        assert_eq!(next.chunk_identifier(), ChunkIdentifier(2));
+        assert_eq!(next.index(), 0);
+        assert_items_eq!(linked_chunk, ['a', 'b'] ['l', 'm'] [-]);
+        assert_eq!(linked_chunk.updates().unwrap().take(), &[RemoveChunk(ChunkIdentifier(1))]);
+
+        // Remove the gap at the end.
+        let next = linked_chunk.remove_gap_at(ChunkIdentifier(3)).unwrap();
+        // It was the last chunk, so there's no next insert position.
+        assert!(next.is_none());
+        assert_items_eq!(linked_chunk, ['a', 'b'] ['l', 'm']);
+        assert_eq!(linked_chunk.updates().unwrap().take(), &[RemoveChunk(ChunkIdentifier(3))]);

        Ok(())
    }
@@ -2683,7 +2878,7 @@ mod tests {

        assert_eq!(Arc::strong_count(&item), 7);
        assert_eq!(Arc::strong_count(&gap), 2);
-        assert_eq!(linked_chunk.len(), 6);
+        assert_eq!(linked_chunk.num_items(), 6);
        assert_eq!(linked_chunk.chunk_identifier_generator.next.load(Ordering::SeqCst), 3);

        // Now, we can clear the linked chunk and see what happens.
@@ -2691,7 +2886,7 @@ mod tests {

        assert_eq!(Arc::strong_count(&item), 1);
        assert_eq!(Arc::strong_count(&gap), 1);
-        assert_eq!(linked_chunk.len(), 0);
+        assert_eq!(linked_chunk.num_items(), 0);
        assert_eq!(linked_chunk.chunk_identifier_generator.next.load(Ordering::SeqCst), 0);
    }

@@ -2700,6 +2895,16 @@ mod tests {
        use super::Update::*;

        let mut linked_chunk = LinkedChunk::<3, char, ()>::new_with_update_history();
+
+        assert_eq!(
+            linked_chunk.updates().unwrap().take(),
+            &[NewItemsChunk {
+                previous: None,
+                new: ChunkIdentifierGenerator::FIRST_IDENTIFIER,
+                next: None
+            }]
+        );
+
        linked_chunk.clear();

        assert_eq!(
@@ -17,6 +17,7 @@

 use ruma::{OwnedRoomId, RoomId};

+use super::{ChunkContent, RawChunk};
 use crate::linked_chunk::{ChunkIdentifier, Position, Update};

 /// A row of the [`RelationalLinkedChunk::chunks`].
@@ -79,6 +80,12 @@ impl<Item, Gap> RelationalLinkedChunk<Item, Gap> {
        Self { chunks: Vec::new(), items: Vec::new() }
    }

+    /// Removes all the chunks and items from this relational linked chunk.
+    pub fn clear(&mut self) {
+        self.chunks.clear();
+        self.items.clear();
+    }
+
    /// Apply [`Update`]s. That's the only way to write data inside this
    /// relational linked chunk.
    pub fn apply_updates(&mut self, room_id: &RoomId, updates: Vec<Update<Item, Gap>>) {
@@ -274,6 +281,97 @@ impl<Item, Gap> RelationalLinkedChunk<Item, Gap> {
    }
 }

+impl<Item, Gap> RelationalLinkedChunk<Item, Gap>
+where
+    Gap: Clone,
+    Item: Clone,
+{
+    /// Reloads the chunks.
+    ///
+    /// Return an error result if the data was malformed in the struct, with a
+    /// string message explaining details about the error.
+    pub fn reload_chunks(&self, room_id: &RoomId) -> Result<Vec<RawChunk<Item, Gap>>, String> {
+        let mut result = Vec::new();
+
+        for chunk_row in self.chunks.iter().filter(|chunk| chunk.room_id == room_id) {
+            // Find all items that correspond to the chunk.
+            let mut items = self
+                .items
+                .iter()
+                .filter(|row| {
+                    row.room_id == room_id && row.position.chunk_identifier() == chunk_row.chunk
+                })
+                .peekable();
+
+            // Look at the first chunk item type, to reconstruct the chunk at hand.
+            let Some(first) = items.peek() else {
+                // The only possibility is that we created an empty items chunk; mark it as
+                // such, and continue.
+                result.push(RawChunk {
+                    content: ChunkContent::Items(Vec::new()),
+                    previous: chunk_row.previous_chunk,
+                    identifier: chunk_row.chunk,
+                    next: chunk_row.next_chunk,
+                });
+                continue;
+            };
+
+            match &first.item {
+                Either::Item(_) => {
+                    // Collect all the related items.
+                    let mut collected_items = Vec::new();
+                    for row in items {
+                        match &row.item {
+                            Either::Item(item) => {
+                                collected_items.push((item.clone(), row.position.index()))
+                            }
+                            Either::Gap(_) => {
+                                return Err(format!(
+                                    "unexpected gap in items chunk {}",
+                                    chunk_row.chunk.index()
+                                ));
+                            }
+                        }
+                    }
+
+                    // Sort them by their position.
+                    collected_items.sort_unstable_by_key(|(_item, index)| *index);
+
+                    result.push(RawChunk {
+                        content: ChunkContent::Items(
+                            collected_items.into_iter().map(|(item, _index)| item).collect(),
+                        ),
+                        previous: chunk_row.previous_chunk,
+                        identifier: chunk_row.chunk,
+                        next: chunk_row.next_chunk,
+                    });
+                }
+
+                Either::Gap(gap) => {
+                    assert!(items.next().is_some(), "we just peeked the gap");
+
+                    // We shouldn't have more than one item row for this chunk.
+                    if items.next().is_some() {
+                        return Err(format!(
+                            "there shouldn't be more than one item row attached in gap chunk {}",
+                            chunk_row.chunk.index()
+                        ));
+                    }
+
+                    result.push(RawChunk {
+                        content: ChunkContent::Gap(gap.clone()),
+                        previous: chunk_row.previous_chunk,
+                        identifier: chunk_row.chunk,
+                        next: chunk_row.next_chunk,
+                    });
+                }
+            }
+        }
+
+        Ok(result)
+    }
+}
+
 impl<Item, Gap> Default for RelationalLinkedChunk<Item, Gap> {
    fn default() -> Self {
        Self::new()
@@ -285,6 +383,7 @@ mod tests {
    use ruma::room_id;

    use super::{ChunkIdentifier as CId, *};
+    use crate::linked_chunk::LinkedChunkBuilder;

    #[test]
    fn test_new_items_chunk() {
@@ -728,4 +827,72 @@ mod tests {
        assert!(relational_linked_chunk.chunks.is_empty());
        assert!(relational_linked_chunk.items.is_empty());
    }
+
+    #[test]
+    fn test_reload_empty_linked_chunk() {
+        let room_id = room_id!("!r0:matrix.org");
+
+        // When I reload the linked chunk components from an empty store,
+        let relational_linked_chunk = RelationalLinkedChunk::<char, char>::new();
+        let result = relational_linked_chunk.reload_chunks(room_id).unwrap();
+        assert!(result.is_empty());
+    }
+
+    #[test]
+    fn test_reload_linked_chunk_with_empty_items() {
+        let room_id = room_id!("!r0:matrix.org");
+
+        let mut relational_linked_chunk = RelationalLinkedChunk::<char, char>::new();
+
+        // When I store an empty items chunks,
+        relational_linked_chunk.apply_updates(
+            room_id,
+            vec![Update::NewItemsChunk { previous: None, new: CId::new(0), next: None }],
+        );
+
+        // It correctly gets reloaded as such.
+        let raws = relational_linked_chunk.reload_chunks(room_id).unwrap();
+        let lc = LinkedChunkBuilder::<3, _, _>::from_raw_parts(raws)
+            .build()
+            .expect("building succeeds")
+            .expect("this leads to a non-empty linked chunk");
+
+        assert_items_eq!(lc, []);
+    }
+
+    #[test]
+    fn test_rebuild_linked_chunk() {
+        let room_id = room_id!("!r0:matrix.org");
+        let mut relational_linked_chunk = RelationalLinkedChunk::<char, char>::new();
+
+        relational_linked_chunk.apply_updates(
+            room_id,
+            vec![
+                // new chunk
+                Update::NewItemsChunk { previous: None, new: CId::new(0), next: None },
+                // new items on 0
+                Update::PushItems { at: Position::new(CId::new(0), 0), items: vec!['a', 'b', 'c'] },
+                // a gap chunk
+                Update::NewGapChunk {
+                    previous: Some(CId::new(0)),
+                    new: CId::new(1),
+                    next: None,
+                    gap: 'g',
+                },
+                // another items chunk
+                Update::NewItemsChunk { previous: Some(CId::new(1)), new: CId::new(2), next: None },
+                // new items on 0
+                Update::PushItems { at: Position::new(CId::new(2), 0), items: vec!['d', 'e', 'f'] },
+            ],
+        );
+
+        let raws = relational_linked_chunk.reload_chunks(room_id).unwrap();
+        let lc = LinkedChunkBuilder::<3, _, _>::from_raw_parts(raws)
+            .build()
+            .expect("building succeeds")
+            .expect("this leads to a non-empty linked chunk");
+
+        // The linked chunk is correctly reloaded.
+        assert_items_eq!(lc, ['a', 'b', 'c'] [-] ['d', 'e', 'f']);
+    }
 }
@@ -129,7 +129,7 @@ impl<Item, Gap> ObservableUpdates<Item, Gap> {
    /// Take new updates.
    ///
    /// Updates that have been taken will not be read again.
-    pub(super) fn take(&mut self) -> Vec<Update<Item, Gap>>
+    pub fn take(&mut self) -> Vec<Update<Item, Gap>>
    where
        Item: Clone,
        Gap: Clone,
@@ -384,7 +384,21 @@ mod tests {
            other_token
        };

-        // There is no new update yet.
+        // There is an initial update.
+        {
+            let updates = linked_chunk.updates().unwrap();
+
+            assert_eq!(
+                updates.take(),
+                &[NewItemsChunk { previous: None, new: ChunkIdentifier(0), next: None }],
+            );
+            assert_eq!(
+                updates.inner.write().unwrap().take_with_token(other_token),
+                &[NewItemsChunk { previous: None, new: ChunkIdentifier(0), next: None }],
+            );
+        }
+
+        // No new update.
        {
            let updates = linked_chunk.updates().unwrap();

@@ -617,7 +631,16 @@ mod tests {
        let updates_subscriber = linked_chunk.updates().unwrap().subscribe();
        pin_mut!(updates_subscriber);

-        // No update, stream is pending.
+        // Initial update, stream is ready.
+        assert_matches!(
+            updates_subscriber.as_mut().poll_next(&mut context),
+            Poll::Ready(Some(items)) => {
+                assert_eq!(
+                    items,
+                    &[NewItemsChunk { previous: None, new: ChunkIdentifier(0), next: None }]
+                );
+            }
+        );
        assert_matches!(updates_subscriber.as_mut().poll_next(&mut context), Poll::Pending);
        assert_eq!(*counter_waker.number_of_wakeup.lock().unwrap(), 0);

@@ -651,6 +674,7 @@ mod tests {
        assert_eq!(
            linked_chunk.updates().unwrap().take(),
            &[
+                NewItemsChunk { previous: None, new: ChunkIdentifier(0), next: None },
                PushItems { at: Position(ChunkIdentifier(0), 0), items: vec!['a'] },
                PushItems { at: Position(ChunkIdentifier(0), 1), items: vec!['b'] },
                PushItems { at: Position(ChunkIdentifier(0), 2), items: vec!['c'] },
@@ -701,9 +725,28 @@ mod tests {
            let updates_subscriber2 = linked_chunk.updates().unwrap().subscribe();
            pin_mut!(updates_subscriber2);

-            // No update, streams are pending.
+            // Initial updates, streams are ready.
+            assert_matches!(
+                updates_subscriber1.as_mut().poll_next(&mut context1),
+                Poll::Ready(Some(items)) => {
+                    assert_eq!(
+                        items,
+                        &[NewItemsChunk { previous: None, new: ChunkIdentifier(0), next: None }]
+                    );
+                }
+            );
            assert_matches!(updates_subscriber1.as_mut().poll_next(&mut context1), Poll::Pending);
            assert_eq!(*counter_waker1.number_of_wakeup.lock().unwrap(), 0);
+
+            assert_matches!(
+                updates_subscriber2.as_mut().poll_next(&mut context2),
+                Poll::Ready(Some(items)) => {
+                    assert_eq!(
+                        items,
+                        &[NewItemsChunk { previous: None, new: ChunkIdentifier(0), next: None }]
+                    );
+                }
+            );
            assert_matches!(updates_subscriber2.as_mut().poll_next(&mut context2), Poll::Pending);
            assert_eq!(*counter_waker2.number_of_wakeup.lock().unwrap(), 0);

@@ -2,6 +2,35 @@

 All notable changes to this project will be documented in this file.

+<!-- next-header -->
+
+## [Unreleased] - ReleaseDate
+
+## [0.9.0] - 2024-12-18
+
+- Expose new API `DehydratedDevices::get_dehydrated_device_pickle_key`, `DehydratedDevices::save_dehydrated_device_pickle_key`
+  and `DehydratedDevices::delete_dehydrated_device_pickle_key` to store/load the dehydrated device pickle key.
+  This allows client to automatically rotate the dehydrated device to avoid one-time-keys exhaustion and to_device accumulation.
+  [**breaking**] `DehydratedDevices::keys_for_upload` and `DehydratedDevices::rehydrate` now use the `DehydratedDeviceKey`
+  as parameter instead of a raw byte array. Use `DehydratedDeviceKey::from_bytes` to migrate.
+  ([#4383](https://github.com/matrix-org/matrix-rust-sdk/pull/4383))
+
+- Add extra logging in `OtherUserIdentity::pin_current_master_key` and
+  `OtherUserIdentity::withdraw_verification`.
+  ([#4415](https://github.com/matrix-org/matrix-rust-sdk/pull/4415))
+
+- Added new `UtdCause` variants `WithheldForUnverifiedOrInsecureDevice` and `WithheldBySender`.
+  These variants provide clearer categorization for expected Unable-To-Decrypt (UTD) errors
+  when the sender either did not wish to share or was unable to share the room_key.
+  ([#4305](https://github.com/matrix-org/matrix-rust-sdk/pull/4305))
+
+- `UtdCause` has two new variants that replace the existing `HistoricalMessage`:
+  `HistoricalMessageAndBackupIsDisabled` and `HistoricalMessageAndDeviceIsUnverified`.
+  These give more detail about what went wrong and allow us to suggest to users
+  what actions they can take to fix the problem. See the doc comments on these
+  variants for suggested wording.
+  ([#4384](https://github.com/matrix-org/matrix-rust-sdk/pull/4384))
+
 ## [0.8.0] - 2024-11-19

 ### Features
@@ -66,6 +95,12 @@ All notable changes to this project will be documented in this file.

 ### Refactor

+- Fix [#4424](https://github.com/matrix-org/matrix-rust-sdk/issues/4424) Failed
+  storage upgrade for "PreviouslyVerifiedButNoLonger". This bug caused errors to
+  occur when loading crypto information from storage, which typically prevented
+  apps from starting correctly.
+  ([#4430](https://github.com/matrix-org/matrix-rust-sdk/pull/4430))
+
 - Add new method `OlmMachine::try_decrypt_room_event`.
  ([#4116](https://github.com/matrix-org/matrix-rust-sdk/pull/4116))

@@ -9,7 +9,7 @@ name = "matrix-sdk-crypto"
 readme = "README.md"
 repository = "https://github.com/matrix-org/matrix-rust-sdk"
 rust-version = { workspace = true }
-version = "0.8.0"
+version = "0.9.0"

 [package.metadata.docs.rs]
 rustdoc-args = ["--cfg", "docsrs"]
@@ -35,39 +35,39 @@ message-ids = []
 testing = ["matrix-sdk-test"]

 [dependencies]
-aes = "0.8.1"
+aes = "0.8.4"
 aquamarine = { workspace = true }
 as_variant = { workspace = true }
 async-trait = { workspace = true }
-bs58 = { version = "0.5.0" }
+bs58 = { version = "0.5.1" }
 byteorder = { workspace = true }
 cfg-if = "1.0"
-ctr = "0.9.1"
+ctr = "0.9.2"
 eyeball = { workspace = true }
 futures-core = { workspace = true }
 futures-util = { workspace = true }
-hkdf = "0.12.3"
-hmac = "0.12.1"
+hkdf = { workspace = true }
+hmac = { workspace = true }
 itertools = { workspace = true }
 js_option = "0.1.1"
 matrix-sdk-qrcode = { workspace = true, optional = true }
 matrix-sdk-common = { workspace = true }
 matrix-sdk-test = { workspace = true, optional = true }  # feature = testing only
-pbkdf2 = { version = "0.12.2", default-features = false }
+pbkdf2 = { workspace = true }
 rand = { workspace = true }
-rmp-serde = "1.1.1"
+rmp-serde = { workspace = true }
 ruma = { workspace = true, features = ["rand", "canonical-json", "unstable-msc3814"] }
 serde = { workspace = true, features = ["derive", "rc"] }
 serde_json = { workspace = true }
 sha2 = { workspace = true }
-subtle = "2.5.0"
-time = { version = "0.3.34", features = ["formatting"] }
+subtle = "2.6.1"
+time = { version = "0.3.36", features = ["formatting"] }
 tokio-stream = { workspace = true, features = ["sync"] }
 tokio = { workspace = true }
 thiserror = { workspace = true }
 tracing = { workspace = true, features = ["attributes"] }
 url = { workspace = true }
-ulid = { version = "1.0.0" }
+ulid = { version = "1.1.3" }
 uniffi = { workspace = true, optional = true }
 vodozemac = { workspace = true }
 zeroize = { workspace = true, features = ["zeroize_derive"] }
@@ -84,10 +84,10 @@ assert_matches = { workspace = true }
 assert_matches2 = { workspace = true }
 futures-executor = { workspace = true }
 http = { workspace = true }
-indoc = "2.0.1"
+indoc = "2.0.5"
 matrix-sdk-test = { workspace = true }
-proptest = { version = "1.0.0", default-features = false, features = ["std"] }
-similar-asserts = "1.5.0"
+proptest = { workspace = true }
+similar-asserts = { workspace = true }
 # required for async_test macro
 stream_assert = { workspace = true }
 tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
@@ -38,8 +38,8 @@ use tracing::{debug, info, instrument, trace, warn};
 use crate::{
    olm::{BackedUpRoomKey, ExportedRoomKey, InboundGroupSession, SignedJsonObject},
    store::{BackupDecryptionKey, BackupKeys, Changes, RoomKeyCounts, Store},
-    types::{MegolmV1AuthData, RoomKeyBackupInfo, Signatures},
-    CryptoStoreError, Device, KeysBackupRequest, RoomKeyImportResult, SignatureError,
+    types::{requests::KeysBackupRequest, MegolmV1AuthData, RoomKeyBackupInfo, Signatures},
+    CryptoStoreError, Device, RoomKeyImportResult, SignatureError,
 };

 mod keys;
@@ -57,7 +57,7 @@ use tracing::{instrument, trace};
 use vodozemac::LibolmPickleError;

 use crate::{
-    store::{CryptoStoreWrapper, MemoryStore, RoomKeyInfo, Store},
+    store::{Changes, CryptoStoreWrapper, DehydratedDeviceKey, MemoryStore, RoomKeyInfo, Store},
    verification::VerificationMachine,
    Account, CryptoStoreError, EncryptionSyncChanges, OlmError, OlmMachine, SignatureError,
 };
@@ -69,6 +69,10 @@ pub enum DehydrationError {
    #[error(transparent)]
    Pickle(#[from] LibolmPickleError),

+    /// The pickle key has an invalid length
+    #[error("The pickle key has an invalid length, expected 32 bytes, got {0}")]
+    PickleKeyLength(usize),
+
    /// The dehydrated device could not be signed by our user identity,
    /// we're missing the self-signing key.
    #[error("The self-signing key is missing, can't create a dehydrated device")]
@@ -132,15 +136,49 @@ impl DehydratedDevices {
    ///   private keys of the device.
    pub async fn rehydrate(
        &self,
-        pickle_key: &[u8; 32],
+        pickle_key: &DehydratedDeviceKey,
        device_id: &DeviceId,
        device_data: Raw<DehydratedDeviceData>,
    ) -> Result<RehydratedDevice, DehydrationError> {
-        let pickle_key = expand_pickle_key(pickle_key, device_id);
+        let pickle_key = expand_pickle_key(pickle_key.inner.as_ref(), device_id);
        let rehydrated = self.inner.rehydrate(&pickle_key, device_id, device_data).await?;

        Ok(RehydratedDevice { rehydrated, original: self.inner.to_owned() })
    }
+
+    /// Get the cached dehydrated device pickle key if any.
+    ///
+    /// None if the key was not previously cached (via
+    /// [`DehydratedDevices::save_dehydrated_device_pickle_key`]).
+    ///
+    /// Should be used to periodically rotate the dehydrated device to avoid
+    /// one-time keys exhaustion and accumulation of to_device messages.
+    pub async fn get_dehydrated_device_pickle_key(
+        &self,
+    ) -> Result<Option<DehydratedDeviceKey>, DehydrationError> {
+        Ok(self.inner.store().load_dehydrated_device_pickle_key().await?)
+    }
+
+    /// Store the dehydrated device pickle key in the crypto store.
+    ///
+    /// This is useful if the client wants to periodically rotate dehydrated
+    /// devices to avoid one-time keys exhaustion and accumulated to_device
+    /// problems.
+    pub async fn save_dehydrated_device_pickle_key(
+        &self,
+        dehydrated_device_pickle_key: &DehydratedDeviceKey,
+    ) -> Result<(), DehydrationError> {
+        let changes = Changes {
+            dehydrated_device_pickle_key: Some(dehydrated_device_pickle_key.clone()),
+            ..Default::default()
+        };
+        Ok(self.inner.store().save_changes(changes).await?)
+    }
+
+    /// Deletes the previously stored dehydrated device pickle key.
+    pub async fn delete_dehydrated_device_pickle_key(&self) -> Result<(), DehydrationError> {
+        Ok(self.inner.store().delete_dehydrated_device_pickle_key().await?)
+    }
 }

 /// A rehydraded device.
@@ -170,7 +208,7 @@ impl RehydratedDevice {
    ///
    /// ```no_run
    /// # use anyhow::Result;
-    /// # use matrix_sdk_crypto::OlmMachine;
+    /// # use matrix_sdk_crypto::{ OlmMachine, store::DehydratedDeviceKey };
    /// # use ruma::{api::client::dehydrated_device, DeviceId};
    /// # async fn example() -> Result<()> {
    /// # let machine: OlmMachine = unimplemented!();
@@ -184,9 +222,9 @@ impl RehydratedDevice {
    /// ) -> Result<dehydrated_device::get_events::unstable::Response> {
    ///     todo!("Download the to-device events of the dehydrated device");
    /// }
-    ///
-    /// // Don't use a zero key for real.
-    /// let pickle_key = [0u8; 32];
+    /// // Get the cached dehydrated key (got it after verification/recovery)
+    /// let pickle_key = machine
+    ///     .dehydrated_devices().get_dehydrated_device_pickle_key().await?.unwrap();
    ///
    /// // Fetch the dehydrated device from the server.
    /// let response = get_dehydrated_device().await?;
@@ -285,11 +323,13 @@ impl DehydratedDevice {
    /// # Examples
    ///
    /// ```no_run
-    /// # use matrix_sdk_crypto::OlmMachine;
-    /// # async fn example() -> anyhow::Result<()> {
+    /// # use matrix_sdk_crypto::OlmMachine;    /// #
+    /// use matrix_sdk_crypto::store::DehydratedDeviceKey;
+    ///
+    /// async fn example() -> anyhow::Result<()> {
    /// # let machine: OlmMachine = unimplemented!();
-    /// // Don't use a zero key for real.
-    /// let pickle_key = [0u8; 32];
+    /// // Create a new random key
+    /// let pickle_key = DehydratedDeviceKey::new()?;
    ///
    /// // Create the dehydrated device.
    /// let device = machine.dehydrated_devices().create().await?;
@@ -299,6 +339,9 @@ impl DehydratedDevice {
    ///     .keys_for_upload("Dehydrated device".to_owned(), &pickle_key)
    ///     .await?;
    ///
+    /// // Save the key if you want to later one rotate the dehydrated device
+    /// machine.dehydrated_devices().save_dehydrated_device_pickle_key(&pickle_key).await.unwrap();
+    ///
    /// // Send the request out using your HTTP client.
    /// // client.send(request).await?;
    /// # Ok(())
@@ -314,7 +357,7 @@ impl DehydratedDevice {
    pub async fn keys_for_upload(
        &self,
        initial_device_display_name: String,
-        pickle_key: &[u8; 32],
+        pickle_key: &DehydratedDeviceKey,
    ) -> Result<put_dehydrated_device::unstable::Request, DehydrationError> {
        let mut transaction = self.store.transaction().await;

@@ -330,7 +373,8 @@ impl DehydratedDevice {

        trace!("Creating an upload request for a dehydrated device");

-        let pickle_key = expand_pickle_key(pickle_key, &self.store.static_account().device_id);
+        let pickle_key =
+            expand_pickle_key(pickle_key.inner.as_ref(), &self.store.static_account().device_id);
        let device_id = self.store.static_account().device_id.clone();
        let device_data = account.dehydrate(&pickle_key);
        let initial_device_display_name = Some(initial_device_display_name);
@@ -393,12 +437,15 @@ mod tests {
            tests::to_device_requests_to_content,
        },
        olm::OutboundGroupSession,
+        store::DehydratedDeviceKey,
        types::{events::ToDeviceEvent, DeviceKeys as DeviceKeysType},
        utilities::json_convert,
        EncryptionSettings, OlmMachine,
    };

-    const PICKLE_KEY: &[u8; 32] = &[0u8; 32];
+    fn pickle_key() -> DehydratedDeviceKey {
+        DehydratedDeviceKey::from_bytes(&[0u8; 32])
+    }

    fn user_id() -> &'static UserId {
        user_id!("@alice:localhost")
@@ -467,7 +514,7 @@ mod tests {
        let dehydrated_device = olm_machine.dehydrated_devices().create().await.unwrap();

        let request = dehydrated_device
-            .keys_for_upload("Foo".to_owned(), PICKLE_KEY)
+            .keys_for_upload("Foo".to_owned(), &pickle_key())
            .await
            .expect("We should be able to create a request to upload a dehydrated device");

@@ -497,7 +544,7 @@ mod tests {
        let dehydrated_device = alice.dehydrated_devices().create().await.unwrap();

        let mut request = dehydrated_device
-            .keys_for_upload("Foo".to_owned(), PICKLE_KEY)
+            .keys_for_upload("Foo".to_owned(), &pickle_key())
            .await
            .expect("We should be able to create a request to upload a dehydrated device");

@@ -531,7 +578,7 @@ mod tests {
        // Rehydrate the device.
        let rehydrated = bob
            .dehydrated_devices()
-            .rehydrate(PICKLE_KEY, &request.device_id, request.device_data)
+            .rehydrate(&pickle_key(), &request.device_id, request.device_data)
            .await
            .expect("We should be able to rehydrate the device");

@@ -561,4 +608,43 @@ mod tests {
            "The session ids of the imported room key and the outbound group session should match"
        );
    }
+
+    #[async_test]
+    async fn test_dehydrated_device_pickle_key_cache() {
+        let alice = get_olm_machine().await;
+
+        let dehydrated_manager = alice.dehydrated_devices();
+
+        let stored_key = dehydrated_manager.get_dehydrated_device_pickle_key().await.unwrap();
+        assert!(stored_key.is_none());
+
+        let pickle_key = DehydratedDeviceKey::new().unwrap();
+
+        dehydrated_manager.save_dehydrated_device_pickle_key(&pickle_key).await.unwrap();
+
+        let stored_key =
+            dehydrated_manager.get_dehydrated_device_pickle_key().await.unwrap().unwrap();
+        assert_eq!(stored_key.to_base64(), pickle_key.to_base64());
+
+        let dehydrated_device = dehydrated_manager.create().await.unwrap();
+
+        let request = dehydrated_device
+            .keys_for_upload("Foo".to_owned(), &stored_key)
+            .await
+            .expect("We should be able to create a request to upload a dehydrated device");
+
+        // Rehydrate the device.
+        dehydrated_manager
+            .rehydrate(&stored_key, &request.device_id, request.device_data)
+            .await
+            .expect("We should be able to rehydrate the device");
+
+        dehydrated_manager
+            .delete_dehydrated_device_pickle_key()
+            .await
+            .expect("Should be able to delete the dehydrated device key");
+
+        let stored_key = dehydrated_manager.get_dehydrated_device_pickle_key().await.unwrap();
+        assert!(stored_key.is_none());
+    }
 }
@@ -14,7 +14,7 @@

 use std::collections::BTreeMap;

-use matrix_sdk_common::deserialized_responses::VerificationLevel;
+use matrix_sdk_common::deserialized_responses::{VerificationLevel, WithheldCode};
 use ruma::{CanonicalJsonError, IdParseError, OwnedDeviceId, OwnedRoomId, OwnedUserId};
 use serde::{ser::SerializeMap, Serializer};
 use serde_json::Error as SerdeError;
@@ -22,10 +22,7 @@ use thiserror::Error;
 use vodozemac::{Curve25519PublicKey, Ed25519PublicKey};

 use super::store::CryptoStoreError;
-use crate::{
-    olm::SessionExportError,
-    types::{events::room_key_withheld::WithheldCode, SignedKey},
-};
+use crate::{olm::SessionExportError, types::SignedKey};
 #[cfg(doc)]
 use crate::{CollectStrategy, Device, LocalTrust, OtherUserIdentity};

@@ -45,16 +45,18 @@ use crate::{
    error::{EventError, OlmError, OlmResult},
    identities::IdentityManager,
    olm::{InboundGroupSession, Session},
-    requests::{OutgoingRequest, ToDeviceRequest},
    session_manager::GroupSessionCache,
    store::{Changes, CryptoStoreError, SecretImportError, Store, StoreCache},
-    types::events::{
-        forwarded_room_key::ForwardedRoomKeyContent,
-        olm_v1::{DecryptedForwardedRoomKeyEvent, DecryptedSecretSendEvent},
-        room::encrypted::EncryptedEvent,
-        room_key_request::RoomKeyRequestEvent,
-        secret_send::SecretSendContent,
-        EventType,
+    types::{
+        events::{
+            forwarded_room_key::ForwardedRoomKeyContent,
+            olm_v1::{DecryptedForwardedRoomKeyEvent, DecryptedSecretSendEvent},
+            room::encrypted::EncryptedEvent,
+            room_key_request::RoomKeyRequestEvent,
+            secret_send::SecretSendContent,
+            EventType,
+        },
+        requests::{OutgoingRequest, ToDeviceRequest},
    },
    Device, MegolmError,
 };
@@ -1115,6 +1117,7 @@ mod tests {
    use crate::{
        gossiping::KeyForwardDecision,
        olm::OutboundGroupSession,
+        types::requests::AnyOutgoingRequest,
        types::{
            events::{
                forwarded_room_key::ForwardedRoomKeyContent, olm_v1::AnyDecryptedOlmEvent,
@@ -1122,7 +1125,7 @@ mod tests {
            },
            EventEncryptionAlgorithm,
        },
-        EncryptionSettings, OutgoingRequests,
+        EncryptionSettings,
    };
    use crate::{
        identities::{DeviceData, IdentityManager, LocalTrust},
@@ -1309,7 +1312,7 @@ mod tests {

    fn extract_content<'a>(
        recipient: &UserId,
-        request: &'a crate::OutgoingRequest,
+        request: &'a crate::types::requests::OutgoingRequest,
    ) -> &'a Raw<ruma::events::AnyToDeviceEventContent> {
        request
            .request()
@@ -1342,7 +1345,7 @@ mod tests {
    fn request_to_event<C>(
        recipient: &UserId,
        sender: &UserId,
-        request: &crate::OutgoingRequest,
+        request: &crate::types::requests::OutgoingRequest,
    ) -> crate::types::events::ToDeviceEvent<C>
    where
        C: crate::types::events::EventType
@@ -2063,7 +2066,7 @@ mod tests {
        assert_eq!(bob_machine.outgoing_to_device_requests().await.unwrap().len(), 1);
        assert_matches!(
            bob_machine.outgoing_to_device_requests().await.unwrap()[0].request(),
-            OutgoingRequests::KeysClaim(_)
+            AnyOutgoingRequest::KeysClaim(_)
        );
        assert!(!bob_machine.inner.users_for_key_claim.read().unwrap().is_empty());
        assert!(!bob_machine.inner.wait_queue.is_empty());
@@ -36,10 +36,12 @@ use ruma::{
 use serde::{Deserialize, Serialize};

 use crate::{
-    requests::{OutgoingRequest, ToDeviceRequest},
-    types::events::{
-        olm_v1::DecryptedSecretSendEvent,
-        room_key_request::{RoomKeyRequestContent, RoomKeyRequestEvent, SupportedKeyInfo},
+    types::{
+        events::{
+            olm_v1::DecryptedSecretSendEvent,
+            room_key_request::{RoomKeyRequestContent, RoomKeyRequestEvent, SupportedKeyInfo},
+        },
+        requests::{OutgoingRequest, ToDeviceRequest},
    },
    Device,
 };
@@ -21,6 +21,7 @@ use std::{
    },
 };

+use matrix_sdk_common::deserialized_responses::WithheldCode;
 use ruma::{
    api::client::keys::upload_signatures::v3::Request as SignatureUploadRequest,
    events::{key::verification::VerificationMethod, AnyToDeviceEventContent},
@@ -48,13 +49,13 @@ use crate::{
    types::{
        events::{
            forwarded_room_key::ForwardedRoomKeyContent,
-            room::encrypted::ToDeviceEncryptedEventContent, room_key_withheld::WithheldCode,
-            EventType,
+            room::encrypted::ToDeviceEncryptedEventContent, EventType,
        },
+        requests::{OutgoingVerificationRequest, ToDeviceRequest},
        DeviceKey, DeviceKeys, EventEncryptionAlgorithm, Signatures, SignedKey,
    },
    verification::VerificationMachine,
-    Account, OutgoingVerificationRequest, Sas, ToDeviceRequest, VerificationRequest,
+    Account, Sas, VerificationRequest,
 };

 pub enum MaybeEncryptedRoomKey {
@@ -33,12 +33,14 @@ use crate::{
    error::OlmResult,
    identities::{DeviceData, OtherUserIdentityData, OwnUserIdentityData, UserIdentityData},
    olm::{InboundGroupSession, PrivateCrossSigningIdentity, SenderDataFinder, SenderDataType},
-    requests::KeysQueryRequest,
    store::{
        caches::SequenceNumber, Changes, DeviceChanges, IdentityChanges, KeyQueryManager,
        Result as StoreResult, Store, StoreCache, StoreCacheGuard, UserKeyQueryResult,
    },
-    types::{CrossSigningKey, DeviceKeys, MasterPubkey, SelfSigningPubkey, UserSigningPubkey},
+    types::{
+        requests::KeysQueryRequest, CrossSigningKey, DeviceKeys, MasterPubkey, SelfSigningPubkey,
+        UserSigningPubkey,
+    },
    CryptoStoreError, LocalTrust, OwnUserIdentity, SignatureError, UserIdentity,
 };

@@ -1228,9 +1230,8 @@ pub(crate) mod testing {
        identities::IdentityManager,
        olm::{Account, PrivateCrossSigningIdentity},
        store::{CryptoStoreWrapper, MemoryStore, PendingChanges, Store},
-        types::DeviceKeys,
+        types::{requests::UploadSigningKeysRequest, DeviceKeys},
        verification::VerificationMachine,
-        UploadSigningKeysRequest,
    };

    pub fn user_id() -> &'static UserId {
@@ -31,14 +31,16 @@ use ruma::{
 };
 use serde::{Deserialize, Deserializer, Serialize};
 use serde_json::Value;
-use tracing::error;
+use tracing::{error, info};

 use crate::{
    error::SignatureError,
    store::{Changes, IdentityChanges, Store},
-    types::{MasterPubkey, SelfSigningPubkey, UserSigningPubkey},
+    types::{
+        requests::OutgoingVerificationRequest, MasterPubkey, SelfSigningPubkey, UserSigningPubkey,
+    },
    verification::VerificationMachine,
-    CryptoStoreError, DeviceData, OutgoingVerificationRequest, VerificationRequest,
+    CryptoStoreError, DeviceData, VerificationRequest,
 };

 /// Enum over the different user identity types we can have.
@@ -390,6 +392,7 @@ impl OtherUserIdentity {

    /// Pin the current identity (public part of the master signing key).
    pub async fn pin_current_master_key(&self) -> Result<(), CryptoStoreError> {
+        info!(master_key = ?self.master_key.get_first_key(), "Pinning current identity for user '{}'", self.user_id());
        self.inner.pin();
        let to_save = UserIdentityData::Other(self.inner.clone());
        let changes = Changes {
@@ -425,6 +428,7 @@ impl OtherUserIdentity {

    /// Remove the requirement for this identity to be verified.
    pub async fn withdraw_verification(&self) -> Result<(), CryptoStoreError> {
+        info!(master_key = ?self.master_key.get_first_key(), "Withdrawing verification status and pinning current identity for user '{}'", self.user_id());
        self.inner.withdraw_verification();
        let to_save = UserIdentityData::Other(self.inner.clone());
        let changes = Changes {
@@ -918,6 +922,7 @@ enum OwnUserIdentityVerifiedState {
    NeverVerified,

    /// We previously verified this identity, but it has changed.
+    #[serde(alias = "PreviouslyVerifiedButNoLonger")]
    VerificationViolation,

    /// We have verified the current identity.
@@ -1537,26 +1542,10 @@ pub(crate) mod tests {
    /// that we can deserialize boolean values.
    #[test]
    fn test_deserialize_own_user_identity_bool_verified() {
-        let mut json = json!({
-            "user_id": "@example:localhost",
-            "master_key": {
-                "user_id":"@example:localhost",
-                "usage":["master"],
-                "keys":{"ed25519:rJ2TAGkEOP6dX41Ksll6cl8K3J48l8s/59zaXyvl2p0":"rJ2TAGkEOP6dX41Ksll6cl8K3J48l8s/59zaXyvl2p0"},
-            },
-            "self_signing_key": {
-                "user_id":"@example:localhost",
-                "usage":["self_signing"],
-                "keys":{"ed25519:0C8lCBxrvrv/O7BQfsKnkYogHZX3zAgw3RfJuyiq210":"0C8lCBxrvrv/O7BQfsKnkYogHZX3zAgw3RfJuyiq210"}
-            },
-            "user_signing_key": {
-                "user_id":"@example:localhost",
-                "usage":["user_signing"],
-                "keys":{"ed25519:DU9z4gBFKFKCk7a13sW9wjT0Iyg7Hqv5f0BPM7DEhPo":"DU9z4gBFKFKCk7a13sW9wjT0Iyg7Hqv5f0BPM7DEhPo"}
-            },
-            "verified": false
-        });
+        let mut json = own_user_identity_data();

+        // Set `"verified": false`
+        *json.get_mut("verified").unwrap() = false.into();
        let id: OwnUserIdentityData = serde_json::from_value(json.clone()).unwrap();
        assert_eq!(*id.verified.read().unwrap(), OwnUserIdentityVerifiedState::NeverVerified);

@@ -1566,6 +1555,38 @@ pub(crate) mod tests {
        assert_eq!(*id.verified.read().unwrap(), OwnUserIdentityVerifiedState::Verified);
    }

+    #[test]
+    fn test_own_user_identity_verified_state_verification_violation_deserializes() {
+        // Given data containing verified: VerificationViolation
+        let mut json = own_user_identity_data();
+        *json.get_mut("verified").unwrap() = "VerificationViolation".into();
+
+        // When we deserialize
+        let id: OwnUserIdentityData = serde_json::from_value(json.clone()).unwrap();
+
+        // Then the value is correctly populated
+        assert_eq!(
+            *id.verified.read().unwrap(),
+            OwnUserIdentityVerifiedState::VerificationViolation
+        );
+    }
+
+    #[test]
+    fn test_own_user_identity_verified_state_previously_verified_deserializes() {
+        // Given data containing verified: PreviouslyVerifiedButNoLonger
+        let mut json = own_user_identity_data();
+        *json.get_mut("verified").unwrap() = "PreviouslyVerifiedButNoLonger".into();
+
+        // When we deserialize
+        let id: OwnUserIdentityData = serde_json::from_value(json.clone()).unwrap();
+
+        // Then the old value is re-interpreted as VerificationViolation
+        assert_eq!(
+            *id.verified.read().unwrap(),
+            OwnUserIdentityVerifiedState::VerificationViolation
+        );
+    }
+
    #[test]
    fn own_identity_check_signatures() {
        let response = own_key_query();
@@ -1941,4 +1962,26 @@ pub(crate) mod tests {
        assert!(!own_identity.was_previously_verified());
        assert!(!own_identity.has_verification_violation());
    }
+
+    fn own_user_identity_data() -> Value {
+        json!({
+            "user_id": "@example:localhost",
+            "master_key": {
+                "user_id":"@example:localhost",
+                "usage":["master"],
+                "keys":{"ed25519:rJ2TAGkEOP6dX41Ksll6cl8K3J48l8s/59zaXyvl2p0":"rJ2TAGkEOP6dX41Ksll6cl8K3J48l8s/59zaXyvl2p0"},
+            },
+            "self_signing_key": {
+                "user_id":"@example:localhost",
+                "usage":["self_signing"],
+                "keys":{"ed25519:0C8lCBxrvrv/O7BQfsKnkYogHZX3zAgw3RfJuyiq210":"0C8lCBxrvrv/O7BQfsKnkYogHZX3zAgw3RfJuyiq210"}
+            },
+            "user_signing_key": {
+                "user_id":"@example:localhost",
+                "usage":["user_signing"],
+                "keys":{"ed25519:DU9z4gBFKFKCk7a13sW9wjT0Iyg7Hqv5f0BPM7DEhPo":"DU9z4gBFKFKCk7a13sW9wjT0Iyg7Hqv5f0BPM7DEhPo"}
+            },
+            "verified": false
+        })
+    }
 }
@@ -17,7 +17,6 @@
 #![cfg_attr(docsrs, feature(doc_auto_cfg))]
 #![warn(missing_docs, missing_debug_implementations)]
 #![cfg_attr(target_arch = "wasm32", allow(clippy::arc_with_non_send_sync))]
-#![cfg_attr(test, allow(unexpected_cfgs))] // Triggered by the init_tracing_for_tests!() invocation.

 pub mod backups;
 mod ciphers;
@@ -28,7 +27,6 @@ mod gossiping;
 mod identities;
 mod machine;
 pub mod olm;
-pub mod requests;
 pub mod secret_storage;
 mod session_manager;
 pub mod store;
@@ -97,10 +95,6 @@ use matrix_sdk_common::deserialized_responses::{DecryptedRoomEvent, UnableToDecr
 #[cfg(feature = "qrcode")]
 pub use matrix_sdk_qrcode;
 pub use olm::{Account, CrossSigningStatus, EncryptionSettings, Session};
-pub use requests::{
-    IncomingResponse, KeysBackupRequest, KeysQueryRequest, OutgoingRequest, OutgoingRequests,
-    OutgoingVerificationRequest, RoomMessageRequest, ToDeviceRequest, UploadSigningKeysRequest,
-};
 use serde::{Deserialize, Serialize};
 pub use session_manager::CollectStrategy;
 pub use store::{
@@ -116,7 +110,7 @@ pub use verification::{QrVerification, QrVerificationState, ScanError};
 pub use vodozemac;

 /// The version of the matrix-sdk-cypto crate being used
-pub static VERSION: &str = env!("CARGO_PKG_VERSION");
+pub const VERSION: &str = env!("CARGO_PKG_VERSION");

 #[cfg(test)]
 matrix_sdk_test::init_tracing_for_tests!();
@@ -454,7 +448,7 @@ pub enum RoomEventDecryptionResult {
 /// # use std::collections::BTreeMap;
 /// # use ruma::api::client::keys::upload_keys::v3::Response;
 /// # use anyhow::Result;
-/// # use matrix_sdk_crypto::{OlmMachine, OutgoingRequest};
+/// # use matrix_sdk_crypto::{OlmMachine, types::requests::OutgoingRequest};
 /// # async fn send_request(request: OutgoingRequest) -> Result<Response> {
 /// #     let response = unimplemented!();
 /// #     Ok(response)
@@ -499,7 +493,7 @@ pub enum RoomEventDecryptionResult {
 /// # use std::collections::BTreeMap;
 /// # use ruma::api::client::keys::upload_keys::v3::Response;
 /// # use anyhow::Result;
-/// # use matrix_sdk_crypto::{OlmMachine, OutgoingRequest};
+/// # use matrix_sdk_crypto::{OlmMachine, types::requests::OutgoingRequest};
 /// # async fn send_request(request: &OutgoingRequest) -> Result<Response> {
 /// #     let response = unimplemented!();
 /// #     Ok(response)
@@ -926,7 +920,7 @@ pub enum RoomEventDecryptionResult {
 /// # use anyhow::Result;
 /// # use ruma::UserId;
 /// # use ruma::api::client::keys::claim_keys::v3::{Response, Request};
-/// # use matrix_sdk_crypto::{OlmMachine, requests::ToDeviceRequest, EncryptionSettings};
+/// # use matrix_sdk_crypto::{OlmMachine, types::requests::ToDeviceRequest, EncryptionSettings};
 /// # async fn send_request(request: &ToDeviceRequest) -> Result<Response> {
 /// #     let response = unimplemented!();
 /// #     Ok(response)
@@ -992,7 +986,7 @@ pub enum RoomEventDecryptionResult {
 /// # use serde_json::json;
 /// # use ruma::{UserId, RoomId, serde::Raw};
 /// # use ruma::api::client::keys::claim_keys::v3::{Response, Request};
-/// # use matrix_sdk_crypto::{EncryptionSettings, OlmMachine, ToDeviceRequest};
+/// # use matrix_sdk_crypto::{EncryptionSettings, OlmMachine, types::requests::ToDeviceRequest};
 /// # use tokio::sync::MutexGuard;
 /// # async fn send_request(request: &Request) -> Result<Response> {
 /// #     let response = unimplemented!();
@@ -70,7 +70,6 @@ use crate::{
        KnownSenderData, OlmDecryptionInfo, PrivateCrossSigningIdentity, SenderData,
        SenderDataFinder, SessionType, StaticAccountData,
    },
-    requests::{IncomingResponse, OutgoingRequest, UploadSigningKeysRequest},
    session_manager::{GroupSessionManager, SessionManager},
    store::{
        Changes, CryptoStoreWrapper, DeviceChanges, IdentityChanges, IntoCryptoStore, MemoryStore,
@@ -90,12 +89,16 @@ use crate::{
            },
            ToDeviceEvents,
        },
+        requests::{
+            AnyIncomingResponse, KeysQueryRequest, OutgoingRequest, ToDeviceRequest,
+            UploadSigningKeysRequest,
+        },
        EventEncryptionAlgorithm, Signatures,
    },
    utilities::timestamp_to_iso8601,
    verification::{Verification, VerificationMachine, VerificationRequest},
-    CrossSigningKeyExport, CryptoStoreError, DecryptionSettings, DeviceData, KeysQueryRequest,
-    LocalTrust, RoomEventDecryptionResult, SignatureError, ToDeviceRequest, TrustRequirement,
+    CrossSigningKeyExport, CryptoStoreError, DecryptionSettings, DeviceData, LocalTrust,
+    RoomEventDecryptionResult, SignatureError, TrustRequirement,
 };

 /// State machine implementation of the Olm/Megolm encryption protocol used for
@@ -576,34 +579,34 @@ impl OlmMachine {
    pub async fn mark_request_as_sent<'a>(
        &self,
        request_id: &TransactionId,
-        response: impl Into<IncomingResponse<'a>>,
+        response: impl Into<AnyIncomingResponse<'a>>,
    ) -> OlmResult<()> {
        match response.into() {
-            IncomingResponse::KeysUpload(response) => {
+            AnyIncomingResponse::KeysUpload(response) => {
                Box::pin(self.receive_keys_upload_response(response)).await?;
            }
-            IncomingResponse::KeysQuery(response) => {
+            AnyIncomingResponse::KeysQuery(response) => {
                Box::pin(self.receive_keys_query_response(request_id, response)).await?;
            }
-            IncomingResponse::KeysClaim(response) => {
+            AnyIncomingResponse::KeysClaim(response) => {
                Box::pin(
                    self.inner.session_manager.receive_keys_claim_response(request_id, response),
                )
                .await?;
            }
-            IncomingResponse::ToDevice(_) => {
+            AnyIncomingResponse::ToDevice(_) => {
                Box::pin(self.mark_to_device_request_as_sent(request_id)).await?;
            }
-            IncomingResponse::SigningKeysUpload(_) => {
+            AnyIncomingResponse::SigningKeysUpload(_) => {
                Box::pin(self.receive_cross_signing_upload_response()).await?;
            }
-            IncomingResponse::SignatureUpload(_) => {
+            AnyIncomingResponse::SignatureUpload(_) => {
                self.inner.verification_machine.mark_request_as_sent(request_id);
            }
-            IncomingResponse::RoomMessage(_) => {
+            AnyIncomingResponse::RoomMessage(_) => {
                self.inner.verification_machine.mark_request_as_sent(request_id);
            }
-            IncomingResponse::KeysBackup(_) => {
+            AnyIncomingResponse::KeysBackup(_) => {
                Box::pin(self.inner.backup_machine.mark_request_as_sent(request_id)).await?;
            }
        };
@@ -2579,7 +2582,9 @@ fn megolm_error_to_utd_info(
    let reason = match error {
        EventError(_) => UnableToDecryptReason::MalformedEncryptedEvent,
        Decode(_) => UnableToDecryptReason::MalformedEncryptedEvent,
-        MissingRoomKey(_) => UnableToDecryptReason::MissingMegolmSession,
+        MissingRoomKey(maybe_withheld) => {
+            UnableToDecryptReason::MissingMegolmSession { withheld_code: maybe_withheld }
+        }
        Decryption(DecryptionError::UnknownMessageIndex(_, _)) => {
            UnableToDecryptReason::UnknownMegolmMessageIndex
        }
@@ -34,8 +34,9 @@ use ruma::{
 use serde_json::json;

 use crate::{
-    store::Changes, types::events::ToDeviceEvent, CrossSigningBootstrapRequests, DeviceData,
-    OlmMachine, OutgoingRequests,
+    store::Changes,
+    types::{events::ToDeviceEvent, requests::AnyOutgoingRequest},
+    CrossSigningBootstrapRequests, DeviceData, OlmMachine,
 };

 /// These keys need to be periodically uploaded to the server.
@@ -214,7 +215,7 @@ pub fn bootstrap_requests_to_keys_query_response(
    // And if we have a device, add that
    if let Some(dk) = bootstrap_requests
        .upload_keys_req
-        .and_then(|req| as_variant!(req.request.as_ref(), OutgoingRequests::KeysUpload).cloned())
+        .and_then(|req| as_variant!(req.request.as_ref(), AnyOutgoingRequest::KeysUpload).cloned())
        .and_then(|keys_upload_request| keys_upload_request.device_keys)
    {
        let user_id: String = dk.get_field("user_id").unwrap().unwrap();
@@ -42,11 +42,12 @@ use crate::{
            room::encrypted::{EncryptedEvent, RoomEventEncryptionScheme},
            ToDeviceEvent,
        },
+        requests::AnyOutgoingRequest,
        CrossSigningKey, DeviceKeys, EventEncryptionAlgorithm, MasterPubkey, SelfSigningPubkey,
    },
    utilities::json_convert,
    CryptoStoreError, DecryptionSettings, DeviceData, EncryptionSettings, LocalTrust, OlmMachine,
-    OtherUserIdentityData, OutgoingRequests, TrustRequirement, UserIdentity,
+    OtherUserIdentityData, TrustRequirement, UserIdentity,
 };

 #[async_test]
@@ -497,7 +498,7 @@ async fn set_up_alice_cross_signing(alice: &OlmMachine, bob: &OlmMachine) {
        upload_signing_keys_req.self_signing_key.unwrap().try_into().unwrap();
    let upload_keys_req = cross_signing_requests.upload_keys_req.unwrap().clone();
    assert_let!(
-        OutgoingRequests::KeysUpload(device_upload_request) = upload_keys_req.request.as_ref()
+        AnyOutgoingRequest::KeysUpload(device_upload_request) = upload_keys_req.request.as_ref()
    );
    bob.store()
        .save_device_data(&[DeviceData::try_from(
@@ -1,18 +1,16 @@
-/*
-Copyright 2024 The Matrix.org Foundation C.I.C.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
+// Copyright 2024 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.

 use std::{fmt::Debug, iter, pin::Pin};

@@ -23,6 +21,7 @@ use matrix_sdk_test::async_test;
 use ruma::{room_id, user_id, RoomId, TransactionId, UserId};
 use serde::Serialize;
 use serde_json::json;
+use tokio_stream::wrappers::errors::BroadcastStreamRecvError;

 use crate::{
    machine::{
@@ -305,13 +304,16 @@ where
 /// Given the `room_keys_received_stream`, check that there is a pending update,
 /// and pop it.
 fn get_room_key_received_update(
-    room_keys_received_stream: &mut Pin<Box<impl Stream<Item = Vec<RoomKeyInfo>>>>,
+    room_keys_received_stream: &mut Pin<
+        Box<impl Stream<Item = Result<Vec<RoomKeyInfo>, BroadcastStreamRecvError>>>,
+    >,
 ) -> RoomKeyInfo {
    room_keys_received_stream
        .next()
        .now_or_never()
        .flatten()
        .expect("We should have received an update of room key infos")
+        .unwrap()
        .pop()
        .expect("Received an empty room key info update")
 }
@@ -19,6 +19,7 @@ use futures_util::{pin_mut, FutureExt, StreamExt};
 use itertools::Itertools;
 use matrix_sdk_common::deserialized_responses::{
    UnableToDecryptInfo, UnableToDecryptReason, UnsignedDecryptionResult, UnsignedEventLocation,
+    WithheldCode,
 };
 use matrix_sdk_test::{async_test, message_like_event_content, ruma_response_from_json, test_json};
 use ruma::{
@@ -61,17 +62,16 @@ use crate::{
    types::{
        events::{
            room::encrypted::{EncryptedToDeviceEvent, ToDeviceEncryptedEventContent},
-            room_key_withheld::{
-                MegolmV1AesSha2WithheldContent, RoomKeyWithheldContent, WithheldCode,
-            },
+            room_key_withheld::{MegolmV1AesSha2WithheldContent, RoomKeyWithheldContent},
            ToDeviceEvent,
        },
+        requests::{AnyOutgoingRequest, ToDeviceRequest},
        DeviceKeys, SignedKey, SigningKeys,
    },
    utilities::json_convert,
    verification::tests::bob_id,
    Account, DecryptionSettings, DeviceData, EncryptionSettings, MegolmError, OlmError,
-    OutgoingRequests, RoomEventDecryptionResult, ToDeviceRequest, TrustRequirement,
+    RoomEventDecryptionResult, TrustRequirement,
 };

 mod decryption_verification_state;
@@ -448,7 +448,7 @@ async fn test_request_missing_secrets() {
        .unwrap()
        .into_iter()
        .filter(|outgoing| match outgoing.request.as_ref() {
-            OutgoingRequests::ToDeviceRequest(request) => {
+            AnyOutgoingRequest::ToDeviceRequest(request) => {
                request.event_type.to_string() == "m.secret.request"
            }
            _ => false,
@@ -479,7 +479,7 @@ async fn test_request_missing_secrets_cross_signed() {
        .unwrap()
        .into_iter()
        .filter(|outgoing| match outgoing.request.as_ref() {
-            OutgoingRequests::ToDeviceRequest(request) => {
+            AnyOutgoingRequest::ToDeviceRequest(request) => {
                request.event_type.to_string() == "m.secret.request"
            }
            _ => false,
@@ -530,7 +530,8 @@ async fn test_megolm_encryption() {
        .next()
        .now_or_never()
        .flatten()
-        .expect("We should have received an update of room key infos");
+        .expect("We should have received an update of room key infos")
+        .unwrap();
    assert_eq!(room_keys.len(), 1);
    assert_eq!(room_keys[0].session_id, group_session.session_id());

@@ -682,7 +683,12 @@ async fn test_withheld_unverified() {
        bob.try_decrypt_room_event(&room_event, room_id, &decryption_settings).await.unwrap();
    assert_let!(RoomEventDecryptionResult::UnableToDecrypt(utd_info) = decrypt_result);
    assert!(utd_info.session_id.is_some());
-    assert_eq!(utd_info.reason, UnableToDecryptReason::MissingMegolmSession);
+    assert_eq!(
+        utd_info.reason,
+        UnableToDecryptReason::MissingMegolmSession {
+            withheld_code: Some(WithheldCode::Unverified)
+        }
+    );
 }

 /// Test what happens when we feed an unencrypted event into the decryption
@@ -1361,7 +1367,7 @@ async fn test_unsigned_decryption() {
        replace_encryption_result,
        UnsignedDecryptionResult::UnableToDecrypt(UnableToDecryptInfo {
            session_id: Some(second_room_key_session_id),
-            reason: UnableToDecryptReason::MissingMegolmSession,
+            reason: UnableToDecryptReason::MissingMegolmSession { withheld_code: None },
        })
    );

@@ -1467,7 +1473,7 @@ async fn test_unsigned_decryption() {
        thread_encryption_result,
        UnsignedDecryptionResult::UnableToDecrypt(UnableToDecryptInfo {
            session_id: Some(third_room_key_session_id),
-            reason: UnableToDecryptReason::MissingMegolmSession,
+            reason: UnableToDecryptReason::MissingMegolmSession { withheld_code: None },
        })
    );

@@ -22,9 +22,9 @@ use crate::{
        test_helpers::{get_machine_pair, get_machine_pair_with_session},
        tests,
    },
-    types::events::ToDeviceEvent,
+    types::{events::ToDeviceEvent, requests::ToDeviceRequest},
    utilities::json_convert,
-    EncryptionSyncChanges, OlmError, ToDeviceRequest,
+    EncryptionSyncChanges, OlmError,
 };

 #[async_test]
@@ -63,7 +63,6 @@ use crate::{
    error::{EventError, OlmResult, SessionCreationError},
    identities::DeviceData,
    olm::SenderData,
-    requests::UploadSigningKeysRequest,
    store::{Changes, DeviceChanges, Store},
    types::{
        events::{
@@ -73,6 +72,7 @@ use crate::{
                ToDeviceEncryptedEventContent,
            },
        },
+        requests::UploadSigningKeysRequest,
        CrossSigningKey, DeviceKeys, EventEncryptionAlgorithm, MasterPubkey, OneTimeKey, SignedKey,
    },
    OlmError, SignatureError,
@@ -23,6 +23,7 @@ use std::{
    time::Duration,
 };

+use matrix_sdk_common::deserialized_responses::WithheldCode;
 use ruma::{
    events::{
        room::{encryption::RoomEncryptionEventContent, history_visibility::HistoryVisibility},
@@ -54,11 +55,12 @@ use crate::{
                MegolmV1AesSha2Content, RoomEncryptedEventContent, RoomEventEncryptionScheme,
            },
            room_key::{MegolmV1AesSha2Content as MegolmV1AesSha2RoomKeyContent, RoomKeyContent},
-            room_key_withheld::{RoomKeyWithheldContent, WithheldCode},
+            room_key_withheld::RoomKeyWithheldContent,
        },
+        requests::ToDeviceRequest,
        EventEncryptionAlgorithm,
    },
-    DeviceData, ToDeviceRequest,
+    DeviceData,
 };

 const ONE_HOUR: Duration = Duration::from_secs(60 * 60);
@@ -215,6 +215,7 @@ enum SenderDataReader {
        legacy_session: bool,
    },

+    #[serde(alias = "SenderUnverifiedButPreviouslyVerified")]
    VerificationViolation(KnownSenderData),

    SenderUnverified(KnownSenderData),
@@ -286,7 +287,10 @@ mod tests {
    use vodozemac::Ed25519PublicKey;

    use super::SenderData;
-    use crate::types::{DeviceKeys, Signatures};
+    use crate::{
+        olm::KnownSenderData,
+        types::{DeviceKeys, Signatures},
+    };

    #[test]
    fn serializing_unknown_device_correctly_preserves_owner_check_failed_if_true() {
@@ -360,6 +364,47 @@ mod tests {
        assert_let!(SenderData::SenderVerified { .. } = end);
    }

+    #[test]
+    fn deserializing_sender_unverified_but_previously_verified_migrates_to_verification_violation()
+    {
+        let json = r#"
+            {
+                "SenderUnverifiedButPreviouslyVerified":{
+                    "user_id":"@u:s.co",
+                    "master_key":[
+                        150,140,249,139,141,29,63,230,179,14,213,175,176,61,11,255,
+                        26,103,10,51,100,154,183,47,181,117,87,204,33,215,241,92
+                    ],
+                    "master_key_verified":true
+                }
+            }
+            "#;
+
+        let end: SenderData = serde_json::from_str(json).expect("Failed to parse!");
+        assert_let!(SenderData::VerificationViolation(KnownSenderData { user_id, .. }) = end);
+        assert_eq!(user_id, owned_user_id!("@u:s.co"));
+    }
+
+    #[test]
+    fn deserializing_verification_violation() {
+        let json = r#"
+            {
+                "VerificationViolation":{
+                    "user_id":"@u:s.co",
+                    "master_key":[
+                        150,140,249,139,141,29,63,230,179,14,213,175,176,61,11,255,
+                        26,103,10,51,100,154,183,47,181,117,87,204,33,215,241,92
+                    ],
+                    "master_key_verified":true
+                }
+            }
+            "#;
+
+        let end: SenderData = serde_json::from_str(json).expect("Failed to parse!");
+        assert_let!(SenderData::VerificationViolation(KnownSenderData { user_id, .. }) = end);
+        assert_eq!(user_id, owned_user_id!("@u:s.co"));
+    }
+
    #[test]
    fn equal_sessions_have_same_trust_level() {
        let unknown = SenderData::unknown();
@@ -32,9 +32,11 @@ use vodozemac::Ed25519Signature;
 use super::StaticAccountData;
 use crate::{
    error::SignatureError,
-    requests::UploadSigningKeysRequest,
    store::SecretImportError,
-    types::{DeviceKeys, MasterPubkey, SelfSigningPubkey, UserSigningPubkey},
+    types::{
+        requests::UploadSigningKeysRequest, DeviceKeys, MasterPubkey, SelfSigningPubkey,
+        UserSigningPubkey,
+    },
    Account, DeviceData, OtherUserIdentityData, OwnUserIdentity, OwnUserIdentityData,
 };

@@ -1,441 +0,0 @@
-// Copyright 2020 The Matrix.org Foundation C.I.C.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-//! Modules containing customized request types.
-
-use std::{collections::BTreeMap, iter, sync::Arc, time::Duration};
-
-#[cfg(test)]
-use as_variant::as_variant;
-use ruma::{
-    api::client::{
-        backup::{add_backup_keys::v3::Response as KeysBackupResponse, RoomKeyBackup},
-        keys::{
-            claim_keys::v3::{Request as KeysClaimRequest, Response as KeysClaimResponse},
-            get_keys::v3::Response as KeysQueryResponse,
-            upload_keys::v3::{Request as KeysUploadRequest, Response as KeysUploadResponse},
-            upload_signatures::v3::{
-                Request as SignatureUploadRequest, Response as SignatureUploadResponse,
-            },
-            upload_signing_keys::v3::Response as SigningKeysUploadResponse,
-        },
-        message::send_message_event::v3::Response as RoomMessageResponse,
-        to_device::send_event_to_device::v3::Response as ToDeviceResponse,
-    },
-    events::{
-        AnyMessageLikeEventContent, AnyToDeviceEventContent, EventContent, ToDeviceEventType,
-    },
-    serde::Raw,
-    to_device::DeviceIdOrAllDevices,
-    OwnedDeviceId, OwnedRoomId, OwnedTransactionId, OwnedUserId, TransactionId, UserId,
-};
-use serde::{Deserialize, Serialize};
-
-use crate::types::CrossSigningKey;
-
-/// Customized version of
-/// `ruma_client_api::to_device::send_event_to_device::v3::Request`
-#[derive(Clone, Debug, Deserialize, Serialize)]
-pub struct ToDeviceRequest {
-    /// Type of event being sent to each device.
-    pub event_type: ToDeviceEventType,
-
-    /// A request identifier unique to the access token used to send the
-    /// request.
-    pub txn_id: OwnedTransactionId,
-
-    /// A map of users to devices to a content for a message event to be
-    /// sent to the user's device. Individual message events can be sent
-    /// to devices, but all events must be of the same type.
-    /// The content's type for this field will be updated in a future
-    /// release, until then you can create a value using
-    /// `serde_json::value::to_raw_value`.
-    pub messages:
-        BTreeMap<OwnedUserId, BTreeMap<DeviceIdOrAllDevices, Raw<AnyToDeviceEventContent>>>,
-}
-
-impl ToDeviceRequest {
-    /// Create a new owned to-device request
-    ///
-    /// # Arguments
-    ///
-    /// * `recipient` - The ID of the user that should receive this to-device
-    ///   event.
-    ///
-    /// * `recipient_device` - The device that should receive this to-device
-    ///   event, or all devices.
-    ///
-    /// * `event_type` - The type of the event content that is getting sent out.
-    ///
-    /// * `content` - The content of the to-device event.
-    pub fn new(
-        recipient: &UserId,
-        recipient_device: impl Into<DeviceIdOrAllDevices>,
-        event_type: &str,
-        content: Raw<AnyToDeviceEventContent>,
-    ) -> Self {
-        let event_type = ToDeviceEventType::from(event_type);
-        let user_messages = iter::once((recipient_device.into(), content)).collect();
-        let messages = iter::once((recipient.to_owned(), user_messages)).collect();
-
-        ToDeviceRequest { event_type, txn_id: TransactionId::new(), messages }
-    }
-
-    pub(crate) fn for_recipients(
-        recipient: &UserId,
-        recipient_devices: Vec<OwnedDeviceId>,
-        content: &AnyToDeviceEventContent,
-        txn_id: OwnedTransactionId,
-    ) -> Self {
-        let event_type = content.event_type();
-        let raw_content = Raw::new(content).expect("Failed to serialize to-device event");
-
-        if recipient_devices.is_empty() {
-            Self::new(
-                recipient,
-                DeviceIdOrAllDevices::AllDevices,
-                &event_type.to_string(),
-                raw_content,
-            )
-        } else {
-            let device_messages = recipient_devices
-                .into_iter()
-                .map(|d| (DeviceIdOrAllDevices::DeviceId(d), raw_content.clone()))
-                .collect();
-
-            let messages = iter::once((recipient.to_owned(), device_messages)).collect();
-
-            ToDeviceRequest { event_type, txn_id, messages }
-        }
-    }
-
-    pub(crate) fn with_id_raw(
-        recipient: &UserId,
-        recipient_device: impl Into<DeviceIdOrAllDevices>,
-        content: Raw<AnyToDeviceEventContent>,
-        event_type: ToDeviceEventType,
-        txn_id: OwnedTransactionId,
-    ) -> Self {
-        let user_messages = iter::once((recipient_device.into(), content)).collect();
-        let messages = iter::once((recipient.to_owned(), user_messages)).collect();
-
-        ToDeviceRequest { event_type, txn_id, messages }
-    }
-
-    pub(crate) fn with_id(
-        recipient: &UserId,
-        recipient_device: impl Into<DeviceIdOrAllDevices>,
-        content: &AnyToDeviceEventContent,
-        txn_id: OwnedTransactionId,
-    ) -> Self {
-        let event_type = content.event_type();
-        let raw_content = Raw::new(content).expect("Failed to serialize to-device event");
-
-        let user_messages = iter::once((recipient_device.into(), raw_content)).collect();
-        let messages = iter::once((recipient.to_owned(), user_messages)).collect();
-
-        ToDeviceRequest { event_type, txn_id, messages }
-    }
-
-    /// Get the number of unique messages this request contains.
-    ///
-    /// *Note*: A single message may be sent to multiple devices, so this may or
-    /// may not be the number of devices that will receive the messages as well.
-    pub fn message_count(&self) -> usize {
-        self.messages.values().map(|d| d.len()).sum()
-    }
-}
-
-/// Request that will publish a cross signing identity.
-///
-/// This uploads the public cross signing key triplet.
-#[derive(Debug, Clone)]
-pub struct UploadSigningKeysRequest {
-    /// The user's master key.
-    pub master_key: Option<CrossSigningKey>,
-    /// The user's self-signing key. Must be signed with the accompanied master,
-    /// or by the user's most recently uploaded master key if no master key
-    /// is included in the request.
-    pub self_signing_key: Option<CrossSigningKey>,
-    /// The user's user-signing key. Must be signed with the accompanied master,
-    /// or by the user's most recently uploaded master key if no master key
-    /// is included in the request.
-    pub user_signing_key: Option<CrossSigningKey>,
-}
-
-/// Customized version of
-/// `ruma_client_api::keys::get_keys::v3::Request`, without any
-/// references.
-#[derive(Clone, Debug)]
-pub struct KeysQueryRequest {
-    /// The time (in milliseconds) to wait when downloading keys from remote
-    /// servers. 10 seconds is the recommended default.
-    pub timeout: Option<Duration>,
-
-    /// The keys to be downloaded. An empty list indicates all devices for
-    /// the corresponding user.
-    pub device_keys: BTreeMap<OwnedUserId, Vec<OwnedDeviceId>>,
-}
-
-impl KeysQueryRequest {
-    pub(crate) fn new(users: impl Iterator<Item = OwnedUserId>) -> Self {
-        let device_keys = users.map(|u| (u, Vec::new())).collect();
-
-        Self { timeout: None, device_keys }
-    }
-}
-
-/// Enum over the different outgoing requests we can have.
-#[derive(Debug)]
-pub enum OutgoingRequests {
-    /// The `/keys/upload` request, uploading device and one-time keys.
-    KeysUpload(KeysUploadRequest),
-    /// The `/keys/query` request, fetching the device and cross signing keys of
-    /// other users.
-    KeysQuery(KeysQueryRequest),
-    /// The request to claim one-time keys for a user/device pair from the
-    /// server, after the response is received an 1-to-1 Olm session will be
-    /// established with the user/device pair.
-    KeysClaim(KeysClaimRequest),
-    /// The to-device requests, this request is used for a couple of different
-    /// things, the main use is key requests/forwards and interactive device
-    /// verification.
-    ToDeviceRequest(ToDeviceRequest),
-    /// Signature upload request, this request is used after a successful device
-    /// or user verification is done.
-    SignatureUpload(SignatureUploadRequest),
-    /// A room message request, usually for sending in-room interactive
-    /// verification events.
-    RoomMessage(RoomMessageRequest),
-}
-
-#[cfg(test)]
-impl OutgoingRequests {
-    /// Test helper to destructure the [`OutgoingRequests`] as a
-    /// [`ToDeviceRequest`].
-    pub fn to_device(&self) -> Option<&ToDeviceRequest> {
-        as_variant!(self, Self::ToDeviceRequest)
-    }
-}
-
-impl From<KeysQueryRequest> for OutgoingRequests {
-    fn from(request: KeysQueryRequest) -> Self {
-        Self::KeysQuery(request)
-    }
-}
-
-impl From<KeysClaimRequest> for OutgoingRequests {
-    fn from(r: KeysClaimRequest) -> Self {
-        Self::KeysClaim(r)
-    }
-}
-
-impl From<KeysUploadRequest> for OutgoingRequests {
-    fn from(request: KeysUploadRequest) -> Self {
-        Self::KeysUpload(request)
-    }
-}
-
-impl From<ToDeviceRequest> for OutgoingRequests {
-    fn from(request: ToDeviceRequest) -> Self {
-        Self::ToDeviceRequest(request)
-    }
-}
-
-impl From<RoomMessageRequest> for OutgoingRequests {
-    fn from(request: RoomMessageRequest) -> Self {
-        Self::RoomMessage(request)
-    }
-}
-
-impl From<SignatureUploadRequest> for OutgoingRequests {
-    fn from(request: SignatureUploadRequest) -> Self {
-        Self::SignatureUpload(request)
-    }
-}
-
-impl From<OutgoingVerificationRequest> for OutgoingRequest {
-    fn from(r: OutgoingVerificationRequest) -> Self {
-        Self { request_id: r.request_id().to_owned(), request: Arc::new(r.into()) }
-    }
-}
-
-impl From<SignatureUploadRequest> for OutgoingRequest {
-    fn from(r: SignatureUploadRequest) -> Self {
-        Self { request_id: TransactionId::new(), request: Arc::new(r.into()) }
-    }
-}
-
-impl From<KeysUploadRequest> for OutgoingRequest {
-    fn from(r: KeysUploadRequest) -> Self {
-        Self { request_id: TransactionId::new(), request: Arc::new(r.into()) }
-    }
-}
-
-/// Enum over all the incoming responses we need to receive.
-#[derive(Debug)]
-pub enum IncomingResponse<'a> {
-    /// The `/keys/upload` response, notifying us about the amount of uploaded
-    /// one-time keys.
-    KeysUpload(&'a KeysUploadResponse),
-    /// The `/keys/query` response, giving us the device and cross signing keys
-    /// of other users.
-    KeysQuery(&'a KeysQueryResponse),
-    /// The to-device response, an empty response.
-    ToDevice(&'a ToDeviceResponse),
-    /// The key claiming requests, giving us new one-time keys of other users so
-    /// new Olm sessions can be created.
-    KeysClaim(&'a KeysClaimResponse),
-    /// The cross signing `/keys/upload` response, marking our private cross
-    /// signing identity as shared.
-    SigningKeysUpload(&'a SigningKeysUploadResponse),
-    /// The cross signing signature upload response.
-    SignatureUpload(&'a SignatureUploadResponse),
-    /// A room message response, usually for interactive verifications.
-    RoomMessage(&'a RoomMessageResponse),
-    /// Response for the server-side room key backup request.
-    KeysBackup(&'a KeysBackupResponse),
-}
-
-impl<'a> From<&'a KeysUploadResponse> for IncomingResponse<'a> {
-    fn from(response: &'a KeysUploadResponse) -> Self {
-        IncomingResponse::KeysUpload(response)
-    }
-}
-
-impl<'a> From<&'a KeysBackupResponse> for IncomingResponse<'a> {
-    fn from(response: &'a KeysBackupResponse) -> Self {
-        IncomingResponse::KeysBackup(response)
-    }
-}
-
-impl<'a> From<&'a KeysQueryResponse> for IncomingResponse<'a> {
-    fn from(response: &'a KeysQueryResponse) -> Self {
-        IncomingResponse::KeysQuery(response)
-    }
-}
-
-impl<'a> From<&'a ToDeviceResponse> for IncomingResponse<'a> {
-    fn from(response: &'a ToDeviceResponse) -> Self {
-        IncomingResponse::ToDevice(response)
-    }
-}
-
-impl<'a> From<&'a RoomMessageResponse> for IncomingResponse<'a> {
-    fn from(response: &'a RoomMessageResponse) -> Self {
-        IncomingResponse::RoomMessage(response)
-    }
-}
-
-impl<'a> From<&'a KeysClaimResponse> for IncomingResponse<'a> {
-    fn from(response: &'a KeysClaimResponse) -> Self {
-        IncomingResponse::KeysClaim(response)
-    }
-}
-
-impl<'a> From<&'a SignatureUploadResponse> for IncomingResponse<'a> {
-    fn from(response: &'a SignatureUploadResponse) -> Self {
-        IncomingResponse::SignatureUpload(response)
-    }
-}
-
-/// Outgoing request type, holds the unique ID of the request and the actual
-/// request.
-#[derive(Debug, Clone)]
-pub struct OutgoingRequest {
-    /// The unique id of a request, needs to be passed when receiving a
-    /// response.
-    pub(crate) request_id: OwnedTransactionId,
-    /// The underlying outgoing request.
-    pub(crate) request: Arc<OutgoingRequests>,
-}
-
-impl OutgoingRequest {
-    /// Get the unique id of this request.
-    pub fn request_id(&self) -> &TransactionId {
-        &self.request_id
-    }
-
-    /// Get the underlying outgoing request.
-    pub fn request(&self) -> &OutgoingRequests {
-        &self.request
-    }
-}
-
-/// Customized owned request type for sending out room messages.
-#[derive(Clone, Debug)]
-pub struct RoomMessageRequest {
-    /// The room to send the event to.
-    pub room_id: OwnedRoomId,
-
-    /// The transaction ID for this event.
-    ///
-    /// Clients should generate an ID unique across requests with the
-    /// same access token; it will be used by the server to ensure
-    /// idempotency of requests.
-    pub txn_id: OwnedTransactionId,
-
-    /// The event content to send.
-    pub content: AnyMessageLikeEventContent,
-}
-
-/// A request that will back up a batch of room keys to the server.
-#[derive(Clone, Debug)]
-pub struct KeysBackupRequest {
-    /// The backup version that these room keys should be part of.
-    pub version: String,
-    /// The map from room id to a backed up room key that we're going to upload
-    /// to the server.
-    pub rooms: BTreeMap<OwnedRoomId, RoomKeyBackup>,
-}
-
-/// An enum over the different outgoing verification based requests.
-#[derive(Clone, Debug)]
-pub enum OutgoingVerificationRequest {
-    /// The to-device verification request variant.
-    ToDevice(ToDeviceRequest),
-    /// The in-room verification request variant.
-    InRoom(RoomMessageRequest),
-}
-
-impl OutgoingVerificationRequest {
-    /// Get the unique id of this request.
-    pub fn request_id(&self) -> &TransactionId {
-        match self {
-            OutgoingVerificationRequest::ToDevice(t) => &t.txn_id,
-            OutgoingVerificationRequest::InRoom(r) => &r.txn_id,
-        }
-    }
-}
-
-impl From<ToDeviceRequest> for OutgoingVerificationRequest {
-    fn from(r: ToDeviceRequest) -> Self {
-        OutgoingVerificationRequest::ToDevice(r)
-    }
-}
-
-impl From<RoomMessageRequest> for OutgoingVerificationRequest {
-    fn from(r: RoomMessageRequest) -> Self {
-        OutgoingVerificationRequest::InRoom(r)
-    }
-}
-
-impl From<OutgoingVerificationRequest> for OutgoingRequests {
-    fn from(request: OutgoingVerificationRequest) -> Self {
-        match request {
-            OutgoingVerificationRequest::ToDevice(r) => OutgoingRequests::ToDeviceRequest(r),
-            OutgoingVerificationRequest::InRoom(r) => OutgoingRequests::RoomMessage(r),
-        }
-    }
-}
@@ -22,7 +22,7 @@ use std::{

 use futures_util::future::join_all;
 use itertools::Itertools;
-use matrix_sdk_common::executor::spawn;
+use matrix_sdk_common::{deserialized_responses::WithheldCode, executor::spawn};
 use ruma::{
    events::{AnyMessageLikeEventContent, ToDeviceEventType},
    serde::Raw,
@@ -41,8 +41,8 @@ use crate::{
        ShareInfo, ShareState,
    },
    store::{Changes, CryptoStoreWrapper, Result as StoreResult, Store},
-    types::events::{room::encrypted::RoomEncryptedEventContent, room_key_withheld::WithheldCode},
-    Device, DeviceData, EncryptionSettings, OlmError, ToDeviceRequest,
+    types::{events::room::encrypted::RoomEncryptedEventContent, requests::ToDeviceRequest},
+    Device, DeviceData, EncryptionSettings, OlmError,
 };

 #[derive(Clone, Debug)]
@@ -779,6 +779,7 @@ mod tests {
    };

    use assert_matches2::assert_let;
+    use matrix_sdk_common::deserialized_responses::WithheldCode;
    use matrix_sdk_test::{async_test, ruma_response_from_json};
    use ruma::{
        api::client::{
@@ -801,14 +802,12 @@ mod tests {
        types::{
            events::{
                room::encrypted::EncryptedToDeviceEvent,
-                room_key_withheld::{
-                    RoomKeyWithheldContent::{self, MegolmV1AesSha2},
-                    WithheldCode,
-                },
+                room_key_withheld::RoomKeyWithheldContent::{self, MegolmV1AesSha2},
            },
+            requests::ToDeviceRequest,
            DeviceKeys, EventEncryptionAlgorithm,
        },
-        EncryptionSettings, LocalTrust, OlmMachine, ToDeviceRequest,
+        EncryptionSettings, LocalTrust, OlmMachine,
    };

    fn alice_id() -> &'static UserId {
@@ -19,6 +19,7 @@ use std::{
 };

 use itertools::{Either, Itertools};
+use matrix_sdk_common::deserialized_responses::WithheldCode;
 use ruma::{DeviceId, OwnedDeviceId, OwnedUserId, UserId};
 use serde::{Deserialize, Serialize};
 use tracing::{debug, instrument, trace};
@@ -27,7 +28,6 @@ use super::OutboundGroupSession;
 use crate::{
    error::{OlmResult, SessionRecipientCollectionError},
    store::Store,
-    types::events::room_key_withheld::WithheldCode,
    DeviceData, EncryptionSettings, LocalTrust, OlmError, OwnUserIdentityData, UserIdentityData,
 };
 #[cfg(doc)]
@@ -517,6 +517,7 @@ mod tests {

    use assert_matches::assert_matches;
    use assert_matches2::assert_let;
+    use matrix_sdk_common::deserialized_responses::WithheldCode;
    use matrix_sdk_test::{
        async_test, test_json,
        test_json::keys_query_sets::{
@@ -536,7 +537,6 @@ mod tests {
            group_sessions::share_strategy::collect_session_recipients, CollectStrategy,
        },
        testing::simulate_key_query_response_for_verification,
-        types::events::room_key_withheld::WithheldCode,
        CrossSigningKeyExport, EncryptionSettings, LocalTrust, OlmError, OlmMachine,
    };

@@ -1368,7 +1368,7 @@ mod tests {
        machine
            .mark_request_as_sent(
                &TransactionId::new(),
-                crate::IncomingResponse::KeysQuery(&kq_response),
+                crate::types::requests::AnyIncomingResponse::KeysQuery(&kq_response),
            )
            .await
            .unwrap();
@@ -34,9 +34,12 @@ use vodozemac::Curve25519PublicKey;
 use crate::{
    error::OlmResult,
    gossiping::GossipMachine,
-    requests::{OutgoingRequest, ToDeviceRequest},
    store::{Changes, Result as StoreResult, Store},
-    types::{events::EventType, EventEncryptionAlgorithm},
+    types::{
+        events::EventType,
+        requests::{OutgoingRequest, ToDeviceRequest},
+        EventEncryptionAlgorithm,
+    },
    DeviceData,
 };

@@ -4,7 +4,10 @@ use futures_core::Stream;
 use futures_util::StreamExt;
 use matrix_sdk_common::store_locks::CrossProcessStoreLock;
 use ruma::{DeviceId, OwnedDeviceId, OwnedUserId, UserId};
-use tokio::sync::{broadcast, Mutex};
+use tokio::sync::{
+    broadcast::{self},
+    Mutex,
+};
 use tokio_stream::wrappers::{errors::BroadcastStreamRecvError, BroadcastStream};
 use tracing::{debug, trace, warn};

@@ -292,11 +295,12 @@ impl CryptoStoreWrapper {
    /// the stream. Updates that happen at the same time are batched into a
    /// [`Vec`].
    ///
-    /// If the reader of the stream lags too far behind, a warning will be
-    /// logged and items will be dropped.
-    pub fn room_keys_received_stream(&self) -> impl Stream<Item = Vec<RoomKeyInfo>> {
-        let stream = BroadcastStream::new(self.room_keys_received_sender.subscribe());
-        Self::filter_errors_out_of_stream(stream, "room_keys_received_stream")
+    /// If the reader of the stream lags too far behind an error will be sent to
+    /// the reader.
+    pub fn room_keys_received_stream(
+        &self,
+    ) -> impl Stream<Item = Result<Vec<RoomKeyInfo>, BroadcastStreamRecvError>> {
+        BroadcastStream::new(self.room_keys_received_sender.subscribe())
    }

    /// Receive notifications of received `m.room_key.withheld` messages.
@@ -44,13 +44,14 @@ macro_rules! cryptostore_integration_tests {
            };
            use serde_json::value::to_raw_value;
            use serde_json::json;
+            use matrix_sdk_common::deserialized_responses::WithheldCode;
            use $crate::{
                olm::{
                    Account, Curve25519PublicKey, InboundGroupSession, OlmMessageHash,
                    PrivateCrossSigningIdentity, SenderData, SenderDataType, Session
                },
                store::{
-                    BackupDecryptionKey, Changes, CryptoStore, DeviceChanges, GossipRequest,
+                    BackupDecryptionKey, Changes, CryptoStore, DehydratedDeviceKey, DeviceChanges, GossipRequest,
                    IdentityChanges, PendingChanges, RoomSettings,
                },
                testing::{get_device, get_other_identity, get_own_identity},
@@ -61,18 +62,17 @@ macro_rules! cryptostore_integration_tests {
                        room_key_request::MegolmV1AesSha2Content,
                        room_key_withheld::{
                            CommonWithheldCodeContent, MegolmV1AesSha2WithheldContent,
-                            RoomKeyWithheldContent, WithheldCode,
+                            RoomKeyWithheldContent,
                        },
                        secret_send::SecretSendContent,
                        ToDeviceEvent,
                    },
+                    requests::ToDeviceRequest,
                    DeviceKeys,
                    EventEncryptionAlgorithm,
                },
-                GossippedSecret, LocalTrust, DeviceData, SecretInfo, ToDeviceRequest, TrackedUser,
-                vodozemac::{
-                    megolm::{GroupSession, SessionConfig},
-                },
+                vodozemac::megolm::{GroupSession, SessionConfig}, DeviceData, GossippedSecret, LocalTrust,  SecretInfo,
+                TrackedUser,
            };

            use super::get_store;
@@ -1217,6 +1217,53 @@ macro_rules! cryptostore_integration_tests {
                assert!(restored.backup_version.is_some(), "The backup version should now be Some as well");
            }

+            #[async_test]
+            async fn test_dehydration_pickle_key_saving() {
+                let (_account, store) = get_loaded_store("dehydration_key_saving").await;
+
+                let restored = store.load_dehydrated_device_pickle_key().await.unwrap();
+                assert!(restored.is_none(), "Initially no pickle key should be present");
+
+                let dehydrated_device_pickle_key = Some(DehydratedDeviceKey::new().unwrap());
+                let exported_base64 = dehydrated_device_pickle_key.clone().unwrap().to_base64();
+
+                let changes = Changes { dehydrated_device_pickle_key, ..Default::default() };
+                store.save_changes(changes).await.unwrap();
+
+                let restored = store.load_dehydrated_device_pickle_key().await.unwrap();
+                assert!(restored.is_some(), "We should be able to restore a pickle key");
+                assert_eq!(restored.unwrap().to_base64(), exported_base64);
+
+                // If None, should not clear the existing saved key
+                let changes = Changes { dehydrated_device_pickle_key: None, ..Default::default() };
+                store.save_changes(changes).await.unwrap();
+
+                let restored = store.load_dehydrated_device_pickle_key().await.unwrap();
+                assert!(restored.is_some(), "We should be able to restore a pickle key");
+                assert_eq!(restored.unwrap().to_base64(), exported_base64);
+
+            }
+
+             #[async_test]
+            async fn test_delete_dehydration_pickle_key() {
+                let (_account, store) = get_loaded_store("dehydration_key_saving").await;
+
+                let dehydrated_device_pickle_key = DehydratedDeviceKey::new().unwrap();
+
+                let changes = Changes { dehydrated_device_pickle_key: Some(dehydrated_device_pickle_key), ..Default::default() };
+                store.save_changes(changes).await.unwrap();
+
+                let restored = store.load_dehydrated_device_pickle_key().await.unwrap();
+                assert!(restored.is_some(), "We should be able to restore a pickle key");
+
+                store.delete_dehydrated_device_pickle_key().await.unwrap();
+
+                let restored = store.load_dehydrated_device_pickle_key().await.unwrap();
+                assert!(restored.is_none(), "The previously saved key should be deleted");
+
+            }
+
+
            #[async_test]
            async fn test_custom_value_saving() {
                let (_, store) = get_loaded_store("custom_value_saving").await;
@@ -31,8 +31,8 @@ use vodozemac::Curve25519PublicKey;

 use super::{
    caches::{DeviceStore, GroupSessionStore},
-    Account, BackupKeys, Changes, CryptoStore, InboundGroupSession, PendingChanges, RoomKeyCounts,
-    RoomSettings, Session,
+    Account, BackupKeys, Changes, CryptoStore, DehydratedDeviceKey, InboundGroupSession,
+    PendingChanges, RoomKeyCounts, RoomSettings, Session,
 };
 use crate::{
    gossiping::{GossipRequest, GossippedSecret, SecretInfo},
@@ -93,6 +93,7 @@ pub struct MemoryStore {
    leases: StdRwLock<HashMap<String, (String, Instant)>>,
    secret_inbox: StdRwLock<HashMap<String, Vec<GossippedSecret>>>,
    backup_keys: RwLock<BackupKeys>,
+    dehydrated_device_pickle_key: RwLock<Option<DehydratedDeviceKey>>,
    next_batch_token: RwLock<Option<String>>,
    room_settings: StdRwLock<HashMap<OwnedRoomId, RoomSettings>>,
 }
@@ -116,6 +117,7 @@ impl Default for MemoryStore {
            custom_values: Default::default(),
            leases: Default::default(),
            backup_keys: Default::default(),
+            dehydrated_device_pickle_key: Default::default(),
            secret_inbox: Default::default(),
            next_batch_token: Default::default(),
            room_settings: Default::default(),
@@ -268,6 +270,11 @@ impl CryptoStore for MemoryStore {
            self.backup_keys.write().await.backup_version = Some(version);
        }

+        if let Some(pickle_key) = changes.dehydrated_device_pickle_key {
+            let mut lock = self.dehydrated_device_pickle_key.write().await;
+            *lock = Some(pickle_key);
+        }
+
        {
            let mut secret_inbox = self.secret_inbox.write().unwrap();
            for secret in changes.secrets {
@@ -486,6 +493,16 @@ impl CryptoStore for MemoryStore {
        Ok(self.backup_keys.read().await.to_owned())
    }

+    async fn load_dehydrated_device_pickle_key(&self) -> Result<Option<DehydratedDeviceKey>> {
+        Ok(self.dehydrated_device_pickle_key.read().await.to_owned())
+    }
+
+    async fn delete_dehydrated_device_pickle_key(&self) -> Result<()> {
+        let mut lock = self.dehydrated_device_pickle_key.write().await;
+        *lock = None;
+        Ok(())
+    }
+
    async fn get_outbound_group_session(
        &self,
        room_id: &RoomId,
@@ -1125,7 +1142,10 @@ mod integration_tests {
            InboundGroupSession, OlmMessageHash, OutboundGroupSession, PrivateCrossSigningIdentity,
            SenderDataType, StaticAccountData,
        },
-        store::{BackupKeys, Changes, CryptoStore, PendingChanges, RoomKeyCounts, RoomSettings},
+        store::{
+            BackupKeys, Changes, CryptoStore, DehydratedDeviceKey, PendingChanges, RoomKeyCounts,
+            RoomSettings,
+        },
        types::events::room_key_withheld::RoomKeyWithheldEvent,
        Account, DeviceData, GossipRequest, GossippedSecret, SecretInfo, Session, TrackedUser,
        UserIdentityData,
@@ -1288,6 +1308,16 @@ mod integration_tests {
            self.0.load_backup_keys().await
        }

+        async fn load_dehydrated_device_pickle_key(
+            &self,
+        ) -> Result<Option<DehydratedDeviceKey>, Self::Error> {
+            self.0.load_dehydrated_device_pickle_key().await
+        }
+
+        async fn delete_dehydrated_device_pickle_key(&self) -> Result<(), Self::Error> {
+            self.0.delete_dehydrated_device_pickle_key().await
+        }
+
        async fn get_outbound_group_session(
            &self,
            room_id: &RoomId,
@@ -57,6 +57,7 @@ use ruma::{
 use serde::{de::DeserializeOwned, Deserialize, Serialize};
 use thiserror::Error;
 use tokio::sync::{Mutex, MutexGuard, Notify, OwnedRwLockReadGuard, OwnedRwLockWriteGuard, RwLock};
+use tokio_stream::wrappers::errors::BroadcastStreamRecvError;
 use tracing::{info, warn};
 use vodozemac::{base64_encode, megolm::SessionOrdering, Curve25519PublicKey};
 use zeroize::{Zeroize, ZeroizeOnDrop};
@@ -96,7 +97,10 @@ use matrix_sdk_common::{store_locks::CrossProcessStoreLock, timeout::timeout};
 pub use memorystore::MemoryStore;
 pub use traits::{CryptoStore, DynCryptoStore, IntoCryptoStore};

-pub use crate::gossiping::{GossipRequest, SecretInfo};
+pub use crate::{
+    dehydrated_devices::DehydrationError,
+    gossiping::{GossipRequest, SecretInfo},
+};

 /// A wrapper for our CryptoStore trait object.
 ///
@@ -518,6 +522,7 @@ pub struct Changes {
    pub private_identity: Option<PrivateCrossSigningIdentity>,
    pub backup_version: Option<String>,
    pub backup_decryption_key: Option<BackupDecryptionKey>,
+    pub dehydrated_device_pickle_key: Option<DehydratedDeviceKey>,
    pub sessions: Vec<Session>,
    pub message_hashes: Vec<OlmMessageHash>,
    pub inbound_group_sessions: Vec<InboundGroupSession>,
@@ -550,6 +555,7 @@ impl Changes {
        self.private_identity.is_none()
            && self.backup_version.is_none()
            && self.backup_decryption_key.is_none()
+            && self.dehydrated_device_pickle_key.is_none()
            && self.sessions.is_empty()
            && self.message_hashes.is_empty()
            && self.inbound_group_sessions.is_empty()
@@ -749,6 +755,76 @@ impl Debug for BackupDecryptionKey {
    }
 }

+/// The pickle key used to safely store the dehydrated device pickle.
+///
+/// This input key material will be expanded using HKDF into an AES key, MAC
+/// key, and an initialization vector (IV).
+#[derive(Clone, Zeroize, ZeroizeOnDrop, Deserialize, Serialize)]
+#[serde(transparent)]
+pub struct DehydratedDeviceKey {
+    pub(crate) inner: Box<[u8; DehydratedDeviceKey::KEY_SIZE]>,
+}
+
+impl DehydratedDeviceKey {
+    /// The number of bytes the encryption key will hold.
+    pub const KEY_SIZE: usize = 32;
+
+    /// Generates a new random pickle key.
+    pub fn new() -> Result<Self, rand::Error> {
+        let mut rng = rand::thread_rng();
+
+        let mut key = Box::new([0u8; Self::KEY_SIZE]);
+        rand::Fill::try_fill(key.as_mut_slice(), &mut rng)?;
+
+        Ok(Self { inner: key })
+    }
+
+    /// Creates a new dehydration pickle key from the given slice.
+    ///
+    /// Fail if the slice length is not 32.
+    pub fn from_slice(slice: &[u8]) -> Result<Self, DehydrationError> {
+        if slice.len() == 32 {
+            let mut key = Box::new([0u8; 32]);
+            key.copy_from_slice(slice);
+            Ok(DehydratedDeviceKey { inner: key })
+        } else {
+            Err(DehydrationError::PickleKeyLength(slice.len()))
+        }
+    }
+
+    /// Creates a dehydration pickle key from the given bytes.
+    pub fn from_bytes(raw_key: &[u8; 32]) -> Self {
+        let mut inner = Box::new([0u8; Self::KEY_SIZE]);
+        inner.copy_from_slice(raw_key);
+
+        Self { inner }
+    }
+
+    /// Export the [`DehydratedDeviceKey`] as a base64 encoded string.
+    pub fn to_base64(&self) -> String {
+        base64_encode(self.inner.as_slice())
+    }
+}
+
+impl From<&[u8; 32]> for DehydratedDeviceKey {
+    fn from(value: &[u8; 32]) -> Self {
+        DehydratedDeviceKey { inner: Box::new(*value) }
+    }
+}
+
+impl From<DehydratedDeviceKey> for Vec<u8> {
+    fn from(key: DehydratedDeviceKey) -> Self {
+        key.inner.to_vec()
+    }
+}
+
+#[cfg(not(tarpaulin_include))]
+impl Debug for DehydratedDeviceKey {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        f.debug_tuple("DehydratedDeviceKey").field(&"...").finish()
+    }
+}
+
 impl DeviceChanges {
    /// Merge the given `DeviceChanges` into this instance of `DeviceChanges`.
    pub fn extend(&mut self, other: DeviceChanges) {
@@ -1518,12 +1594,14 @@ impl Store {
    /// the stream. Updates that happen at the same time are batched into a
    /// [`Vec`].
    ///
-    /// If the reader of the stream lags too far behind, a warning will be
-    /// logged and items will be dropped.
+    /// If the reader of the stream lags too far behind an error will be sent to
+    /// the reader.
    ///
    /// The stream will terminate once all references to the underlying
    /// `CryptoStoreWrapper` are dropped.
-    pub fn room_keys_received_stream(&self) -> impl Stream<Item = Vec<RoomKeyInfo>> {
+    pub fn room_keys_received_stream(
+        &self,
+    ) -> impl Stream<Item = Result<Vec<RoomKeyInfo>, BroadcastStreamRecvError>> {
        self.inner.store.room_keys_received_stream()
    }

@@ -1945,7 +2023,10 @@ mod tests {
    use matrix_sdk_test::async_test;
    use ruma::{room_id, user_id};

-    use crate::{machine::test_helpers::get_machine_pair, types::EventEncryptionAlgorithm};
+    use crate::{
+        machine::test_helpers::get_machine_pair, store::DehydratedDeviceKey,
+        types::EventEncryptionAlgorithm,
+    };

    #[async_test]
    async fn test_import_room_keys_notifies_stream() {
@@ -1965,7 +2046,8 @@ mod tests {
            .next()
            .now_or_never()
            .flatten()
-            .expect("We should have received an update of room key infos");
+            .expect("We should have received an update of room key infos")
+            .unwrap();
        assert_eq!(room_keys.len(), 1);
        assert_eq!(room_keys[0].room_id, "!room1:localhost");
    }
@@ -2082,4 +2164,29 @@ mod tests {

        assert!(status.is_complete(), "We should have imported all the cross-signing keys");
    }
+
+    #[async_test]
+    async fn test_create_dehydrated_device_key() {
+        let pickle_key = DehydratedDeviceKey::new()
+            .expect("Should be able to create a random dehydrated device key");
+
+        let to_vec = pickle_key.inner.to_vec();
+        let pickle_key_from_slice = DehydratedDeviceKey::from_slice(to_vec.as_slice())
+            .expect("Should be able to create a dehydrated device key from slice");
+
+        assert_eq!(pickle_key_from_slice.to_base64(), pickle_key.to_base64());
+    }
+
+    #[async_test]
+    async fn test_create_dehydrated_errors() {
+        let too_small = [0u8; 22];
+        let pickle_key = DehydratedDeviceKey::from_slice(&too_small);
+
+        assert!(pickle_key.is_err());
+
+        let too_big = [0u8; 40];
+        let pickle_key = DehydratedDeviceKey::from_slice(&too_big);
+
+        assert!(pickle_key.is_err());
+    }
 }
@@ -22,7 +22,8 @@ use ruma::{
 use vodozemac::Curve25519PublicKey;

 use super::{
-    BackupKeys, Changes, CryptoStoreError, PendingChanges, Result, RoomKeyCounts, RoomSettings,
+    BackupKeys, Changes, CryptoStoreError, DehydratedDeviceKey, PendingChanges, Result,
+    RoomKeyCounts, RoomSettings,
 };
 #[cfg(doc)]
 use crate::olm::SenderData;
@@ -195,6 +196,14 @@ pub trait CryptoStore: AsyncTraitDeps {
    /// Get the backup keys we have stored.
    async fn load_backup_keys(&self) -> Result<BackupKeys, Self::Error>;

+    /// Get the dehydrated device pickle key we have stored.
+    async fn load_dehydrated_device_pickle_key(
+        &self,
+    ) -> Result<Option<DehydratedDeviceKey>, Self::Error>;
+
+    /// Deletes the previously stored dehydrated device pickle key.
+    async fn delete_dehydrated_device_pickle_key(&self) -> Result<(), Self::Error>;
+
    /// Get the outbound group session we have stored that is used for the
    /// given room.
    async fn get_outbound_group_session(
@@ -465,6 +474,14 @@ impl<T: CryptoStore> CryptoStore for EraseCryptoStoreError<T> {
        self.0.load_backup_keys().await.map_err(Into::into)
    }

+    async fn load_dehydrated_device_pickle_key(&self) -> Result<Option<DehydratedDeviceKey>> {
+        self.0.load_dehydrated_device_pickle_key().await.map_err(Into::into)
+    }
+
+    async fn delete_dehydrated_device_pickle_key(&self) -> Result<(), Self::Error> {
+        self.0.delete_dehydrated_device_pickle_key().await.map_err(Into::into)
+    }
+
    async fn get_outbound_group_session(
        &self,
        room_id: &RoomId,
@@ -31,7 +31,7 @@ mod utd_cause;

 use ruma::serde::Raw;
 pub use to_device::{ToDeviceCustomEvent, ToDeviceEvent, ToDeviceEvents};
-pub use utd_cause::UtdCause;
+pub use utd_cause::{CryptoContextInfo, UtdCause};

 /// A trait for event contents to define their event type.
 pub trait EventType {
@@ -16,19 +16,14 @@

 use std::collections::BTreeMap;

-use ruma::{
-    exports::ruma_macros::AsStrAsRefStr,
-    serde::{AsRefStr, DebugAsRefStr, DeserializeFromCowStr, FromString, SerializeAsRefStr},
-    OwnedDeviceId, OwnedRoomId,
-};
+use matrix_sdk_common::deserialized_responses::WithheldCode;
+use ruma::{OwnedDeviceId, OwnedRoomId};
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
 use vodozemac::Curve25519PublicKey;

 use super::{EventType, ToDeviceEvent};
-use crate::types::{
-    deserialize_curve_key, serialize_curve_key, EventEncryptionAlgorithm, PrivOwnedStr,
-};
+use crate::types::{deserialize_curve_key, serialize_curve_key, EventEncryptionAlgorithm};

 /// The `m.room_key_request` to-device event.
 pub type RoomKeyWithheldEvent = ToDeviceEvent<RoomKeyWithheldContent>;
@@ -160,65 +155,6 @@ impl EventType for RoomKeyWithheldContent {
    const EVENT_TYPE: &'static str = "m.room_key.withheld";
 }

-/// A machine-readable code for why the megolm key was not sent.
-#[derive(
-    Clone,
-    PartialEq,
-    Eq,
-    Hash,
-    AsStrAsRefStr,
-    AsRefStr,
-    FromString,
-    DebugAsRefStr,
-    SerializeAsRefStr,
-    DeserializeFromCowStr,
-)]
-#[non_exhaustive]
-pub enum WithheldCode {
-    /// the user/device was blacklisted.
-    #[ruma_enum(rename = "m.blacklisted")]
-    Blacklisted,
-
-    /// the user/devices is unverified.
-    #[ruma_enum(rename = "m.unverified")]
-    Unverified,
-
-    /// The user/device is not allowed have the key. For example, this would
-    /// usually be sent in response to a key request if the user was not in
-    /// the room when the message was sent.
-    #[ruma_enum(rename = "m.unauthorised")]
-    Unauthorised,
-
-    /// Sent in reply to a key request if the device that the key is requested
-    /// from does not have the requested key.
-    #[ruma_enum(rename = "m.unavailable")]
-    Unavailable,
-
-    /// An olm session could not be established.
-    /// This may happen, for example, if the sender was unable to obtain a
-    /// one-time key from the recipient.
-    #[ruma_enum(rename = "m.no_olm")]
-    NoOlm,
-
-    #[doc(hidden)]
-    _Custom(PrivOwnedStr),
-}
-
-impl std::fmt::Display for WithheldCode {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> Result<(), std::fmt::Error> {
-        let string = match self {
-            WithheldCode::Blacklisted => "The sender has blocked you.",
-            WithheldCode::Unverified => "The sender has disabled encrypting to unverified devices.",
-            WithheldCode::Unauthorised => "You are not authorised to read the message.",
-            WithheldCode::Unavailable => "The requested key was not found.",
-            WithheldCode::NoOlm => "Unable to establish a secure channel.",
-            _ => self.as_str(),
-        };
-
-        f.write_str(string)
-    }
-}
-
 #[derive(Debug, Deserialize, Serialize)]
 struct WithheldHelper {
    pub algorithm: EventEncryptionAlgorithm,
@@ -490,15 +426,14 @@ pub(super) mod tests {

    use assert_matches::assert_matches;
    use assert_matches2::assert_let;
+    use matrix_sdk_common::deserialized_responses::WithheldCode;
    use ruma::{device_id, room_id, serde::Raw, to_device::DeviceIdOrAllDevices, user_id};
    use serde_json::{json, Value};
    use vodozemac::Curve25519PublicKey;

    use super::RoomKeyWithheldEvent;
    use crate::types::{
-        events::room_key_withheld::{
-            MegolmV1AesSha2WithheldContent, RoomKeyWithheldContent, WithheldCode,
-        },
+        events::room_key_withheld::{MegolmV1AesSha2WithheldContent, RoomKeyWithheldContent},
        EventEncryptionAlgorithm,
    };

@@ -13,9 +13,9 @@
 // limitations under the License.

 use matrix_sdk_common::deserialized_responses::{
-    UnableToDecryptInfo, UnableToDecryptReason, VerificationLevel,
+    UnableToDecryptInfo, UnableToDecryptReason, VerificationLevel, WithheldCode,
 };
-use ruma::{events::AnySyncTimelineEvent, serde::Raw};
+use ruma::{events::AnySyncTimelineEvent, serde::Raw, MilliSecondsSinceUnixEpoch};
 use serde::Deserialize;

 /// Our best guess at the reason why an event can't be decrypted.
@@ -24,6 +24,15 @@ use serde::Deserialize;
 pub enum UtdCause {
    /// We don't have an explanation for why this UTD happened - it is probably
    /// a bug, or a network split between the two homeservers.
+    ///
+    /// For example:
+    ///
+    /// - the keys for this event are missing, but a key storage backup exists
+    ///   and is working, so we should be able to find the keys in the backup.
+    ///
+    /// - the keys for this event are missing, and a key storage backup exists
+    ///   on the server, but that backup is not working on this client even
+    ///   though this device is verified.
    #[default]
    Unknown = 0,

@@ -47,6 +56,45 @@ pub enum UtdCause {
    /// data was obtained from an insecure source (imported from a file,
    /// obtained from a legacy (asymmetric) backup, unsafe key forward, etc.)
    UnknownDevice = 4,
+
+    /// We are missing the keys for this event, but it is a "device-historical"
+    /// message and there is no key storage backup on the server, presumably
+    /// because the user has turned it off.
+    ///
+    /// Device-historical means that the message was sent before the current
+    /// device existed (but the current user was probably a member of the room
+    /// at the time the message was sent). Not to
+    /// be confused with pre-join or pre-invite messages (see
+    /// [`UtdCause::SentBeforeWeJoined`] for that).
+    ///
+    /// Expected message to user: "History is not available on this device".
+    HistoricalMessageAndBackupIsDisabled = 5,
+
+    /// The keys for this event are intentionally withheld.
+    ///
+    /// The sender has refused to share the key because our device does not meet
+    /// the sender's security requirements.
+    WithheldForUnverifiedOrInsecureDevice = 6,
+
+    /// The keys for this event are missing, likely because the sender was
+    /// unable to share them (e.g., failure to establish an Olm 1:1
+    /// channel). Alternatively, the sender may have deliberately excluded
+    /// this device by cherry-picking and blocking it, in which case, no action
+    /// can be taken on our side.
+    WithheldBySender = 7,
+
+    /// We are missing the keys for this event, but it is a "device-historical"
+    /// message, and even though a key storage backup does exist, we can't use
+    /// it because our device is unverified.
+    ///
+    /// Device-historical means that the message was sent before the current
+    /// device existed (but the current user was probably a member of the room
+    /// at the time the message was sent). Not to
+    /// be confused with pre-join or pre-invite messages (see
+    /// [`UtdCause::SentBeforeWeJoined`] for that).
+    ///
+    /// Expected message to user: "You need to verify this device".
+    HistoricalMessageAndDeviceIsUnverified = 8,
 }

 /// MSC4115 membership info in the unsigned area.
@@ -65,27 +113,66 @@ enum Membership {
    Join,
 }

+/// Contextual crypto information used by [`UtdCause::determine`] to properly
+/// identify an Unable-To-Decrypt cause in addition to the
+/// [`UnableToDecryptInfo`] and raw event info.
+#[derive(Debug, Clone, Copy)]
+pub struct CryptoContextInfo {
+    /// The current device creation timestamp, used as a heuristic to determine
+    /// if an event is device-historical or not (sent before the current device
+    /// existed).
+    pub device_creation_ts: MilliSecondsSinceUnixEpoch,
+
+    /// True if this device is secure because it has been verified by us
+    pub this_device_is_verified: bool,
+
+    /// True if key storage exists on the server, even if we are unable to use
+    /// it
+    pub backup_exists_on_server: bool,
+
+    /// True if key storage is correctly set up and can be used by the current
+    /// client to download and decrypt message keys.
+    pub is_backup_configured: bool,
+}
+
 impl UtdCause {
    /// Decide the cause of this UTD, based on the evidence we have.
    pub fn determine(
-        raw_event: Option<&Raw<AnySyncTimelineEvent>>,
+        raw_event: &Raw<AnySyncTimelineEvent>,
+        crypto_context_info: CryptoContextInfo,
        unable_to_decrypt_info: &UnableToDecryptInfo,
    ) -> Self {
        // TODO: in future, use more information to give a richer answer. E.g.
-        match unable_to_decrypt_info.reason {
-            UnableToDecryptReason::MissingMegolmSession
+        match &unable_to_decrypt_info.reason {
+            UnableToDecryptReason::MissingMegolmSession { withheld_code: Some(reason) } => {
+                match reason {
+                    WithheldCode::Unverified => UtdCause::WithheldForUnverifiedOrInsecureDevice,
+                    WithheldCode::Blacklisted
+                    | WithheldCode::Unauthorised
+                    | WithheldCode::Unavailable
+                    | WithheldCode::NoOlm
+                    | WithheldCode::_Custom(_) => UtdCause::WithheldBySender,
+                }
+            }
+            UnableToDecryptReason::MissingMegolmSession { withheld_code: None }
            | UnableToDecryptReason::UnknownMegolmMessageIndex => {
                // Look in the unsigned area for a `membership` field.
-                if let Some(raw_event) = raw_event {
-                    if let Ok(Some(unsigned)) =
-                        raw_event.get_field::<UnsignedWithMembership>("unsigned")
-                    {
-                        if let Membership::Leave = unsigned.membership {
-                            // We were not a member - this is the cause of the UTD
-                            return UtdCause::SentBeforeWeJoined;
-                        }
+                if let Some(unsigned) =
+                    raw_event.get_field::<UnsignedWithMembership>("unsigned").ok().flatten()
+                {
+                    if let Membership::Leave = unsigned.membership {
+                        // We were not a member - this is the cause of the UTD
+                        return UtdCause::SentBeforeWeJoined;
                    }
                }
+
+                if let Ok(timeline_event) = raw_event.deserialize() {
+                    if timeline_event.origin_server_ts() < crypto_context_info.device_creation_ts {
+                        // This event was sent before this device existed, so it is "historical"
+                        return UtdCause::determine_historical(crypto_context_info);
+                    }
+                }
+
                UtdCause::Unknown
            }

@@ -104,6 +191,50 @@ impl UtdCause {
            _ => UtdCause::Unknown,
        }
    }
+
+    /**
+     * Below is the flow chart we follow for deciding whether historical
+     * UTDs are expected. This function starts at position `B`.
+     *
+     * ```text
+     * A: Is the message newer than the device?
+     *   No -> B
+     *   Yes - Normal UTD error
+     *
+     * B: Is there a backup on the server?
+     *   No -> History is not available on this device
+     *   Yes -> C
+     *
+     * C: Is backup working on this device?
+     *   No -> D
+     *   Yes -> Normal UTD error
+     *
+     * D: Is this device verified?
+     *   No -> You need to verify this device
+     *   Yes -> Normal UTD error
+     * ```
+     */
+    fn determine_historical(crypto_context_info: CryptoContextInfo) -> UtdCause {
+        let backup_disabled = !crypto_context_info.backup_exists_on_server;
+        let backup_failing = !crypto_context_info.is_backup_configured;
+        let unverified = !crypto_context_info.this_device_is_verified;
+
+        if backup_disabled {
+            UtdCause::HistoricalMessageAndBackupIsDisabled
+        } else if backup_failing && unverified {
+            UtdCause::HistoricalMessageAndDeviceIsUnverified
+        } else {
+            // We didn't get the key from key storage backup, but we think we should have,
+            // because either:
+            //
+            // * backup is working (so why didn't we get it?), or
+            // * backup is not working for an unknown reason (because the device is
+            //   verified, and that is the only reason we check).
+            //
+            // In either case, we shrug and give an `Unknown` cause.
+            UtdCause::Unknown
+        }
+    }
 }

 #[cfg(test)]
@@ -111,38 +242,20 @@ mod tests {
    use matrix_sdk_common::deserialized_responses::{
        DeviceLinkProblem, UnableToDecryptInfo, UnableToDecryptReason, VerificationLevel,
    };
-    use ruma::{events::AnySyncTimelineEvent, serde::Raw};
+    use ruma::{events::AnySyncTimelineEvent, serde::Raw, MilliSecondsSinceUnixEpoch};
    use serde_json::{json, value::to_raw_value};

-    use crate::types::events::UtdCause;
+    use crate::types::events::{utd_cause::CryptoContextInfo, UtdCause};

-    #[test]
-    fn test_a_missing_raw_event_means_we_guess_unknown() {
-        // When we don't provide any JSON to check for membership, then we guess the UTD
-        // is unknown.
-        assert_eq!(
-            UtdCause::determine(
-                None,
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::MissingMegolmSession,
-                }
-            ),
-            UtdCause::Unknown
-        );
-    }
+    const EVENT_TIME: usize = 5555;
+    const BEFORE_EVENT_TIME: usize = 1111;
+    const AFTER_EVENT_TIME: usize = 9999;

    #[test]
    fn test_if_there_is_no_membership_info_we_guess_unknown() {
        // If our JSON contains no membership info, then we guess the UTD is unknown.
        assert_eq!(
-            UtdCause::determine(
-                Some(&raw_event(json!({}))),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::MissingMegolmSession
-                }
-            ),
+            UtdCause::determine(&raw_event(json!({})), device_old(), &missing_megolm_session()),
            UtdCause::Unknown
        );
    }
@@ -153,11 +266,9 @@ mod tests {
        // we guess the UTD is unknown.
        assert_eq!(
            UtdCause::determine(
-                Some(&raw_event(json!({ "unsigned": { "membership": 3 } }))),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::MissingMegolmSession
-                }
+                &raw_event(json!({ "unsigned": { "membership": 3 } })),
+                device_old(),
+                &missing_megolm_session()
            ),
            UtdCause::Unknown
        );
@@ -169,11 +280,9 @@ mod tests {
        // UTD is unknown.
        assert_eq!(
            UtdCause::determine(
-                Some(&raw_event(json!({ "unsigned": { "membership": "invite" } }),)),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::MissingMegolmSession
-                }
+                &raw_event(json!({ "unsigned": { "membership": "invite" } }),),
+                device_old(),
+                &missing_megolm_session()
            ),
            UtdCause::Unknown
        );
@@ -185,11 +294,9 @@ mod tests {
        // UTD is unknown.
        assert_eq!(
            UtdCause::determine(
-                Some(&raw_event(json!({ "unsigned": { "membership": "join" } }))),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::MissingMegolmSession
-                }
+                &raw_event(json!({ "unsigned": { "membership": "join" } })),
+                device_old(),
+                &missing_megolm_session()
            ),
            UtdCause::Unknown
        );
@@ -201,11 +308,9 @@ mod tests {
        // until we have MSC3061.
        assert_eq!(
            UtdCause::determine(
-                Some(&raw_event(json!({ "unsigned": { "membership": "leave" } }))),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::MissingMegolmSession
-                }
+                &raw_event(json!({ "unsigned": { "membership": "leave" } })),
+                device_old(),
+                &missing_megolm_session()
            ),
            UtdCause::SentBeforeWeJoined
        );
@@ -218,11 +323,9 @@ mod tests {
        // even if membership=leave.
        assert_eq!(
            UtdCause::determine(
-                Some(&raw_event(json!({ "unsigned": { "membership": "leave" } }))),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::MalformedEncryptedEvent
-                }
+                &raw_event(json!({ "unsigned": { "membership": "leave" } })),
+                device_old(),
+                &malformed_encrypted_event()
            ),
            UtdCause::Unknown
        );
@@ -233,13 +336,9 @@ mod tests {
        // Before MSC4115 is merged, we support the unstable prefix too.
        assert_eq!(
            UtdCause::determine(
-                Some(&raw_event(
-                    json!({ "unsigned": { "io.element.msc4115.membership": "leave" } })
-                )),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::MissingMegolmSession
-                }
+                &raw_event(json!({ "unsigned": { "io.element.msc4115.membership": "leave" } })),
+                device_old(),
+                &missing_megolm_session()
            ),
            UtdCause::SentBeforeWeJoined
        );
@@ -248,15 +347,7 @@ mod tests {
    #[test]
    fn test_verification_violation_is_passed_through() {
        assert_eq!(
-            UtdCause::determine(
-                Some(&raw_event(json!({}))),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::SenderIdentityNotTrusted(
-                        VerificationLevel::VerificationViolation,
-                    )
-                }
-            ),
+            UtdCause::determine(&raw_event(json!({})), device_old(), &verification_violation()),
            UtdCause::VerificationViolation
        );
    }
@@ -264,15 +355,7 @@ mod tests {
    #[test]
    fn test_unsigned_device_is_passed_through() {
        assert_eq!(
-            UtdCause::determine(
-                Some(&raw_event(json!({}))),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::SenderIdentityNotTrusted(
-                        VerificationLevel::UnsignedDevice,
-                    )
-                }
-            ),
+            UtdCause::determine(&raw_event(json!({})), device_old(), &unsigned_device()),
            UtdCause::UnsignedDevice
        );
    }
@@ -280,20 +363,251 @@ mod tests {
    #[test]
    fn test_unknown_device_is_passed_through() {
        assert_eq!(
-            UtdCause::determine(
-                Some(&raw_event(json!({}))),
-                &UnableToDecryptInfo {
-                    session_id: None,
-                    reason: UnableToDecryptReason::SenderIdentityNotTrusted(
-                        VerificationLevel::None(DeviceLinkProblem::MissingDevice)
-                    )
-                }
-            ),
+            UtdCause::determine(&raw_event(json!({})), device_old(), &missing_device()),
            UtdCause::UnknownDevice
        );
    }

+    #[test]
+    fn test_old_devices_dont_cause_historical_utds() {
+        // Message key is missing.
+        let info = missing_megolm_session();
+
+        // The device is old.
+        let context = device_old();
+
+        // So we have no explanation for this UTD.
+        assert_eq!(UtdCause::determine(&utd_event(), context, &info), UtdCause::Unknown);
+
+        // Same for unknown megolm message index
+        let info = unknown_megolm_message_index();
+        assert_eq!(UtdCause::determine(&utd_event(), context, &info), UtdCause::Unknown);
+    }
+
+    #[test]
+    fn test_if_backup_is_disabled_historical_utd_is_expected() {
+        // Message key is missing.
+        let info = missing_megolm_session();
+
+        // The device is new.
+        let mut context = device_new();
+
+        // There is no key storage backup on the server.
+        context.backup_exists_on_server = false;
+
+        // So this UTD is expected, and the solution (for future messages!) is to turn
+        // on key storage backups.
+        assert_eq!(
+            UtdCause::determine(&utd_event(), context, &info),
+            UtdCause::HistoricalMessageAndBackupIsDisabled
+        );
+
+        // Same for unknown megolm message index
+        let info = unknown_megolm_message_index();
+        assert_eq!(
+            UtdCause::determine(&utd_event(), context, &info),
+            UtdCause::HistoricalMessageAndBackupIsDisabled
+        );
+    }
+
+    #[test]
+    fn test_malformed_events_are_never_expected_utds() {
+        // The event was malformed.
+        let info = malformed_encrypted_event();
+
+        // The device is new.
+        let mut context = device_new();
+
+        // There is no key storage backup on the server.
+        context.backup_exists_on_server = false;
+
+        // So this could be expected historical like the previous test, but because the
+        // encrypted event is malformed, that takes precedence, and it's unexpected.
+        assert_eq!(UtdCause::determine(&utd_event(), context, &info), UtdCause::Unknown);
+
+        // Same for decryption failures
+        let info = megolm_decryption_failure();
+        assert_eq!(UtdCause::determine(&utd_event(), context, &info), UtdCause::Unknown);
+    }
+
+    #[test]
+    fn test_new_devices_with_nonworking_backups_because_unverified_cause_expected_utds() {
+        // Message key is missing.
+        let info = missing_megolm_session();
+
+        // The device is new.
+        let mut context = device_new();
+
+        // There is a key storage backup on the server.
+        context.backup_exists_on_server = true;
+
+        // The key storage backup is not working,
+        context.is_backup_configured = false;
+
+        // probably because...
+        // Our device is not verified.
+        context.this_device_is_verified = false;
+
+        // So this UTD is expected, and the solution is (hopefully) to verify.
+        assert_eq!(
+            UtdCause::determine(&utd_event(), context, &info),
+            UtdCause::HistoricalMessageAndDeviceIsUnverified
+        );
+
+        // Same for unknown megolm message index
+        let info = unknown_megolm_message_index();
+        assert_eq!(
+            UtdCause::determine(&utd_event(), context, &info),
+            UtdCause::HistoricalMessageAndDeviceIsUnverified
+        );
+    }
+
+    #[test]
+    fn test_if_backup_is_working_then_historical_utd_is_unexpected() {
+        // Message key is missing.
+        let info = missing_megolm_session();
+
+        // The device is new.
+        let mut context = device_new();
+
+        // There is a key storage backup on the server.
+        context.backup_exists_on_server = true;
+
+        // The key storage backup is working.
+        context.is_backup_configured = true;
+
+        // So this UTD is unexpected since we should be able to fetch the key from
+        // storage.
+        assert_eq!(UtdCause::determine(&utd_event(), context, &info), UtdCause::Unknown);
+
+        // Same for unknown megolm message index
+        let info = unknown_megolm_message_index();
+        assert_eq!(UtdCause::determine(&utd_event(), context, &info), UtdCause::Unknown);
+    }
+
+    #[test]
+    fn test_if_backup_is_not_working_even_though_verified_then_historical_utd_is_unexpected() {
+        // Message key is missing.
+        let info = missing_megolm_session();
+
+        // The device is new.
+        let mut context = device_new();
+
+        // There is a key storage backup on the server.
+        context.backup_exists_on_server = true;
+
+        // The key storage backup is working.
+        context.is_backup_configured = false;
+
+        // even though...
+        // Our device is verified.
+        context.this_device_is_verified = true;
+
+        // So this UTD is unexpected since we can't explain why our backup is not
+        // working.
+        //
+        // TODO: it might be nice to tell the user that our backup is not working!
+        // Currently we don't distinguish between Unknown cases, since we want
+        // to make sure they are all reported as unexpected UTDs.
+        assert_eq!(UtdCause::determine(&utd_event(), context, &info), UtdCause::Unknown);
+
+        // Same for unknown megolm message index
+        let info = unknown_megolm_message_index();
+        assert_eq!(UtdCause::determine(&utd_event(), context, &info), UtdCause::Unknown);
+    }
+
+    fn utd_event() -> Raw<AnySyncTimelineEvent> {
+        raw_event(json!({
+            "type": "m.room.encrypted",
+            "event_id": "$0",
+            // the values don't matter much but the expected fields should be there.
+            "content": {
+                "algorithm": "m.megolm.v1.aes-sha2",
+                "ciphertext": "FOO",
+                "sender_key": "SENDERKEYSENDERKEY",
+                "device_id": "ABCDEFGH",
+                "session_id": "A0",
+            },
+            "sender": "@bob:localhost",
+            "origin_server_ts": EVENT_TIME,
+            "unsigned": { "membership": "join" }
+        }))
+    }
+
    fn raw_event(value: serde_json::Value) -> Raw<AnySyncTimelineEvent> {
        Raw::from_json(to_raw_value(&value).unwrap())
    }
+
+    fn device_old() -> CryptoContextInfo {
+        CryptoContextInfo {
+            device_creation_ts: MilliSecondsSinceUnixEpoch((BEFORE_EVENT_TIME).try_into().unwrap()),
+            this_device_is_verified: false,
+            is_backup_configured: false,
+            backup_exists_on_server: false,
+        }
+    }
+
+    fn device_new() -> CryptoContextInfo {
+        CryptoContextInfo {
+            device_creation_ts: MilliSecondsSinceUnixEpoch((AFTER_EVENT_TIME).try_into().unwrap()),
+            this_device_is_verified: false,
+            is_backup_configured: false,
+            backup_exists_on_server: false,
+        }
+    }
+
+    fn missing_megolm_session() -> UnableToDecryptInfo {
+        UnableToDecryptInfo {
+            session_id: None,
+            reason: UnableToDecryptReason::MissingMegolmSession { withheld_code: None },
+        }
+    }
+
+    fn malformed_encrypted_event() -> UnableToDecryptInfo {
+        UnableToDecryptInfo {
+            session_id: None,
+            reason: UnableToDecryptReason::MalformedEncryptedEvent,
+        }
+    }
+
+    fn unknown_megolm_message_index() -> UnableToDecryptInfo {
+        UnableToDecryptInfo {
+            session_id: None,
+            reason: UnableToDecryptReason::UnknownMegolmMessageIndex,
+        }
+    }
+
+    fn megolm_decryption_failure() -> UnableToDecryptInfo {
+        UnableToDecryptInfo {
+            session_id: None,
+            reason: UnableToDecryptReason::MegolmDecryptionFailure,
+        }
+    }
+
+    fn verification_violation() -> UnableToDecryptInfo {
+        UnableToDecryptInfo {
+            session_id: None,
+            reason: UnableToDecryptReason::SenderIdentityNotTrusted(
+                VerificationLevel::VerificationViolation,
+            ),
+        }
+    }
+
+    fn unsigned_device() -> UnableToDecryptInfo {
+        UnableToDecryptInfo {
+            session_id: None,
+            reason: UnableToDecryptReason::SenderIdentityNotTrusted(
+                VerificationLevel::UnsignedDevice,
+            ),
+        }
+    }
+
+    fn missing_device() -> UnableToDecryptInfo {
+        UnableToDecryptInfo {
+            session_id: None,
+            reason: UnableToDecryptReason::SenderIdentityNotTrusted(VerificationLevel::None(
+                DeviceLinkProblem::MissingDevice,
+            )),
+        }
+    }
 }
@@ -34,6 +34,7 @@ use std::{
 };

 use as_variant::as_variant;
+use matrix_sdk_common::deserialized_responses::PrivOwnedStr;
 use ruma::{
    serde::StringEnum, DeviceKeyAlgorithm, DeviceKeyId, OwnedDeviceKeyId, OwnedUserId, UserId,
 };
@@ -47,6 +48,7 @@ mod device_keys;
 pub mod events;
 mod one_time_keys;
 pub mod qr_login;
+pub mod requests;

 pub use self::{backup::*, cross_signing::*, device_keys::*, one_time_keys::*};
 use crate::store::BackupDecryptionKey;
@@ -424,20 +426,6 @@ impl Algorithm for DeviceKeyAlgorithm {
    }
 }

-// Wrapper around `Box<str>` that cannot be used in a meaningful way outside of
-// this crate. Used for string enums because their `_Custom` variant can't be
-// truly private (only `#[doc(hidden)]`).
-#[doc(hidden)]
-#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
-pub struct PrivOwnedStr(Box<str>);
-
-#[cfg(not(tarpaulin_include))]
-impl std::fmt::Debug for PrivOwnedStr {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
-        self.0.fmt(f)
-    }
-}
-
 /// An encryption algorithm to be used to encrypt messages sent to a room.
 #[derive(Clone, PartialEq, Eq, PartialOrd, Ord, StringEnum)]
 #[non_exhaustive]
@@ -0,0 +1,201 @@
+// Copyright 2020 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use std::sync::Arc;
+
+use ruma::{
+    api::client::{
+        backup::add_backup_keys::v3::Response as KeysBackupResponse,
+        keys::{
+            claim_keys::v3::{Request as KeysClaimRequest, Response as KeysClaimResponse},
+            get_keys::v3::Response as KeysQueryResponse,
+            upload_keys::v3::{Request as KeysUploadRequest, Response as KeysUploadResponse},
+            upload_signatures::v3::{
+                Request as SignatureUploadRequest, Response as SignatureUploadResponse,
+            },
+            upload_signing_keys::v3::Response as SigningKeysUploadResponse,
+        },
+        message::send_message_event::v3::Response as RoomMessageResponse,
+        to_device::send_event_to_device::v3::Response as ToDeviceResponse,
+    },
+    TransactionId,
+};
+
+use super::{
+    KeysQueryRequest, OutgoingRequest, OutgoingVerificationRequest, RoomMessageRequest,
+    ToDeviceRequest,
+};
+
+/// Enum over the different outgoing requests we can have.
+#[derive(Debug)]
+pub enum AnyOutgoingRequest {
+    /// The `/keys/upload` request, uploading device and one-time keys.
+    KeysUpload(KeysUploadRequest),
+    /// The `/keys/query` request, fetching the device and cross signing keys of
+    /// other users.
+    KeysQuery(KeysQueryRequest),
+    /// The request to claim one-time keys for a user/device pair from the
+    /// server, after the response is received an 1-to-1 Olm session will be
+    /// established with the user/device pair.
+    KeysClaim(KeysClaimRequest),
+    /// The to-device requests, this request is used for a couple of different
+    /// things, the main use is key requests/forwards and interactive device
+    /// verification.
+    ToDeviceRequest(ToDeviceRequest),
+    /// Signature upload request, this request is used after a successful device
+    /// or user verification is done.
+    SignatureUpload(SignatureUploadRequest),
+    /// A room message request, usually for sending in-room interactive
+    /// verification events.
+    RoomMessage(RoomMessageRequest),
+}
+
+#[cfg(test)]
+impl AnyOutgoingRequest {
+    /// Test helper to destructure the [`OutgoingRequests`] as a
+    /// [`ToDeviceRequest`].
+    pub fn to_device(&self) -> Option<&ToDeviceRequest> {
+        as_variant::as_variant!(self, Self::ToDeviceRequest)
+    }
+}
+
+impl From<KeysQueryRequest> for AnyOutgoingRequest {
+    fn from(request: KeysQueryRequest) -> Self {
+        Self::KeysQuery(request)
+    }
+}
+
+impl From<KeysClaimRequest> for AnyOutgoingRequest {
+    fn from(r: KeysClaimRequest) -> Self {
+        Self::KeysClaim(r)
+    }
+}
+
+impl From<KeysUploadRequest> for AnyOutgoingRequest {
+    fn from(request: KeysUploadRequest) -> Self {
+        Self::KeysUpload(request)
+    }
+}
+
+impl From<ToDeviceRequest> for AnyOutgoingRequest {
+    fn from(request: ToDeviceRequest) -> Self {
+        Self::ToDeviceRequest(request)
+    }
+}
+
+impl From<RoomMessageRequest> for AnyOutgoingRequest {
+    fn from(request: RoomMessageRequest) -> Self {
+        Self::RoomMessage(request)
+    }
+}
+
+impl From<SignatureUploadRequest> for AnyOutgoingRequest {
+    fn from(request: SignatureUploadRequest) -> Self {
+        Self::SignatureUpload(request)
+    }
+}
+
+impl From<OutgoingVerificationRequest> for OutgoingRequest {
+    fn from(r: OutgoingVerificationRequest) -> Self {
+        Self { request_id: r.request_id().to_owned(), request: Arc::new(r.into()) }
+    }
+}
+
+impl From<SignatureUploadRequest> for OutgoingRequest {
+    fn from(r: SignatureUploadRequest) -> Self {
+        Self { request_id: TransactionId::new(), request: Arc::new(r.into()) }
+    }
+}
+
+impl From<KeysUploadRequest> for OutgoingRequest {
+    fn from(r: KeysUploadRequest) -> Self {
+        Self { request_id: TransactionId::new(), request: Arc::new(r.into()) }
+    }
+}
+
+impl From<OutgoingVerificationRequest> for AnyOutgoingRequest {
+    fn from(request: OutgoingVerificationRequest) -> Self {
+        match request {
+            OutgoingVerificationRequest::ToDevice(r) => AnyOutgoingRequest::ToDeviceRequest(r),
+            OutgoingVerificationRequest::InRoom(r) => AnyOutgoingRequest::RoomMessage(r),
+        }
+    }
+}
+
+/// Enum over all the incoming responses we need to receive.
+#[derive(Debug)]
+pub enum AnyIncomingResponse<'a> {
+    /// The `/keys/upload` response, notifying us about the amount of uploaded
+    /// one-time keys.
+    KeysUpload(&'a KeysUploadResponse),
+    /// The `/keys/query` response, giving us the device and cross signing keys
+    /// of other users.
+    KeysQuery(&'a KeysQueryResponse),
+    /// The to-device response, an empty response.
+    ToDevice(&'a ToDeviceResponse),
+    /// The key claiming requests, giving us new one-time keys of other users so
+    /// new Olm sessions can be created.
+    KeysClaim(&'a KeysClaimResponse),
+    /// The cross signing `/keys/upload` response, marking our private cross
+    /// signing identity as shared.
+    SigningKeysUpload(&'a SigningKeysUploadResponse),
+    /// The cross signing signature upload response.
+    SignatureUpload(&'a SignatureUploadResponse),
+    /// A room message response, usually for interactive verifications.
+    RoomMessage(&'a RoomMessageResponse),
+    /// Response for the server-side room key backup request.
+    KeysBackup(&'a KeysBackupResponse),
+}
+
+impl<'a> From<&'a KeysUploadResponse> for AnyIncomingResponse<'a> {
+    fn from(response: &'a KeysUploadResponse) -> Self {
+        AnyIncomingResponse::KeysUpload(response)
+    }
+}
+
+impl<'a> From<&'a KeysBackupResponse> for AnyIncomingResponse<'a> {
+    fn from(response: &'a KeysBackupResponse) -> Self {
+        AnyIncomingResponse::KeysBackup(response)
+    }
+}
+
+impl<'a> From<&'a KeysQueryResponse> for AnyIncomingResponse<'a> {
+    fn from(response: &'a KeysQueryResponse) -> Self {
+        AnyIncomingResponse::KeysQuery(response)
+    }
+}
+
+impl<'a> From<&'a ToDeviceResponse> for AnyIncomingResponse<'a> {
+    fn from(response: &'a ToDeviceResponse) -> Self {
+        AnyIncomingResponse::ToDevice(response)
+    }
+}
+
+impl<'a> From<&'a RoomMessageResponse> for AnyIncomingResponse<'a> {
+    fn from(response: &'a RoomMessageResponse) -> Self {
+        AnyIncomingResponse::RoomMessage(response)
+    }
+}
+
+impl<'a> From<&'a KeysClaimResponse> for AnyIncomingResponse<'a> {
+    fn from(response: &'a KeysClaimResponse) -> Self {
+        AnyIncomingResponse::KeysClaim(response)
+    }
+}
+
+impl<'a> From<&'a SignatureUploadResponse> for AnyIncomingResponse<'a> {
+    fn from(response: &'a SignatureUploadResponse) -> Self {
+        AnyIncomingResponse::SignatureUpload(response)
+    }
+}
@@ -0,0 +1,27 @@
+// Copyright 2020 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use std::collections::BTreeMap;
+
+use ruma::{api::client::backup::RoomKeyBackup, OwnedRoomId};
+
+/// A request that will back up a batch of room keys to the server.
+#[derive(Clone, Debug)]
+pub struct KeysBackupRequest {
+    /// The backup version that these room keys should be part of.
+    pub version: String,
+    /// The map from room id to a backed up room key that we're going to upload
+    /// to the server.
+    pub rooms: BTreeMap<OwnedRoomId, RoomKeyBackup>,
+}
@@ -0,0 +1,38 @@
+// Copyright 2020 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use std::{collections::BTreeMap, time::Duration};
+
+use ruma::{OwnedDeviceId, OwnedUserId};
+
+/// Customized version of `ruma_client_api::keys::get_keys::v3::Request`,
+/// without any references.
+#[derive(Clone, Debug)]
+pub struct KeysQueryRequest {
+    /// The time (in milliseconds) to wait when downloading keys from remote
+    /// servers. 10 seconds is the recommended default.
+    pub timeout: Option<Duration>,
+
+    /// The keys to be downloaded. An empty list indicates all devices for
+    /// the corresponding user.
+    pub device_keys: BTreeMap<OwnedUserId, Vec<OwnedDeviceId>>,
+}
+
+impl KeysQueryRequest {
+    pub(crate) fn new(users: impl Iterator<Item = OwnedUserId>) -> Self {
+        let device_keys = users.map(|u| (u, Vec::new())).collect();
+
+        Self { timeout: None, device_keys }
+    }
+}
@@ -0,0 +1,58 @@
+// Copyright 2020 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Modules containing customized request types.
+
+use std::sync::Arc;
+
+use ruma::{OwnedTransactionId, TransactionId};
+
+mod enums;
+mod keys_backup;
+mod keys_query;
+mod room_message;
+mod signing_keys;
+mod to_device;
+mod verification;
+
+pub use enums::*;
+pub use keys_backup::*;
+pub use keys_query::*;
+pub use room_message::*;
+pub use signing_keys::*;
+pub use to_device::*;
+pub use verification::*;
+
+/// Outgoing request type, holds the unique ID of the request and the actual
+/// request.
+#[derive(Debug, Clone)]
+pub struct OutgoingRequest {
+    /// The unique id of a request, needs to be passed when receiving a
+    /// response.
+    pub(crate) request_id: OwnedTransactionId,
+    /// The underlying outgoing request.
+    pub(crate) request: Arc<AnyOutgoingRequest>,
+}
+
+impl OutgoingRequest {
+    /// Get the unique id of this request.
+    pub fn request_id(&self) -> &TransactionId {
+        &self.request_id
+    }
+
+    /// Get the underlying outgoing request.
+    pub fn request(&self) -> &AnyOutgoingRequest {
+        &self.request
+    }
+}
@@ -0,0 +1,32 @@
+// Copyright 2020 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use ruma::{events::AnyMessageLikeEventContent, OwnedRoomId, OwnedTransactionId};
+
+/// Customized owned request type for sending out room messages.
+#[derive(Clone, Debug)]
+pub struct RoomMessageRequest {
+    /// The room to send the event to.
+    pub room_id: OwnedRoomId,
+
+    /// The transaction ID for this event.
+    ///
+    /// Clients should generate an ID unique across requests with the
+    /// same access token; it will be used by the server to ensure
+    /// idempotency of requests.
+    pub txn_id: OwnedTransactionId,
+
+    /// The event content to send.
+    pub content: AnyMessageLikeEventContent,
+}
@@ -0,0 +1,32 @@
+// Copyright 2020 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use crate::types::CrossSigningKey;
+
+/// Request that will publish a cross signing identity.
+///
+/// This uploads the public cross signing key triplet.
+#[derive(Debug, Clone)]
+pub struct UploadSigningKeysRequest {
+    /// The user's master key.
+    pub master_key: Option<CrossSigningKey>,
+    /// The user's self-signing key. Must be signed with the accompanied master,
+    /// or by the user's most recently uploaded master key if no master key
+    /// is included in the request.
+    pub self_signing_key: Option<CrossSigningKey>,
+    /// The user's user-signing key. Must be signed with the accompanied master,
+    /// or by the user's most recently uploaded master key if no master key
+    /// is included in the request.
+    pub user_signing_key: Option<CrossSigningKey>,
+}
@@ -0,0 +1,136 @@
+// Copyright 2020 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use std::{collections::BTreeMap, iter};
+
+use ruma::{
+    events::{AnyToDeviceEventContent, EventContent, ToDeviceEventType},
+    serde::Raw,
+    to_device::DeviceIdOrAllDevices,
+    OwnedDeviceId, OwnedTransactionId, OwnedUserId, TransactionId, UserId,
+};
+use serde::{Deserialize, Serialize};
+
+/// Customized version of
+/// `ruma_client_api::to_device::send_event_to_device::v3::Request`
+#[derive(Clone, Debug, Deserialize, Serialize)]
+pub struct ToDeviceRequest {
+    /// Type of event being sent to each device.
+    pub event_type: ToDeviceEventType,
+
+    /// A request identifier unique to the access token used to send the
+    /// request.
+    pub txn_id: OwnedTransactionId,
+
+    /// A map of users to devices to a content for a message event to be
+    /// sent to the user's device. Individual message events can be sent
+    /// to devices, but all events must be of the same type.
+    /// The content's type for this field will be updated in a future
+    /// release, until then you can create a value using
+    /// `serde_json::value::to_raw_value`.
+    pub messages:
+        BTreeMap<OwnedUserId, BTreeMap<DeviceIdOrAllDevices, Raw<AnyToDeviceEventContent>>>,
+}
+
+impl ToDeviceRequest {
+    /// Create a new owned to-device request
+    ///
+    /// # Arguments
+    ///
+    /// * `recipient` - The ID of the user that should receive this to-device
+    ///   event.
+    ///
+    /// * `recipient_device` - The device that should receive this to-device
+    ///   event, or all devices.
+    ///
+    /// * `event_type` - The type of the event content that is getting sent out.
+    ///
+    /// * `content` - The content of the to-device event.
+    pub fn new(
+        recipient: &UserId,
+        recipient_device: impl Into<DeviceIdOrAllDevices>,
+        event_type: &str,
+        content: Raw<AnyToDeviceEventContent>,
+    ) -> Self {
+        let event_type = ToDeviceEventType::from(event_type);
+        let user_messages = iter::once((recipient_device.into(), content)).collect();
+        let messages = iter::once((recipient.to_owned(), user_messages)).collect();
+
+        ToDeviceRequest { event_type, txn_id: TransactionId::new(), messages }
+    }
+
+    pub(crate) fn for_recipients(
+        recipient: &UserId,
+        recipient_devices: Vec<OwnedDeviceId>,
+        content: &AnyToDeviceEventContent,
+        txn_id: OwnedTransactionId,
+    ) -> Self {
+        let event_type = content.event_type();
+        let raw_content = Raw::new(content).expect("Failed to serialize to-device event");
+
+        if recipient_devices.is_empty() {
+            Self::new(
+                recipient,
+                DeviceIdOrAllDevices::AllDevices,
+                &event_type.to_string(),
+                raw_content,
+            )
+        } else {
+            let device_messages = recipient_devices
+                .into_iter()
+                .map(|d| (DeviceIdOrAllDevices::DeviceId(d), raw_content.clone()))
+                .collect();
+
+            let messages = iter::once((recipient.to_owned(), device_messages)).collect();
+
+            ToDeviceRequest { event_type, txn_id, messages }
+        }
+    }
+
+    pub(crate) fn with_id_raw(
+        recipient: &UserId,
+        recipient_device: impl Into<DeviceIdOrAllDevices>,
+        content: Raw<AnyToDeviceEventContent>,
+        event_type: ToDeviceEventType,
+        txn_id: OwnedTransactionId,
+    ) -> Self {
+        let user_messages = iter::once((recipient_device.into(), content)).collect();
+        let messages = iter::once((recipient.to_owned(), user_messages)).collect();
+
+        ToDeviceRequest { event_type, txn_id, messages }
+    }
+
+    pub(crate) fn with_id(
+        recipient: &UserId,
+        recipient_device: impl Into<DeviceIdOrAllDevices>,
+        content: &AnyToDeviceEventContent,
+        txn_id: OwnedTransactionId,
+    ) -> Self {
+        let event_type = content.event_type();
+        let raw_content = Raw::new(content).expect("Failed to serialize to-device event");
+
+        let user_messages = iter::once((recipient_device.into(), raw_content)).collect();
+        let messages = iter::once((recipient.to_owned(), user_messages)).collect();
+
+        ToDeviceRequest { event_type, txn_id, messages }
+    }
+
+    /// Get the number of unique messages this request contains.
+    ///
+    /// *Note*: A single message may be sent to multiple devices, so this may or
+    /// may not be the number of devices that will receive the messages as well.
+    pub fn message_count(&self) -> usize {
+        self.messages.values().map(|d| d.len()).sum()
+    }
+}
@@ -0,0 +1,48 @@
+// Copyright 2020 The Matrix.org Foundation C.I.C.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use ruma::TransactionId;
+
+use super::{RoomMessageRequest, ToDeviceRequest};
+
+/// An enum over the different outgoing verification based requests.
+#[derive(Clone, Debug)]
+pub enum OutgoingVerificationRequest {
+    /// The to-device verification request variant.
+    ToDevice(ToDeviceRequest),
+    /// The in-room verification request variant.
+    InRoom(RoomMessageRequest),
+}
+
+impl OutgoingVerificationRequest {
+    /// Get the unique id of this request.
+    pub fn request_id(&self) -> &TransactionId {
+        match self {
+            OutgoingVerificationRequest::ToDevice(t) => &t.txn_id,
+            OutgoingVerificationRequest::InRoom(r) => &r.txn_id,
+        }
+    }
+}
+
+impl From<ToDeviceRequest> for OutgoingVerificationRequest {
+    fn from(r: ToDeviceRequest) -> Self {
+        OutgoingVerificationRequest::ToDevice(r)
+    }
+}
+
+impl From<RoomMessageRequest> for OutgoingVerificationRequest {
+    fn from(r: RoomMessageRequest) -> Self {
+        OutgoingVerificationRequest::InRoom(r)
+    }
+}
@@ -24,9 +24,11 @@ use tracing::debug;
 use tracing::{trace, warn};

 use super::{event_enums::OutgoingContent, FlowId, Sas, Verification};
+use crate::types::requests::{
+    OutgoingRequest, OutgoingVerificationRequest, RoomMessageRequest, ToDeviceRequest,
+};
 #[cfg(feature = "qrcode")]
 use crate::QrVerification;
-use crate::{OutgoingRequest, OutgoingVerificationRequest, RoomMessageRequest, ToDeviceRequest};

 #[derive(Clone, Debug, Default)]
 pub struct VerificationCache {
@@ -679,9 +679,9 @@ impl From<(OwnedRoomId, AnyMessageLikeEventContent)> for OutgoingContent {
    }
 }

-use crate::{
-    types::events::ToDeviceEvents, OutgoingRequest, OutgoingVerificationRequest,
-    RoomMessageRequest, ToDeviceRequest,
+use crate::types::{
+    events::ToDeviceEvents,
+    requests::{OutgoingRequest, OutgoingVerificationRequest, RoomMessageRequest, ToDeviceRequest},
 };

 impl TryFrom<OutgoingVerificationRequest> for OutgoingContent {
@@ -765,13 +765,15 @@ impl TryFrom<OutgoingRequest> for OutgoingContent {
    type Error = String;

    fn try_from(value: OutgoingRequest) -> Result<Self, Self::Error> {
+        use crate::types::requests::AnyOutgoingRequest;
+
        match value.request() {
-            crate::OutgoingRequests::KeysUpload(_)
-            | crate::OutgoingRequests::KeysQuery(_)
-            | crate::OutgoingRequests::SignatureUpload(_)
-            | crate::OutgoingRequests::KeysClaim(_) => Err("Invalid request type".to_owned()),
-            crate::OutgoingRequests::ToDeviceRequest(r) => Self::try_from(r.clone()),
-            crate::OutgoingRequests::RoomMessage(r) => Ok(Self::from(r.clone())),
+            AnyOutgoingRequest::KeysUpload(_)
+            | AnyOutgoingRequest::KeysQuery(_)
+            | AnyOutgoingRequest::SignatureUpload(_)
+            | AnyOutgoingRequest::KeysClaim(_) => Err("Invalid request type".to_owned()),
+            AnyOutgoingRequest::ToDeviceRequest(r) => Self::try_from(r.clone()),
+            AnyOutgoingRequest::RoomMessage(r) => Ok(Self::from(r.clone())),
        }
    }
 }
@@ -38,10 +38,11 @@ use super::{
 };
 use crate::{
    olm::{PrivateCrossSigningIdentity, StaticAccountData},
-    requests::OutgoingRequest,
    store::{CryptoStoreError, CryptoStoreWrapper},
-    DeviceData, OtherUserIdentityData, OutgoingVerificationRequest, RoomMessageRequest,
-    ToDeviceRequest,
+    types::requests::{
+        OutgoingRequest, OutgoingVerificationRequest, RoomMessageRequest, ToDeviceRequest,
+    },
+    DeviceData, OtherUserIdentityData,
 };

 #[derive(Clone, Debug)]
--- a/Show More
+++ b/Show More