55 Commits

Author SHA1 Message Date
Skye Elliot 640fa4854f feat(sqlite): Add query_many helper trait method
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
2026-02-25 16:02:35 +00:00
Ivan Enderlin a8e4630f56 chore(sqlite): Update deadpool to 0.13 and deadpool-sync to 0.2.
These releases include our patch to fix a panic (see
https://github.com/deadpool-rs/deadpool/pull/461).
2026-02-24 16:26:45 +01:00
Ivan Enderlin 09a1bf8ec2 fix(sqlite): Replace unwrap when using interact.
This patch replaces the `interact(…).unwrap()` by a proper error.

So far, `interact()` was only returning `InteractError::Panic`
despites `InteractError::Aborted` exists. With
https://github.com/deadpool-rs/deadpool/pull/461, we now get
`InteractError::Aborted` when the SDK is shutdown, sometimes. This
results in hitting the `unwrap` and having a panic again. This patch
solves the problem by changing the `unwrap` to a proper error. Note: in
case of `InteractError::Panic`, we continue to panic.

This patch makes sense with or without the merge of the PR on
`deadpool`.
2026-01-30 15:08:39 +01:00
Jorge Martín 6b0c1e2992 doc: Add doc and inline comments 2026-01-07 16:11:40 +01:00
Jorge Martín 3be6fb1a80 fix(sqlite): Add WAL checkpoints when vacuuming
For some reason, the automatic WAL checkpoints don't seem to be working as expected. Since we should periodically run VACUUM operations, we might as well add checkpoints before vacuuming (so the WAL size is reset and can grow to fit the whole DB) and after (so we clean up after that).
2026-01-07 16:11:40 +01:00
Jonas Platte eae3006f8d Reformat matrix-sdk-sqlite 2025-12-11 16:18:42 +01:00
Jorge Martín bc45457d0e feat: add Client::get_store_sizes
This method will retrieve the database sizes if available and expose it in the client.

Note: the actual database size measuring is only implemented for the SQLite based stores
2025-12-03 15:41:42 +01:00
Jorge Martín 1e7bc1286e refactor: Add trace log to ensure the VACUUM operation has finished successfully
This was a bit confusing, because I treated a lack of logs as success when in reality my code was calling an empty implementation
2025-12-03 15:41:42 +01:00
Ivan Enderlin 6a828e31dd feat(sqlite): Replace deadpool-sqlite by our own implementation.
This patch replaces `deadpool-sqlite` by our own implementation in
`crate::connection`. It still uses `deadpool` but the object manager has
a different implementation.
2025-11-11 11:09:18 +01:00
multi prise 0d0e2aa472 Add ZeroiseOnDrop trait to secret and make the key a Box 2025-09-24 12:35:17 +02:00
multi prise c51536a054 reformat 2025-09-24 12:35:17 +02:00
multi prise 6099928b40 Remove conditional logic for running tests 2025-09-24 12:35:17 +02:00
multi prise fac1f295b2 Correct wrong borrow 2025-09-24 12:35:17 +02:00
multi prise 24d02a72e3 implement zeroizing of secrets after use 2025-09-24 12:35:17 +02:00
multi prise 2a073043fd Revert "Use of lifetime in order to not clone/copy the data"
This reverts commit 009ee3a0e5fdff1332aaf0b1e62ab2577d728b82.
2025-09-24 12:35:17 +02:00
multi prise 8e759befd3 Refactorize tests config to correspond with the new api 2025-09-24 12:35:17 +02:00
multi prise 9faffa5b10 Temporary comment insecure function 2025-09-24 12:35:17 +02:00
multi prise 75b8c9fe93 Use of lifetime in order to not clone/copy the data 2025-09-24 12:35:17 +02:00
multi prise 004d98230c Uncomment the config directives and allows test to run faster by usinf an insecure export function 2025-09-24 12:35:17 +02:00
multi prise 6e1e0981b1 remove typo 2025-09-24 12:35:17 +02:00
multi prise 5d169ae765 Comment the use 2025-09-24 12:35:17 +02:00
multi prise 34dd7ea3cd Revert get_or_create_store_cypher to use 2025-09-24 12:35:17 +02:00
multi prise f6d2e73cab More reformarting of files 2025-09-24 12:35:17 +02:00
multi prise 0dfecd78d6 Updated the store encryption to use a enum Secret instead of passphrase 2025-09-24 12:35:17 +02:00
multi prise c17dbf9ebe Replace the passphrase logic with a key logic in the implementation of encrypted store 2025-09-24 12:35:17 +02:00
Benjamin Bouvier c92a89d571 chore(sqlite): reorder methods and add doc comment for encode_key
This reorders methods so that they're grouped in "dual" pairs
(encode/decode, serialize/deserialize). Also adds a doc comment to
`encode_key`, as I've wondered in the past what it did.
2025-07-09 12:37:34 +02:00
Benjamin Bouvier 684f228e70 refactor(sqlite): share implementations of the encode/decode/serialize/deserialize sqlite store helpers 2025-07-09 12:37:34 +02:00
Damir Jelić 511cf78d51 chore: Bump the rusqlite version 2025-04-09 16:37:40 +02:00
Ivan Enderlin 339b220488 feat(sqlite): Introduce RuntimeConfig which includes cache_size
This patch updates `StoreOpenConfig` to hold a new type: `RuntimeConfig`.

This `RuntimeConfig` type is passed to a new `SqliteAsyncConnExt`
method, named `apply_runtime_config`. Depending on the values passed
here, the `optimize`, `cache_size` (new!) and `journal_size_limit`
methods will be called automatically.

The goal of this type is to automate a flow we keep repeating in
all the stores. This is error-prone. This type brings uniformity and
consistency.

This patch also makes all `open_with_pool` methods on the stores private
(they were public before):

1. they were never used as far as I know because getting a `SqlitePool`
   isn't possible since the `pool` attribute is private…
2. it's better to keep control of this flow.
2025-03-28 13:36:32 +01:00
Kévin Commaille 65ee18a52d feat(sqlite): Run VACUUM operation after removing a room
A room can be associated to a lot of data, depending on the number of members in the room.
So freeing space on the filesystem should be worth it in some cases.

An (extreme) example: I have a test account that is in ~60 rooms, a few of those big public rooms, including Matrix HQ. The size of the matrix-sdk-state.sqlite3 file is 542 MB. Using this PR and leaving, then forgetting Matrix HQ brings the DB down to 255 MB.
2025-02-11 14:13:25 +00:00
Kévin Commaille f641a639cd Merge branch 'main' into media-cache-auto-cleanup 2025-02-04 11:56:32 +01:00
Kévin Commaille 6ebd4295b9 feat(sqlite): Limit size of WAL file
The WAL file can grow depending on the transactions that are run. A
critical case is VACUUM which basically writes the content of the DB
file to the WAL file before writing it back to the DB file.

SQLite doesn't try to reduce the size of the file after that unless we
set an explicit limit,
so we could end up taking twice the size of the database on the
filesystem.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2025-02-03 11:22:23 +01:00
Kévin Commaille c5104d68fd feat(sqlite): Run PRAGMA optimize regularly
As recommended by the SQLite docs.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2025-02-03 11:22:23 +01:00
Kévin Commaille 582b3a91d6 refactor(sqlite): Add methods to get and set values in the kv table by (de)serializing them
Since it's a common occurrence, it will reduce duplication.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2025-02-02 11:47:43 +01:00
Ivan Enderlin 5049d1a3b6 chore(sqlite): Use repeat_n(…, n) instead of repeat(…).take(n).
Thanks Clippy!
2025-01-31 14:07:43 +01:00
Kévin Commaille 8ca5983093 feat(sqlite): Implement EventCacheStoreMedia for SqliteEventCacheStore
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2025-01-28 15:38:18 +01:00
Valere 2b39476d9b feat(crypto): Support storing the dehydrated device pickle key 2024-12-13 13:05:19 +01:00
Damir Jelić a1b7906a7d chore: Fix some clippy lints around lifetimes 2024-11-27 13:23:34 +01:00
Kévin Commaille 66e901bb9b sqlite: Make migrations atomic
Setting the version number only when all migrations are done
means that the version will be wrong if a migration fails.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-29 09:31:40 +03:00
Kévin Commaille d17a49e827 sqlite: Add more methods to SqliteKeyValueStoreAsyncConnExt
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-28 15:45:12 +02:00
Kévin Commaille 1321e92b30 sqlite: Rename SqliteObjectStoreExt to SqliteKeyValueStoreAsyncConnExt
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-28 15:45:12 +02:00
Kévin Commaille 6e62f8b269 sqlite: Rename utils::ConnectionExt to SqliteKeyValueStoreConnExt
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-28 15:45:12 +02:00
Kévin Commaille 6e369aecc9 sqlite: Rename SqliteConn to SqliteAsyncConn and SqliteObjectExt to SqliteAsyncConnExt
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-28 15:45:12 +02:00
Kévin Commaille b9d49b85c3 sqlite: Use transaction for chunked queries
Allows the operation to be atomic.
Also allows to chunk part of a transaction.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-26 18:20:41 +02:00
Kévin Commaille 154f86aa20 sqlite: Rename deadpool_sqlite::Object to SqliteConn consistently
So we know that its always the same type when reading the code.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-26 10:40:12 +02:00
Andy Balaam dfb33c9534 Move chunk_large_query_over into SqliteObjectExt so it can be reused
Signed-off-by: Andy Balaam <andy.balaam@matrix.org>
2023-12-14 16:55:36 +00:00
Andy Balaam 03aad4e965 Move repeat_vars into utils so it can be reused
Signed-off-by: Andy Balaam <andy.balaam@matrix.org>
2023-12-14 16:34:45 +00:00
Andy Balaam 17ebc23719 Provide a limit() method in sqlite to find limit values
Signed-off-by: Andy Balaam <andy.balaam@matrix.org>
2023-12-14 11:21:25 +00:00
Kévin Commaille b8abd1c022 base: Add state store method to fetch several profiles at once
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2023-06-14 13:51:52 +02:00
Kévin Commaille ea219d836e base: Do not separate stripped room info
Since stripped and non-stripped room infos use the same types,
the separation is not necessary anymore.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2023-06-14 10:45:37 +02:00