Commit Graph

3965 Commits

Author SHA1 Message Date
3nprob e87ce873b0 utils: Fix bug in deepCompare which would incorrectly return objects with disjoint keys as equal (#2586)
* utils: Fix bug in deepCompare which would incorrectly return objects with disjoint keys as equal

* Fix bugs in sync test

This test wrongly asserted that `initialSyncLimit` would be used to make a filter
It is used only for the initial sync inline filter, and not in POST /filter

Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2022-09-01 21:36:24 +00:00
RiotRobot bbaa0e6536 Merge branch 'master' into develop 2022-08-31 16:26:39 +01:00
Travis Ralston 8716c1ab9b Convert several internal maps to real maps 2022-08-31 09:21:46 -06:00
Faye Duxovni 1c77816dbd Use deep equality comparisons when searching for outgoing key requests by target (#2623) 2022-08-27 03:13:08 +00:00
Michael Telatynski b789cc5933 Refactor Sync and fix initialSyncLimit (#2587)
* Small tidy-up to sync.ts

* Convert doSync into a while loop

* Apply `initialSyncLimit` only to initial syncs

* Convert matrix-client-syncing spec to TS

* Add tests around initial sync filtering

* Switch confusing filterId field for `filter`

* Tweak doSync error control flow

* Fix error control flow intricacies

* use includes

* Add tests

* Fix some strict mode errors

* Fix more strict mode errors

* Fix some strict mode errors
2022-08-23 16:25:54 +01:00
Jonathan Otto 5e4474b959 Fix room membership race with PREPARED event (#2613)
* Fix room membership race with PREPARED event

See the call site of the original triggering event of this function: https://github.com/matrix-org/matrix-js-sdk/blob/b265d795a427c6d30ccdf279a09f7836509df863/src/sliding-sync.ts#L789-L806

I think the bug is current code assumes downstream event listeners of `SlidingSyncEvent.RoomData` have synchronous execution so that by the time it emits `SlidingSyncState.Complete`, and eventually `SyncState.Prepared` the room state is correct. But since SlidingSyncSdk's `processRoomData` is async, and the membership field was being set after the async, it looks like `SlidingSyncState.Complete` was being fired before the membership field was set.

* Rm whitespace
2022-08-23 15:01:33 +00:00
Michael Telatynski 8e646ea584 Add static analysis for tsc --strict (#2615)
* Initial attempt at CI to annotate new TSC errors

* Make tsconfig file valid

* enable debug

* Specify commit

* Fix commit specification

* Switch back to main

* Tweak permissions

* Add strict mode failure

* Attempt number two

* Fix ts-extra-args

* Add static analysis for tsc --strict
2022-08-23 14:02:50 +01:00
kegsay 1df329df7c Merge pull request #2610 from matrix-org/kegan/ss-bugfix
sliding sync: handle lone DELETE and INSERT operations
2022-08-23 08:42:34 +01:00
Kegan Dougal de3b3960d2 Add tags and not_tags to the list of valid sliding sync filters 2022-08-22 18:20:51 +01:00
Robin b265d795a4 Re-emit room state events on rooms (#2607)
* Re-emit room state events on rooms

This also fixes some potential memory leaks and abuse of
removeAllListeners in sync.ts.

* Remove some stray whitespace

* Deduplicate some code to appease SonarCloud

* Name helper function more explicitly
2022-08-22 17:04:32 +02:00
Michael Telatynski eb79f6246d Add ability to override built in room name generator for an i18n'able one (#2609)
* Add ability to override built in room name generator for an i18n'able one

* Add tests
2022-08-22 14:39:04 +01:00
Kegan Dougal 37f8f736e0 sliding sync: handle lone DELETE and INSERT operations
If you leave a room you can get a lone DELETE op.
If you join a room you can get a lone INSERT op.

Up until now, we've assumed these operations happen at the ends
of the list (e.g [0] or [length-1]) which is not guaranteed as it
depends on the sort order (e.g sort alphabetically and join a room
called 'D'). In this scenario, the indexes would not be tracked
correctly. Fixed with integration tests.
2022-08-22 14:37:10 +01:00
Šimon Brandner 3ae974e23e Remove duplicate log of answering a call (#2595) 2022-08-17 10:23:06 +00:00
Kerry be3e731499 Fix: Handle parsing of a beacon info event without asset (#2591)
* test case

* handle missing beacon info asset

* default beacon info asset type to self

* make BeaconLocationState.assetType optional
2022-08-16 15:33:19 +02:00
3nprob 3f6f5b69c7 Improve test coverage and modernize style for interactive-auth (#2574)
* style: address no-mixed-operators errors,minor style improvements

* test: Fix async interactive-auth tests, add test case

* tests: Fix incorrectly stringified mock response

* pushprocessor: style update

* use async primitives in interactive-auth-spec

* lint

* fixup: remove duplicate test

* add test case for no-flow-with-session for interactive-auth

* interactive-auth: handle non-existing error.data

* async test fix

* test: add dummyauth test

* add testing for errcode

* Revert "pushprocessor: style update"

This reverts commit 3ed0fdfb73ae55b725aa7c74d9cab35fb96c9178.

* add testcase for missing error data

* test: move sessionId assignment

* Add tests to improve coverage for interactive-auth

* pushprocessor: style update
2022-08-11 15:29:53 +01:00
Šimon Brandner 478270b225 Fix finding event read up to if stable private read receipts is missing (#2585)
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
2022-08-11 13:30:51 +01:00
Kegan Dougal 2728d74771 Review comments 2022-08-10 19:53:36 +01:00
Kegan Dougal edcef9364c Only add events if there are some; set the pagination token for faster scrollback 2022-08-10 12:43:47 +01:00
Kegan Dougal 1635ac9971 sliding sync bugfix: ensure history is treated as history and not live events
with tests
2022-08-10 12:32:28 +01:00
kegsay 8f13df2dd9 Merge pull request #2567 from matrix-org/kegan/sync-v3
Add txn_id support to sliding sync
2022-08-10 12:16:47 +01:00
Kegan Dougal fa9f078a75 Review comments 2022-08-10 11:53:13 +01:00
Faye Duxovni 3762c20aad Revert "Always block sending keys to unverified devices of verified users (#2562)" (#2571)
This will be rolled out again later with more accompanying UI adjustments, including clearer error messages and possibly the option to disable it per-room.
2022-08-08 12:27:41 -04:00
Kegan Dougal 2596999cb8 Review comments 2022-08-08 14:26:24 +01:00
Kegan Dougal a3248c0aa1 Merge branch 'develop' into kegan/sync-v3 2022-08-08 12:51:03 +01:00
Matthew Hodgson 43c81358b2 don't load the sync accumulator if there's already a sync persist in flight
this should hopefully reduce chances of
https://github.com/vector-im/element-web/issues/21541 a bit more
as we were incorrectly loading the sync accumulator even
if a sync persist was already in flight, thus wasting RAM
and increasing the chance of the renderer process OOMing
2022-08-07 01:13:06 +01:00
Kegan Dougal e05f9b5815 Add txn_id support to sliding sync
This allows clients to know when a request has been applied
on the server. This allows us to change `resend(): void` to
`resend(): Promise<string>` which resolves/rejects with the
transaction ID when it has been applied/not.
2022-08-05 17:28:02 +01:00
Šimon Brandner 6316a6ae44 Add support for stable prefixes for MSC2285 (#2524)
Co-authored-by: Travis Ralston <travisr@matrix.org>
2022-08-05 17:33:49 +02:00
David Baker 575b416856 Simplify encryptAndSendToDevices (#2566)
It went to quite a lot of effort to gather a bunch of information to
return, but the only thing using it already had all that info anyway.
2022-08-05 15:58:33 +01:00
Robin c629d2f60e Emit an event when the client receives TURN servers (#2529)
* Emit an event when the client receives TURN servers

* Add tests

* Fix lints
2022-08-04 11:44:10 -04:00
Faye Duxovni 43b453804b Always block sending keys to unverified devices of verified users (#2562) 2022-08-04 11:11:12 -04:00
Šimon Brandner d867affc40 Remove stream-replacement (#2551) 2022-08-03 21:45:37 +02:00
Robin c36bfc821c Add support for sending user-defined encrypted to-device messages (#2528)
* Add support for sending user-defined encrypted to-device messages

This is a port of the same change from the robertlong/group-call branch.

* Fix tests

* Expose the method in MatrixClient

* Fix a code smell

* Fix types

* Test the MatrixClient method

* Fix some types in Crypto test suite

* Test the Crypto method

* Fix tests

* Upgrade matrix-mock-request

* Move useRealTimers to afterEach
2022-08-03 16:16:48 +00:00
David Baker 7e784da00a Retry to-device messages (#2549)
* Retry to-device messages

This adds a queueToDevice API alongside sendToDevice which is a
much higher-level API that adds the messages to a queue, stored in
persistent storage, and retries them periodically. Also converts
sending of megolm keys to use the new API.

Other uses of sendToDevice are nopt converted in this PR, but could
be later.

Requires https://github.com/matrix-org/matrix-mock-request/pull/17

* Bump matrix-mock-request

* Add more waits to make indexeddb tests pass

* Switch some test expectations to queueToDevice

* Stop straight away if the client has been stopped

Hopefully will fix tests being flakey and logging after tests have
finished.

* Add return types & fix constant usage

* Fix return type

Co-authored-by: Germain <germains@element.io>

* Fix return type

Co-authored-by: Germain <germains@element.io>

* Fix return type

Co-authored-by: Germain <germains@element.io>

* Stop the client in all test cases

Co-authored-by: Germain <germains@element.io>
2022-08-03 13:32:58 +01:00
Germain b79f469008 Use EventType enum values instead of hardcoded strings (#2557) 2022-08-03 08:54:11 +00:00
Kegan Dougal fa6708c27e Gracefully handle missing room_ids 2022-08-01 16:34:11 +01:00
Kegan Dougal 4427201326 Sliding sync: add missing filters from latest MSC 2022-08-01 16:30:33 +01:00
David Baker 3824f65d15 Prevent double mute status changed events (#2502) (#2522)
Audio & video mute status were set in separate calls but share a
mute status changed event, so you'd always get two mute status
changed events emitted. We could suppress events where the mute
status didn't change, but this would still get two events saying
the same thing when they both changed. Instead, merge setAudioMuted
& setVideoMuted into a single call that sets either or both.

Port of https://github.com/matrix-org/matrix-js-sdk/pull/2502 from
group call branch
2022-07-28 16:13:00 +01:00
Kerry 75513d08de test typescriptification - misc (#2547)
* renamed:    spec/unit/login.spec.js -> spec/unit/login.spec.ts

* type test client

* renamed:    spec/unit/interactive-auth.spec.js -> spec/unit/interactive-auth.spec.ts

* fix ts issues in interactive-auth.spec

* renamed:    spec/unit/filter.spec.js -> spec/unit/filter.spec.ts

* fix ts in filter.spec

* renamed:    spec/unit/event.spec.js -> spec/unit/event.spec.ts

* ts in event.spec

* renamed:    spec/unit/pushprocessor.spec.js -> spec/unit/pushprocessor.spec.ts

* fix ts in pushprocessor.spec

* fix ts in realtime-callbacks.spec

* renamed:    spec/unit/content-repo.spec.js -> spec/unit/content-repo.spec.ts

* fix signature for getHttpUriForMxc

* pr fixes
2022-07-28 08:09:21 +02:00
Šimon Brandner 7cb3b40493 Use stable prefixes for MSC3827 (#2537) 2022-07-27 20:10:39 +02:00
Faye Duxovni 5367ee18fb Re-insert room IDs when decrypting bundled redaction events returned by /sync (#2531) 2022-07-21 10:55:20 +00:00
Hubert Chathi 7e8dfa56d0 Support fixed base64 in SAS verification (#2320) 2022-07-20 09:16:40 -04:00
Travis Ralston 08ab51eeac Remove unstable support for m.room_key.withheld (#2512)
We no longer send or receive the unstable type.
2022-07-13 08:56:01 -06:00
Robin 5112340040 Correct the units in TURN servers expiry documentation (#2520)
As shown elsewhere in client.ts, turnServersExpiry really is in
milliseconds rather than seconds. It seems that other libraries like
matrix-react-sdk were already expecting it to be in milliseconds
anyways, so it's just the documentation that was wrong.
2022-07-12 18:48:44 +00:00
kegsay 8d7eaa769a Add support for MSC3575: Sliding Sync (#2242)
* sliding sync: add client function and add stub sliding-sync.ts

Mostly c/p from sync.ts. Define interfaces for MSC3575 sliding
sync types. Complete WIP!

* Add core sliding sync classes

* Add integration tests for sliding sync api basics

* gut unused code; add more types

* Use SlidingSync in MatrixClient; stub functions for Sync

Enough to make ele-web actually load okay with 0 rooms.

* Start feeding through room data to the client

* Bugfixes so it sorta ish works

* Refactor the public API for sliding sync

Still needs some work but it's a start.

* Use EventEmitter for callbacks. Add ability to adjust lists and listen for list updates.

- Have atomic getList/setList operations on SlidingSync to update windows etc
- Add a list callback which is invoked with the list indicies and joined count.

* Add stub tests; add listenUntil to make tests easier to read

* No need to resend now

* Add more sliding sync tests; add new setListRanges function

* build tests upon one another to reduce boilerplate and c/p

* More thorough sliding sync tests

* Dependency inject SlidingSync in Client opts when calling startClient()

* Linting

* Fix crash when opts is undefined

* Fix up docs to make CI happy

* Remove all listeners when stop()d to allow for GC

* Add support for extensions

* Add ExtensionE2EE automatically if opts.crypto is present

* Add ExtensionToDevice automatically

* Bugfixes for to_device message processing

* default events to []

* bugfix: don't tightloop when the server is down

Caused by not detecting abort() correctly

* Return null for bad index positions

* Add getListData to get the initial calculated list response

* Add is_tombstoned

* More comments

* Add support for account data extension; rejig extension interface

* Handle invite_state

* Feed through prev_batch tokens

* Linting

* Fix tests

* Linting

* Iterate PR

* Iterate tests and remove unused code

* Update matrix-mock-request

* Make tests happier

* Remove DEBUG/debuglog and use logger.debug

* Update the API to the latest MSC; fixup tests

* Use undefined not null to make it work with the latest changes

* Don't recreate rooms when initial: true

* Add defensive code when unsigned.transaction_id is missing

We can still pair up events by looking at the event_id. We need
to do this in Sliding Sync because the proxy has limitations that
means it cannot guarantee it will always incude a transaction_id
in unsigned. The main reason why is due to the following race condition:
 - A and B are in a DM room.
 - Both are using the proxy.
 - A says "hello".
 - B's sync stream gets "hello" on the proxy. At this point the proxy
   knows it needs to deliver it to A. It does so, but this event has
   no transaction_id as it came down B's sync stream, not A's.
 - If instead, A's sync stream gets "hello" on the proxy, the proxy
   will deliver this message with the transaction_id correctly set.

There are no guarantees that A's sync stream will get the event in a
timely manner, hence the decision to just deliver the events as soon
as the proxy gets the event. This will not be an issue for native
Sliding Sync implementations; this is just a proxy issue.

* Linting

* Add additional sliding sync tests

* Begin adding SlidingSyncSdk tests

* Linting

* Add more sliding sync sdk tests

* Prep work for extension tests

* Linting

* Add account data extension tests

* add to-device tests

* Add E2EE extension tests

* Code smell fixes and extra tests

* Add test for no-txn-id local echo

* Add tests for resolveProfilesToInvites

* Add tests for moving entries down as well as up the list

* Remove conn-management.ts

* Actually verify the event was removed from the txn map

* Handle the case when /sync returns before /send without a txn_id

And ensure all the tests actually test the right things.

* Linting

Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2022-07-12 14:09:58 +00:00
Šimon Brandner 1fbfdaf221 Don't crash with undefined room in processBeaconEvents() (#2500) 2022-07-11 10:03:44 +02:00
Šimon Brandner c4f7e4d5aa Remove dead code (#2510) 2022-07-11 09:46:50 +02:00
Šimon Brandner 9a6dccb79b Remove setNow from realtime-callbacks.ts (#2509)
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
2022-07-10 14:31:48 +02:00
Faye Duxovni 3935152d08 Properly re-insert room ID in bundled thread relation messages from sync (#2505)
Events returned by the `/sync` endpoint, including relations bundled with other events, may have their `room_id`s stripped out. This causes decryption errors if the IDs aren't repopulated.

Fixes vector-im/element-web#22094.
2022-07-08 22:43:38 +00:00
Travis Ralston 72f9a51c27 Actually store the identity server in the client when given as an option (#2503)
* Actually store the identity server in the client when given as an option

* Update requestRegisterEmailToken to a modern spec version too
2022-07-08 01:07:28 -06:00
Travis Ralston efdda8425d Remove MSC3244 support (#2504) 2022-07-08 00:32:27 -06:00