Šimon Brandner bf81c4bfeb Add E2EE for embedded mode of Element Call (#3667)
* WIP refactor for removing m.call events

* Always remember rtcsessions since we need to only have one instance

* Fix tests

* Fix import loop

* Fix more cyclic imports & tests

* Test session joining

* Attempt to make tests happy

* Always leave calls in the tests to clean up

* comment + desperate attempt to work out what's failing

* More test debugging

* Okay, so these ones are fine?

* Stop more timers and hopefully have happy tests

* Test no rejoin

* Test malformed m.call.member events

* Test event emitting

and also move some code to a more sensible place in the file

* Test getActiveFoci()

* Test event emitting (and also fix it)

* Test membership updating & pruning on join

* Test getOldestMembership()

* Test member event renewal

* Don't start the rtc manager until the client has synced

Then we can initialise from the state once it's completed.

* Fix type

* Remove listeners added in constructor

* Stop the client here too

* Stop the client here also also

* ARGH. Disable tests to work out which one is causing the exception

* Disable everything

* Re-jig to avoid setting listeners in the constructor

and re-enable tests

* No need to rename this anymore

* argh, remove the right listener

* Is it this test???

* Re-enable some tests

* Try mocking getRooms to return something valid

* Re-enable other tests

* Give up trying to get the tests to work sensibly and deal with getRooms() returning nothing

* Oops, don't enable the ones that were skipped before

* One more try at the sensible way

* Didn't work, go back to the hack way.

* Log when we manage to send the member event update

* Support `getOpenIdToken()` in embedded mode (#3676)

* Call `sendContentLoaded()` (#3677)

* Start MatrixRTC in embedded mode (#3679)

* Reschedule the membership event check

* Bump widget api version

* Add mock for sendContentLoaded()

* Embeded mode pre-requisites

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Embeded mode E2EE

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Encryption condition

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert "Embeded mode pre-requisites"

This reverts commit 8cd73702052609c995ad754e31f85d0da0be4aa9.

* Get back event type

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

fds

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Change embedded E2EE implementation

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* More log detail

* Fix tests

and also better assert because the tests were passing undefined which
was considered fine because we were only checking for null.

* Simplify updateCallMembershipEvent a bit

* Split up updateCallMembershipEvent some more

* Use `crypto.getRandomValues()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Rename to `membershipToUserAndDeviceId()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Better error

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add log line

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add comment

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Send call ID in enc events

(also a small refactor)

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert making `joinRoomSession()` async

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `client` `private` again

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Just use `toString()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix `callId` check

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix map

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix map compare

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix emitting

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Explicit logging

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Refactor

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `updateEncryptionKeyEvent()` public

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Only update keys based on others

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix call order

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Improve logging

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Avoid races

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert "Avoid races"

This reverts commit f65ed72d6e.

* Add try-catch

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `updateEncryptionKeyEvent()` private

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Handle indices and throttling

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix merge mistakes

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Mort post-merge fixes

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Split out key generation from key sending

And send all keys in a key event (changes the format of the key event)
rather than just the one we just generated.

* Remember and clear the timeout for the send key event

So we don't schedule more key updates if one is already pending.
Also don't update the last sent time when we didn't actually send the
keys.

* Make key event resends more robust

* Attempt to make tests pass

* crypto wasn't defined at all

* Hopefully get interface right

* Fix key format on the wire to base64

* Add comment

* More standard method order

* Rename encryptMedia

The js-sdk doesn't do media and therefore doesn't do media encryption

* Stop logging encryption keys now

* Use regular base64

It's not going in a URL, so no need

* Re-add base64url

randomstring was using it. Also give it a test.

* Add tests for randomstring

* Switch between either browser or node crypto

Let's see if this will work...

* Obviously crypto has already solved this

* Some tests for MatrixRTCSession key stuff

* Test keys object contents

* Change keys event format

To move away from m. keys

* Test key event retries

* Test onCallEncryption

* Test event sending & spam prevention

* Test event cancelation

* Test onCallEncryption called

* Some errors didn't have data

* Fix binary key comparison

& add log line

* Fix compare function with undefined values

* Remove more key logging

* Check content.keys is an array

* Check key index & key

* Better function name

* Tests too

---------

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
Co-authored-by: David Baker <dave@matrix.org>
Co-authored-by: David Baker <dbkr@users.noreply.github.com>
2023-10-31 16:01:46 +00:00
2023-10-17 10:27:19 +01:00
2023-10-03 10:23:11 +01:00
2022-12-09 09:38:20 +01:00
2022-12-09 09:34:01 +01:00
2023-10-24 15:27:01 +01:00
2022-12-09 09:38:20 +01:00
2015-03-04 17:37:46 +00:00
2023-05-16 13:58:05 +01:00

npm Tests Static Analysis Quality Gate Status Coverage Vulnerabilities Bugs

Matrix JavaScript SDK

This is the Matrix Client-Server SDK for JavaScript and TypeScript. This SDK can be run in a browser or in Node.js.

Minimum Matrix server version: v1.1

The Matrix specification is constantly evolving - while this SDK aims for maximum backwards compatibility, it only guarantees that a feature will be supported for at least 4 spec releases. For example, if a feature the js-sdk supports is removed in v1.4 then the feature is eligible for removal from the SDK when v1.8 is released. This SDK has no guarantee on implementing all features of any particular spec release, currently. This can mean that the SDK will call endpoints from before Matrix 1.1, for example.

Quickstart

In a browser

Note, the browserify build has been removed. Please use a bundler like webpack or vite instead.

In Node.js

Ensure you have the latest LTS version of Node.js installed. This library relies on fetch which is available in Node from v18.0.0 - it should work fine also with polyfills. If you wish to use a ponyfill or adapter of some sort then pass it as fetchFn to the MatrixClient constructor options.

Using yarn instead of npm is recommended. Please see the Yarn install guide if you do not have it already.

yarn add matrix-js-sdk

import * as sdk from "matrix-js-sdk";
const client = sdk.createClient({ baseUrl: "https://matrix.org" });
client.publicRooms(function (err, data) {
    console.log("Public Rooms: %s", JSON.stringify(data));
});

See below for how to include libolm to enable end-to-end-encryption. Please check the Node.js terminal app for a more complex example.

You can also use the sdk with Deno (import npm:matrix-js-sdk) but its not officialy supported.

To start the client:

await client.startClient({ initialSyncLimit: 10 });

You can perform a call to /sync to get the current state of the client:

client.once("sync", function (state, prevState, res) {
    if (state === "PREPARED") {
        console.log("prepared");
    } else {
        console.log(state);
        process.exit(1);
    }
});

To send a message:

const content = {
    body: "message text",
    msgtype: "m.text",
};
client.sendEvent("roomId", "m.room.message", content, "", (err, res) => {
    console.log(err);
});

To listen for message events:

client.on("Room.timeline", function (event, room, toStartOfTimeline) {
    if (event.getType() !== "m.room.message") {
        return; // only use messages
    }
    console.log(event.event.content.body);
});

By default, the matrix-js-sdk client uses the MemoryStore to store events as they are received. For example to iterate through the currently stored timeline for a room:

Object.keys(client.store.rooms).forEach((roomId) => {
    client.getRoom(roomId).timeline.forEach((t) => {
        console.log(t.event);
    });
});

What does this SDK do?

This SDK provides a full object model around the Matrix Client-Server API and emits events for incoming data and state changes. Aside from wrapping the HTTP API, it:

  • Handles syncing (via /initialSync and /events)
  • Handles the generation of "friendly" room and member names.
  • Handles historical RoomMember information (e.g. display names).
  • Manages room member state across multiple events (e.g. it handles typing, power levels and membership changes).
  • Exposes high-level objects like Rooms, RoomState, RoomMembers and Users which can be listened to for things like name changes, new messages, membership changes, presence changes, and more.
  • Handle "local echo" of messages sent using the SDK. This means that messages that have just been sent will appear in the timeline as 'sending', until it completes. This is beneficial because it prevents there being a gap between hitting the send button and having the "remote echo" arrive.
  • Mark messages which failed to send as not sent.
  • Automatically retry requests to send messages due to network errors.
  • Automatically retry requests to send messages due to rate limiting errors.
  • Handle queueing of messages.
  • Handles pagination.
  • Handle assigning push actions for events.
  • Handles room initial sync on accepting invites.
  • Handles WebRTC calling.

Later versions of the SDK will:

  • Expose a RoomSummary which would be suitable for a recents page.
  • Provide different pluggable storage layers (e.g. local storage, database-backed)

Usage

Conventions

Emitted events

The SDK will emit events using an EventEmitter. It also emits object models (e.g. Rooms, RoomMembers) when they are updated.

// Listen for low-level MatrixEvents
client.on("event", function (event) {
    console.log(event.getType());
});

// Listen for typing changes
client.on("RoomMember.typing", function (event, member) {
    if (member.typing) {
        console.log(member.name + " is typing...");
    } else {
        console.log(member.name + " stopped typing.");
    }
});

// start the client to setup the connection to the server
client.startClient();

Promises and Callbacks

Most of the methods in the SDK are asynchronous: they do not directly return a result, but instead return a Promise which will be fulfilled in the future.

The typical usage is something like:

  matrixClient.someMethod(arg1, arg2).then(function(result) {
    ...
  });

Alternatively, if you have a Node.js-style callback(err, result) function, you can pass the result of the promise into it with something like:

matrixClient.someMethod(arg1, arg2).nodeify(callback);

The main thing to note is that it is problematic to discard the result of a promise-returning function, as that will cause exceptions to go unobserved.

Methods which return a promise show this in their documentation.

Many methods in the SDK support both Node.js-style callbacks and Promises, via an optional callback argument. The callback support is now deprecated: new methods do not include a callback argument, and in the future it may be removed from existing methods.

Examples

This section provides some useful code snippets which demonstrate the core functionality of the SDK. These examples assume the SDK is setup like this:

import * as sdk from "matrix-js-sdk";
const myUserId = "@example:localhost";
const myAccessToken = "QGV4YW1wbGU6bG9jYWxob3N0.qPEvLuYfNBjxikiCjP";
const matrixClient = sdk.createClient({
    baseUrl: "http://localhost:8008",
    accessToken: myAccessToken,
    userId: myUserId,
});

Automatically join rooms when invited

matrixClient.on("RoomMember.membership", function (event, member) {
    if (member.membership === "invite" && member.userId === myUserId) {
        matrixClient.joinRoom(member.roomId).then(function () {
            console.log("Auto-joined %s", member.roomId);
        });
    }
});

matrixClient.startClient();

Print out messages for all rooms

matrixClient.on("Room.timeline", function (event, room, toStartOfTimeline) {
    if (toStartOfTimeline) {
        return; // don't print paginated results
    }
    if (event.getType() !== "m.room.message") {
        return; // only print messages
    }
    console.log(
        // the room name will update with m.room.name events automatically
        "(%s) %s :: %s",
        room.name,
        event.getSender(),
        event.getContent().body,
    );
});

matrixClient.startClient();

Output:

  (My Room) @megan:localhost :: Hello world
  (My Room) @megan:localhost :: how are you?
  (My Room) @example:localhost :: I am good
  (My Room) @example:localhost :: change the room name
  (My New Room) @megan:localhost :: done

Print out membership lists whenever they are changed

matrixClient.on("RoomState.members", function (event, state, member) {
    const room = matrixClient.getRoom(state.roomId);
    if (!room) {
        return;
    }
    const memberList = state.getMembers();
    console.log(room.name);
    console.log(Array(room.name.length + 1).join("=")); // underline
    for (var i = 0; i < memberList.length; i++) {
        console.log("(%s) %s", memberList[i].membership, memberList[i].name);
    }
});

matrixClient.startClient();

Output:

  My Room
  =======
  (join) @example:localhost
  (leave) @alice:localhost
  (join) Bob
  (invite) @charlie:localhost

API Reference

A hosted reference can be found at http://matrix-org.github.io/matrix-js-sdk/index.html

This SDK uses Typedoc doc comments. You can manually build and host the API reference from the source files like this:

  $ yarn gendoc
  $ cd _docs
  $ python -m http.server 8005

Then visit http://localhost:8005 to see the API docs.

End-to-end encryption support

The SDK supports end-to-end encryption via the Olm and Megolm protocols, using libolm. It is left up to the application to make libolm available, via the Olm global.

It is also necessary to call await matrixClient.initCrypto() after creating a new MatrixClient (but before calling matrixClient.startClient()) to initialise the crypto layer.

If the Olm global is not available, the SDK will show a warning, as shown below; initCrypto() will also fail.

Unable to load crypto module: crypto will be disabled: Error: global.Olm is not defined

If the crypto layer is not (successfully) initialised, the SDK will continue to work for unencrypted rooms, but it will not support the E2E parts of the Matrix specification.

To provide the Olm library in a browser application:

To provide the Olm library in a node.js application:

  • yarn add https://packages.matrix.org/npm/olm/olm-3.1.4.tgz (replace the URL with the latest version you want to use from https://packages.matrix.org/npm/olm/)
  • global.Olm = require('olm'); before loading matrix-js-sdk.

If you want to package Olm as dependency for your node.js application, you can use yarn add https://packages.matrix.org/npm/olm/olm-3.1.4.tgz. If your application also works without e2e crypto enabled, add --optional to mark it as an optional dependency.

Contributing

This section is for people who want to modify the SDK. If you just want to use this SDK, skip this section.

First, you need to pull in the right build tools:

 $ yarn install

Building

To build a browser version from scratch when developing:

 $ yarn build

To run tests (Jest):

 $ yarn test

To run linting:

 $ yarn lint
S
Description
No description provided
Readme 57 MiB
Languages
TypeScript 99.4%
Python 0.4%
JavaScript 0.2%