kegsay 8d7eaa769a Add support for MSC3575: Sliding Sync (#2242)
* sliding sync: add client function and add stub sliding-sync.ts

Mostly c/p from sync.ts. Define interfaces for MSC3575 sliding
sync types. Complete WIP!

* Add core sliding sync classes

* Add integration tests for sliding sync api basics

* gut unused code; add more types

* Use SlidingSync in MatrixClient; stub functions for Sync

Enough to make ele-web actually load okay with 0 rooms.

* Start feeding through room data to the client

* Bugfixes so it sorta ish works

* Refactor the public API for sliding sync

Still needs some work but it's a start.

* Use EventEmitter for callbacks. Add ability to adjust lists and listen for list updates.

- Have atomic getList/setList operations on SlidingSync to update windows etc
- Add a list callback which is invoked with the list indicies and joined count.

* Add stub tests; add listenUntil to make tests easier to read

* No need to resend now

* Add more sliding sync tests; add new setListRanges function

* build tests upon one another to reduce boilerplate and c/p

* More thorough sliding sync tests

* Dependency inject SlidingSync in Client opts when calling startClient()

* Linting

* Fix crash when opts is undefined

* Fix up docs to make CI happy

* Remove all listeners when stop()d to allow for GC

* Add support for extensions

* Add ExtensionE2EE automatically if opts.crypto is present

* Add ExtensionToDevice automatically

* Bugfixes for to_device message processing

* default events to []

* bugfix: don't tightloop when the server is down

Caused by not detecting abort() correctly

* Return null for bad index positions

* Add getListData to get the initial calculated list response

* Add is_tombstoned

* More comments

* Add support for account data extension; rejig extension interface

* Handle invite_state

* Feed through prev_batch tokens

* Linting

* Fix tests

* Linting

* Iterate PR

* Iterate tests and remove unused code

* Update matrix-mock-request

* Make tests happier

* Remove DEBUG/debuglog and use logger.debug

* Update the API to the latest MSC; fixup tests

* Use undefined not null to make it work with the latest changes

* Don't recreate rooms when initial: true

* Add defensive code when unsigned.transaction_id is missing

We can still pair up events by looking at the event_id. We need
to do this in Sliding Sync because the proxy has limitations that
means it cannot guarantee it will always incude a transaction_id
in unsigned. The main reason why is due to the following race condition:
 - A and B are in a DM room.
 - Both are using the proxy.
 - A says "hello".
 - B's sync stream gets "hello" on the proxy. At this point the proxy
   knows it needs to deliver it to A. It does so, but this event has
   no transaction_id as it came down B's sync stream, not A's.
 - If instead, A's sync stream gets "hello" on the proxy, the proxy
   will deliver this message with the transaction_id correctly set.

There are no guarantees that A's sync stream will get the event in a
timely manner, hence the decision to just deliver the events as soon
as the proxy gets the event. This will not be an issue for native
Sliding Sync implementations; this is just a proxy issue.

* Linting

* Add additional sliding sync tests

* Begin adding SlidingSyncSdk tests

* Linting

* Add more sliding sync sdk tests

* Prep work for extension tests

* Linting

* Add account data extension tests

* add to-device tests

* Add E2EE extension tests

* Code smell fixes and extra tests

* Add test for no-txn-id local echo

* Add tests for resolveProfilesToInvites

* Add tests for moving entries down as well as up the list

* Remove conn-management.ts

* Actually verify the event was removed from the txn map

* Handle the case when /sync returns before /send without a txn_id

And ensure all the tests actually test the right things.

* Linting

Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2022-07-12 14:09:58 +00:00
2021-07-13 20:11:58 +02:00
2022-07-05 14:06:48 +01:00
2015-03-04 17:37:46 +00:00
2022-05-03 14:52:44 -06:00
2022-05-31 10:43:46 +01:00

npm Tests Static Analysis Quality Gate Status Coverage Vulnerabilities Bugs

Matrix Javascript SDK

This is the Matrix Client-Server r0 SDK for JavaScript. This SDK can be run in a browser or in Node.js.

Quickstart

In a browser

Download the browser version from https://github.com/matrix-org/matrix-js-sdk/releases/latest and add that as a <script> to your page. There will be a global variable matrixcs attached to window through which you can access the SDK. See below for how to include libolm to enable end-to-end-encryption.

The browser bundle supports recent versions of browsers. Typically this is ES2015 or > 0.5%, last 2 versions, Firefox ESR, not dead if using browserlists.

Please check the working browser example for more information.

In Node.js

Ensure you have the latest LTS version of Node.js installed.

This SDK targets Node 12 for compatibility, which translates to ES6. If you're using a bundler like webpack you'll likely have to transpile dependencies, including this SDK, to match your target browsers.

Using yarn instead of npm is recommended. Please see the Yarn install guide if you do not have it already.

yarn add matrix-js-sdk

  import * as sdk from "matrix-js-sdk";
  const client = sdk.createClient("https://matrix.org");
  client.publicRooms(function(err, data) {
    console.log("Public Rooms: %s", JSON.stringify(data));
  });

See below for how to include libolm to enable end-to-end-encryption. Please check the Node.js terminal app for a more complex example.

To start the client:

await client.startClient({initialSyncLimit: 10});

You can perform a call to /sync to get the current state of the client:

client.once('sync', function(state, prevState, res) {
    if(state === 'PREPARED') {
        console.log("prepared");
    } else {
        console.log(state);
        process.exit(1);
    }
});

To send a message:

const content = {
    "body": "message text",
    "msgtype": "m.text"
};
client.sendEvent("roomId", "m.room.message", content, "", (err, res) => {
    console.log(err);
});

To listen for message events:

client.on("Room.timeline", function(event, room, toStartOfTimeline) {
  if (event.getType() !== "m.room.message") {
    return; // only use messages
  }
  console.log(event.event.content.body);
});

By default, the matrix-js-sdk client uses the MemoryStore to store events as they are received. For example to iterate through the currently stored timeline for a room:

Object.keys(client.store.rooms).forEach((roomId) => {
  client.getRoom(roomId).timeline.forEach(t => {
      console.log(t.event);
  });
});

What does this SDK do?

This SDK provides a full object model around the Matrix Client-Server API and emits events for incoming data and state changes. Aside from wrapping the HTTP API, it:

  • Handles syncing (via /initialSync and /events)
  • Handles the generation of "friendly" room and member names.
  • Handles historical RoomMember information (e.g. display names).
  • Manages room member state across multiple events (e.g. it handles typing, power levels and membership changes).
  • Exposes high-level objects like Rooms, RoomState, RoomMembers and Users which can be listened to for things like name changes, new messages, membership changes, presence changes, and more.
  • Handle "local echo" of messages sent using the SDK. This means that messages that have just been sent will appear in the timeline as 'sending', until it completes. This is beneficial because it prevents there being a gap between hitting the send button and having the "remote echo" arrive.
  • Mark messages which failed to send as not sent.
  • Automatically retry requests to send messages due to network errors.
  • Automatically retry requests to send messages due to rate limiting errors.
  • Handle queueing of messages.
  • Handles pagination.
  • Handle assigning push actions for events.
  • Handles room initial sync on accepting invites.
  • Handles WebRTC calling.

Later versions of the SDK will:

  • Expose a RoomSummary which would be suitable for a recents page.
  • Provide different pluggable storage layers (e.g. local storage, database-backed)

Usage

Conventions

Emitted events

The SDK will emit events using an EventEmitter. It also emits object models (e.g. Rooms, RoomMembers) when they are updated.

  // Listen for low-level MatrixEvents
  client.on("event", function(event) {
    console.log(event.getType());
  });

  // Listen for typing changes
  client.on("RoomMember.typing", function(event, member) {
    if (member.typing) {
      console.log(member.name + " is typing...");
    }
    else {
      console.log(member.name + " stopped typing.");
    }
  });

  // start the client to setup the connection to the server
  client.startClient();

Promises and Callbacks

Most of the methods in the SDK are asynchronous: they do not directly return a result, but instead return a Promise which will be fulfilled in the future.

The typical usage is something like:

  matrixClient.someMethod(arg1, arg2).then(function(result) {
    ...
  });

Alternatively, if you have a Node.js-style callback(err, result) function, you can pass the result of the promise into it with something like:

  matrixClient.someMethod(arg1, arg2).nodeify(callback);

The main thing to note is that it is problematic to discard the result of a promise-returning function, as that will cause exceptions to go unobserved.

Methods which return a promise show this in their documentation.

Many methods in the SDK support both Node.js-style callbacks and Promises, via an optional callback argument. The callback support is now deprecated: new methods do not include a callback argument, and in the future it may be removed from existing methods.

Examples

This section provides some useful code snippets which demonstrate the core functionality of the SDK. These examples assume the SDK is setup like this:

   import * as sdk from "matrix-js-sdk";
   const myUserId = "@example:localhost";
   const myAccessToken = "QGV4YW1wbGU6bG9jYWxob3N0.qPEvLuYfNBjxikiCjP";
   const matrixClient = sdk.createClient({
       baseUrl: "http://localhost:8008",
       accessToken: myAccessToken,
       userId: myUserId
   });

Automatically join rooms when invited

   matrixClient.on("RoomMember.membership", function(event, member) {
       if (member.membership === "invite" && member.userId === myUserId) {
           matrixClient.joinRoom(member.roomId).then(function() {
               console.log("Auto-joined %s", member.roomId);
           });
       }
   });

   matrixClient.startClient();

Print out messages for all rooms

   matrixClient.on("Room.timeline", function(event, room, toStartOfTimeline) {
       if (toStartOfTimeline) {
           return; // don't print paginated results
       }
       if (event.getType() !== "m.room.message") {
           return; // only print messages
       }
       console.log(
           // the room name will update with m.room.name events automatically
           "(%s) %s :: %s", room.name, event.getSender(), event.getContent().body
       );
   });

   matrixClient.startClient();

Output:

  (My Room) @megan:localhost :: Hello world
  (My Room) @megan:localhost :: how are you?
  (My Room) @example:localhost :: I am good
  (My Room) @example:localhost :: change the room name
  (My New Room) @megan:localhost :: done

Print out membership lists whenever they are changed

   matrixClient.on("RoomState.members", function(event, state, member) {
       const room = matrixClient.getRoom(state.roomId);
       if (!room) {
           return;
       }
       const memberList = state.getMembers();
       console.log(room.name);
       console.log(Array(room.name.length + 1).join("="));  // underline
       for (var i = 0; i < memberList.length; i++) {
           console.log(
               "(%s) %s",
               memberList[i].membership,
               memberList[i].name
           );
       }
   });

   matrixClient.startClient();

Output:

  My Room
  =======
  (join) @example:localhost
  (leave) @alice:localhost
  (join) Bob
  (invite) @charlie:localhost

API Reference

A hosted reference can be found at http://matrix-org.github.io/matrix-js-sdk/index.html

This SDK uses JSDoc3 style comments. You can manually build and host the API reference from the source files like this:

  $ yarn gendoc
  $ cd .jsdoc
  $ python -m SimpleHTTPServer 8005

Then visit http://localhost:8005 to see the API docs.

End-to-end encryption support

The SDK supports end-to-end encryption via the Olm and Megolm protocols, using libolm. It is left up to the application to make libolm available, via the Olm global.

It is also necessary to call await matrixClient.initCrypto() after creating a new MatrixClient (but before calling matrixClient.startClient()) to initialise the crypto layer.

If the Olm global is not available, the SDK will show a warning, as shown below; initCrypto() will also fail.

Unable to load crypto module: crypto will be disabled: Error: global.Olm is not defined

If the crypto layer is not (successfully) initialised, the SDK will continue to work for unencrypted rooms, but it will not support the E2E parts of the Matrix specification.

To provide the Olm library in a browser application:

To provide the Olm library in a node.js application:

  • yarn add https://packages.matrix.org/npm/olm/olm-3.1.4.tgz (replace the URL with the latest version you want to use from https://packages.matrix.org/npm/olm/)
  • global.Olm = require('olm'); before loading matrix-js-sdk.

If you want to package Olm as dependency for your node.js application, you can use yarn add https://packages.matrix.org/npm/olm/olm-3.1.4.tgz. If your application also works without e2e crypto enabled, add --optional to mark it as an optional dependency.

Contributing

This section is for people who want to modify the SDK. If you just want to use this SDK, skip this section.

First, you need to pull in the right build tools:

 $ yarn install

Building

To build a browser version from scratch when developing::

 $ yarn build

To run tests (Jasmine)::

 $ yarn test

To run linting:

 $ yarn lint
S
Description
No description provided
Readme 57 MiB
Languages
TypeScript 99.4%
Python 0.4%
JavaScript 0.2%