v0.9.2

Prepare changelog for v0.9.2
Send correct m.visibility flag in js-sdk
2017-12-04 11:57:05 +00:00 · 2017-12-04 11:57:04 +00:00 · 2017-12-04 11:54:42 +00:00 · 2017-11-17 15:55:21 +00:00 · 2017-11-17 15:55:20 +00:00 · 2017-11-17 15:51:01 +00:00
28 changed files with 1866 additions and 557 deletions
@@ -9,6 +9,10 @@ module.exports = {
    env: {
        browser: true,
        node: true,
+
+        // babel's transform-runtime converts references to ES6 globals such as
+        // Promise and Map to core-js polyfills, so we can use ES6 globals.
+        es6: true,
    },
    extends: ["eslint:recommended", "google"],
    rules: {
@@ -1,3 +1,118 @@
+Changes in [0.9.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.9.2) (2017-12-04)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.9.1...v0.9.2)
+
+
+Changes in [0.9.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.9.1) (2017-11-17)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.9.0...v0.9.1)
+
+ * Fix the force TURN option
+   [\#577](https://github.com/matrix-org/matrix-js-sdk/pull/577)
+
+Changes in [0.9.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.9.0) (2017-11-15)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.9.0-rc.1...v0.9.0)
+
+
+Changes in [0.9.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.9.0-rc.1) (2017-11-10)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.5...v0.9.0-rc.1)
+
+ * Modify addRoomToGroup to allow setting isPublic, create alias
+   updateGroupRoomAssociation
+   [\#567](https://github.com/matrix-org/matrix-js-sdk/pull/567)
+ * Expose more functionality of pushprocessor
+   [\#565](https://github.com/matrix-org/matrix-js-sdk/pull/565)
+ * Function for working out notif trigger permission
+   [\#566](https://github.com/matrix-org/matrix-js-sdk/pull/566)
+ * keep track of event ID and timestamp of decrypted messages
+   [\#555](https://github.com/matrix-org/matrix-js-sdk/pull/555)
+ * Fix notifEvent computation
+   [\#564](https://github.com/matrix-org/matrix-js-sdk/pull/564)
+ * Fix power level of sentinel members
+   [\#563](https://github.com/matrix-org/matrix-js-sdk/pull/563)
+ * don't try to decrypt a redacted message (fixes vector-im/riot-web#3744)
+   [\#554](https://github.com/matrix-org/matrix-js-sdk/pull/554)
+ * Support room notifs
+   [\#562](https://github.com/matrix-org/matrix-js-sdk/pull/562)
+ * Fix the glob-to-regex code
+   [\#558](https://github.com/matrix-org/matrix-js-sdk/pull/558)
+
+Changes in [0.8.5](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.8.5) (2017-10-16)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.5-rc.1...v0.8.5)
+
+ * Make unknown pushrule conditions not match
+   [\#559](https://github.com/matrix-org/matrix-js-sdk/pull/559)
+
+Changes in [0.8.5-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.8.5-rc.1) (2017-10-13)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.4...v0.8.5-rc.1)
+
+ * Implement wrapper API for removing a room from a group
+   [\#553](https://github.com/matrix-org/matrix-js-sdk/pull/553)
+ * Fix typo which resulted in stuck key download requests
+   [\#552](https://github.com/matrix-org/matrix-js-sdk/pull/552)
+ * Store group when it's created
+   [\#549](https://github.com/matrix-org/matrix-js-sdk/pull/549)
+ * Luke/groups remove rooms users from summary
+   [\#548](https://github.com/matrix-org/matrix-js-sdk/pull/548)
+ * Clean on prepublish
+   [\#546](https://github.com/matrix-org/matrix-js-sdk/pull/546)
+ * Implement wrapper APIs for adding rooms to group summary
+   [\#545](https://github.com/matrix-org/matrix-js-sdk/pull/545)
+
+Changes in [0.8.4](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.8.4) (2017-09-21)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.3...v0.8.4)
+
+ * Fix build issue
+
+Changes in [0.8.3](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.8.3) (2017-09-20)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.3-rc.1...v0.8.3)
+
+ * No changes
+
+Changes in [0.8.3-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.8.3-rc.1) (2017-09-19)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.2...v0.8.3-rc.1)
+
+ * consume trailing slash when creating Matrix Client in HS and IS urls
+   [\#526](https://github.com/matrix-org/matrix-js-sdk/pull/526)
+ * Add ignore users API
+   [\#539](https://github.com/matrix-org/matrix-js-sdk/pull/539)
+ * Upgrade to jsdoc 3.5.5
+   [\#540](https://github.com/matrix-org/matrix-js-sdk/pull/540)
+ * Make re-emitting events much more memory efficient
+   [\#538](https://github.com/matrix-org/matrix-js-sdk/pull/538)
+ * Only re-emit events from Event objects if needed
+   [\#536](https://github.com/matrix-org/matrix-js-sdk/pull/536)
+ * Handle 'left' users in the deviceList mananagement
+   [\#535](https://github.com/matrix-org/matrix-js-sdk/pull/535)
+ * Factor out devicelist integration tests to a separate file
+   [\#534](https://github.com/matrix-org/matrix-js-sdk/pull/534)
+ * Refactor sync._sync as an async function
+   [\#533](https://github.com/matrix-org/matrix-js-sdk/pull/533)
+ * Add es6 to eslint environments
+   [\#532](https://github.com/matrix-org/matrix-js-sdk/pull/532)
+
+Changes in [0.8.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.8.2) (2017-08-24)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.1...v0.8.2)
+
+ * Handle m.call.* events which are decrypted asynchronously
+   [\#530](https://github.com/matrix-org/matrix-js-sdk/pull/530)
+ * Re-emit events from, er, Event objects
+   [\#529](https://github.com/matrix-org/matrix-js-sdk/pull/529)
+
+Changes in [0.8.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.8.1) (2017-08-23)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.1-rc.1...v0.8.1)
+
+ * [No changes]
+
 Changes in [0.8.1-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.8.1-rc.1) (2017-08-22)
 ==========================================================================================================
 [Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.8.0...v0.8.1-rc.1)
@@ -1,6 +1,6 @@
 {
  "name": "matrix-js-sdk",
-  "version": "0.8.1-rc.1",
+  "version": "0.9.2",
  "description": "Matrix Client-Server SDK for Javascript",
  "main": "index.js",
  "scripts": {
@@ -11,11 +11,12 @@
    "check": "npm run test:build && _mocha --recursive specbuild --colors",
    "gendoc": "babel --no-babelrc -d .jsdocbuild src && jsdoc -r .jsdocbuild -P package.json -R README.md -d .jsdoc",
    "start": "babel -s -w -d lib src",
+    "clean": "rimraf lib dist",
    "build": "babel -s -d lib src && rimraf dist && mkdir dist && browserify -d browser-index.js | exorcist dist/browser-matrix.js.map > dist/browser-matrix.js && uglifyjs -c -m -o dist/browser-matrix.min.js --source-map dist/browser-matrix.min.js.map --in-source-map dist/browser-matrix.js.map dist/browser-matrix.js",
    "dist": "npm run build",
    "watch": "watchify -d browser-index.js -o 'exorcist dist/browser-matrix.js.map > dist/browser-matrix.js' -v",
    "lint": "eslint --max-warnings 109 src spec",
-    "prepublish": "npm run build && git rev-parse HEAD > git-revision.txt"
+    "prepublish": "npm run clean && npm run build && git rev-parse HEAD > git-revision.txt"
  },
  "repository": {
    "url": "https://github.com/matrix-org/matrix-js-sdk"
@@ -49,6 +50,7 @@
  ],
  "dependencies": {
    "another-json": "^0.2.0",
+    "babel-runtime": "^6.26.0",
    "bluebird": "^3.5.0",
    "browser-request": "^0.3.3",
    "content-type": "^1.0.2",
@@ -67,7 +69,7 @@
    "exorcist": "^0.4.0",
    "expect": "^1.20.2",
    "istanbul": "^0.4.5",
-    "jsdoc": "^3.5.0",
+    "jsdoc": "^3.5.5",
    "lolex": "^1.5.2",
    "matrix-mock-request": "^1.2.0",
    "mocha": "^3.2.0",
@@ -33,12 +33,20 @@ import Promise from 'bluebird';
 * @param {string} userId
 * @param {string} deviceId
 * @param {string} accessToken
+ *
+ * @param {WebStorage=} sessionStoreBackend a web storage object to use for the
+ *     session store. If undefined, we will create a MockStorageApi.
 */
-export default function TestClient(userId, deviceId, accessToken) {
+export default function TestClient(
+    userId, deviceId, accessToken, sessionStoreBackend,
+) {
    this.userId = userId;
    this.deviceId = deviceId;

-    this.storage = new sdk.WebStorageSessionStore(new testUtils.MockStorageApi());
+    if (sessionStoreBackend === undefined) {
+        sessionStoreBackend = new testUtils.MockStorageApi();
+    }
+    this.storage = new sdk.WebStorageSessionStore(sessionStoreBackend);
    this.httpBackend = new MockHttpBackend();
    this.client = sdk.createClient({
        baseUrl: "http://" + userId + ".test.server",
@@ -0,0 +1,369 @@
+import expect from 'expect';
+import Promise from 'bluebird';
+
+import TestClient from '../TestClient';
+import testUtils from '../test-utils';
+
+const ROOM_ID = "!room:id";
+
+/**
+ * get a /sync response which contains a single e2e room (ROOM_ID), with the
+ * members given
+ *
+ * @param {string[]} roomMembers
+ *
+ * @return {object} sync response
+ */
+function getSyncResponse(roomMembers) {
+    const stateEvents = [
+        testUtils.mkEvent({
+            type: 'm.room.encryption',
+            skey: '',
+            content: {
+                algorithm: 'm.megolm.v1.aes-sha2',
+            },
+        }),
+    ];
+
+    Array.prototype.push.apply(
+        stateEvents,
+        roomMembers.map(
+            (m) => testUtils.mkMembership({
+                mship: 'join',
+                sender: m,
+            }),
+        ),
+    );
+
+    const syncResponse = {
+        next_batch: 1,
+        rooms: {
+            join: {
+                [ROOM_ID]: {
+                    state: {
+                        events: stateEvents,
+                    },
+                },
+            },
+        },
+    };
+
+    return syncResponse;
+}
+
+
+describe("DeviceList management:", function() {
+    if (!global.Olm) {
+        console.warn('not running deviceList tests: Olm not present');
+        return;
+    }
+
+    let sessionStoreBackend;
+    let aliceTestClient;
+
+    async function createTestClient() {
+        const testClient = new TestClient(
+            "@alice:localhost", "xzcvb", "akjgkrgjs", sessionStoreBackend,
+        );
+        await testClient.client.initCrypto();
+        return testClient;
+    }
+
+    beforeEach(async function() {
+        testUtils.beforeEach(this); // eslint-disable-line no-invalid-this
+
+        // we create our own sessionStoreBackend so that we can use it for
+        // another TestClient.
+        sessionStoreBackend = new testUtils.MockStorageApi();
+
+        aliceTestClient = await createTestClient();
+    });
+
+    afterEach(function() {
+        aliceTestClient.stop();
+    });
+
+    it("Alice shouldn't do a second /query for non-e2e-capable devices", function() {
+        return aliceTestClient.start().then(function() {
+            const syncResponse = getSyncResponse(['@bob:xyz']);
+            aliceTestClient.httpBackend.when('GET', '/sync').respond(200, syncResponse);
+
+            return aliceTestClient.flushSync();
+        }).then(function() {
+            console.log("Forcing alice to download our device keys");
+
+            aliceTestClient.httpBackend.when('POST', '/keys/query').respond(200, {
+                device_keys: {
+                    '@bob:xyz': {},
+                },
+            });
+
+            return Promise.all([
+                aliceTestClient.client.downloadKeys(['@bob:xyz']),
+                aliceTestClient.httpBackend.flush('/keys/query', 1),
+            ]);
+        }).then(function() {
+            console.log("Telling alice to send a megolm message");
+
+            aliceTestClient.httpBackend.when(
+                'PUT', '/send/',
+            ).respond(200, {
+                    event_id: '$event_id',
+            });
+
+            return Promise.all([
+                aliceTestClient.client.sendTextMessage(ROOM_ID, 'test'),
+
+                // the crypto stuff can take a while, so give the requests a whole second.
+                aliceTestClient.httpBackend.flushAllExpected({
+                    timeout: 1000,
+                }),
+            ]);
+        });
+    });
+
+
+    it("We should not get confused by out-of-order device query responses",
+       () => {
+           // https://github.com/vector-im/riot-web/issues/3126
+           return aliceTestClient.start().then(() => {
+               aliceTestClient.httpBackend.when('GET', '/sync').respond(
+                   200, getSyncResponse(['@bob:xyz', '@chris:abc']));
+               return aliceTestClient.flushSync();
+           }).then(() => {
+               // to make sure the initial device queries are flushed out, we
+               // attempt to send a message.
+
+               aliceTestClient.httpBackend.when('POST', '/keys/query').respond(
+                   200, {
+                       device_keys: {
+                           '@bob:xyz': {},
+                           '@chris:abc': {},
+                       },
+                   },
+               );
+
+               aliceTestClient.httpBackend.when('PUT', '/send/').respond(
+                   200, {event_id: '$event1'});
+
+               return Promise.all([
+                   aliceTestClient.client.sendTextMessage(ROOM_ID, 'test'),
+                   aliceTestClient.httpBackend.flush('/keys/query', 1).then(
+                       () => aliceTestClient.httpBackend.flush('/send/', 1),
+                   ),
+               ]);
+           }).then(() => {
+               expect(aliceTestClient.storage.getEndToEndDeviceSyncToken()).toEqual(1);
+
+               // invalidate bob's and chris's device lists in separate syncs
+               aliceTestClient.httpBackend.when('GET', '/sync').respond(200, {
+                   next_batch: '2',
+                   device_lists: {
+                       changed: ['@bob:xyz'],
+                   },
+               });
+               aliceTestClient.httpBackend.when('GET', '/sync').respond(200, {
+                   next_batch: '3',
+                   device_lists: {
+                       changed: ['@chris:abc'],
+                   },
+               });
+               // flush both syncs
+               return aliceTestClient.flushSync().then(() => {
+                   return aliceTestClient.flushSync();
+               });
+           }).then(() => {
+               // check that we don't yet have a request for chris's devices.
+               aliceTestClient.httpBackend.when('POST', '/keys/query', {
+                   device_keys: {
+                       '@chris:abc': {},
+                   },
+                   token: '3',
+               }).respond(200, {
+                   device_keys: {'@chris:abc': {}},
+               });
+               return aliceTestClient.httpBackend.flush('/keys/query', 1);
+           }).then((flushed) => {
+               expect(flushed).toEqual(0);
+               const bobStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+               if (bobStat != 1 && bobStat != 2) {
+                   throw new Error('Unexpected status for bob: wanted 1 or 2, got ' +
+                                   bobStat);
+               }
+
+               const chrisStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];
+               if (chrisStat != 1 && chrisStat != 2) {
+                   throw new Error('Unexpected status for chris: wanted 1 or 2, got ' +
+                                   chrisStat);
+               }
+
+               // now add an expectation for a query for bob's devices, and let
+               // it complete.
+               aliceTestClient.httpBackend.when('POST', '/keys/query', {
+                   device_keys: {
+                       '@bob:xyz': {},
+                   },
+                   token: '2',
+               }).respond(200, {
+                   device_keys: {'@bob:xyz': {}},
+               });
+               return aliceTestClient.httpBackend.flush('/keys/query', 1);
+           }).then((flushed) => {
+               expect(flushed).toEqual(1);
+
+               // wait for the client to stop processing the response
+               return aliceTestClient.client.downloadKeys(['@bob:xyz']);
+           }).then(() => {
+               const bobStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+               expect(bobStat).toEqual(3);
+               const chrisStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];
+               if (chrisStat != 1 && chrisStat != 2) {
+                   throw new Error('Unexpected status for chris: wanted 1 or 2, got ' +
+                                   bobStat);
+               }
+
+               // now let the query for chris's devices complete.
+               return aliceTestClient.httpBackend.flush('/keys/query', 1);
+           }).then((flushed) => {
+               expect(flushed).toEqual(1);
+
+               // wait for the client to stop processing the response
+               return aliceTestClient.client.downloadKeys(['@chris:abc']);
+           }).then(() => {
+               const bobStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+               const chrisStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];
+
+               expect(bobStat).toEqual(3);
+               expect(chrisStat).toEqual(3);
+               expect(aliceTestClient.storage.getEndToEndDeviceSyncToken()).toEqual(3);
+           });
+       });
+
+    // https://github.com/vector-im/riot-web/issues/4983
+    describe("Alice should know she has stale device lists", () => {
+        beforeEach(async function() {
+            await aliceTestClient.start();
+
+            aliceTestClient.httpBackend.when('GET', '/sync').respond(
+                200, getSyncResponse(['@bob:xyz']));
+            await aliceTestClient.flushSync();
+
+            aliceTestClient.httpBackend.when('POST', '/keys/query').respond(
+                200, {
+                    device_keys: {
+                        '@bob:xyz': {},
+                    },
+                },
+            );
+            await aliceTestClient.httpBackend.flush('/keys/query', 1);
+
+            const bobStat = aliceTestClient.storage
+                      .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+
+            expect(bobStat).toBeGreaterThan(
+                0, "Alice should be tracking bob's device list",
+            );
+        });
+
+        it("when Bob leaves", async function() {
+            aliceTestClient.httpBackend.when('GET', '/sync').respond(
+                200, {
+                    next_batch: 2,
+                    device_lists: {
+                        left: ['@bob:xyz'],
+                    },
+                    rooms: {
+                        join: {
+                            [ROOM_ID]: {
+                                timeline: {
+                                    events: [
+                                        testUtils.mkMembership({
+                                            mship: 'leave',
+                                            sender: '@bob:xyz',
+                                        }),
+                                    ],
+                                },
+                            },
+                        },
+                    },
+                },
+            );
+
+
+            await aliceTestClient.flushSync();
+
+            const bobStat = aliceTestClient.storage
+                      .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+            expect(bobStat).toEqual(
+                0, "Alice should have marked bob's device list as untracked",
+            );
+        });
+
+        it("when Alice leaves", async function() {
+            aliceTestClient.httpBackend.when('GET', '/sync').respond(
+                200, {
+                    next_batch: 2,
+                    device_lists: {
+                        left: ['@bob:xyz'],
+                    },
+                    rooms: {
+                        leave: {
+                            [ROOM_ID]: {
+                                timeline: {
+                                    events: [
+                                        testUtils.mkMembership({
+                                            mship: 'leave',
+                                            sender: '@bob:xyz',
+                                        }),
+                                    ],
+                                },
+                            },
+                        },
+                    },
+                },
+            );
+
+            await aliceTestClient.flushSync();
+
+            const bobStat = aliceTestClient.storage
+                      .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+            expect(bobStat).toEqual(
+                0, "Alice should have marked bob's device list as untracked",
+            );
+        });
+
+        it("when Bob leaves whilst Alice is offline", async function() {
+            aliceTestClient.stop();
+
+            const anotherTestClient = await createTestClient();
+
+            try {
+                anotherTestClient.httpBackend.when('GET', '/keys/changes').respond(
+                    200, {
+                        changed: [],
+                        left: ['@bob:xyz'],
+                    },
+                );
+                await anotherTestClient.start();
+                anotherTestClient.httpBackend.when('GET', '/sync').respond(
+                    200, getSyncResponse([]));
+                await anotherTestClient.flushSync();
+
+                const bobStat = anotherTestClient.storage
+                      .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+
+                expect(bobStat).toEqual(
+                    0, "Alice should have marked bob's device list as untracked",
+                );
+            } finally {
+                anotherTestClient.stop();
+            }
+        });
+    });
+});
@@ -580,46 +580,6 @@ describe("megolm", function() {
        });
    });

-    it("Alice shouldn't do a second /query for non-e2e-capable devices", function() {
-        return aliceTestClient.start().then(function() {
-            const syncResponse = getSyncResponse(['@bob:xyz']);
-            aliceTestClient.httpBackend.when('GET', '/sync').respond(200, syncResponse);
-
-            return aliceTestClient.flushSync();
-        }).then(function() {
-            console.log("Forcing alice to download our device keys");
-
-            aliceTestClient.httpBackend.when('POST', '/keys/query').respond(200, {
-                device_keys: {
-                    '@bob:xyz': {},
-                },
-            });
-
-            return Promise.all([
-                aliceTestClient.client.downloadKeys(['@bob:xyz']),
-                aliceTestClient.httpBackend.flush('/keys/query', 1),
-            ]);
-        }).then(function() {
-            console.log("Telling alice to send a megolm message");
-
-            aliceTestClient.httpBackend.when(
-                'PUT', '/send/',
-            ).respond(200, {
-                    event_id: '$event_id',
-            });
-
-            return Promise.all([
-                aliceTestClient.client.sendTextMessage(ROOM_ID, 'test'),
-
-                // the crypto stuff can take a while, so give the requests a whole second.
-                aliceTestClient.httpBackend.flushAllExpected({
-                    timeout: 1000,
-                }),
-            ]);
-        });
-    });
-
-
    it("We shouldn't attempt to send to blocked devices", function() {
        return aliceTestClient.start().then(() => {
            // establish an olm session with alice
@@ -917,128 +877,6 @@ describe("megolm", function() {
    });


-    it("We should not get confused by out-of-order device query responses",
-       () => {
-           // https://github.com/vector-im/riot-web/issues/3126
-           return aliceTestClient.start().then(() => {
-               aliceTestClient.httpBackend.when('GET', '/sync').respond(
-                   200, getSyncResponse(['@bob:xyz', '@chris:abc']));
-               return aliceTestClient.flushSync();
-           }).then(() => {
-               // to make sure the initial device queries are flushed out, we
-               // attempt to send a message.
-
-               aliceTestClient.httpBackend.when('POST', '/keys/query').respond(
-                   200, {
-                       device_keys: {
-                           '@bob:xyz': {},
-                           '@chris:abc': {},
-                       },
-                   },
-               );
-
-               aliceTestClient.httpBackend.when('PUT', '/send/').respond(
-                   200, {event_id: '$event1'});
-
-               return Promise.all([
-                   aliceTestClient.client.sendTextMessage(ROOM_ID, 'test'),
-                   aliceTestClient.httpBackend.flush('/keys/query', 1).then(
-                       () => aliceTestClient.httpBackend.flush('/send/', 1),
-                   ),
-               ]);
-           }).then(() => {
-               expect(aliceTestClient.storage.getEndToEndDeviceSyncToken()).toEqual(1);
-
-               // invalidate bob's and chris's device lists in separate syncs
-               aliceTestClient.httpBackend.when('GET', '/sync').respond(200, {
-                   next_batch: '2',
-                   device_lists: {
-                       changed: ['@bob:xyz'],
-                   },
-               });
-               aliceTestClient.httpBackend.when('GET', '/sync').respond(200, {
-                   next_batch: '3',
-                   device_lists: {
-                       changed: ['@chris:abc'],
-                   },
-               });
-               // flush both syncs
-               return aliceTestClient.flushSync().then(() => {
-                   return aliceTestClient.flushSync();
-               });
-           }).then(() => {
-               // check that we don't yet have a request for chris's devices.
-               aliceTestClient.httpBackend.when('POST', '/keys/query', {
-                   device_keys: {
-                       '@chris:abc': {},
-                   },
-                   token: '3',
-               }).respond(200, {
-                   device_keys: {'@chris:abc': {}},
-               });
-               return aliceTestClient.httpBackend.flush('/keys/query', 1);
-           }).then((flushed) => {
-               expect(flushed).toEqual(0);
-               const bobStat = aliceTestClient.storage
-                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
-               if (bobStat != 1 && bobStat != 2) {
-                   throw new Error('Unexpected status for bob: wanted 1 or 2, got ' +
-                                   bobStat);
-               }
-
-               const chrisStat = aliceTestClient.storage
-                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];
-               if (chrisStat != 1 && chrisStat != 2) {
-                   throw new Error('Unexpected status for chris: wanted 1 or 2, got ' +
-                                   chrisStat);
-               }
-
-               // now add an expectation for a query for bob's devices, and let
-               // it complete.
-               aliceTestClient.httpBackend.when('POST', '/keys/query', {
-                   device_keys: {
-                       '@bob:xyz': {},
-                   },
-                   token: '2',
-               }).respond(200, {
-                   device_keys: {'@bob:xyz': {}},
-               });
-               return aliceTestClient.httpBackend.flush('/keys/query', 1);
-           }).then((flushed) => {
-               expect(flushed).toEqual(1);
-
-               // wait for the client to stop processing the response
-               return aliceTestClient.client.downloadKeys(['@bob:xyz']);
-           }).then(() => {
-               const bobStat = aliceTestClient.storage
-                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
-               expect(bobStat).toEqual(3);
-               const chrisStat = aliceTestClient.storage
-                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];
-               if (chrisStat != 1 && chrisStat != 2) {
-                   throw new Error('Unexpected status for chris: wanted 1 or 2, got ' +
-                                   bobStat);
-               }
-
-               // now let the query for chris's devices complete.
-               return aliceTestClient.httpBackend.flush('/keys/query', 1);
-           }).then((flushed) => {
-               expect(flushed).toEqual(1);
-
-               // wait for the client to stop processing the response
-               return aliceTestClient.client.downloadKeys(['@chris:abc']);
-           }).then(() => {
-               const bobStat = aliceTestClient.storage
-                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
-               const chrisStat = aliceTestClient.storage
-                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];
-
-               expect(bobStat).toEqual(3);
-               expect(chrisStat).toEqual(3);
-               expect(aliceTestClient.storage.getEndToEndDeviceSyncToken()).toEqual(3);
-           });
-       });
-
    it("Alice exports megolm keys and imports them to a new device", function() {
        let messageEncrypted;

@@ -181,5 +181,88 @@ describe("MegolmDecryption", function() {
                expect(payload.content.session_key).toExist();
            });
        });
+
+        it("can detect replay attacks", function() {
+            // trying to decrypt two different messages (marked by different
+            // event IDs or timestamps) using the same (sender key, session id,
+            // message index) triple should result in an exception being thrown
+            // as it should be detected as a replay attack.
+            const sessionId = groupSession.session_id();
+            const cipherText = groupSession.encrypt(JSON.stringify({
+                room_id: ROOM_ID,
+                content: 'testytest',
+            }));
+            const event1 = new MatrixEvent({
+                type: 'm.room.encrypted',
+                room_id: ROOM_ID,
+                content: {
+                    algorithm: 'm.megolm.v1.aes-sha2',
+                    sender_key: "SENDER_CURVE25519",
+                    session_id: sessionId,
+                    ciphertext: cipherText,
+                },
+                event_id: "$event1",
+                origin_server_ts: 1507753886000,
+            });
+
+            const successHandler = expect.createSpy();
+            const failureHandler = expect.createSpy()
+                .andCall((err) => {
+                    expect(err.toString()).toMatch(
+                        /Duplicate message index, possible replay attack/,
+                    );
+                });
+
+            return megolmDecryption.decryptEvent(event1).then((res) => {
+                const event2 = new MatrixEvent({
+                    type: 'm.room.encrypted',
+                    room_id: ROOM_ID,
+                    content: {
+                        algorithm: 'm.megolm.v1.aes-sha2',
+                        sender_key: "SENDER_CURVE25519",
+                        session_id: sessionId,
+                        ciphertext: cipherText,
+                    },
+                    event_id: "$event2",
+                    origin_server_ts: 1507754149000,
+                });
+
+                return megolmDecryption.decryptEvent(event2);
+            }).then(
+                successHandler,
+                failureHandler,
+            ).then(() => {
+                expect(successHandler).toNotHaveBeenCalled();
+                expect(failureHandler).toHaveBeenCalled();
+            });
+        });
+
+        it("allows re-decryption of the same event", function() {
+            // in contrast with the previous test, if the event ID and
+            // timestamp are the same, then it should not be considered a
+            // replay attack
+            const sessionId = groupSession.session_id();
+            const cipherText = groupSession.encrypt(JSON.stringify({
+                room_id: ROOM_ID,
+                content: 'testytest',
+            }));
+            const event = new MatrixEvent({
+                type: 'm.room.encrypted',
+                room_id: ROOM_ID,
+                content: {
+                    algorithm: 'm.megolm.v1.aes-sha2',
+                    sender_key: "SENDER_CURVE25519",
+                    session_id: sessionId,
+                    ciphertext: cipherText,
+                },
+                event_id: "$event1",
+                origin_server_ts: 1507753886000,
+            });
+
+            return megolmDecryption.decryptEvent(event).then((res) => {
+                return megolmDecryption.decryptEvent(event);
+                // test is successful if no exception is thrown
+            });
+        });
    });
 });
@@ -886,7 +886,7 @@ describe("Room", function() {
                expect(name).toEqual("Empty room");
            });

-            it("should return 'Invite from [inviter display name] if state event " +
+            it("should return '[inviter display name] if state event " +
               "available",
            function() {
                setJoinRule("invite");
@@ -894,7 +894,7 @@ describe("Room", function() {
                addMember(userB, "invite", {user: userA});
                room.recalculate(userB);
                const name = room.name;
-                expect(name).toEqual("Invite from Alice");
+                expect(name).toEqual("Alice");
            });

            it("should return inviter mxid if display name not available",
@@ -904,7 +904,7 @@ describe("Room", function() {
                addMember(userB, "invite", {user: userA});
                room.recalculate(userB);
                const name = room.name;
-                expect(name).toEqual("Invite from " + userA);
+                expect(name).toEqual(userA);
            });
        });
    });
@@ -0,0 +1,46 @@
+/*
+Copyright 2015, 2016 OpenMarket Ltd
+Copyright 2017 Vector Creations Ltd
+Copyright 2017 New Vector Ltd
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+/**
+ * @module
+ */
+
+export default class Reemitter {
+    constructor(target) {
+        this.target = target;
+
+        // We keep one bound event handler for each event name so we know
+        // what event is arriving
+        this.boundHandlers = {};
+    }
+
+    _handleEvent(eventName, ...args) {
+        this.target.emit(eventName, ...args);
+    }
+
+    reEmit(source, eventNames) {
+        for (const eventName of eventNames) {
+            if (this.boundHandlers[eventName] === undefined) {
+                this.boundHandlers[eventName] = this._handleEvent.bind(this, eventName);
+            }
+            const boundHandler = this.boundHandlers[eventName];
+
+            source.on(eventName, boundHandler);
+        }
+    }
+}
@@ -411,6 +411,16 @@ MatrixBaseApis.prototype.getGroupSummary = function(groupId) {
    return this._http.authedRequest(undefined, "GET", path);
 };

+/**
+ * @param {string} groupId
+ * @return {module:client.Promise} Resolves: Group profile object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.getGroupProfile = function(groupId) {
+    const path = utils.encodeUri("/groups/$groupId/profile", {$groupId: groupId});
+    return this._http.authedRequest(undefined, "GET", path);
+};
+
 /**
 * @param {string} groupId
 * @param {Object} profile The group profile object
@@ -438,6 +448,26 @@ MatrixBaseApis.prototype.getGroupUsers = function(groupId) {
    return this._http.authedRequest(undefined, "GET", path);
 };

+/**
+ * @param {string} groupId
+ * @return {module:client.Promise} Resolves: Group users list object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.getGroupInvitedUsers = function(groupId) {
+    const path = utils.encodeUri("/groups/$groupId/invited_users", {$groupId: groupId});
+    return this._http.authedRequest(undefined, "GET", path);
+};
+
+/**
+ * @param {string} groupId
+ * @return {module:client.Promise} Resolves: Group rooms list object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.getGroupRooms = function(groupId) {
+    const path = utils.encodeUri("/groups/$groupId/rooms", {$groupId: groupId});
+    return this._http.authedRequest(undefined, "GET", path);
+};
+
 /**
 * @param {string} groupId
 * @param {string} userId
@@ -466,6 +496,150 @@ MatrixBaseApis.prototype.removeUserFromGroup = function(groupId, userId) {
    return this._http.authedRequest(undefined, "PUT", path, undefined, {});
 };

+/**
+ * @param {string} groupId
+ * @param {string} userId
+ * @param {string} roleId Optional.
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.addUserToGroupSummary = function(groupId, userId, roleId) {
+    const path = utils.encodeUri(
+        roleId ?
+            "/groups/$groupId/summary/$roleId/users/$userId" :
+            "/groups/$groupId/summary/users/$userId",
+        {$groupId: groupId, $roleId: roleId, $userId: userId},
+    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined, {});
+};
+
+/**
+ * @param {string} groupId
+ * @param {string} userId
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.removeUserFromGroupSummary = function(groupId, userId) {
+    const path = utils.encodeUri(
+        "/groups/$groupId/summary/users/$userId",
+        {$groupId: groupId, $userId: userId},
+    );
+    return this._http.authedRequest(undefined, "DELETE", path, undefined, {});
+};
+
+/**
+ * @param {string} groupId
+ * @param {string} roomId
+ * @param {string} categoryId Optional.
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.addRoomToGroupSummary = function(groupId, roomId, categoryId) {
+    const path = utils.encodeUri(
+        categoryId ?
+            "/groups/$groupId/summary/$categoryId/rooms/$roomId" :
+            "/groups/$groupId/summary/rooms/$roomId",
+        {$groupId: groupId, $categoryId: categoryId, $roomId: roomId},
+    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined, {});
+};
+
+/**
+ * @param {string} groupId
+ * @param {string} roomId
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.removeRoomFromGroupSummary = function(groupId, roomId) {
+    const path = utils.encodeUri(
+        "/groups/$groupId/summary/rooms/$roomId",
+        {$groupId: groupId, $roomId: roomId},
+    );
+    return this._http.authedRequest(undefined, "DELETE", path, undefined, {});
+};
+
+/**
+ * @param {string} groupId
+ * @param {string} roomId
+ * @param {bool} isPublic Whether the room-group association is visible to non-members
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.addRoomToGroup = function(groupId, roomId, isPublic) {
+    if (isPublic === undefined) {
+        isPublic = true;
+    }
+    const path = utils.encodeUri(
+        "/groups/$groupId/admin/rooms/$roomId",
+        {$groupId: groupId, $roomId: roomId},
+    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined,
+        { "m.visibility": { type: isPublic ? "public" : "private" } },
+    );
+};
+
+/**
+ * Configure the visibility of a room-group association.
+ * @param {string} groupId
+ * @param {string} roomId
+ * @param {bool} isPublic Whether the room-group association is visible to non-members
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.updateGroupRoomVisibility = function(groupId, roomId, isPublic) {
+    // NB: The /config API is generic but there's not much point in exposing this yet as synapse
+    //     is the only server to implement this. In future we should consider an API that allows
+    //     arbitrary configuration, i.e. "config/$configKey".
+
+    const path = utils.encodeUri(
+        "/groups/$groupId/admin/rooms/$roomId/config/m.visibility",
+        {$groupId: groupId, $roomId: roomId},
+    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined,
+        { type: isPublic ? "public" : "private" },
+    );
+};
+
+/**
+ * @param {string} groupId
+ * @param {string} roomId
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.removeRoomFromGroup = function(groupId, roomId) {
+    const path = utils.encodeUri(
+        "/groups/$groupId/admin/rooms/$roomId",
+        {$groupId: groupId, $roomId: roomId},
+    );
+    return this._http.authedRequest(undefined, "DELETE", path, undefined, {});
+};
+
+/**
+ * @param {string} groupId
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.acceptGroupInvite = function(groupId) {
+    const path = utils.encodeUri(
+        "/groups/$groupId/self/accept_invite",
+        {$groupId: groupId},
+    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined, {});
+};
+
+/**
+ * @param {string} groupId
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.leaveGroup = function(groupId) {
+    const path = utils.encodeUri(
+        "/groups/$groupId/self/leave",
+        {$groupId: groupId},
+    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined, {});
+};
+
 /**
 * @return {module:client.Promise} Resolves: The groups to which the user is joined
 * @return {module:http-api.MatrixError} Rejects: with an error response.
@@ -489,6 +663,42 @@ MatrixBaseApis.prototype.createGroup = function(content) {
    );
 };

+/**
+ * @param {string[]} userIds List of user IDs
+ * @return {module:client.Promise} Resolves: Object as exmaple below
+ *
+ *     {
+ *         "users": {
+ *             "@bob:example.com": {
+ *                 "+example:example.com"
+ *             }
+ *         }
+ *     }
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.getPublicisedGroups = function(userIds) {
+    const path = utils.encodeUri("/publicised_groups");
+    return this._http.authedRequest(
+        undefined, "POST", path, undefined, { user_ids: userIds },
+    );
+};
+
+/**
+ * @param {string} groupId
+ * @param {bool} isPublic Whether the user's membership of this group is made public
+ * @return {module:client.Promise} Resolves: Empty object
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.setGroupPublicity = function(groupId, isPublic) {
+    const path = utils.encodeUri(
+        "/groups/$groupId/self/update_publicity",
+        {$groupId: groupId},
+    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined, {
+        publicise: isPublic,
+    });
+};
+
 /**
 * Retrieve a state event.
 * @param {string} roomId
@@ -40,7 +40,7 @@ const SyncApi = require("./sync");
 const MatrixBaseApis = require("./base-apis");
 const MatrixError = httpApi.MatrixError;

-import reEmit from './reemit';
+import ReEmitter from './ReEmitter';

 const SCROLLBACK_DELAY_MS = 3000;
 let CRYPTO_ENABLED = false;
@@ -113,8 +113,20 @@ try {
 *    crypto store implementation.
 */
 function MatrixClient(opts) {
+    // Allow trailing slash in HS url
+    if (opts.baseUrl && opts.baseUrl.endsWith("/")) {
+        opts.baseUrl = opts.baseUrl.substr(0, opts.baseUrl.length - 1);
+    }
+
+    // Allow trailing slash in IS url
+    if (opts.idBaseUrl && opts.idBaseUrl.endsWith("/")) {
+        opts.idBaseUrl = opts.idBaseUrl.substr(0, opts.idBaseUrl.length - 1);
+    }
+
    MatrixBaseApis.call(this, opts);

+    this.reEmitter = new ReEmitter(this);
+
    this.store = opts.store || new StubStore();

    this.deviceId = opts.deviceId || null;
@@ -163,6 +175,8 @@ function MatrixClient(opts) {
    this._cryptoStore = opts.cryptoStore;
    this._sessionStore = opts.sessionStore;

+    this._forceTURN = opts.forceTURN || false;
+
    if (CRYPTO_ENABLED) {
        this.olmVersion = Crypto.getOlmVersion();
    }
@@ -240,6 +254,16 @@ MatrixClient.prototype.supportsVoip = function() {
    return this._supportsVoip;
 };

+/**
+ * Set whether VoIP calls are forced to use only TURN
+ * candidates. This is the same as the forceTURN option
+ * when creating the client.
+ * @param {bool} forceTURN True to force use of TURN servers
+ */
+MatrixClient.prototype.setForceTURN = function(forceTURN) {
+    this._forceTURN = forceTURN;
+};
+
 /**
 * Get the current sync state.
 * @return {?string} the sync state, which may be null.
@@ -364,7 +388,7 @@ MatrixClient.prototype.initCrypto = async function() {
        this._cryptoStore,
    );

-    reEmit(this, crypto, [
+    this.reEmitter.reEmit(crypto, [
        "crypto.roomKeyRequest",
        "crypto.roomKeyRequestCancellation",
    ]);
@@ -659,6 +683,30 @@ MatrixClient.prototype.importRoomKeys = function(keys) {
    return this._crypto.importRoomKeys(keys);
 };

+// Group ops
+// =========
+// Operations on groups that come down the sync stream (ie. ones the
+// user is a member of or invited to)
+
+/**
+ * Get the group for the given group ID.
+ * This function will return a valid group for any group for which a Group event
+ * has been emitted.
+ * @param {string} groupId The group ID
+ * @return {Group} The Group or null if the group is not known or there is no data store.
+ */
+MatrixClient.prototype.getGroup = function(groupId) {
+    return this.store.getGroup(groupId);
+};
+
+/**
+ * Retrieve all known groups.
+ * @return {Groups[]} A list of groups, or an empty list if there is no data store.
+ */
+MatrixClient.prototype.getGroups = function() {
+    return this.store.getGroups();
+};
+
 // Room ops
 // ========

@@ -731,6 +779,38 @@ MatrixClient.prototype.getAccountData = function(eventType) {
    return this.store.getAccountData(eventType);
 };

+/**
+ * Gets the users that are ignored by this client
+ * @returns {string[]} The array of users that are ignored (empty if none)
+ */
+MatrixClient.prototype.getIgnoredUsers = function() {
+    const event = this.getAccountData("m.ignored_user_list");
+    if (!event || !event.getContent() || !event.getContent()["ignored_users"]) return [];
+    return Object.keys(event.getContent()["ignored_users"]);
+};
+
+/**
+ * Sets the users that the current user should ignore.
+ * @param {string[]} userIds the user IDs to ignore
+ * @param {module:client.callback} [callback] Optional.
+ * @return {module:client.Promise} Resolves: Account data event
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixClient.prototype.setIgnoredUsers = function(userIds, callback) {
+    const content = {ignored_users: {}};
+    userIds.map((u) => content.ignored_users[u] = {});
+    return this.setAccountData("m.ignored_user_list", content, callback);
+};
+
+/**
+ * Gets whether or not a specific user is being ignored by this client.
+ * @param {string} userId the user ID to check
+ * @returns {boolean} true if the user is ignored, false otherwise
+ */
+MatrixClient.prototype.isUserIgnored = function(userId) {
+    return this.getIgnoredUsers().indexOf(userId) !== -1;
+};
+
 // Room operations
 // ===============

@@ -3029,6 +3109,10 @@ function setupCallEventHandler(client) {
            // now loop through the buffer chronologically and inject them
            callEventBuffer.forEach(function(e) {
                if (ignoreCallIds[e.getContent().call_id]) {
+                    console.log(
+                        'Ignoring previously answered/hungup call ' +
+                            e.getContent().call_id,
+                    );
                    return;
                }
                callEventHandler(e);
@@ -3037,20 +3121,25 @@ function setupCallEventHandler(client) {
        }
    });

-    client.on("event", function(event) {
-        if (!isClientPrepared) {
-            if (event.getType().indexOf("m.call.") === 0) {
-                callEventBuffer.push(event);
+    client.on("event", onEvent);
+
+    function onEvent(event) {
+        if (event.getType().indexOf("m.call.") !== 0) {
+            // not a call event
+            if (event.isBeingDecrypted() || event.isDecryptionFailure()) {
+                // not *yet* a call event, but might become one...
+                event.once("Event.decrypted", onEvent);
            }
            return;
        }
+        if (!isClientPrepared) {
+            callEventBuffer.push(event);
+            return;
+        }
        callEventHandler(event);
-    });
+    }

    function callEventHandler(event) {
-        if (event.getType().indexOf("m.call.") !== 0) {
-            return; // not a call event
-        }
        const content = event.getContent();
        let call = content.call_id ? client.callList[content.call_id] : undefined;
        let i;
@@ -3076,7 +3165,9 @@ function setupCallEventHandler(client) {
                );
            }

-            call = webRtcCall.createNewMatrixCall(client, event.getRoomId());
+            call = webRtcCall.createNewMatrixCall(client, event.getRoomId(), {
+                forceTURN: client._forceTURN,
+            });
            if (!call) {
                console.log(
                    "Incoming call ID " + content.call_id + " but this client " +
@@ -3242,6 +3333,9 @@ function _PojoToMatrixEventMapper(client) {
    function mapper(plainOldJsObject) {
        const event = new MatrixEvent(plainOldJsObject);
        if (event.isEncrypted()) {
+            client.reEmitter.reEmit(event, [
+                "Event.decrypted",
+            ]);
            event.attemptDecryption(client._crypto);
        }
        return event;
@@ -3423,6 +3517,17 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED;
 * });
 */

+ /**
+ * Fires whenever the sdk learns about a new group. <strong>This event
+ * is experimental and may change.</strong>
+ * @event module:client~MatrixClient#"Group"
+ * @param {Group} group The newly created, fully populated group.
+ * @example
+ * matrixClient.on("Group", function(group){
+ *   var groupId = group.groupId;
+ * });
+ */
+
 /**
 * Fires whenever a new Room is added. This will fire when you are invited to a
 * room, as well as when you join a room. <strong>This event is experimental and
@@ -26,8 +26,30 @@ import Promise from 'bluebird';
 import DeviceInfo from './deviceinfo';
 import olmlib from './olmlib';

+
+/* State transition diagram for DeviceList._deviceTrackingStatus
+ *
+ *                                |
+ *     stopTrackingDeviceList     V
+ *   +---------------------> NOT_TRACKED
+ *   |                            |
+ *   +<--------------------+      | startTrackingDeviceList
+ *   |                     |      V
+ *   |   +-------------> PENDING_DOWNLOAD <--------------------+-+
+ *   |   |                      ^ |                            | |
+ *   |   | restart     download | |  start download            | | invalidateUserDeviceList
+ *   |   | client        failed | |                            | |
+ *   |   |                      | V                            | |
+ *   |   +------------ DOWNLOAD_IN_PROGRESS -------------------+ |
+ *   |                    |       |                              |
+ *   +<-------------------+       |  download successful         |
+ *   ^                            V                              |
+ *   +----------------------- UP_TO_DATE ------------------------+
+ */
+
+
 // constants for DeviceList._deviceTrackingStatus
-// const TRACKING_STATUS_NOT_TRACKED = 0;
+const TRACKING_STATUS_NOT_TRACKED = 0;
 const TRACKING_STATUS_PENDING_DOWNLOAD = 1;
 const TRACKING_STATUS_DOWNLOAD_IN_PROGRESS = 2;
 const TRACKING_STATUS_UP_TO_DATE = 3;
@@ -236,6 +258,26 @@ export default class DeviceList {
        // refreshOutdatedDeviceLists.
    }

+    /**
+     * Mark the given user as no longer being tracked for device-list updates.
+     *
+     * This won't affect any in-progress downloads, which will still go on to
+     * complete; it will just mean that we don't think that we have an up-to-date
+     * list for future calls to downloadKeys.
+     *
+     * @param {String} userId
+     */
+    stopTrackingDeviceList(userId) {
+        if (this._deviceTrackingStatus[userId]) {
+            console.log('No longer tracking device list for ' + userId);
+            this._deviceTrackingStatus[userId] = TRACKING_STATUS_NOT_TRACKED;
+        }
+        // we don't yet persist the tracking status, since there may be a lot
+        // of calls; instead we wait for the forthcoming
+        // refreshOutdatedDeviceLists.
+    }
+
+
    /**
     * Mark the cached device list for the given user outdated.
     *
@@ -283,9 +325,6 @@ export default class DeviceList {
                usersToDownload.push(userId);
            }
        }
-        if (usersToDownload.length == 0) {
-            return;
-        }

        // we didn't persist the tracking status during
        // invalidateUserDeviceList, so do it now.
@@ -479,7 +518,7 @@ class DeviceListUpdateSerialiser {
            }
        }, (e) => {
            console.warn('Error downloading keys for ' + downloadUsers + ':', e);
-            this._downloadInProgressInProgress = false;
+            this._downloadInProgress = false;
            deferred.reject(e);
        });

@@ -96,12 +96,18 @@ function OlmDevice(sessionStore) {
    // This partially mitigates a replay attack where a MITM resends a group
    // message into the room.
    //
-    // TODO: If we ever remove an event from memory we will also need to remove
-    // it from this map. Otherwise if we download the event from the server we
-    // will think that it is a duplicate.
+    // When we decrypt a message and the message index matches a previously
+    // decrypted message, one possible cause of that is that we are decrypting
+    // the same event, and may not indicate an actual replay attack.  For
+    // example, this could happen if we receive events, forget about them, and
+    // then re-fetch them when we backfill.  So we store the event ID and
+    // timestamp corresponding to each message index when we first decrypt it,
+    // and compare these against the event ID and timestamp every time we use
+    // that same index.  If they match, then we're probably decrypting the same
+    // event and we don't consider it a replay attack.
    //
    // Keys are strings of form "<senderKey>|<session_id>|<message_index>"
-    // Values are true.
+    // Values are objects of the form "{id: <event id>, timestamp: <ts>}"
    this._inboundGroupSessionMessageIndexes = {};
 }

@@ -794,6 +800,8 @@ OlmDevice.prototype.importInboundGroupSession = async function(data) {
 * @param {string} senderKey base64-encoded curve25519 key of the sender
 * @param {string} sessionId session identifier
 * @param {string} body      base64-encoded body of the encrypted message
+ * @param {string} eventId   ID of the event being decrypted
+ * @param {Number} timestamp timestamp of the event being decrypted
 *
 * @return {null} the sessionId is unknown
 *
@@ -802,7 +810,7 @@ OlmDevice.prototype.importInboundGroupSession = async function(data) {
 *    keysClaimed: Object<string, string>}>}
 */
 OlmDevice.prototype.decryptGroupMessage = async function(
-    roomId, senderKey, sessionId, body,
+    roomId, senderKey, sessionId, body, eventId, timestamp,
 ) {
    const self = this;

@@ -815,14 +823,23 @@ OlmDevice.prototype.decryptGroupMessage = async function(
            plaintext = res;
        } else {
            // Check if we have seen this message index before to detect replay attacks.
+            // If the event ID and timestamp are specified, and the match the event ID
+            // and timestamp from the last time we used this message index, then we
+            // don't consider it a replay attack.
            const messageIndexKey = senderKey + "|" + sessionId + "|" + res.message_index;
            if (messageIndexKey in self._inboundGroupSessionMessageIndexes) {
-                throw new Error(
-                    "Duplicate message index, possible replay attack: " +
-                    messageIndexKey,
-                );
+                const msgInfo = self._inboundGroupSessionMessageIndexes[messageIndexKey];
+                if (msgInfo.id !== eventId || msgInfo.timestamp !== timestamp) {
+                    throw new Error(
+                        "Duplicate message index, possible replay attack: " +
+                        messageIndexKey,
+                    );
+                }
            }
-            self._inboundGroupSessionMessageIndexes[messageIndexKey] = true;
+            self._inboundGroupSessionMessageIndexes[messageIndexKey] = {
+                id: eventId,
+                timestamp: timestamp,
+            };
        }

        sessionData.session = session.pickle(self._pickleKey);
@@ -74,6 +74,14 @@ OutboundSessionInfo.prototype.needsRotation = function(
    return false;
 };

+OutboundSessionInfo.prototype.markSharedWithDevice = function(
+    userId, deviceId, chainIndex,
+) {
+    if (!this.sharedWithDevices[userId]) {
+        this.sharedWithDevices[userId] = {};
+    }
+    this.sharedWithDevices[userId][deviceId] = chainIndex;
+};

 /**
 * Determine if this session has been shared with devices which it shouldn't
@@ -262,15 +270,151 @@ MegolmEncryption.prototype._prepareNewSession = async function() {
 *
 * @param {module:crypto/algorithms/megolm.OutboundSessionInfo} session
 *
+ * @param {number} chainIndex current chain index
+ *
+ * @param {object<userId, deviceId>} devicemap
+ *   mapping from userId to deviceId to {@link module:crypto~OlmSessionResult}
+ *
 * @param {object<string, module:crypto/deviceinfo[]>} devicesByUser
 *    map from userid to list of devices
 *
- * @return {module:client.Promise} Promise which resolves once the key sharing
- *     message has been sent.
+ * @return {array<object<userid, deviceInfo>>}
 */
-MegolmEncryption.prototype._shareKeyWithDevices = function(session, devicesByUser) {
-    const self = this;
+MegolmEncryption.prototype._splitUserDeviceMap = function(
+    session, chainIndex, devicemap, devicesByUser,
+) {
+    const maxToDeviceMessagesPerRequest = 20;

+    // use an array where the slices of a content map gets stored
+    const mapSlices = [];
+    let currentSliceId = 0; // start inserting in the first slice
+    let entriesInCurrentSlice = 0;
+
+    for (const userId of Object.keys(devicesByUser)) {
+        const devicesToShareWith = devicesByUser[userId];
+        const sessionResults = devicemap[userId];
+
+        for (let i = 0; i < devicesToShareWith.length; i++) {
+            const deviceInfo = devicesToShareWith[i];
+            const deviceId = deviceInfo.deviceId;
+
+            const sessionResult = sessionResults[deviceId];
+            if (!sessionResult.sessionId) {
+                // no session with this device, probably because there
+                // were no one-time keys.
+                //
+                // we could send them a to_device message anyway, as a
+                // signal that they have missed out on the key sharing
+                // message because of the lack of keys, but there's not
+                // much point in that really; it will mostly serve to clog
+                // up to_device inboxes.
+
+                // mark this device as "handled" because we don't want to try
+                // to claim a one-time-key for dead devices on every message.
+                session.markSharedWithDevice(userId, deviceId, chainIndex);
+
+                // ensureOlmSessionsForUsers has already done the logging,
+                // so just skip it.
+                continue;
+            }
+
+            console.log(
+                "share keys with device " + userId + ":" + deviceId,
+            );
+
+            if (entriesInCurrentSlice > maxToDeviceMessagesPerRequest) {
+                // the current slice is filled up. Start inserting into the next slice
+                entriesInCurrentSlice = 0;
+                currentSliceId++;
+            }
+            if (!mapSlices[currentSliceId]) {
+                mapSlices[currentSliceId] = [];
+            }
+
+            mapSlices[currentSliceId].push({
+                userId: userId,
+                deviceInfo: deviceInfo,
+            });
+
+            entriesInCurrentSlice++;
+        }
+    }
+    return mapSlices;
+};
+
+/**
+ * @private
+ *
+ * @param {module:crypto/algorithms/megolm.OutboundSessionInfo} session
+ *
+ * @param {number} chainIndex current chain index
+ *
+ * @param {object<userId, deviceInfo>} userDeviceMap
+ *   mapping from userId to deviceInfo
+ *
+ * @param {object} payload fields to include in the encrypted payload
+ *
+ * @return {module:client.Promise} Promise which resolves once the key sharing
+ *     for the given userDeviceMap is generated and has been sent.
+ */
+MegolmEncryption.prototype._encryptAndSendKeysToDevices = function(
+    session, chainIndex, userDeviceMap, payload,
+) {
+    const encryptedContent = {
+        algorithm: olmlib.OLM_ALGORITHM,
+        sender_key: this._olmDevice.deviceCurve25519Key,
+        ciphertext: {},
+    };
+    const contentMap = {};
+
+    const promises = [];
+    for (let i = 0; i < userDeviceMap.length; i++) {
+        const val = userDeviceMap[i];
+        const userId = val.userId;
+        const deviceInfo = val.deviceInfo;
+        const deviceId = deviceInfo.deviceId;
+
+        if (!contentMap[userId]) {
+            contentMap[userId] = {};
+        }
+        contentMap[userId][deviceId] = encryptedContent;
+
+        promises.push(
+            olmlib.encryptMessageForDevice(
+                encryptedContent.ciphertext,
+                this._userId,
+                this._deviceId,
+                this._olmDevice,
+                userId,
+                deviceInfo,
+                payload,
+            ),
+        );
+    }
+
+    return Promise.all(promises).then(() => {
+        return this._baseApis.sendToDevice("m.room.encrypted", contentMap).then(() => {
+            // store that we successfully uploaded the keys of the current slice
+            for (const userId of Object.keys(contentMap)) {
+                for (const deviceId of Object.keys(contentMap[userId])) {
+                    session.markSharedWithDevice(
+                        userId, deviceId, chainIndex,
+                    );
+                }
+            }
+        });
+    });
+};
+
+/**
+ * @private
+ *
+ * @param {module:crypto/algorithms/megolm.OutboundSessionInfo} session
+ *
+ * @param {object<string, module:crypto/deviceinfo[]>} devicesByUser
+ *    map from userid to list of devices
+ */
+MegolmEncryption.prototype._shareKeyWithDevices = async function(session, devicesByUser) {
    const key = this._olmDevice.getOutboundGroupSessionKey(session.sessionId);
    const payload = {
        type: "m.room_key",
@@ -283,104 +427,28 @@ MegolmEncryption.prototype._shareKeyWithDevices = function(session, devicesByUse
        },
    };

-    const contentMap = {};
-
-    return olmlib.ensureOlmSessionsForDevices(
+    const devicemap = await olmlib.ensureOlmSessionsForDevices(
        this._olmDevice, this._baseApis, devicesByUser,
-    ).then(function(devicemap) {
-        const promises = [];
+    );

-        for (const userId in devicesByUser) {
-            if (!devicesByUser.hasOwnProperty(userId)) {
-                continue;
-            }
+    const userDeviceMaps = this._splitUserDeviceMap(
+        session, key.chain_index, devicemap, devicesByUser,
+    );

-            const devicesToShareWith = devicesByUser[userId];
-            const sessionResults = devicemap[userId];
+    for (let i = 0; i < userDeviceMaps.length; i++) {
+        try {
+            await this._encryptAndSendKeysToDevices(
+                session, key.chain_index, userDeviceMaps[i], payload,
+            );
+            console.log(`Completed megolm keyshare in ${this._roomId} `
+                + `(slice ${i + 1}/${userDeviceMaps.length})`);
+        } catch (e) {
+            console.log(`megolm keyshare in ${this._roomId} `
+                + `(slice ${i + 1}/${userDeviceMaps.length}) failed`);

-            for (let i = 0; i < devicesToShareWith.length; i++) {
-                const deviceInfo = devicesToShareWith[i];
-                const deviceId = deviceInfo.deviceId;
-
-                const sessionResult = sessionResults[deviceId];
-                if (!sessionResult.sessionId) {
-                    // no session with this device, probably because there
-                    // were no one-time keys.
-                    //
-                    // we could send them a to_device message anyway, as a
-                    // signal that they have missed out on the key sharing
-                    // message because of the lack of keys, but there's not
-                    // much point in that really; it will mostly serve to clog
-                    // up to_device inboxes.
-                    //
-                    // ensureOlmSessionsForUsers has already done the logging,
-                    // so just skip it.
-                    continue;
-                }
-
-                console.log(
-                    "sharing keys with device " + userId + ":" + deviceId,
-                );
-
-                const encryptedContent = {
-                    algorithm: olmlib.OLM_ALGORITHM,
-                    sender_key: self._olmDevice.deviceCurve25519Key,
-                    ciphertext: {},
-                };
-
-                if (!contentMap[userId]) {
-                    contentMap[userId] = {};
-                }
-
-                contentMap[userId][deviceId] = encryptedContent;
-
-                promises.push(
-                    olmlib.encryptMessageForDevice(
-                        encryptedContent.ciphertext,
-                        self._userId,
-                        self._deviceId,
-                        self._olmDevice,
-                        userId,
-                        deviceInfo,
-                        payload,
-                    ),
-                );
-            }
+            throw e;
        }
-
-        if (promises.length === 0) {
-            // no devices to send to
-            return Promise.resolve();
-        }
-
-        return Promise.all(promises).then(() => {
-            // TODO: retries
-            return self._baseApis.sendToDevice("m.room.encrypted", contentMap);
-        });
-    }).then(function() {
-        console.log(`Completed megolm keyshare in ${self._roomId}`);
-
-        // Add the devices we have shared with to session.sharedWithDevices.
-        //
-        // we deliberately iterate over devicesByUser (ie, the devices we
-        // attempted to share with) rather than the contentMap (those we did
-        // share with), because we don't want to try to claim a one-time-key
-        // for dead devices on every message.
-        for (const userId in devicesByUser) {
-            if (!devicesByUser.hasOwnProperty(userId)) {
-                continue;
-            }
-            if (!session.sharedWithDevices[userId]) {
-                session.sharedWithDevices[userId] = {};
-            }
-            const devicesToShareWith = devicesByUser[userId];
-            for (let i = 0; i < devicesToShareWith.length; i++) {
-                const deviceInfo = devicesToShareWith[i];
-                session.sharedWithDevices[userId][deviceInfo.deviceId] =
-                    key.chain_index;
-            }
-        }
-    });
+    }
 };

 /**
@@ -522,7 +590,7 @@ function MegolmDecryption(params) {
    base.DecryptionAlgorithm.call(this, params);

    // events which we couldn't decrypt due to unknown sessions / indexes: map from
-    // senderKey|sessionId to list of MatrixEvents
+    // senderKey|sessionId to Set of MatrixEvents
    this._pendingEvents = {};

    // this gets stubbed out by the unit tests.
@@ -549,14 +617,21 @@ MegolmDecryption.prototype.decryptEvent = async function(event) {
        throw new base.DecryptionError("Missing fields in input");
    }

+    // we add the event to the pending list *before* we start decryption.
+    //
+    // then, if the key turns up while decryption is in progress (and
+    // decryption fails), we will schedule a retry.
+    // (fixes https://github.com/vector-im/riot-web/issues/5001)
+    this._addEventToPendingList(event);
+
    let res;
    try {
        res = await this._olmDevice.decryptGroupMessage(
            event.getRoomId(), content.sender_key, content.session_id, content.ciphertext,
+            event.getId(), event.getTs(),
        );
    } catch (e) {
        if (e.message === 'OLM.UNKNOWN_MESSAGE_INDEX') {
-            this._addEventToPendingList(event);
            this._requestKeysForEvent(event);
        }
        throw new base.DecryptionError(
@@ -568,7 +643,12 @@ MegolmDecryption.prototype.decryptEvent = async function(event) {

    if (res === null) {
        // We've got a message for a session we don't have.
-        this._addEventToPendingList(event);
+        //
+        // (XXX: We might actually have received this key since we started
+        // decrypting, in which case we'll have scheduled a retry, and this
+        // request will be redundant. We could probably check to see if the
+        // event is still in the pending list; if not, a retry will have been
+        // scheduled, so we needn't send out the request here.)
        this._requestKeysForEvent(event);
        throw new base.DecryptionError(
            "The sender's device has not sent us the keys for this message.",
@@ -578,6 +658,10 @@ MegolmDecryption.prototype.decryptEvent = async function(event) {
        );
    }

+    // success. We can remove the event from the pending list, if that hasn't
+    // already happened.
+    this._removeEventFromPendingList(event);
+
    const payload = JSON.parse(res.result);

    // belt-and-braces check that the room id matches that indicated by the HS
@@ -621,8 +705,7 @@ MegolmDecryption.prototype._requestKeysForEvent = function(event) {
 };

 /**
- * Add an event to the list of those we couldn't decrypt the first time we
- * saw them.
+ * Add an event to the list of those awaiting their session keys.
 *
 * @private
 *
@@ -632,11 +715,32 @@ MegolmDecryption.prototype._addEventToPendingList = function(event) {
    const content = event.getWireContent();
    const k = content.sender_key + "|" + content.session_id;
    if (!this._pendingEvents[k]) {
-        this._pendingEvents[k] = [];
+        this._pendingEvents[k] = new Set();
    }
-    this._pendingEvents[k].push(event);
+    this._pendingEvents[k].add(event);
 };

+/**
+ * Remove an event from the list of those awaiting their session keys.
+ *
+ * @private
+ *
+ * @param {module:models/event.MatrixEvent} event
+ */
+MegolmDecryption.prototype._removeEventFromPendingList = function(event) {
+    const content = event.getWireContent();
+    const k = content.sender_key + "|" + content.session_id;
+    if (!this._pendingEvents[k]) {
+        return;
+    }
+
+    this._pendingEvents[k].delete(event);
+    if (this._pendingEvents[k].size === 0) {
+        delete this._pendingEvents[k];
+    }
+};
+
+
 /**
 * @inheritdoc
 *
@@ -841,8 +945,8 @@ MegolmDecryption.prototype._retryDecryption = function(senderKey, sessionId) {

    delete this._pendingEvents[k];

-    for (let i = 0; i < pending.length; i++) {
-        pending[i].attemptDecryption(this._crypto);
+    for (const ev of pending) {
+        ev.attemptDecryption(this._crypto);
    }
 };

@@ -69,7 +69,6 @@ function Crypto(baseApis, sessionStore, userId, deviceId,

    this._olmDevice = new OlmDevice(sessionStore);
    this._deviceList = new DeviceList(baseApis, sessionStore, this._olmDevice);
-    this._initialDeviceListInvalidationPending = false;

    // the last time we did a check for the number of one-time-keys on the
    // server.
@@ -150,15 +149,6 @@ Crypto.prototype.init = async function() {
 */
 Crypto.prototype.registerEventHandlers = function(eventEmitter) {
    const crypto = this;
-    eventEmitter.on("sync", function(syncState, oldState, data) {
-        try {
-            if (syncState === "SYNCING") {
-                crypto._onSyncCompleted(data);
-            }
-        } catch (e) {
-            console.error("Error handling sync", e);
-        }
-    });

    eventEmitter.on("RoomMember.membership", function(event, member, oldMembership) {
        try {
@@ -248,7 +238,7 @@ Crypto.prototype.uploadDeviceKeys = function() {

 /**
 * Stores the current one_time_key count which will be handled later (in a call of
- * _onSyncCompleted). The count is e.g. coming from a /sync response.
+ * onSyncCompleted). The count is e.g. coming from a /sync response.
 *
 * @param {Number} currentCount The current count of one_time_keys to be stored
 */
@@ -779,18 +769,39 @@ Crypto.prototype.encryptEvent = function(event, room) {
 *  is a problem decrypting the event.
 */
 Crypto.prototype.decryptEvent = function(event) {
+    if (event.isRedacted()) {
+        return Promise.resolve({
+            clearEvent: {
+                room_id: event.getRoomId(),
+                type: "m.room.message",
+                content: {},
+            },
+        });
+    }
    const content = event.getWireContent();
    const alg = this._getRoomDecryptor(event.getRoomId(), content.algorithm);
    return alg.decryptEvent(event);
 };

 /**
- * Handle the notification from /sync that a user has updated their device list.
+ * Handle the notification from /sync or /keys/changes that device lists have
+ * been changed.
 *
- * @param {String} userId
+ * @param {Object} deviceLists device_lists field from /sync, or response from
+ * /keys/changes
 */
-Crypto.prototype.userDeviceListChanged = function(userId) {
-    this._deviceList.invalidateUserDeviceList(userId);
+Crypto.prototype.handleDeviceListChanges = async function(deviceLists) {
+    if (deviceLists.changed && Array.isArray(deviceLists.changed)) {
+        deviceLists.changed.forEach((u) => {
+            this._deviceList.invalidateUserDeviceList(u);
+        });
+    }
+
+    if (deviceLists.left && Array.isArray(deviceLists.left)) {
+        deviceLists.left.forEach((u) => {
+            this._deviceList.stopTrackingDeviceList(u);
+        });
+    }

    // don't flush the outdated device list yet - we do it once we finish
    // processing the sync.
@@ -837,7 +848,7 @@ Crypto.prototype.onCryptoEvent = async function(event) {

    try {
        // inhibit the device list refresh for now - it will happen once we've
-        // finished processing the sync, in _onSyncCompleted.
+        // finished processing the sync, in onSyncCompleted.
        await this.setRoomEncryption(roomId, content, true);
    } catch (e) {
        console.error("Error configuring encryption in room " + roomId +
@@ -853,7 +864,7 @@ Crypto.prototype.onCryptoEvent = async function(event) {
 *
 * @param {Object} syncData  the data from the 'MatrixClient.sync' event
 */
-Crypto.prototype._onSyncCompleted = function(syncData) {
+Crypto.prototype.onSyncCompleted = async function(syncData) {
    const nextSyncToken = syncData.nextSyncToken;

    if (!syncData.oldSyncToken) {
@@ -863,18 +874,15 @@ Crypto.prototype._onSyncCompleted = function(syncData) {
        // invalidate devices which have changed since then.
        const oldSyncToken = this._sessionStore.getEndToEndDeviceSyncToken();
        if (oldSyncToken !== null) {
-            this._initialDeviceListInvalidationPending = true;
-            this._invalidateDeviceListsSince(
-                oldSyncToken, nextSyncToken,
-            ).catch((e) => {
+            try {
+                await this._invalidateDeviceListsSince(
+                    oldSyncToken, nextSyncToken,
+                );
+            } catch (e) {
                // if that failed, we fall back to invalidating everyone.
                console.warn("Error fetching changed device list", e);
                this._deviceList.invalidateAllDeviceLists();
-            }).done(() => {
-                this._initialDeviceListInvalidationPending = false;
-                this._deviceList.lastKnownSyncToken = nextSyncToken;
-                this._deviceList.refreshOutdatedDeviceLists();
-            });
+            }
        } else {
            // otherwise, we have to invalidate all devices for all users we
            // are tracking.
@@ -884,14 +892,12 @@ Crypto.prototype._onSyncCompleted = function(syncData) {
        }
    }

-    if (!this._initialDeviceListInvalidationPending) {
-        // we can now store our sync token so that we can get an update on
-        // restart rather than having to invalidate everyone.
-        //
-        // (we don't really need to do this on every sync - we could just
-        // do it periodically)
-        this._sessionStore.storeEndToEndDeviceSyncToken(nextSyncToken);
-    }
+    // we can now store our sync token so that we can get an update on
+    // restart rather than having to invalidate everyone.
+    //
+    // (we don't really need to do this on every sync - we could just
+    // do it periodically)
+    this._sessionStore.storeEndToEndDeviceSyncToken(nextSyncToken);

    // catch up on any new devices we got told about during the sync.
    this._deviceList.lastKnownSyncToken = nextSyncToken;
@@ -914,25 +920,19 @@ Crypto.prototype._onSyncCompleted = function(syncData) {
 * @param {String} oldSyncToken
 * @param {String} lastKnownSyncToken
 *
- * @returns {Promise} resolves once the query is complete. Rejects if the
+ * Returns a Promise which resolves once the query is complete. Rejects if the
 *   keyChange query fails.
 */
-Crypto.prototype._invalidateDeviceListsSince = function(
+Crypto.prototype._invalidateDeviceListsSince = async function(
    oldSyncToken, lastKnownSyncToken,
 ) {
-    return this._baseApis.getKeyChanges(
+    const r = await this._baseApis.getKeyChanges(
        oldSyncToken, lastKnownSyncToken,
-    ).then((r) => {
-        console.log("got key changes since", oldSyncToken, ":", r.changed);
+    );

-        if (!r.changed || !Array.isArray(r.changed)) {
-            return;
-        }
+    console.log("got key changes since", oldSyncToken, ":", r);

-        r.changed.forEach((u) => {
-            this._deviceList.invalidateUserDeviceList(u);
-        });
-    });
+    await this.handleDeviceListChanges(r);
 };

 /**
@@ -318,6 +318,19 @@ utils.extend(module.exports.MatrixEvent.prototype, {
        return this._decryptionPromise != null;
    },

+    /**
+     * Check if this event is an encrypted event which we failed to decrypt
+     *
+     * (This implies that we might retry decryption at some point in the future)
+     *
+     * @return {boolean} True if this event is an encrypted event which we
+     *     couldn't decrypt.
+     */
+    isDecryptionFailure: function() {
+        return this._clearEvent && this._clearEvent.content &&
+            this._clearEvent.content.msgtype === "m.bad.encrypted";
+    },
+
    /**
     * Start the process of trying to decrypt this event.
     *
@@ -0,0 +1,95 @@
+/*
+Copyright 2017 New Vector Ltd
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+/**
+ * @module models/group
+ */
+const EventEmitter = require("events").EventEmitter;
+
+const utils = require("../utils");
+
+/**
+ * Construct a new Group.
+ *
+ * @param {string} groupId The ID of this group.
+ *
+ * @prop {string} groupId The ID of this group.
+ * @prop {string} name The human-readable display name for this group.
+ * @prop {string} avatarUrl The mxc URL for this group's avatar.
+ * @prop {string} myMembership The logged in user's membership of this group
+ * @prop {Object} inviter Infomation about the user who invited the logged in user
+ *       to the group, if myMembership is 'invite'.
+ * @prop {string} inviter.userId The user ID of the inviter
+ */
+function Group(groupId) {
+    this.groupId = groupId;
+    this.name = null;
+    this.avatarUrl = null;
+    this.myMembership = null;
+    this.inviter = null;
+}
+utils.inherits(Group, EventEmitter);
+
+Group.prototype.setProfile = function(name, avatarUrl) {
+    if (this.name === name && this.avatarUrl === avatarUrl) return;
+
+    this.name = name || this.groupId;
+    this.avatarUrl = avatarUrl;
+
+    this.emit("Group.profile", this);
+};
+
+Group.prototype.setMyMembership = function(membership) {
+    if (this.myMembership === membership) return;
+
+    this.myMembership = membership;
+
+    this.emit("Group.myMembership", this);
+};
+
+/**
+ * Sets the 'inviter' property. This does not emit an event (the inviter
+ * will only change when the user is revited / reinvited to a room),
+ * so set this before setting myMembership.
+ * @param {Object} inviter Infomation about who invited us to the room
+ */
+Group.prototype.setInviter = function(inviter) {
+    this.inviter = inviter;
+};
+
+module.exports = Group;
+
+/**
+ * Fires whenever a group's profile information is updated.
+ * This means the 'name' and 'avatarUrl' properties.
+ * @event module:client~MatrixClient#"Group.profile"
+ * @param {Group} group The group whose profile was updated.
+ * @example
+ * matrixClient.on("Group.profile", function(group){
+ *   var name = group.name;
+ * });
+ */
+
+/**
+ * Fires whenever the logged in user's membership status of
+ * the group is updated.
+ * @event module:client~MatrixClient#"Group.myMembership"
+ * @param {Group} group The group in which the user's membership changed
+ * @example
+ * matrixClient.on("Group.myMembership", function(group){
+ *   var myMembership = group.myMembership;
+ * });
+ */
@@ -104,17 +104,20 @@ RoomMember.prototype.setPowerLevelEvent = function(powerLevelEvent) {
    if (powerLevelEvent.getType() !== "m.room.power_levels") {
        return;
    }
-    let maxLevel = powerLevelEvent.getContent().users_default || 0;
-    utils.forEach(utils.values(powerLevelEvent.getContent().users), function(lvl) {
+
+    const evContent = powerLevelEvent.getDirectionalContent();
+
+    let maxLevel = evContent.users_default || 0;
+    utils.forEach(utils.values(evContent.users), function(lvl) {
        maxLevel = Math.max(maxLevel, lvl);
    });
    const oldPowerLevel = this.powerLevel;
    const oldPowerLevelNorm = this.powerLevelNorm;

-    if (powerLevelEvent.getContent().users[this.userId] !== undefined) {
-        this.powerLevel = powerLevelEvent.getContent().users[this.userId];
-    } else if (powerLevelEvent.getContent().users_default !== undefined) {
-        this.powerLevel = powerLevelEvent.getContent().users_default;
+    if (evContent.users && evContent.users[this.userId] !== undefined) {
+        this.powerLevel = evContent.users[this.userId];
+    } else if (evContent.users_default !== undefined) {
+        this.powerLevel = evContent.users_default;
    } else {
        this.powerLevel = 0;
    }
@@ -197,6 +197,16 @@ RoomState.prototype.setStateEvents = function(stateEvents) {
                member.setPowerLevelEvent(event);
                self.emit("RoomState.members", event, self, member);
            });
+
+            // Go through the sentinel members and see if any of them would be
+            // affected by the new power levels. If so, replace the sentinel.
+            for (const userId of Object.keys(self._sentinels)) {
+                const oldSentinel = self._sentinels[userId];
+                const newSentinel = new RoomMember(event.getRoomId(), userId);
+                newSentinel.setMembershipEvent(oldSentinel.events.member, self);
+                newSentinel.setPowerLevelEvent(event);
+                self._sentinels[userId] = newSentinel;
+            }
        }
    });
 };
@@ -383,6 +393,35 @@ RoomState.prototype._maySendEventOfType = function(eventType, userId, state) {
    return member.powerLevel >= required_level;
 };

+/**
+ * Returns true if the given user ID has permission to trigger notification
+ * of type `notifLevelKey`
+ * @param {string} notifLevelKey The level of notification to test (eg. 'room')
+ * @param {string} userId The user ID of the user to test permission for
+ * @return {boolean} true if the given user ID has permission to trigger a
+ *                        notification of this type.
+ */
+RoomState.prototype.mayTriggerNotifOfType = function(notifLevelKey, userId) {
+    const member = this.getMember(userId);
+    if (!member) {
+        return false;
+    }
+
+    const powerLevelsEvent = this.getStateEvents('m.room.power_levels', '');
+
+    let notifLevel = 50;
+    if (
+        powerLevelsEvent &&
+        powerLevelsEvent.getContent() &&
+        powerLevelsEvent.getContent().notifications &&
+        powerLevelsEvent.getContent().notifications[notifLevelKey]
+    ) {
+        notifLevel = powerLevelsEvent.getContent().notifications[notifLevelKey];
+    }
+
+    return member.powerLevel >= notifLevel;
+};
+
 /**
 * The RoomState class.
 */
@@ -27,7 +27,7 @@ const ContentRepo = require("../content-repo");
 const EventTimeline = require("./event-timeline");
 const EventTimelineSet = require("./event-timeline-set");

-import reEmit from '../reemit';
+import ReEmitter from '../ReEmitter';

 function synthesizeReceipt(userId, event, receiptType) {
    // console.log("synthesizing receipt for "+event.getId());
@@ -106,6 +106,8 @@ function Room(roomId, opts) {
    opts = opts || {};
    opts.pendingEventOrdering = opts.pendingEventOrdering || "chronological";

+    this.reEmitter = new ReEmitter(this);
+
    if (["chronological", "detached"].indexOf(opts.pendingEventOrdering) === -1) {
        throw new Error(
            "opts.pendingEventOrdering MUST be either 'chronological' or " +
@@ -153,7 +155,7 @@ function Room(roomId, opts) {
    // all our per-room timeline sets. the first one is the unfiltered ones;
    // the subsequent ones are the filtered ones in no particular order.
    this._timelineSets = [new EventTimelineSet(this, opts)];
-    reEmit(this, this.getUnfilteredTimelineSet(),
+    this.reEmitter.reEmit(this.getUnfilteredTimelineSet(),
           ["Room.timeline", "Room.timelineReset"]);

    this._fixUpLegacyTimelineFields();
@@ -490,7 +492,7 @@ Room.prototype.getOrCreateFilteredTimelineSet = function(filter) {
    }
    const opts = Object.assign({ filter: filter }, this._opts);
    const timelineSet = new EventTimelineSet(this, opts);
-    reEmit(this, timelineSet, ["Room.timeline", "Room.timelineReset"]);
+    this.reEmitter.reEmit(timelineSet, ["Room.timeline", "Room.timelineReset"]);
    this._filteredTimelineSets[filter.filterId] = timelineSet;
    this._timelineSets.push(timelineSet);

@@ -1203,13 +1205,13 @@ function calculateRoomName(room, userId, ignoreRoomNameEvent) {
    if (myMemberEvent && myMemberEvent.content.membership == "invite") {
        if (room.currentState.getMember(myMemberEvent.sender)) {
            // extract who invited us to the room
-            return "Invite from " + room.currentState.getMember(
+            return room.currentState.getMember(
                myMemberEvent.sender,
            ).name;
        } else if (allMembers[0].events.member) {
            // use the sender field from the invite event, although this only
            // gets us the mxid
-            return "Invite from " + myMemberEvent.sender;
+            return myMemberEvent.sender;
        } else {
            return "Room Invite";
        }
@@ -1,5 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
+Copyright 2017 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -17,6 +18,8 @@ limitations under the License.
 * @module pushprocessor
 */

+const RULEKINDS_IN_ORDER = ['override', 'content', 'room', 'sender', 'underride'];
+
 /**
 * Construct a Push Processor.
 * @constructor
@@ -27,12 +30,11 @@ function PushProcessor(client) {
        return string.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
    };

-    const matchingRuleFromKindSet = function(ev, kindset, device) {
-        const rulekinds_in_order = ['override', 'content', 'room', 'sender', 'underride'];
+    const matchingRuleFromKindSet = (ev, kindset, device) => {
        for (let ruleKindIndex = 0;
-                ruleKindIndex < rulekinds_in_order.length;
+                ruleKindIndex < RULEKINDS_IN_ORDER.length;
                ++ruleKindIndex) {
-            const kind = rulekinds_in_order[ruleKindIndex];
+            const kind = RULEKINDS_IN_ORDER[ruleKindIndex];
            const ruleset = kindset[kind];

            for (let ruleIndex = 0; ruleIndex < ruleset.length; ++ruleIndex) {
@@ -46,7 +48,7 @@ function PushProcessor(client) {
                    continue;
                }

-                if (ruleMatchesEvent(rawrule, ev)) {
+                if (this.ruleMatchesEvent(rawrule, ev)) {
                    rule.kind = kind;
                    return rule;
                }
@@ -106,27 +108,38 @@ function PushProcessor(client) {
        return rawrule;
    };

-    const ruleMatchesEvent = function(rule, ev) {
-        let ret = true;
-        for (let i = 0; i < rule.conditions.length; ++i) {
-            const cond = rule.conditions[i];
-            ret &= eventFulfillsCondition(cond, ev);
-        }
-        //console.log("Rule "+rule.rule_id+(ret ? " matches" : " doesn't match"));
-        return ret;
-    };
-
    const eventFulfillsCondition = function(cond, ev) {
        const condition_functions = {
            "event_match": eventFulfillsEventMatchCondition,
            "device": eventFulfillsDeviceCondition,
            "contains_display_name": eventFulfillsDisplayNameCondition,
            "room_member_count": eventFulfillsRoomMemberCountCondition,
+            "sender_notification_permission": eventFulfillsSenderNotifPermCondition,
        };
        if (condition_functions[cond.kind]) {
            return condition_functions[cond.kind](cond, ev);
        }
-        return true;
+        // unknown conditions: we previously matched all unknown conditions,
+        // but given that rules can be added to the base rules on a server,
+        // it's probably better to not match unknown conditions.
+        return false;
+    };
+
+    const eventFulfillsSenderNotifPermCondition = function(cond, ev) {
+        const notifLevelKey = cond['key'];
+        if (!notifLevelKey) {
+            return false;
+        }
+
+        const room = client.getRoom(ev.getRoomId());
+        if (!room || !room.currentState) {
+            return false;
+        }
+
+        // Note that this should not be the current state of the room but the state at
+        // the point the event is in the DAG. Unfortunately the js-sdk does not store
+        // this.
+        return room.currentState.mayTriggerNotifOfType(notifLevelKey, ev.getSender());
    };

    const eventFulfillsRoomMemberCountCondition = function(cond, ev) {
@@ -215,9 +228,9 @@ function PushProcessor(client) {
        // Because micromatch is about 130KB with dependencies,
        // and minimatch is not much better.
        let pat = escapeRegExp(glob);
-        pat = pat.replace(/\\\*/, '.*');
-        pat = pat.replace(/\?/, '.');
-        pat = pat.replace(/\\\[(!|)(.*)\\]/, function(match, p1, p2, offset, string) {
+        pat = pat.replace(/\\\*/g, '.*');
+        pat = pat.replace(/\?/g, '.');
+        pat = pat.replace(/\\\[(!|)(.*)\\]/g, function(match, p1, p2, offset, string) {
            const first = p1 && '^' || '';
            const second = p2.replace(/\\\-/, '-');
            return '[' + first + second + ']';
@@ -291,6 +304,17 @@ function PushProcessor(client) {
        return actionObj;
    };

+    this.ruleMatchesEvent = function(rule, ev) {
+        let ret = true;
+        for (let i = 0; i < rule.conditions.length; ++i) {
+            const cond = rule.conditions[i];
+            ret &= eventFulfillsCondition(cond, ev);
+        }
+        //console.log("Rule "+rule.rule_id+(ret ? " matches" : " doesn't match"));
+        return ret;
+    };
+
+
    /**
     * Get the user's push actions for the given event
     *
@@ -301,6 +325,27 @@ function PushProcessor(client) {
    this.actionsForEvent = function(ev) {
        return pushActionsForEventAndRulesets(ev, client.pushRules);
    };
+
+    /**
+     * Get one of the users push rules by its ID
+     *
+     * @param {string} ruleId The ID of the rule to search for
+     * @return {object} The push rule, or null if no such rule was found
+     */
+    this.getPushRuleById = function(ruleId) {
+        for (const scope of ['device', 'global']) {
+            if (client.pushRules[scope] === undefined) continue;
+
+            for (const kind of RULEKINDS_IN_ORDER) {
+                if (client.pushRules[scope][kind] === undefined) continue;
+
+                for (const rule of client.pushRules[scope][kind]) {
+                    if (rule.rule_id === ruleId) return rule;
+                }
+            }
+        }
+        return null;
+    };
 }

 /**
@@ -1,44 +0,0 @@
-/*
-Copyright 2015, 2016 OpenMarket Ltd
-Copyright 2017 Vector Creations Ltd
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-/**
- * @module
- */
-
-/**
- * re-emit events raised by one EventEmitter from another
- *
- * @param {external:EventEmitter} reEmitEntity
- *     entity from which we want events to be emitted
- * @param {external:EventEmitter} emittableEntity
- *     entity from which events are currently emitted
- * @param {Array<string>} eventNames
- *     list of events to be reemitted
- */
-export default function reEmit(reEmitEntity, emittableEntity, eventNames) {
-    for (const eventName of eventNames) {
-        // setup a listener on the entity (the Room, User, etc) for this event
-        emittableEntity.on(eventName, function(...args) {
-            // take the args from the listener and reuse them, adding the
-            // event name to the arg list so it works with .emit()
-            // Transformation Example:
-            // listener on "foo" => function(a,b) { ... }
-            // Re-emit on "thing" => thing.emit("foo", a, b)
-            reEmitEntity.emit(eventName, ...args);
-        });
-    }
-}
@@ -175,6 +175,7 @@ LocalIndexedDBStoreBackend.prototype = {
            this._syncAccumulator.accumulate({
                next_batch: syncData.nextBatch,
                rooms: syncData.roomsData,
+                groups: syncData.groupsData,
                account_data: {
                    events: accountData,
                },
@@ -251,7 +252,9 @@ LocalIndexedDBStoreBackend.prototype = {
        return Promise.all([
            this._persistUserPresenceEvents(userTuples),
            this._persistAccountData(syncData.accountData),
-            this._persistSyncData(syncData.nextBatch, syncData.roomsData),
+            this._persistSyncData(
+                syncData.nextBatch, syncData.roomsData, syncData.groupsData,
+            ),
        ]);
    },

@@ -259,9 +262,10 @@ LocalIndexedDBStoreBackend.prototype = {
     * Persist rooms /sync data along with the next batch token.
     * @param {string} nextBatch The next_batch /sync value.
     * @param {Object} roomsData The 'rooms' /sync data from a SyncAccumulator
+     * @param {Object} groupsData The 'groups' /sync data from a SyncAccumulator
     * @return {Promise} Resolves if the data was persisted.
     */
-    _persistSyncData: function(nextBatch, roomsData) {
+    _persistSyncData: function(nextBatch, roomsData, groupsData) {
        console.log("Persisting sync data up to ", nextBatch);
        return Promise.try(() => {
            const txn = this.db.transaction(["sync"], "readwrite");
@@ -270,6 +274,7 @@ LocalIndexedDBStoreBackend.prototype = {
                clobber: "-", // constant key so will always clobber
                nextBatch: nextBatch,
                roomsData: roomsData,
+                groupsData: groupsData,
            }); // put == UPSERT
            return promiseifyTxn(txn);
        });
@@ -19,8 +19,8 @@ limitations under the License.
 * This is an internal module. See {@link MatrixInMemoryStore} for the public class.
 * @module store/memory
 */
- const utils = require("../utils");
- const User = require("../models/user");
+const utils = require("../utils");
+const User = require("../models/user");
 import Promise from 'bluebird';

 /**
@@ -35,6 +35,9 @@ module.exports.MatrixInMemoryStore = function MatrixInMemoryStore(opts) {
    this.rooms = {
        // roomId: Room
    };
+    this.groups = {
+        // groupId: Group
+    };
    this.users = {
        // userId: User
    };
@@ -69,6 +72,31 @@ module.exports.MatrixInMemoryStore.prototype = {
        this.syncToken = token;
    },

+    /**
+     * Store the given room.
+     * @param {Group} group The group to be stored
+     */
+    storeGroup: function(group) {
+        this.groups[group.groupId] = group;
+    },
+
+    /**
+     * Retrieve a group by its group ID.
+     * @param {string} groupId The group ID.
+     * @return {Group} The group or null.
+     */
+    getGroup: function(groupId) {
+        return this.groups[groupId] || null;
+    },
+
+    /**
+     * Retrieve all known groups.
+     * @return {Group[]} A list of groups, which may be empty.
+     */
+    getGroups: function() {
+        return utils.values(this.groups);
+    },
+
    /**
     * Store the given room.
     * @param {Room} room The room to be stored. All properties must be stored.
@@ -47,6 +47,30 @@ StubStore.prototype = {
        this.fromToken = token;
    },

+    /**
+     * No-op.
+     * @param {Group} group
+     */
+    storeGroup: function(group) {
+    },
+
+    /**
+     * No-op.
+     * @param {string} groupId
+     * @return {null}
+     */
+    getGroup: function(groupId) {
+        return null;
+    },
+
+    /**
+     * No-op.
+     * @return {Array} An empty array.
+     */
+    getGroups: function() {
+        return [];
+    },
+
    /**
     * No-op.
     * @param {Room} room
@@ -72,10 +72,18 @@ class SyncAccumulator {
        // coherent /sync response and know at what point they should be
        // streaming from without losing events.
        this.nextBatch = null;
+
+        // { ('invite'|'join'|'leave'): $groupId: { ... sync 'group' data } }
+        this.groups = {
+            invite: {},
+            join: {},
+            leave: {},
+        };
    }

    accumulate(syncResponse) {
        this._accumulateRooms(syncResponse);
+        this._accumulateGroups(syncResponse);
        this._accumulateAccountData(syncResponse);
        this.nextBatch = syncResponse.next_batch;
    }
@@ -336,6 +344,44 @@ class SyncAccumulator {
        }
    }

+    /**
+     * Accumulate incremental /sync group data.
+     * @param {Object} syncResponse the complete /sync JSON
+     */
+    _accumulateGroups(syncResponse) {
+        if (!syncResponse.groups) {
+            return;
+        }
+        if (syncResponse.groups.invite) {
+            Object.keys(syncResponse.groups.invite).forEach((groupId) => {
+                this._accumulateGroup(
+                    groupId, "invite", syncResponse.groups.invite[groupId],
+                );
+            });
+        }
+        if (syncResponse.groups.join) {
+            Object.keys(syncResponse.groups.join).forEach((groupId) => {
+                this._accumulateGroup(
+                    groupId, "join", syncResponse.groups.join[groupId],
+                );
+            });
+        }
+        if (syncResponse.groups.leave) {
+            Object.keys(syncResponse.groups.leave).forEach((groupId) => {
+                this._accumulateGroup(
+                    groupId, "leave", syncResponse.groups.leave[groupId],
+                );
+            });
+        }
+    }
+
+    _accumulateGroup(groupId, category, data) {
+        for (const cat of ['invite', 'join', 'leave']) {
+            delete this.groups[cat][groupId];
+        }
+        this.groups[category][groupId] = data;
+    }
+
    /**
     * Return everything under the 'rooms' key from a /sync response which
     * represents all room data that should be stored. This should be paired
@@ -470,6 +516,7 @@ class SyncAccumulator {
        return {
            nextBatch: this.nextBatch,
            roomsData: data,
+            groupsData: this.groups,
            accountData: accData,
        };
    }
@@ -27,12 +27,11 @@ limitations under the License.
 import Promise from 'bluebird';
 const User = require("./models/user");
 const Room = require("./models/room");
+const Group = require('./models/group');
 const utils = require("./utils");
 const Filter = require("./filter");
 const EventTimeline = require("./models/event-timeline");

-import reEmit from './reemit';
-
 const DEBUG = true;

 // /sync requests allow you to set a timeout= but the request may continue
@@ -99,7 +98,7 @@ function SyncApi(client, opts) {
    this._failedSyncCount = 0; // Number of consecutive failed /sync requests

    if (client.getNotifTimelineSet()) {
-        reEmit(client, client.getNotifTimelineSet(),
+        client.reEmitter.reEmit(client.getNotifTimelineSet(),
               ["Room.timeline", "Room.timelineReset"]);
    }
 }
@@ -114,7 +113,7 @@ SyncApi.prototype.createRoom = function(roomId) {
        pendingEventOrdering: this.opts.pendingEventOrdering,
        timelineSupport: client.timelineSupport,
    });
-    reEmit(client, room, ["Room.name", "Room.timeline", "Room.redaction",
+    client.reEmitter.reEmit(room, ["Room.name", "Room.timeline", "Room.redaction",
                          "Room.receipt", "Room.tags",
                          "Room.timelineReset",
                          "Room.localEchoUpdated",
@@ -124,6 +123,18 @@ SyncApi.prototype.createRoom = function(roomId) {
    return room;
 };

+/**
+ * @param {string} groupId
+ * @return {Group}
+ */
+SyncApi.prototype.createGroup = function(groupId) {
+    const client = this.client;
+    const group = new Group(groupId);
+    client.reEmitter.reEmit(group, ["Group.profile", "Group.myMembership"]);
+    client.store.storeGroup(group);
+    return group;
+};
+
 /**
 * @param {Room} room
 * @private
@@ -133,13 +144,13 @@ SyncApi.prototype._registerStateListeners = function(room) {
    // we need to also re-emit room state and room member events, so hook it up
    // to the client now. We need to add a listener for RoomState.members in
    // order to hook them correctly. (TODO: find a better way?)
-    reEmit(client, room.currentState, [
+    client.reEmitter.reEmit(room.currentState, [
        "RoomState.events", "RoomState.members", "RoomState.newMember",
    ]);
    room.currentState.on("RoomState.newMember", function(event, state, member) {
        member.user = client.getUser(member.userId);
-        reEmit(
-            client, member,
+        client.reEmitter.reEmit(
+            member,
            [
                "RoomMember.name", "RoomMember.typing", "RoomMember.powerLevel",
                "RoomMember.membership",
@@ -220,6 +231,8 @@ SyncApi.prototype.syncLeftRooms = function() {
            room.recalculate(client.credentials.userId);
            client.store.storeRoom(room);
            client.emit("Room", room);
+
+            self._processEventsForNotifs(room, timelineEvents);
        });
        return rooms;
    });
@@ -486,15 +499,14 @@ SyncApi.prototype.retryImmediately = function() {
 * @param {string} syncOptions.filterId
 * @param {boolean} syncOptions.hasSyncedBefore
 */
-SyncApi.prototype._sync = function(syncOptions) {
+SyncApi.prototype._sync = async function(syncOptions) {
    const client = this.client;
-    const self = this;

    if (!this._running) {
        debuglog("Sync no longer running: exiting.");
-        if (self._connectionReturnedDefer) {
-            self._connectionReturnedDefer.reject();
-            self._connectionReturnedDefer = null;
+        if (this._connectionReturnedDefer) {
+            this._connectionReturnedDefer.reject();
+            this._connectionReturnedDefer = null;
        }
        this._updateSyncState("STOPPED");
        return;
@@ -550,123 +562,134 @@ SyncApi.prototype._sync = function(syncOptions) {
        qps.timeout = 0;
    }

-    let isCachedResponse = false;
-
-    let syncPromise;
+    let savedSync;
    if (!syncOptions.hasSyncedBefore) {
        // Don't do an HTTP hit to /sync. Instead, load up the persisted /sync data,
        // if there is data there.
-        syncPromise = client.store.getSavedSync();
-    } else {
-        syncPromise = Promise.resolve(null);
+        savedSync = await client.store.getSavedSync();
    }

-    syncPromise.then((savedSync) => {
-        if (savedSync) {
-            debuglog("sync(): not doing HTTP hit, instead returning stored /sync data");
-            isCachedResponse = true;
-            return {
-                next_batch: savedSync.nextBatch,
-                rooms: savedSync.roomsData,
-                account_data: {
-                    events: savedSync.accountData,
-                },
-            };
-        } else {
+    let isCachedResponse = false;
+    let data;
+
+    if (savedSync) {
+        debuglog("sync(): not doing HTTP hit, instead returning stored /sync data");
+        isCachedResponse = true;
+        data = {
+            next_batch: savedSync.nextBatch,
+            rooms: savedSync.roomsData,
+            groups: savedSync.groupsData,
+            account_data: {
+                events: savedSync.accountData,
+            },
+        };
+    } else {
+        try {
            //debuglog('Starting sync since=' + syncToken);
            this._currentSyncRequest = client._http.authedRequest(
                undefined, "GET", "/sync", qps, undefined, clientSideTimeoutMs,
            );
-            return this._currentSyncRequest;
-        }
-    }).then(function(data) {
-        //debuglog('Completed sync, next_batch=' + data.next_batch);
-
-        // set the sync token NOW *before* processing the events. We do this so
-        // if something barfs on an event we can skip it rather than constantly
-        // polling with the same token.
-        client.store.setSyncToken(data.next_batch);
-
-        // Reset after a successful sync
-        self._failedSyncCount = 0;
-
-        // We need to wait until the sync data has been sent to the backend
-        // because it appears that the sync data gets modified somewhere in
-        // processing it in such a way as to make it no longer cloneable.
-        // XXX: Find out what is modifying it!
-        if (!isCachedResponse) {
-            // Don't give the store back its own cached data
-            return client.store.setSyncData(data).then(() => {
-                return data;
-            });
-        } else {
-            return Promise.resolve(data);
-        }
-    }).done((data) => {
-        self._processSyncResponse(syncToken, data).catch((e) => {
-            // log the exception with stack if we have it, else fall back
-            // to the plain description
-            console.error("Caught /sync error", e.stack || e);
-        }).then(() => {
-            // emit synced events
-            const syncEventData = {
-                oldSyncToken: syncToken,
-                nextSyncToken: data.next_batch,
-                catchingUp: self._catchingUp,
-            };
-
-            if (!syncOptions.hasSyncedBefore) {
-                self._updateSyncState("PREPARED", syncEventData);
-                syncOptions.hasSyncedBefore = true;
-            }
-
-            // keep emitting SYNCING -> SYNCING for clients who want to do bulk updates
-            if (!isCachedResponse) {
-                self._updateSyncState("SYNCING", syncEventData);
-
-                // tell databases that everything is now in a consistent state and can be
-                // saved (no point doing so if we only have the data we just got out of the
-                // store).
-                client.store.save();
-            }
-
-            // Begin next sync
-            self._sync(syncOptions);
-        });
-    }, function(err) {
-        if (!self._running) {
-            debuglog("Sync no longer running: exiting");
-            if (self._connectionReturnedDefer) {
-                self._connectionReturnedDefer.reject();
-                self._connectionReturnedDefer = null;
-            }
-            self._updateSyncState("STOPPED");
+            data = await this._currentSyncRequest;
+        } catch (e) {
+            this._onSyncError(e, syncOptions);
            return;
        }
-        console.error("/sync error %s", err);
-        console.error(err);
+    }

-        self._failedSyncCount++;
-        console.log('Number of consecutive failed sync requests:', self._failedSyncCount);
+    //debuglog('Completed sync, next_batch=' + data.next_batch);

-        debuglog("Starting keep-alive");
-        // Note that we do *not* mark the sync connection as
-        // lost yet: we only do this if a keepalive poke
-        // fails, since long lived HTTP connections will
-        // go away sometimes and we shouldn't treat this as
-        // erroneous. We set the state to 'reconnecting'
-        // instead, so that clients can onserve this state
-        // if they wish.
-        self._startKeepAlives().done(function() {
-            self._sync(syncOptions);
-        });
-        self._currentSyncRequest = null;
-        // Transition from RECONNECTING to ERROR after a given number of failed syncs
-        self._updateSyncState(
-            self._failedSyncCount >= FAILED_SYNC_ERROR_THRESHOLD ?
-                "ERROR" : "RECONNECTING",
-        );
+    // set the sync token NOW *before* processing the events. We do this so
+    // if something barfs on an event we can skip it rather than constantly
+    // polling with the same token.
+    client.store.setSyncToken(data.next_batch);
+
+    // Reset after a successful sync
+    this._failedSyncCount = 0;
+
+    // We need to wait until the sync data has been sent to the backend
+    // because it appears that the sync data gets modified somewhere in
+    // processing it in such a way as to make it no longer cloneable.
+    // XXX: Find out what is modifying it!
+    if (!isCachedResponse) {
+        // Don't give the store back its own cached data
+        await client.store.setSyncData(data);
+    }
+
+    try {
+        await this._processSyncResponse(syncToken, data, isCachedResponse);
+    } catch(e) {
+        // log the exception with stack if we have it, else fall back
+        // to the plain description
+        console.error("Caught /sync error", e.stack || e);
+    }
+
+    // emit synced events
+    const syncEventData = {
+        oldSyncToken: syncToken,
+        nextSyncToken: data.next_batch,
+        catchingUp: this._catchingUp,
+    };
+
+    if (!syncOptions.hasSyncedBefore) {
+        this._updateSyncState("PREPARED", syncEventData);
+        syncOptions.hasSyncedBefore = true;
+    }
+
+    if (!isCachedResponse) {
+        // tell the crypto module to do its processing. It may block (to do a
+        // /keys/changes request).
+        if (this.opts.crypto) {
+            await this.opts.crypto.onSyncCompleted(syncEventData);
+        }
+
+        // keep emitting SYNCING -> SYNCING for clients who want to do bulk updates
+        this._updateSyncState("SYNCING", syncEventData);
+
+        // tell databases that everything is now in a consistent state and can be
+        // saved (no point doing so if we only have the data we just got out of the
+        // store).
+        client.store.save();
+    }
+
+    // Begin next sync
+    this._sync(syncOptions);
+};
+
+SyncApi.prototype._onSyncError = function(err, syncOptions) {
+    if (!this._running) {
+        debuglog("Sync no longer running: exiting");
+        if (this._connectionReturnedDefer) {
+            this._connectionReturnedDefer.reject();
+            this._connectionReturnedDefer = null;
+        }
+        this._updateSyncState("STOPPED");
+        return;
+    }
+
+    console.error("/sync error %s", err);
+    console.error(err);
+
+    this._failedSyncCount++;
+    console.log('Number of consecutive failed sync requests:', this._failedSyncCount);
+
+    debuglog("Starting keep-alive");
+    // Note that we do *not* mark the sync connection as
+    // lost yet: we only do this if a keepalive poke
+    // fails, since long lived HTTP connections will
+    // go away sometimes and we shouldn't treat this as
+    // erroneous. We set the state to 'reconnecting'
+    // instead, so that clients can onserve this state
+    // if they wish.
+    this._startKeepAlives().then(() => {
+        this._sync(syncOptions);
    });
+
+    this._currentSyncRequest = null;
+    // Transition from RECONNECTING to ERROR after a given number of failed syncs
+    this._updateSyncState(
+        this._failedSyncCount >= FAILED_SYNC_ERROR_THRESHOLD ?
+            "ERROR" : "RECONNECTING",
+    );
 };

 /**
@@ -676,8 +699,11 @@ SyncApi.prototype._sync = function(syncOptions) {
 * @param {string} syncToken the old next_batch token sent to this
 *    sync request.
 * @param {Object} data The response from /sync
+ * @param {bool} isCachedResponse True if this response is from our local cache
 */
-SyncApi.prototype._processSyncResponse = async function(syncToken, data) {
+SyncApi.prototype._processSyncResponse = async function(
+    syncToken, data, isCachedResponse,
+) {
    const client = this.client;
    const self = this;

@@ -714,6 +740,19 @@ SyncApi.prototype._processSyncResponse = async function(syncToken, data) {
    //        }
    //      }
    //    },
+    //    groups: {
+    //        invite: {
+    //            $groupId: {
+    //                inviter: $inviter,
+    //                profile: {
+    //                    avatar_url: $avatarUrl,
+    //                    name: $groupName,
+    //                },
+    //            },
+    //        },
+    //        join: {},
+    //        leave: {},
+    //    },
    // }

    // TODO-arch:
@@ -743,7 +782,13 @@ SyncApi.prototype._processSyncResponse = async function(syncToken, data) {
        client.store.storeAccountDataEvents(events);
        events.forEach(
            function(accountDataEvent) {
-                if (accountDataEvent.getType() == 'm.push_rules') {
+                // XXX: This is awful: ignore push rules from our cached sync. We fetch the
+                // push rules before syncing so we actually have up-to-date ones. We do want
+                // to honour new push rules that come down the sync but synapse doesn't
+                // put new push rules in the sync stream when the base rules change, so
+                // if the base rules change, we do need to refresh. We therefore ignore
+                // the push rules in our cached sync response.
+                if (accountDataEvent.getType() == 'm.push_rules' && !isCachedResponse) {
                    client.pushRules = accountDataEvent.getContent();
                }
                client.emit("accountData", accountDataEvent);
@@ -781,6 +826,20 @@ SyncApi.prototype._processSyncResponse = async function(syncToken, data) {
        this._catchingUp = false;
    }

+    if (data.groups) {
+        if (data.groups.invite) {
+            this._processGroupSyncEntry(data.groups.invite, 'invite');
+        }
+
+        if (data.groups.join) {
+            this._processGroupSyncEntry(data.groups.join, 'join');
+        }
+
+        if (data.groups.leave) {
+            this._processGroupSyncEntry(data.groups.leave, 'leave');
+        }
+    }
+
    // the returned json structure is a bit crap, so make it into a
    // nicer form (array) after applying sanity to make sure we don't fail
    // on missing keys (on the off chance)
@@ -913,6 +972,8 @@ SyncApi.prototype._processSyncResponse = async function(syncToken, data) {
            client.emit("Room", room);
        }

+        self._processEventsForNotifs(room, timelineEvents);
+
        async function processRoomEvent(e) {
            client.emit("event", e);
            if (e.isState() && e.getType() == "m.room.encryption" && self.opts.crypto) {
@@ -949,6 +1010,8 @@ SyncApi.prototype._processSyncResponse = async function(syncToken, data) {
            client.emit("Room", room);
        }

+        self._processEventsForNotifs(room, timelineEvents);
+
        stateEvents.forEach(function(e) {
            client.emit("event", e);
        });
@@ -975,10 +1038,14 @@ SyncApi.prototype._processSyncResponse = async function(syncToken, data) {
    }

    // Handle device list updates
-    if (this.opts.crypto && data.device_lists && data.device_lists.changed) {
-        data.device_lists.changed.forEach((u) => {
-            this.opts.crypto.userDeviceListChanged(u);
-        });
+    if (data.device_lists) {
+        if (this.opts.crypto) {
+            await this.opts.crypto.handleDeviceListChanges(data.device_lists);
+        } else {
+            // FIXME if we *don't* have a crypto module, we still need to
+            // invalidate the device lists. But that would require a
+            // substantial bit of rework :/.
+        }
    }

    // Handle one_time_keys_count
@@ -1070,6 +1137,35 @@ SyncApi.prototype._pokeKeepAlive = function() {
    });
 };

+/**
+ * @param {Object} groupsSection Groups section object, eg. response.groups.invite
+ * @param {string} sectionName Which section this is ('invite', 'join' or 'leave')
+ */
+SyncApi.prototype._processGroupSyncEntry = function(groupsSection, sectionName) {
+    // Processes entries from 'groups' section of the sync stream
+    for (const groupId of Object.keys(groupsSection)) {
+        const groupInfo = groupsSection[groupId];
+        let group = this.client.store.getGroup(groupId);
+        const isBrandNew = group === null;
+        if (group === null) {
+            group = this.createGroup(groupId);
+        }
+        if (groupInfo.profile) {
+            group.setProfile(
+                groupInfo.profile.name, groupInfo.profile.avatar_url,
+            );
+        }
+        if (groupInfo.inviter) {
+            group.setInviter({userId: groupInfo.inviter});
+        }
+        group.setMyMembership(sectionName);
+        if (isBrandNew) {
+            // Now we've filled in all the fields, emit the Group event
+            this.client.emit("Group", group);
+        }
+    }
+};
+
 /**
 * @param {Object} obj
 * @return {Object[]}
@@ -1196,20 +1292,33 @@ SyncApi.prototype._processRoomEvents = function(room, stateEventList,
    // may make notifications appear which should have the right name.
    room.recalculate(this.client.credentials.userId);

+    // execute the timeline events, this will begin to diverge the current state
+    // if the timeline has any state events in it.
+    // This also needs to be done before running push rules on the events as they need
+    // to be decorated with sender etc.
+    room.addLiveEvents(timelineEventList);
+};
+
+/**
+ * Takes a list of timelineEvents and adds and adds to _notifEvents
+ * as appropriate.
+ * This must be called after the room the events belong to has been stored.
+ *
+ * @param {Room} room
+ * @param {MatrixEvent[]} [timelineEventList] A list of timeline events. Lower index
+ * is earlier in time. Higher index is later.
+ */
+SyncApi.prototype._processEventsForNotifs = function(room, timelineEventList) {
    // gather our notifications into this._notifEvents
-    if (client.getNotifTimelineSet()) {
+    if (this.client.getNotifTimelineSet()) {
        for (let i = 0; i < timelineEventList.length; i++) {
-            const pushActions = client.getPushActionsForEvent(timelineEventList[i]);
+            const pushActions = this.client.getPushActionsForEvent(timelineEventList[i]);
            if (pushActions && pushActions.notify &&
                pushActions.tweaks && pushActions.tweaks.highlight) {
                this._notifEvents.push(timelineEventList[i]);
            }
        }
    }
-
-    // execute the timeline events, this will begin to diverge the current state
-    // if the timeline has any state events in it.
-    room.addLiveEvents(timelineEventList);
 };

 /**
@@ -1255,7 +1364,7 @@ SyncApi.prototype._onOnline = function() {

 function createNewUser(client, userId) {
    const user = new User(userId);
-    reEmit(client, user, [
+    client.reEmitter.reEmit(user, [
        "User.avatarUrl", "User.displayName", "User.presence",
        "User.currentlyActive", "User.lastPresenceTs",
    ]);
@@ -1296,8 +1296,8 @@ module.exports.setVideoInput = function(deviceId) { videoInput = deviceId; };
 * Create a new Matrix call for the browser.
 * @param {MatrixClient} client The client instance to use.
 * @param {string} roomId The room the call is in.
- * @param {Object?} options optional options map.
- * @param {boolean} options.forceTURN whether relay through TURN should be forced.
+ * @param {Object?} options DEPRECATED optional options map.
+ * @param {boolean} options.forceTURN DEPRECATED whether relay through TURN should be forced. This option is deprecated - use opts.forceTURN when creating the matrix client since it's only possible to set this option on outbound calls.
 * @return {MatrixCall} the call or null if the browser doesn't support calling.
 */
 module.exports.createNewMatrixCall = function(client, roomId, options) {
@@ -1350,6 +1350,9 @@ module.exports.createNewMatrixCall = function(client, roomId, options) {
            !webRtc.RtcPeerConnection || !webRtc.getUserMedia) {
        return null; // WebRTC is not supported.
    }
+
+    const optionsForceTURN = options ? options.forceTURN : false;
+
    const opts = {
        webRtc: webRtc,
        client: client,
@@ -1357,7 +1360,7 @@ module.exports.createNewMatrixCall = function(client, roomId, options) {
        roomId: roomId,
        turnServers: client.getTurnServers(),
        // call level options
-        forceTURN: options ? options.forceTURN : false,
+        forceTURN: client._forceTURN || optionsForceTURN,
    };
    return new MatrixCall(opts);
 };
Author	SHA1	Message	Date
Luke Barnard	678ff23bcb	v0.9.2	2017-12-04 11:57:05 +00:00
Luke Barnard	f93d50dcd0	Prepare changelog for v0.9.2	2017-12-04 11:57:04 +00:00
Luke Barnard	aa3201ebb0	Send correct m.visibility flag in js-sdk	2017-12-04 11:54:42 +00:00
David Baker	5b1a5b7dd0	v0.9.1	2017-11-17 15:55:21 +00:00
David Baker	f25324fb1c	Prepare changelog for v0.9.1	2017-11-17 15:55:20 +00:00
David Baker	2a7f35a633	Merge pull request #577 from matrix-org/dbkr/fix_force_turn Fix the force TURN option	2017-11-17 15:51:01 +00:00
David Baker	eb2d5484b8	jsdoc and remove pointless return	2017-11-17 15:31:50 +00:00
David Baker	40cbd5ec9d	Honour old forceTURN option for now	2017-11-17 15:03:12 +00:00
David Baker	10680ace17	Fix the force TURN option Option needed to be passed in when creating a webrtc call, but for incoming calls the js-sdk creates the call itself, so the app never gets a chance to set the option.	2017-11-17 14:51:37 +00:00
Luke Barnard	3091a76702	v0.9.0	2017-11-15 10:56:31 +00:00
Luke Barnard	117e2370d7	Prepare changelog for v0.9.0	2017-11-15 10:56:31 +00:00
Luke Barnard	a60e4efe6d	v0.9.0-rc.1	2017-11-10 13:30:24 +00:00
Luke Barnard	4c0ebeab58	Prepare changelog for v0.9.0-rc.1	2017-11-10 13:30:24 +00:00
David Baker	0143ac2a86	Ignore pushrules in our cached sync response (#571 ) * Ignore pushrules in our cached sync response As hopefully explained in the comment	2017-11-10 13:14:54 +00:00
Luke Barnard	497a2bd057	Update API wrapper for group room association configuration (#569 ) This is now a generic API on the homeserver. This is wrapped in a non-generic js-sdk function for now. In future, other group server implementations may want specific configuration and the configuration keys are namespaced to prevent collisions there.	2017-11-09 17:34:26 +00:00
Travis Ralston	bf462e2840	Install babel-runtime, as it is required for operation (#561 ) Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-11-08 14:34:08 +00:00
Matthew Hodgson	fb75134179	unbreak tests	2017-11-06 15:17:06 +00:00
Matthew Hodgson	68d67b7fc2	Remove ugly english non-i18n invite descriptions any sensible app should be synthesising a better name for invites which is i18n-aware.	2017-11-06 11:18:24 +00:00
David Baker	bbf412f3ad	Merge pull request #567 from matrix-org/luke/groups-update-group-room-assoc Modify addRoomToGroup to allow setting isPublic, create alias updateGroupRoomAssociation	2017-11-02 14:55:01 +00:00
David Baker	52b575296c	Merge pull request #565 from matrix-org/dbkr/get_rule_by_id Expose more functionality of pushprocessor	2017-11-02 13:49:00 +00:00
David Baker	c1652f4898	Merge pull request #566 from matrix-org/dbkr/may_trigger_notif Function for working out notif trigger permission	2017-11-02 13:48:07 +00:00
Luke Barnard	d40cc795f7	Modify addRoomToGroup to allow setting isPublic, create alias updateGroupRoomAssociation The API on synapse was modified to be an upsert, which means the same API call for adding a room to a group can be used to update the visibility of that association.	2017-11-02 13:34:34 +00:00
David Baker	4035d933ad	Use the default power level if there's no PL event	2017-11-02 13:28:58 +00:00
David Baker	d255348762	Remove leave check 1. It's wrong because it doesn't consider the ban state 2. This should be caught by whether they have permission to send the event in the first place rather than worrying about whether it has notification permission or not	2017-11-02 13:26:39 +00:00
David Baker	978db89deb	Function for working out notif trigger permission And make pushprocessor use this function	2017-11-02 12:04:55 +00:00
David Baker	b867afe772	lint	2017-11-01 19:48:12 +00:00
David Baker	e34fd89bb6	Expose more functionality of pushprocessor This exposes a function to get rules by their ID and ruleMatchesEvent to test whether a rule matches a given event.	2017-11-01 19:40:50 +00:00
Richard van der Hoff	51883b8f11	Merge pull request #555 from uhoreg/e2e_replay keep track of event ID and timestamp of decrypted messages	2017-10-25 14:14:33 +01:00
David Baker	ff5f95227a	Merge pull request #564 from matrix-org/dbkr/fix_notifevents Fix notifEvent computation	2017-10-25 12:06:05 +01:00
David Baker	dc929236a4	oops, events not event	2017-10-25 12:03:17 +01:00
David Baker	b09e20747a	Oops, missing this	2017-10-25 12:00:11 +01:00
David Baker	e8fc857dbc	Don't calculate notifEvents until ready We were previously computing notifEvents at the point where we processed them but before the room they belong to was stored. This was problematic because some push conditions try to get the room and therefore failed. Since the push actions are cached, this spurious calculation also got cached. This moves the calculation out to a separate function that gets called only after the room has been stored.	2017-10-25 11:54:44 +01:00
David Baker	b42a13cc3b	Merge pull request #563 from matrix-org/dbkr/fix_sentinels_on_power_levels Fix power level of sentinel members	2017-10-25 11:47:18 +01:00
David Baker	37ed9800c5	Fix sender_notification_permission cond Don't assume all events have a sender, because they don't.	2017-10-25 11:10:55 +01:00
David Baker	99f4968888	Replace all the sentinel members on power_levels Rather than trying to be clever and keep ones whose power level isn't affected, since they'll still have an associated power_levels event that isn't the one actually in effect, which could be confusing.	2017-10-25 10:57:28 +01:00
David Baker	34428a811c	Fix setPowerLevelEvent() semantics * Make setPowerLevelEvent() use getDirectionalContent() rather than getContent(), otherwise power level events from pagination set power levels the wrong way * Don't assume the presence of a `users` key since the first power_levels event in a room will have no prev_content	2017-10-25 10:40:15 +01:00
Richard van der Hoff	da7104b00d	Merge pull request #554 from uhoreg/e2e_redact don't try to decrypt a redacted message (fixes vector-im/riot-web#3744)	2017-10-24 21:02:01 +01:00
Hubert Chathi	b373601e13	make sure redacted e2e messages are returned as a promise Signed-off-by: Hubert Chathi <hubert@uhoreg.ca>	2017-10-24 13:11:48 -04:00
David Baker	fcc90e884b	Fix power level of sentinel members When a ppower_levels event arrived, the current member objects were being updated but the sentinel members were not, meaning that the power levels of RoomMember objects associated with events had stale power levels. This was causing power level based push rules to be wrong.	2017-10-23 19:07:04 +01:00
David Baker	7a90096077	Merge pull request #562 from matrix-org/dbkr/room_notifs Support room notifs	2017-10-23 17:01:23 +01:00
David Baker	e7886a55fe	lint	2017-10-23 16:50:01 +01:00
David Baker	e33b786f65	Support room notifs Specifically, add support for the sender_notification_permission push rule condition. Also delays calculating notification events until after they're added to the timeline, as per comment.	2017-10-23 16:02:06 +01:00
Luke Barnard	b0b50f4ef9	Implement API wrapper for GET /group/$groupId/invited_users (#560 )	2017-10-16 18:23:40 +02:00
David Baker	37c8e6dc07	Merge branch 'master' into develop	2017-10-16 14:37:39 +01:00
David Baker	675441fe93	v0.8.5	2017-10-16 14:35:20 +01:00
David Baker	2495d97862	Prepare changelog for v0.8.5	2017-10-16 14:35:20 +01:00
David Baker	70c998dd55	Merge pull request #558 from matrix-org/dbkr/fix_glob_to_regex Fix the glob-to-regex code	2017-10-16 14:04:14 +01:00
David Baker	ec60e46611	Merge remote-tracking branch 'origin/develop' into release-v0.8.5	2017-10-16 14:03:04 +01:00
David Baker	bb2dc618f7	Merge pull request #559 from matrix-org/dbkr/unknown_condition_dont_match Make unknown pushrule conditions not match	2017-10-16 14:02:02 +01:00
David Baker	d9addf84ef	Make unknown pushrule conditions not match Unknown conditions previously were assumed to match, but given rules can be added on the server as base rules, it's probably better to not match unknown conditions.	2017-10-16 13:55:07 +01:00
David Baker	d89163110e	Fix the glob-to-regex code It was only making one replacement so would fail for anything with more than ine glob special char (eg. foo would be fine, but foo* was broken)	2017-10-16 13:40:49 +01:00
David Baker	93b722af3c	v0.8.5-rc.1	2017-10-13 11:01:58 +01:00
David Baker	ee0e036aa5	Prepare changelog for v0.8.5-rc.1	2017-10-13 11:01:57 +01:00
David Baker	24ef004adc	remove debug logging	2017-10-12 16:03:08 +01:00
Hubert Chathi	dcab4eb70b	fix lint error, and incorporate suggestions from richvdh and krombel Signed-off-by: Hubert Chathi <hubert@uhoreg.ca>	2017-10-12 09:04:12 -04:00
Hubert Chathi	8f252992e4	keep track of event ID and timestamp of decrypted messages This is to avoid false positives when detecting replay attacks. fixes: vector-im/riot-web#3712 Signed-off-by: Hubert Chathi <hubert@uhoreg.ca>	2017-10-11 23:27:36 -04:00
Hubert Chathi	a3a3e32e21	don't try to decrypt a redacted message (fixes vector-im/riot-web#3744 ) set the decrypted data to an empty message Signed-off-by: Hubert Chathi <hubert@uhoreg.ca>	2017-10-04 12:26:08 -04:00
David Baker	f5f8867326	Merge pull request #553 from matrix-org/luke/groups-remove-room Implement wrapper API for removing a room from a group	2017-10-04 10:58:28 +01:00
Luke Barnard	5494a9dd3b	Implement wrapper API for removing a room from a group	2017-09-29 17:57:53 +01:00
David Baker	89f7857c84	Merge pull request #552 from matrix-org/rav/stuck_key_download Fix typo which resulted in stuck key download requests	2017-09-29 10:22:26 +01:00
Richard van der Hoff	3140c6526a	Fix typo which resulted in stuck key download requests (... and hence stuck outgoing e2e messages) Fixes https://github.com/vector-im/riot-web/issues/4278	2017-09-29 10:02:16 +01:00
Luke Barnard	96d828d22b	Implement API wrapper for adding a room to a group (#551 )	2017-09-27 14:27:19 +01:00
David Baker	a8a50384b8	Add wrapper for group update_publicity API (#550 ) Add wrapper for group update_publicity API	2017-09-25 15:27:59 +01:00
David Baker	787af6448d	Merge pull request #549 from matrix-org/dbkr/groups_store_when_created Store group when it's created	2017-09-22 18:02:17 +01:00
David Baker	4633135322	Store group when it's created Otherwise we emit events when we set properties on the group but getGroup() for that group returns null which is very confusing.	2017-09-22 17:57:08 +01:00
David Baker	050d522735	Merge pull request #548 from matrix-org/luke/groups-remove-rooms-users-from-summary Luke/groups remove rooms users from summary	2017-09-22 16:23:47 +01:00
Luke Barnard	809b202e70	Merge branch 'develop' into luke/groups-remove-rooms-users-from-summary	2017-09-22 16:19:17 +01:00
Luke Barnard	f1555dbb5d	Implement wrapper APIs for removing users and rooms from group summary	2017-09-22 16:18:18 +01:00
krombel	d1d0266a10	reduce sendToDevice payload (#522 ) instead of sending one huge request split them up.	2017-09-22 14:29:23 +01:00
David Baker	ee37ed0697	Add a clean target (#547 )	2017-09-21 21:45:55 +01:00
David Baker	e3972dee2c	Merge branch 'master' into develop	2017-09-21 21:41:48 +01:00
David Baker	7a727e7eda	v0.8.4	2017-09-21 21:39:37 +01:00
David Baker	d24c822f68	Prepare changelog for v0.8.4	2017-09-21 21:39:37 +01:00
Richard van der Hoff	8d804013f3	Merge pull request #546 from matrix-org/dbkr/clean_on_prerelease Clean on prepublish	2017-09-21 18:06:28 +01:00
David Baker	d73a115436	Clean on prerelease Otherwise you can make broken releases on case insensitive file systems	2017-09-21 18:02:30 +01:00
Luke Barnard	0b4ff731e8	Implement wrapper APIs for adding rooms to group summary (#545 )	2017-09-21 17:05:36 +01:00
Luke Barnard	a58ec3f192	Implement wrapper APIs for adding rooms to group summary	2017-09-21 17:01:33 +01:00
Richard van der Hoff	868c20b161	Fix a race in decrypting megolm messages (#544 ) * Fix a race in decrypting megolm messages This fixes a race wherein it was possible for us to fail to decrypt a message, if the keys arrived immediately after our attempt to decrypt it. In that case, a retry should have been scheduled, but was not. Fixes https://github.com/vector-im/riot-web/issues/5001. * WORDS	2017-09-21 15:17:28 +01:00
Luke Barnard	c2cd050419	Implement API to add users to group summaries (#543 ) (and call it addUserToGroupSummary)	2017-09-20 16:31:28 +01:00
David Baker	7d5c107fb8	v0.8.3	2017-09-20 15:04:11 +01:00
David Baker	d855a6ea0f	Prepare changelog for v0.8.3	2017-09-20 15:04:10 +01:00
David Baker	464f84d8cd	v0.8.3-rc.1	2017-09-19 10:44:58 +01:00
David Baker	fe0ee6402a	Prepare changelog for v0.8.3-rc.1	2017-09-19 10:44:57 +01:00
David Baker	068939f790	APIs for flair (#542 )	2017-09-18 14:44:40 +01:00
Matthew Hodgson	35f48d1c8e	Merge pull request #526 from t3chguy/allow_trailing_slash consume trailing slash when creating Matrix Client in HS and IS urls	2017-09-17 22:53:48 +01:00
Matthew Hodgson	52adde2501	Merge pull request #539 from turt2live/travis/ignored_users Add ignore users API	2017-09-17 21:47:03 +01:00
David Baker	0ddc4eceaf	Merge pull request #540 from matrix-org/dbkr/jsdoc_355 Upgrade to jsdoc 3.5.5	2017-09-15 09:19:21 +01:00
turt2live	b0ab8c750d	Add isUserIgnored convenience method Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-09-14 19:47:16 -06:00
Travis Ralston	b17dd8351f	Make linter happy Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-09-14 19:24:18 -06:00
Travis Ralston	0ceb8d159a	Generate the ignored users event content correctly Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-09-14 16:06:55 -06:00
David Baker	402b943ddb	Upgrade to jsdoc 3.5.5 Because of https://github.com/jsdoc3/jsdoc/issues/1438	2017-09-14 22:33:10 +01:00
Travis Ralston	be55451c90	Add ignore users API Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-09-14 12:55:48 -06:00
Matthew Hodgson	c51c1a2ae6	Merge pull request #538 from matrix-org/dbkr/reemit_use_fewer_closures Make re-emitting events much more memory efficient	2017-09-13 12:51:26 +01:00
David Baker	845c796b96	Make re-emitting events much more memory efficient The previous impl bluntly created a closure for every event type and source emitter we set up a re-emit for. We can do much better than this fairly easily by having one bound handler for each event name and moving it into a class so we have one emitter per target, since 99% of the time the target is the client object.	2017-09-13 11:55:03 +01:00
David Baker	b0918ef293	Merge pull request #536 from matrix-org/dbkr/only_reemit_events_when_needed Only re-emit events from Event objects if needed	2017-09-12 19:26:58 +01:00
David Baker	102572b088	Empty commit to get Travis to re-test	2017-09-12 17:37:16 +01:00
David Baker	63076e77f5	Only re-emit events from Event objects if needed The only event and Event emits is 'Event.decrypted', so don't bother to add listeners if the event isn't encrypted.	2017-09-12 17:09:01 +01:00
Richard van der Hoff	8e48ee5f66	Merge pull request #535 from matrix-org/rav/devicelist_bits Handle 'left' users in the deviceList mananagement	2017-09-08 16:21:58 +01:00
Richard van der Hoff	1a55f550c0	Handle 'left' users in the deviceList mananagement When we no longer share any rooms with a given user, the server will stop sending us updates on their device list, and will (once synapse is updated) send us a notification of that fact via the 'left' field in the device_lists field in /sync, or the response from /keys/changes.	2017-09-08 15:40:26 +01:00
Richard van der Hoff	ae8fc64394	Do /keys/changes before second /sync This will avoid races between /keys/changes and /syncs.	2017-09-08 15:40:26 +01:00
David Baker	5e8e56caf9	Merge pull request #534 from matrix-org/rav/factor_out_device_tests Factor out devicelist integration tests to a separate file	2017-09-08 13:51:11 +01:00
David Baker	c075c161c2	Merge pull request #533 from matrix-org/rav/refactor_sync Refactor sync._sync as an async function	2017-09-08 13:51:05 +01:00
Richard van der Hoff	237a553d15	sync: s/self/this/	2017-09-08 12:07:19 +01:00
Richard van der Hoff	ca8674e0de	Factor out devicelist integration tests to a separate file There's a fuzzy line between the megolm tests and the devicelist ones, but since I want to add more tests for devicelists, we might as well put the ones which are definitely about devicelists in their own file	2017-09-08 11:54:16 +01:00
Richard van der Hoff	0511a1172f	Refactor sync._sync as an async function This stuff is much more clearly expressed as an async function than as a promise chain, and I'm about to add more stuff to the chain, so let's refactor now. There is plenty more that could be done here (self -> this, for instance), but I've gone for keeping the diff minimal.	2017-09-08 11:53:10 +01:00
David Baker	e07b304914	Merge pull request #532 from matrix-org/rav/eslint_es6 Add es6 to eslint environments	2017-09-07 10:33:52 +01:00
Richard van der Hoff	17364e72ec	Add es6 to eslint environments	2017-09-07 10:29:27 +01:00
David Baker	22b213ae26	v0.8.2	2017-08-24 14:45:24 +01:00
David Baker	7d5936a9e9	Prepare changelog for v0.8.2	2017-08-24 14:45:24 +01:00
David Baker	ab8f466f53	Merge pull request #530 from matrix-org/rav/fix_encrypted_calls Handle m.call.* events which are decrypted asynchronously	2017-08-24 14:09:12 +01:00
David Baker	201177e7f0	Merge pull request #529 from matrix-org/dbkr/event_object_reemit Re-emit events from, er, Event objects	2017-08-24 13:35:30 +01:00
Richard van der Hoff	ec5f9a2892	Handle m.call.* events which are decrypted asynchronously Handle the case where received m.call.* events are not decrypted at the point of the 'event' notification by adding an 'Event.decrypted' listener for them.	2017-08-24 13:35:02 +01:00
Richard van der Hoff	ee5b8748b5	Add MatrixEvent.isDecryptionFailure()	2017-08-24 13:35:02 +01:00
David Baker	8d04f8b8b5	Re-emit events from, er, Event objects We do create Events in more places, but this is probably the only place that matters since the only event is 'decrypted' which won't fire for, eg. events we send.	2017-08-24 11:29:48 +01:00
David Baker	033babfbfc	Groups: Sync Stream, Accept Invite & Leave (#528 ) * WIP support for reading groups from sync stream Only does invites currently * More support for parsing groups in the sync stream * Fix jsdoc	2017-08-24 10:24:24 +01:00
David Baker	15b77861ea	v0.8.1	2017-08-23 15:51:31 +01:00
David Baker	c4721850ce	Prepare changelog for v0.8.1	2017-08-23 15:51:31 +01:00
Michael Telatynski	01f93e0970	consume trailing slash when creating Matrix Client in HS and IS urls Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>	2017-08-19 11:00:46 +01:00