v0.12.0

Prepare changelog for v0.12.0
oops - fix changelog format in retrospect
2018-10-16 10:47:33 +01:00 · 2018-10-16 10:47:32 +01:00 · 2018-10-11 15:38:34 +01:00 · 2018-10-11 14:42:29 +01:00 · 2018-10-11 14:42:29 +01:00 · 2018-10-11 14:01:38 +01:00
25 changed files with 788 additions and 353 deletions
@@ -1,3 +1,134 @@
+Changes in [0.12.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.12.0) (2018-10-16)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.12.0-rc.1...v0.12.0)
+
+ * No changes since rc.1
+
+Changes in [0.12.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.12.0-rc.1) (2018-10-11)
+============================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.11.1...v0.12.0-rc.1)
+
+BREAKING CHANGES
+----------------
+ * If js-sdk finds data in the store that is incompatible with the options currently being used,
+   it will emit sync state ERROR with an error of type InvalidStoreError. It will also stop trying
+   to sync in this situation: the app must stop the client and then either clear the store or
+   change the options (in this case, enable or disable lazy loading of members) and then start
+   the client again.
+
+All Changes
+-----------
+
+ * never replace /sync'ed memberships with OOB ones
+   [\#760](https://github.com/matrix-org/matrix-js-sdk/pull/760)
+ * Don't fail to start up if lazy load check fails
+   [\#759](https://github.com/matrix-org/matrix-js-sdk/pull/759)
+ * Make e2e work on Edge
+   [\#754](https://github.com/matrix-org/matrix-js-sdk/pull/754)
+ * throw error with same name and message over idb worker boundary
+   [\#758](https://github.com/matrix-org/matrix-js-sdk/pull/758)
+ * Default to a room version of 1 when there is no room create event
+   [\#755](https://github.com/matrix-org/matrix-js-sdk/pull/755)
+ * Silence bluebird warnings
+   [\#757](https://github.com/matrix-org/matrix-js-sdk/pull/757)
+ * allow non-ff merge from release branch into master
+   [\#750](https://github.com/matrix-org/matrix-js-sdk/pull/750)
+ * Reject with the actual error on indexeddb error
+   [\#751](https://github.com/matrix-org/matrix-js-sdk/pull/751)
+ * Update mocha to v5
+   [\#744](https://github.com/matrix-org/matrix-js-sdk/pull/744)
+ * disable lazy loading for guests as they cant create filters
+   [\#748](https://github.com/matrix-org/matrix-js-sdk/pull/748)
+ * Revert "Add getMediaLimits to client"
+   [\#745](https://github.com/matrix-org/matrix-js-sdk/pull/745)
+
+Changes in [0.11.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.1) (2018-10-01)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.11.1-rc.1...v0.11.1)
+
+ * No changes since rc.1
+
+Changes in [0.11.1-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.1-rc.1) (2018-09-27)
+============================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.11.0...v0.11.1-rc.1)
+
+ * make usage of hub compatible with latest version (2.5)
+   [\#747](https://github.com/matrix-org/matrix-js-sdk/pull/747)
+ * Detect when lazy loading has been toggled in client.startClient
+   [\#746](https://github.com/matrix-org/matrix-js-sdk/pull/746)
+ * Add getMediaLimits to client
+   [\#644](https://github.com/matrix-org/matrix-js-sdk/pull/644)
+ * Split npm start into an init and watch script
+   [\#742](https://github.com/matrix-org/matrix-js-sdk/pull/742)
+ * Revert "room name should only take canonical alias into account"
+   [\#738](https://github.com/matrix-org/matrix-js-sdk/pull/738)
+ * fix display name disambiguation with LL
+   [\#737](https://github.com/matrix-org/matrix-js-sdk/pull/737)
+ * Introduce Room.myMembership event
+   [\#735](https://github.com/matrix-org/matrix-js-sdk/pull/735)
+ * room name should only take canonical alias into account
+   [\#733](https://github.com/matrix-org/matrix-js-sdk/pull/733)
+ * state events from context response were not wrapped in a MatrixEvent
+   [\#732](https://github.com/matrix-org/matrix-js-sdk/pull/732)
+ * Reduce amount of promises created when inserting members
+   [\#724](https://github.com/matrix-org/matrix-js-sdk/pull/724)
+ * dont wait for LL members to be stored to resolve the members
+   [\#726](https://github.com/matrix-org/matrix-js-sdk/pull/726)
+ * RoomState.members emitted with wrong argument order for OOB members
+   [\#728](https://github.com/matrix-org/matrix-js-sdk/pull/728)
+
+Changes in [0.11.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.0) (2018-09-10)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.11.0-rc.1...v0.11.0)
+
+BREAKING CHANGES
+----------------
+ * v0.11.0-rc.1 introduced some breaking changes - see the respective release notes.
+
+No changes since rc.1
+
+Changes in [0.11.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.0-rc.1) (2018-09-07)
+============================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.9...v0.11.0-rc.1)
+
+ * Support for lazy loading members. This should improve performance for
+   users who joined big rooms a lot. Pass to `lazyLoadMembers = true` option when calling `startClient`.
+
+BREAKING CHANGES
+----------------
+
+ * `MatrixClient::startClient` now returns a Promise. No method should be called on the client before that promise resolves. Before this method didn't return anything.
+ * A new `CATCHUP` sync state, emitted by `MatrixClient#"sync"` and returned by `MatrixClient::getSyncState()`, when doing initial sync after the `ERROR` state. See `MatrixClient` documentation for details.
+ * `RoomState::maySendEvent('m.room.message', userId)` & `RoomState::maySendMessage(userId)` do not check the membership of the user anymore, only the power level. To check if the syncing user is allowed to write in a room, use `Room::maySendMessage()` as `RoomState` is not always aware of the syncing user's membership anymore, in case lazy loading of members is enabled.
+
+All Changes
+-----------
+
+ * Only emit CATCHUP if recovering from conn error
+   [\#727](https://github.com/matrix-org/matrix-js-sdk/pull/727)
+ * Fix docstring for sync data.error
+   [\#725](https://github.com/matrix-org/matrix-js-sdk/pull/725)
+ * Re-apply "Don't rely on members to query if syncing user can post to room"
+   [\#723](https://github.com/matrix-org/matrix-js-sdk/pull/723)
+ * Revert "Don't rely on members to query if syncing user can post to room"
+   [\#721](https://github.com/matrix-org/matrix-js-sdk/pull/721)
+ * Don't rely on members to query if syncing user can post to room
+   [\#717](https://github.com/matrix-org/matrix-js-sdk/pull/717)
+ * Fixes for room.guessDMUserId
+   [\#719](https://github.com/matrix-org/matrix-js-sdk/pull/719)
+ * Fix filepanel also filtering main timeline with LL turned on.
+   [\#716](https://github.com/matrix-org/matrix-js-sdk/pull/716)
+ * Remove lazy loaded members when leaving room
+   [\#711](https://github.com/matrix-org/matrix-js-sdk/pull/711)
+ * Fix: show spinner again while recovering from connection error
+   [\#702](https://github.com/matrix-org/matrix-js-sdk/pull/702)
+ * Add method to query LL state in client
+   [\#714](https://github.com/matrix-org/matrix-js-sdk/pull/714)
+ * Fix: also load invited members when lazy loading members
+   [\#707](https://github.com/matrix-org/matrix-js-sdk/pull/707)
+ * Pass through function to discard megolm session
+   [\#704](https://github.com/matrix-org/matrix-js-sdk/pull/704)
+
 Changes in [0.10.9](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.9) (2018-09-03)
 ==================================================================================================
 [Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.9-rc.2...v0.10.9)
@@ -36,18 +167,6 @@ Changes in [0.10.9-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/ta
 * Lazy loading of room members
   [\#691](https://github.com/matrix-org/matrix-js-sdk/pull/691)

-BREAKING CHANGE
---------------
-
- * `MatrixClient::startClient` now returns a Promise. No method should be called on the client before that promise resolves. Before this method didn't return anything.
-
-Changes in [0.11.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.0) (TDB)
-==================================================================================================
-[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.6...v0.11.0)
-
- * Support for lazy loading members. This should improve performance for
-   users who joined big rooms a lot. Pass to `lazyLoadMembers = true` option when calling `startClient`.
-
 Changes in [0.10.8](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.8) (2018-08-20)
 ==================================================================================================
 [Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.8-rc.1...v0.10.8)
@@ -1,6 +1,6 @@
 {
  "name": "matrix-js-sdk",
-  "version": "0.10.9",
+  "version": "0.12.0",
  "description": "Matrix Client-Server SDK for Javascript",
  "main": "index.js",
  "scripts": {
@@ -10,7 +10,9 @@
    "test": "npm run test:build && npm run test:run",
    "check": "npm run test:build && _mocha --recursive specbuild --colors",
    "gendoc": "babel --no-babelrc -d .jsdocbuild src && jsdoc -r .jsdocbuild -P package.json -R README.md -d .jsdoc",
-    "start": "babel -s -w -d lib src",
+    "start": "npm run start:init && npm run start:watch",
+    "start:watch": "babel -s -w --skip-initial-build -d lib src",
+    "start:init": "babel -s -d lib src",
    "clean": "rimraf lib dist",
    "build": "babel -s -d lib src && rimraf dist && mkdir dist && browserify -d browser-index.js | exorcist dist/browser-matrix.js.map > dist/browser-matrix.js && uglifyjs -c -m -o dist/browser-matrix.min.js --source-map dist/browser-matrix.min.js.map --in-source-map dist/browser-matrix.js.map dist/browser-matrix.js",
    "dist": "npm run build",
@@ -72,8 +74,8 @@
    "jsdoc": "^3.5.5",
    "lolex": "^1.5.2",
    "matrix-mock-request": "^1.2.0",
-    "mocha": "^3.2.0",
-    "mocha-jenkins-reporter": "^0.3.6",
+    "mocha": "^5.2.0",
+    "mocha-jenkins-reporter": "^0.4.0",
    "rimraf": "^2.5.4",
    "source-map-support": "^0.4.11",
    "sourceify": "^0.1.0",
@@ -245,7 +245,7 @@ release_text=`mktemp`
 echo "$tag" > "${release_text}"
 echo >> "${release_text}"
 cat "${latest_changes}" >> "${release_text}"
-hub release create $hubflags $assets -f "${release_text}" "$tag"
+hub release create $hubflags $assets -F "${release_text}" "$tag"

 if [ $dodist -eq 0 ]; then
    rm -rf "$builddir"
@@ -281,7 +281,7 @@ fi
 echo "updating master branch"
 git checkout master
 git pull
-git merge --ff-only "$rel_branch"
+git merge "$rel_branch"

 # push master  and docs (if generated) to github
 git push origin master
@@ -94,7 +94,7 @@ describe("MatrixClient opts", function() {
            httpBackend.flush("/txn1", 1);
        });

-        it("should be able to sync / get new events", function(done) {
+        it("should be able to sync / get new events", async function() {
            const expectedEventTypes = [ // from /initialSync
                "m.room.message", "m.room.name", "m.room.member", "m.room.member",
                "m.room.create",
@@ -110,20 +110,16 @@ describe("MatrixClient opts", function() {
            httpBackend.when("GET", "/pushrules").respond(200, {});
            httpBackend.when("POST", "/filter").respond(200, { filter_id: "foo" });
            httpBackend.when("GET", "/sync").respond(200, syncData);
-            client.startClient();
-            httpBackend.flush("/pushrules", 1).then(function() {
-                return httpBackend.flush("/filter", 1);
-            }).then(function() {
-                return Promise.all([
-                    httpBackend.flush("/sync", 1),
-                    utils.syncPromise(client),
-                ]);
-            }).done(function() {
-                expect(expectedEventTypes.length).toEqual(
-                    0, "Expected to see event types: " + expectedEventTypes,
-                );
-                done();
-            });
+            await client.startClient();
+            await httpBackend.flush("/pushrules", 1);
+            await httpBackend.flush("/filter", 1);
+            await Promise.all([
+                httpBackend.flush("/sync", 1),
+                utils.syncPromise(client),
+            ]);
+            expect(expectedEventTypes.length).toEqual(
+                0, "Expected to see event types: " + expectedEventTypes,
+            );
        });
    });

@@ -139,6 +139,9 @@ describe("MatrixClient", function() {
        store.getSavedSync = expect.createSpy().andReturn(Promise.resolve(null));
        store.getSavedSyncToken = expect.createSpy().andReturn(Promise.resolve(null));
        store.setSyncData = expect.createSpy().andReturn(Promise.resolve(null));
+        store.getClientOptions = expect.createSpy().andReturn(Promise.resolve(null));
+        store.storeClientOptions = expect.createSpy().andReturn(Promise.resolve(null));
+        store.isNewlyCreated = expect.createSpy().andReturn(Promise.resolve(true));
        client = new MatrixClient({
            baseUrl: "https://my.home.server",
            idBaseUrl: identityServerUrl,
@@ -182,7 +185,7 @@ describe("MatrixClient", function() {
        });
    });

-    it("should not POST /filter if a matching filter already exists", function(done) {
+    it("should not POST /filter if a matching filter already exists", async function() {
        httpLookups = [];
        httpLookups.push(PUSH_RULES_RESPONSE);
        httpLookups.push(SYNC_RESPONSE);
@@ -191,15 +194,19 @@ describe("MatrixClient", function() {
        const filter = new sdk.Filter(0, filterId);
        filter.setDefinition({"room": {"timeline": {"limit": 8}}});
        store.getFilter.andReturn(filter);
-        client.startClient();
-
-        client.on("sync", function syncListener(state) {
-            if (state === "SYNCING") {
-                expect(httpLookups.length).toEqual(0);
-                client.removeListener("sync", syncListener);
-                done();
-            }
+        const syncPromise = new Promise((resolve, reject) => {
+            client.on("sync", function syncListener(state) {
+                if (state === "SYNCING") {
+                    expect(httpLookups.length).toEqual(0);
+                    client.removeListener("sync", syncListener);
+                    resolve();
+                } else if (state === "ERROR") {
+                    reject(new Error("sync error"));
+                }
+            });
        });
+        await client.startClient();
+        await syncPromise;
    });

    describe("getSyncState", function() {
@@ -207,15 +214,18 @@ describe("MatrixClient", function() {
            expect(client.getSyncState()).toBe(null);
        });

-        it("should return the same sync state as emitted sync events", function(done) {
-            client.on("sync", function syncListener(state) {
-                expect(state).toEqual(client.getSyncState());
-                if (state === "SYNCING") {
-                    client.removeListener("sync", syncListener);
-                    done();
-                }
+        it("should return the same sync state as emitted sync events", async function() {
+            const syncingPromise = new Promise((resolve) => {
+                client.on("sync", function syncListener(state) {
+                    expect(state).toEqual(client.getSyncState());
+                    if (state === "SYNCING") {
+                        client.removeListener("sync", syncListener);
+                        resolve();
+                    }
+                });
            });
-            client.startClient();
+            await client.startClient();
+            await syncingPromise;
        });
    });

@@ -258,8 +268,8 @@ describe("MatrixClient", function() {
    });

    describe("retryImmediately", function() {
-        it("should return false if there is no request waiting", function() {
-            client.startClient();
+        it("should return false if there is no request waiting", async function() {
+            await client.startClient();
            expect(client.retryImmediately()).toBe(false);
        });

@@ -380,7 +390,7 @@ describe("MatrixClient", function() {
            client.startClient();
        });

-        it("should transition ERROR -> PREPARED after /sync if prev failed",
+        it("should transition ERROR -> CATCHUP after /sync if prev failed",
        function(done) {
            const expectedStates = [];
            acceptKeepalives = false;
@@ -403,7 +413,7 @@ describe("MatrixClient", function() {

            expectedStates.push(["RECONNECTING", null]);
            expectedStates.push(["ERROR", "RECONNECTING"]);
-            expectedStates.push(["PREPARED", "ERROR"]);
+            expectedStates.push(["CATCHUP", "ERROR"]);
            client.on("sync", syncChecker(expectedStates, done));
            client.startClient();
        });
@@ -201,22 +201,6 @@ describe("RoomMember", function() {
        });
    });

-    describe("supersedesOutOfBand", function() {
-        it("should be set by markSupersedesOutOfBand", function() {
-            const member = new RoomMember();
-            expect(member.supersedesOutOfBand()).toEqual(false);
-            member.markSupersedesOutOfBand();
-            expect(member.supersedesOutOfBand()).toEqual(true);
-        });
-        it("should be cleared by clearSupersedesOutOfBand", function() {
-            const member = new RoomMember();
-            member.markSupersedesOutOfBand();
-            expect(member.supersedesOutOfBand()).toEqual(true);
-            member.clearSupersedesOutOfBand();
-            expect(member.supersedesOutOfBand()).toEqual(false);
-        });
-    });
-
    describe("setMembershipEvent", function() {
        const joinEvent = utils.mkMembership({
            event: true,
@@ -299,41 +299,25 @@ describe("RoomState", function() {
            expect(eventReceived).toEqual(true);
        });

-        it("should overwrite existing members", function() {
+        it("should never overwrite existing members", function() {
            const oobMemberEvent = utils.mkMembership({
                user: userA, mship: "join", room: roomId, event: true,
            });
            state.markOutOfBandMembersStarted();
            state.setOutOfBandMembers([oobMemberEvent]);
            const memberA = state.getMember(userA);
-            expect(memberA.events.member.getId()).toEqual(oobMemberEvent.getId());
-            expect(memberA.isOutOfBand()).toEqual(true);
-        });
-
-        it("should allow later state events to overwrite", function() {
-            const oobMemberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            });
-            const memberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            });
-
-            state.markOutOfBandMembersStarted();
-            state.setOutOfBandMembers([oobMemberEvent]);
-            state.setStateEvents([memberEvent]);
-
-            const memberA = state.getMember(userA);
-            expect(memberA.events.member.getId()).toEqual(memberEvent.getId());
+            expect(memberA.events.member.getId()).toNotEqual(oobMemberEvent.getId());
            expect(memberA.isOutOfBand()).toEqual(false);
        });

        it("should emit members when updating a member", function() {
+            const doesntExistYetUserId = "@doesntexistyet:hs";
            const oobMemberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
+                user: doesntExistYetUserId, mship: "join", room: roomId, event: true,
            });
            let eventReceived = false;
            state.once('RoomState.members', (_, __, member) => {
-                expect(member.userId).toEqual(userA);
+                expect(member.userId).toEqual(doesntExistYetUserId);
                eventReceived = true;
            });

@@ -341,28 +325,6 @@ describe("RoomState", function() {
            state.setOutOfBandMembers([oobMemberEvent]);
            expect(eventReceived).toEqual(true);
        });
-
-
-        it("should not overwrite members updated since starting loading oob",
-        function() {
-            const oobMemberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            });
-
-            const existingMemberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            });
-
-            state.markOutOfBandMembersStarted();
-            state.setStateEvents([existingMemberEvent]);
-            expect(state.getMember(userA).supersedesOutOfBand()).toEqual(true);
-            state.setOutOfBandMembers([oobMemberEvent]);
-
-            const memberA = state.getMember(userA);
-            expect(memberA.events.member.getId()).toEqual(existingMemberEvent.getId());
-            expect(memberA.isOutOfBand()).toEqual(false);
-            expect(memberA.supersedesOutOfBand()).toEqual(false);
-        });
    });

    describe("clone", function() {
@@ -386,20 +348,6 @@ describe("RoomState", function() {
            expect(state.getJoinedMemberCount()).toEqual(copy.getJoinedMemberCount());
        });

-        it("should copy supersedes flag when OOB loading is progress",
-        function() {
-            // include OOB members in copy
-            state.markOutOfBandMembersStarted();
-            state.setStateEvents([utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            })]);
-            const copy = state.clone();
-            const memberA = state.getMember(userA);
-            const memberACopy = copy.getMember(userA);
-            expect(memberA.supersedesOutOfBand()).toEqual(true);
-            expect(memberACopy.supersedesOutOfBand()).toEqual(true);
-        });
-
        it("should mark old copy as not waiting for out of band anymore", function() {
            state.markOutOfBandMembersStarted();
            const copy = state.clone();
@@ -447,13 +395,6 @@ describe("RoomState", function() {
    });

    describe("maySendStateEvent", function() {
-        it("should say non-joined members may not send state",
-        function() {
-            expect(state.maySendStateEvent(
-                'm.room.name', "@nobody:nowhere",
-            )).toEqual(false);
-        });
-
        it("should say any member may send state with no power level event",
        function() {
            expect(state.maySendStateEvent('m.room.name', userA)).toEqual(true);
@@ -640,14 +581,6 @@ describe("RoomState", function() {
    });

    describe("maySendEvent", function() {
-        it("should say non-joined members may not send events",
-        function() {
-            expect(state.maySendEvent(
-                'm.room.message', "@nobody:nowhere",
-            )).toEqual(false);
-            expect(state.maySendMessage("@nobody:nowhere")).toEqual(false);
-        });
-
        it("should say any member may send events with no power level event",
        function() {
            expect(state.maySendEvent('m.room.message', userA)).toEqual(true);
@@ -1400,15 +1400,63 @@ describe("Room", function() {

    describe("getMyMembership", function() {
        it("should return synced membership if membership isn't available yet",
-        async function() {
+        function() {
            const room = new Room(roomId, null, userA);
-            room.setSyncedMembership("invite");
+            room.updateMyMembership("invite");
            expect(room.getMyMembership()).toEqual("invite");
+        });
+        it("should emit a Room.myMembership event on a change",
+        function() {
+            const room = new Room(roomId, null, userA);
+            const events = [];
+            room.on("Room.myMembership", (_room, membership, oldMembership) => {
+                events.push({membership, oldMembership});
+            });
+            room.updateMyMembership("invite");
+            expect(room.getMyMembership()).toEqual("invite");
+            expect(events[0]).toEqual({membership: "invite", oldMembership: null});
+            events.splice(0);   //clear
+            room.updateMyMembership("invite");
+            expect(events.length).toEqual(0);
+            room.updateMyMembership("join");
+            expect(room.getMyMembership()).toEqual("join");
+            expect(events[0]).toEqual({membership: "join", oldMembership: "invite"});
+        });
+    });
+
+    describe("guessDMUserId", function() {
+        it("should return first hero id",
+        function() {
+            const room = new Room(roomId, null, userA);
+            room.setSummary({'m.heroes': [userB]});
+            expect(room.guessDMUserId()).toEqual(userB);
+        });
+        it("should return first member that isn't self",
+        function() {
+            const room = new Room(roomId, null, userA);
            room.addLiveEvents([utils.mkMembership({
-                user: userA, mship: "join",
+                user: userB, mship: "join",
                room: roomId, event: true,
            })]);
-            expect(room.getMyMembership()).toEqual("join");
+            expect(room.guessDMUserId()).toEqual(userB);
+        });
+        it("should return self if only member present",
+        function() {
+            const room = new Room(roomId, null, userA);
+            expect(room.guessDMUserId()).toEqual(userA);
+        });
+    });
+
+    describe("maySendMessage", function() {
+        it("should return false if synced membership not join",
+        function() {
+            const room = new Room(roomId, null, userA);
+            room.updateMyMembership("invite");
+            expect(room.maySendMessage()).toEqual(false);
+            room.updateMyMembership("leave");
+            expect(room.maySendMessage()).toEqual(false);
+            room.updateMyMembership("join");
+            expect(room.maySendMessage()).toEqual(true);
        });
    });
 });
@@ -45,16 +45,9 @@ const ContentHelpers = require("./content-helpers");
 import ReEmitter from './ReEmitter';
 import RoomList from './crypto/RoomList';

-
-const LAZY_LOADING_MESSAGES_FILTER = {
-    lazy_load_members: true,
-};
-
-const LAZY_LOADING_SYNC_FILTER = {
-    room: {
-        state: LAZY_LOADING_MESSAGES_FILTER,
-    },
-};
+// Disable warnings for now: we use deprecated bluebird functions
+// and need to migrate, but they spam the console with warnings.
+Promise.config({warnings: false});


 const SCROLLBACK_DELAY_MS = 3000;
@@ -702,6 +695,21 @@ MatrixClient.prototype.isRoomEncrypted = function(roomId) {
    return this._roomList.isRoomEncrypted(roomId);
 };

+/**
+ * Forces the current outbound group session to be discarded such
+ * that another one will be created next time an event is sent.
+ *
+ * @param {string} roomId The ID of the room to discard the session for
+ *
+ * This should not normally be necessary.
+ */
+MatrixClient.prototype.forceDiscardSession = function(roomId) {
+    if (!this._crypto) {
+        throw new Error("End-to-End encryption disabled");
+    }
+    this._crypto.forceDiscardSession(roomId);
+};
+
 /**
 * Get a list containing all of the room keys
 *
@@ -2052,7 +2060,7 @@ MatrixClient.prototype.getEventTimeline = function(timelineSet, eventId) {

    let params = undefined;
    if (this._clientOpts.lazyLoadMembers) {
-        params = {filter: JSON.stringify(LAZY_LOADING_MESSAGES_FILTER)};
+        params = {filter: JSON.stringify(Filter.LAZY_LOADING_MESSAGES_FILTER)};
    }

    // TODO: we should implement a backoff (as per scrollback()) to deal more
@@ -2087,7 +2095,8 @@ MatrixClient.prototype.getEventTimeline = function(timelineSet, eventId) {
                                               self.getEventMapper()));
            timeline.getState(EventTimeline.FORWARDS).paginationToken = res.end;
        } else {
-            timeline.getState(EventTimeline.BACKWARDS).setUnknownStateEvents(res.state);
+            const stateEvents = utils.map(res.state, self.getEventMapper());
+            timeline.getState(EventTimeline.BACKWARDS).setUnknownStateEvents(stateEvents);
        }
        timelineSet.addEventsToTimeline(matrixEvents, true, timeline, res.start);

@@ -2130,7 +2139,9 @@ function(roomId, fromToken, limit, dir, timelineFilter = undefined) {

    let filter = null;
    if (this._clientOpts.lazyLoadMembers) {
-        filter = LAZY_LOADING_MESSAGES_FILTER;
+        // create a shallow copy of LAZY_LOADING_MESSAGES_FILTER,
+        // so the timelineFilter doesn't get written into it below
+        filter = Object.assign({}, Filter.LAZY_LOADING_MESSAGES_FILTER);
    }
    if (timelineFilter) {
        // XXX: it's horrific that /messages' filter parameter doesn't match
@@ -3087,21 +3098,6 @@ MatrixClient.prototype.startClient = async function(opts) {
    // shallow-copy the opts dict before modifying and storing it
    opts = Object.assign({}, opts);

-    if (opts.lazyLoadMembers) {
-        const supported = await this.doesServerSupportLazyLoading();
-        if (supported) {
-            opts.filter = await this.createFilter(LAZY_LOADING_SYNC_FILTER);
-        } else {
-            console.log("LL: lazy loading requested but not supported " +
-                "by server, so disabling");
-            opts.lazyLoadMembers = false;
-        }
-    }
-
-    if (opts.lazyLoadMembers && this._crypto) {
-        this._crypto.enableLazyLoading();
-    }
-
    opts.crypto = this._crypto;
    opts.canResetEntireTimeline = (roomId) => {
        if (!this._canResetTimelineCallback) {
@@ -3110,11 +3106,29 @@ MatrixClient.prototype.startClient = async function(opts) {
        return this._canResetTimelineCallback(roomId);
    };
    this._clientOpts = opts;
-
    this._syncApi = new SyncApi(this, opts);
    this._syncApi.sync();
 };

+/**
+ * store client options with boolean/string/numeric values
+ * to know in the next session what flags the sync data was
+ * created with (e.g. lazy loading)
+ * @param {object} opts the complete set of client options
+ * @return {Promise} for store operation */
+MatrixClient.prototype._storeClientOptions = function() {
+    const primTypes = ["boolean", "string", "number"];
+    const serializableOpts = Object.entries(this._clientOpts)
+        .filter(([key, value]) => {
+            return primTypes.includes(typeof value);
+        })
+        .reduce((obj, [key, value]) => {
+            obj[key] = value;
+            return obj;
+        }, {});
+    return this.store.storeClientOptions(serializableOpts);
+};
+
 /**
 * High level helper method to stop the client from polling and allow a
 * clean shutdown.
@@ -3159,6 +3173,14 @@ MatrixClient.prototype.doesServerSupportLazyLoading = async function() {
    return this._serverSupportsLazyLoading;
 };

+/*
+ * Get if lazy loading members is being used.
+ * @return {boolean} Whether or not members are lazy loaded by this client
+ */
+MatrixClient.prototype.hasLazyLoadMembersEnabled = function() {
+    return !!this._clientOpts.lazyLoadMembers;
+};
+
 /*
 * Set a function which is called when /sync returns a 'limited' response.
 * It is called with a room ID and returns a boolean. It should return 'true' if the SDK
@@ -3504,6 +3526,12 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED;
 * a state of SYNCING. <i>This is the equivalent of "syncComplete" in the
 * previous API.</i></li>
 *
+ * <li>CATCHUP: The client has detected the connection to the server might be
+ * available again and will now try to do a sync again. As this sync might take
+ * a long time (depending how long ago was last synced, and general server
+ * performance) the client is put in this mode so the UI can reflect trying
+ * to catch up with the server after losing connection.</li>
+ *
 * <li>SYNCING : The client is currently polling for new events from the server.
 * This will be called <i>after</i> processing latest events from a sync.</li>
 *
@@ -3527,11 +3555,11 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED;
 *                                          +---->STOPPED
 *                                          |
 *              +----->PREPARED -------> SYNCING <--+
- *              |        ^                |  ^      |
- *              |        |                |  |      |
- *              |        |                V  |      |
- *   null ------+        |  +--------RECONNECTING   |
- *              |        |  V                       |
+ *              |                        ^  |  ^    |
+ *              |      CATCHUP ----------+  |  |    |
+ *              |        ^                  V  |    |
+ *   null ------+        |  +------- RECONNECTING   |
+ *              |        V  V                       |
 *              +------->ERROR ---------------------+
 *
 * NB: 'null' will never be emitted by this event.
@@ -3581,7 +3609,7 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED;
 *
 * @param {?Object} data Data about this transition.
 *
- * @param {MatrixError} data.err The matrix error if <code>state=ERROR</code>.
+ * @param {MatrixError} data.error The matrix error if <code>state=ERROR</code>.
 *
 * @param {String} data.oldSyncToken The 'since' token passed to /sync.
 *    <code>null</code> for the first successful sync since this client was
@@ -1,5 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
+Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -498,6 +499,16 @@ MegolmEncryption.prototype.encryptMessage = function(room, eventType, content) {
    });
 };

+/**
+ * Forces the current outbound group session to be discarded such
+ * that another one will be created next time an event is sent.
+ *
+ * This should not normally be necessary.
+ */
+MegolmEncryption.prototype.forceDiscardSession = function() {
+    this._setupPromise = this._setupPromise.then(() => null);
+};
+
 /**
 * Checks the devices we're about to send to and see if any are entirely
 * unknown to the user.  If so, warn the user, and mark them as known to
@@ -621,6 +621,23 @@ Crypto.prototype.getEventSenderDeviceInfo = function(event) {
    return device;
 };

+/**
+ * Forces the current outbound group session to be discarded such
+ * that another one will be created next time an event is sent.
+ *
+ * @param {string} roomId The ID of the room to discard the session for
+ *
+ * This should not normally be necessary.
+ */
+Crypto.prototype.forceDiscardSession = function(roomId) {
+    const alg = this._roomEncryptors[roomId];
+    if (alg === undefined) throw new Error("Room not encrypted");
+    if (alg.forceDiscardSession === undefined) {
+        throw new Error("Room encryption algorithm doesn't support session discarding");
+    }
+    alg.forceDiscardSession();
+};
+
 /**
 * Configure a room to use encryption (ie, save a flag in the sessionstore).
 *
@@ -92,6 +92,19 @@ export default class IndexedDBCryptoStore {
                console.log(`connected to indexeddb ${this._dbName}`);
                resolve(new IndexedDBCryptoStoreBackend.Backend(db));
            };
+        }).then((backend) => {
+            // Edge has IndexedDB but doesn't support compund keys which we use fairly extensively.
+            // Try a dummy query which will fail if the browser doesn't support compund keys, so
+            // we can fall back to a different backend.
+            return backend.doTxn(
+                'readonly',
+                [IndexedDBCryptoStore.STORE_INBOUND_GROUP_SESSIONS],
+                (txn) => {
+                    backend.getEndToEndInboundGroupSession('', '', txn, () => {});
+                }).then(() => {
+                    return backend;
+                },
+            );
        }).catch((e) => {
            console.warn(
                `unable to connect to indexeddb ${this._dbName}` +
@@ -0,0 +1,25 @@
+// can't just do InvalidStoreError extends Error
+// because of http://babeljs.io/docs/usage/caveats/#classes
+function InvalidStoreError(reason, value) {
+    const message = `Store is invalid because ${reason}, ` +
+        `please stop the client, delete all data and start the client again`;
+    const instance = Reflect.construct(Error, [message]);
+    Reflect.setPrototypeOf(instance, Reflect.getPrototypeOf(this));
+    instance.reason = reason;
+    instance.value = value;
+    return instance;
+}
+
+InvalidStoreError.TOGGLED_LAZY_LOADING = "TOGGLED_LAZY_LOADING";
+
+InvalidStoreError.prototype = Object.create(Error.prototype, {
+  constructor: {
+    value: Error,
+    enumerable: false,
+    writable: true,
+    configurable: true,
+  },
+});
+Reflect.setPrototypeOf(InvalidStoreError, Error);
+
+module.exports.InvalidStoreError = InvalidStoreError;
@@ -51,6 +51,17 @@ function Filter(userId, filterId) {
    this.definition = {};
 }

+Filter.LAZY_LOADING_MESSAGES_FILTER = {
+    lazy_load_members: true,
+};
+
+Filter.LAZY_LOADING_SYNC_FILTER = {
+    room: {
+        state: Filter.LAZY_LOADING_MESSAGES_FILTER,
+    },
+};
+
+
 /**
 * Get the ID of this filter on your homeserver (if known)
 * @return {?Number} The filter ID
@@ -34,6 +34,8 @@ module.exports.SyncAccumulator = require("./sync-accumulator");
 module.exports.MatrixHttpApi = require("./http-api").MatrixHttpApi;
 /** The {@link module:http-api.MatrixError|MatrixError} class. */
 module.exports.MatrixError = require("./http-api").MatrixError;
+/** The {@link module:errors.InvalidStoreError|InvalidStoreError} class. */
+module.exports.InvalidStoreError = require("./errors").InvalidStoreError;
 /** The {@link module:client.MatrixClient|MatrixClient} class. */
 module.exports.MatrixClient = require("./client").MatrixClient;
 /** The {@link module:models/room|Room} class. */
@@ -59,7 +59,6 @@ function RoomMember(roomId, userId) {
        member: null,
    };
    this._isOutOfBand = false;
-    this._supersedesOutOfBand = false;
    this._updateModifiedTime();
 }
 utils.inherits(RoomMember, EventEmitter);
@@ -80,31 +79,6 @@ RoomMember.prototype.isOutOfBand = function() {
    return this._isOutOfBand;
 };

-/**
- * Does the member supersede an incoming out-of-band
- * member? If so the out-of-band member should be ignored.
- * @return {bool}
- */
-RoomMember.prototype.supersedesOutOfBand = function() {
-    return this._supersedesOutOfBand;
-};
-
-/**
- * Mark the member as superseding the future incoming
- * out-of-band members.
- */
-RoomMember.prototype.markSupersedesOutOfBand = function() {
-    this._supersedesOutOfBand = true;
-};
-
-/**
- * Clear the member superseding the future incoming
- * out-of-band members, as loading finished or failed.
- */
-RoomMember.prototype.clearSupersedesOutOfBand = function() {
-    this._supersedesOutOfBand = false;
-};
-
 /**
 * Update this room member's membership event. May fire "RoomMember.name" if
 * this event updates this member's name.
@@ -180,7 +180,7 @@ RoomState.prototype.getSentinelMember = function(userId) {
        sentinel = new RoomMember(this.roomId, userId);
        const member = this.members[userId];
        if (member) {
-            sentinel.setMembershipEvent(member.events.member);
+            sentinel.setMembershipEvent(member.events.member, this);
        }
        this._sentinels[userId] = sentinel;
    }
@@ -220,8 +220,7 @@ RoomState.prototype.clone = function() {
    // if loading is in progress (through _oobMemberFlags)
    // since these are not new members, we're merely copying them
    // set the status to not started
-    // after copying, we set back the status and
-    // copy the superseding flag from the current state
+    // after copying, we set back the status
    const status = this._oobMemberFlags.status;
    this._oobMemberFlags.status = OOB_STATUS_NOTSTARTED;

@@ -249,14 +248,6 @@ RoomState.prototype.clone = function() {
                copyMember.markOutOfBand();
            }
        });
-    } else if (this._oobMemberFlags.status == OOB_STATUS_INPROGRESS) {
-        // copy markSupersedesOutOfBand flags
-        this.getMembers().forEach((member) => {
-            if (member.supersedesOutOfBand()) {
-                const copyMember = copy.getMember(member.userId);
-                copyMember.markSupersedesOutOfBand();
-            }
-        });
    }

    return copy;
@@ -341,11 +332,6 @@ RoomState.prototype.setStateEvents = function(stateEvents) {

            const member = self._getOrCreateMember(userId, event);
            member.setMembershipEvent(event, self);
-            // if out of band members are loading,
-            // mark the member as more recent
-            if (self._oobMemberFlags.status == OOB_STATUS_INPROGRESS) {
-                member.markSupersedesOutOfBand();
-            }

            self._updateMember(member);
            self.emit("RoomState.members", event, self, member);
@@ -434,12 +420,22 @@ RoomState.prototype.markOutOfBandMembersFailed = function() {
    if (this._oobMemberFlags.status !== OOB_STATUS_INPROGRESS) {
        return;
    }
-    // the request failed, there is nothing to supersede
-    // in case of a retry, these event would not supersede the
-    // retry anymore.
-    this.getMembers().forEach((m) => {
-        m.clearSupersedesOutOfBand();
+    this._oobMemberFlags.status = OOB_STATUS_NOTSTARTED;
+};
+
+/**
+ * Clears the loaded out-of-band members
+ */
+RoomState.prototype.clearOutOfBandMembers = function() {
+    let count = 0;
+    Object.keys(this.members).forEach((userId) => {
+        const member = this.members[userId];
+        if (member.isOutOfBand()) {
+            ++count;
+            delete this.members[userId];
+        }
    });
+    console.log(`LL: RoomState removed ${count} members...`);
    this._oobMemberFlags.status = OOB_STATUS_NOTSTARTED;
 };

@@ -467,27 +463,15 @@ RoomState.prototype._setOutOfBandMember = function(stateEvent) {
    }
    const userId = stateEvent.getStateKey();
    const existingMember = this.getMember(userId);
-    if (existingMember) {
-        const existingMemberEvent = existingMember.events.member;
-        // ignore out of band members with events we are
-        // already aware of.
-        if (existingMemberEvent.getId() === stateEvent.getId()) {
-            return;
-        }
-        // this member was updated since we started
-        // loading the out of band members.
-        // Ignore the out of band member and clear
-        // the "supersedes" flag as the out of members are now loaded
-        if (existingMember.supersedesOutOfBand()) {
-            existingMember.clearSupersedesOutOfBand();
-            return;
-        }
+    // never replace members received as part of the sync
+    if (existingMember && !existingMember.isOutOfBand()) {
+        return;
    }

    const member = this._getOrCreateMember(userId, stateEvent);
-    member.setMembershipEvent(stateEvent);
+    member.setMembershipEvent(stateEvent, this);
    // needed to know which members need to be stored seperately
-    // as the are not part of the sync accumulator
+    // as they are not part of the sync accumulator
    // this is cleared by setMembershipEvent so when it's updated through /sync
    member.markOutOfBand();

@@ -495,7 +479,7 @@ RoomState.prototype._setOutOfBandMember = function(stateEvent) {

    this._setStateEvent(stateEvent);
    this._updateMember(member);
-    this.emit("RoomState.members", {}, stateEvent, member);
+    this.emit("RoomState.members", stateEvent, this, member);
 };

 /**
@@ -651,11 +635,6 @@ RoomState.prototype.maySendStateEvent = function(stateEventType, userId) {
 *                        according to the room's state.
 */
 RoomState.prototype._maySendEventOfType = function(eventType, userId, state) {
-    const member = this.getMember(userId);
-    if (!member || member.membership == 'leave') {
-        return false;
-    }
-
    const power_levels_event = this.getStateEvents('m.room.power_levels', '');

    let power_levels;
@@ -663,25 +642,34 @@ RoomState.prototype._maySendEventOfType = function(eventType, userId, state) {

    let state_default = 0;
    let events_default = 0;
+    let powerLevel = 0;
    if (power_levels_event) {
        power_levels = power_levels_event.getContent();
        events_levels = power_levels.events || {};

-        if (utils.isNumber(power_levels.state_default)) {
+        if (Number.isFinite(power_levels.state_default)) {
            state_default = power_levels.state_default;
        } else {
            state_default = 50;
        }
-        if (utils.isNumber(power_levels.events_default)) {
+
+        const userPowerLevel = power_levels.users && power_levels.users[userId];
+        if (Number.isFinite(userPowerLevel)) {
+            powerLevel = userPowerLevel;
+        } else if(Number.isFinite(power_levels.users_default)) {
+            powerLevel = power_levels.users_default;
+        }
+
+        if (Number.isFinite(power_levels.events_default)) {
            events_default = power_levels.events_default;
        }
    }

    let required_level = state ? state_default : events_default;
-    if (utils.isNumber(events_levels[eventType])) {
+    if (Number.isFinite(events_levels[eventType])) {
        required_level = events_levels[eventType];
    }
-    return member.powerLevel >= required_level;
+    return powerLevel >= required_level;
 };

 /**
@@ -178,7 +178,7 @@ function Room(roomId, client, myUserId, opts) {

    // read by megolm; boolean value - null indicates "use global value"
    this._blacklistUnverifiedDevices = null;
-    this._syncedMembership = null;
+    this._selfMembership = null;
    this._summaryHeroes = null;
    // awaited by getEncryptionTargetMembers while room members are loading

@@ -198,7 +198,10 @@ utils.inherits(Room, EventEmitter);
 */
 Room.prototype.getVersion = function() {
    const createEvent = this.currentState.getStateEvents("m.room.create", "");
-    if (!createEvent) return null;
+    if (!createEvent) {
+        console.warn("Room " + this.room_id + " does not have an m.room.create event");
+        return '1';
+    }
    const ver = createEvent.getContent()['room_version'];
    if (ver === undefined) return '1';
    return ver;
@@ -257,13 +260,7 @@ Room.prototype.getLiveTimeline = function() {
 * @return {string} the membership type (join | leave | invite) for the logged in user
 */
 Room.prototype.getMyMembership = function() {
-    if (this.myUserId) {
-        const me = this.getMember(this.myUserId);
-        if (me) {
-            return me.membership;
-        }
-    }
-    return this._syncedMembership;
+    return this._selfMembership;
 };

 /**
@@ -278,7 +275,7 @@ Room.prototype.getDMInviter = function() {
            return me.getDMInviter();
        }
    }
-    if (this._syncedMembership === "invite") {
+    if (this._selfMembership === "invite") {
        // fall back to summary information
        const memberCount = this.getInvitedAndJoinedMemberCount();
        if (memberCount == 2 && this._summaryHeroes.length) {
@@ -299,27 +296,17 @@ Room.prototype.guessDMUserId = function() {
            return inviterId;
        }
    }
-    const fallbackMember = this.getAvatarFallbackMember();
-    if (fallbackMember) {
-        return fallbackMember.userId;
-    }
-    // now we're getting into sketchy territory,
-    // but we're assuming this room is marked as a DM
-    // so we're going to make a wild-ish guess with whom
-    if (this._summaryHeroes.length) {
+    // remember, we're assuming this room is a DM,
+    // so returning the first member we find should be fine
+    const hasHeroes = Array.isArray(this._summaryHeroes) &&
+        this._summaryHeroes.length;
+    if (hasHeroes) {
        return this._summaryHeroes[0];
    }
    const members = this.currentState.getMembers();
-    const anyMember = members.filter((m) => m.userId !== this.myUserId);
+    const anyMember = members.find((m) => m.userId !== this.myUserId);
    if (anyMember) {
-        return anyMember;
-    }
-    const createEvent = this.currentState.getStateEvents("m.room.create", "");
-    if (createEvent) {
-        const sender = createEvent.getSender();
-        if (sender !== this.myUserId) {
-            return sender;
-        }
+        return anyMember.userId;
    }
    // it really seems like I'm the only user in the room
    // so I probably created a room with just me in it
@@ -372,14 +359,20 @@ Room.prototype.getAvatarFallbackMember = function() {
 * Sets the membership this room was received as during sync
 * @param {string} membership join | leave | invite
 */
-Room.prototype.setSyncedMembership = function(membership) {
-    this._syncedMembership = membership;
+Room.prototype.updateMyMembership = function(membership) {
+    const prevMembership = this._selfMembership;
+    this._selfMembership = membership;
+    if (prevMembership !== membership) {
+        if (membership === "leave") {
+            this._cleanupAfterLeaving();
+        }
+        this.emit("Room.myMembership", this, membership, prevMembership);
+    }
 };

 Room.prototype._loadMembersFromServer = async function() {
    const lastSyncToken = this._client.store.getSyncToken();
    const queryString = utils.encodeParams({
-        membership: "join",
        not_membership: "leave",
        at: lastSyncToken,
    });
@@ -424,8 +417,21 @@ Room.prototype.loadMembersIfNeeded = function() {
    // the OOB members
    this.currentState.markOutOfBandMembersStarted();

-    const promise = this._loadMembers().then(({memberEvents, fromServer}) => {
-        this.currentState.setOutOfBandMembers(memberEvents);
+    const inMemoryUpdate = this._loadMembers().then((result) => {
+        this.currentState.setOutOfBandMembers(result.memberEvents);
+        // now the members are loaded, start to track the e2e devices if needed
+        if (this._client.isRoomEncrypted(this.roomId)) {
+            this._client._crypto.trackRoomDevices(this.roomId);
+        }
+        return result.fromServer;
+    }).catch((err) => {
+        // allow retries on fail
+        this._membersPromise = null;
+        this.currentState.markOutOfBandMembersFailed();
+        throw err;
+    });
+    // update members in storage, but don't wait for it
+    inMemoryUpdate.then((fromServer) => {
        if (fromServer) {
            const oobMembers = this.currentState.getMembers()
                .filter((m) => m.isOutOfBand())
@@ -442,21 +448,40 @@ Room.prototype.loadMembersIfNeeded = function() {
                });
        }
    }).catch((err) => {
-        // allow retries on fail
-        this._membersPromise = null;
-        this.currentState.markOutOfBandMembersFailed();
-        throw err;
+        // as this is not awaited anywhere,
+        // at least show the error in the console
+        console.error(err);
    });

-    this._membersPromise = promise;
-    // now the members are loaded, start to track the e2e devices if needed
-    if (this._client.isRoomEncrypted(this.roomId)) {
-        this._client._crypto.trackRoomDevices(this.roomId);
-    }
+    this._membersPromise = inMemoryUpdate;

    return this._membersPromise;
 };

+/**
+ * Removes the lazily loaded members from storage if needed
+ */
+Room.prototype.clearLoadedMembersIfNeeded = async function() {
+    if (this._opts.lazyLoadMembers && this._membersPromise) {
+        await this.loadMembersIfNeeded();
+        await this._client.store.clearOutOfBandMembers(this.roomId);
+        this.currentState.clearOutOfBandMembers();
+        this._membersPromise = null;
+    }
+};
+
+/**
+ * called when sync receives this room in the leave section
+ * to do cleanup after leaving a room. Possibly called multiple times.
+ */
+Room.prototype._cleanupAfterLeaving = function() {
+    this.clearLoadedMembersIfNeeded().catch((err) => {
+        console.error(`error after clearing loaded members from ` +
+            `room ${this.roomId} after leaving`);
+        console.dir(err);
+    });
+};
+
 /**
 * Reset the live timeline of all timelineSets, and start new ones.
 *
@@ -1472,6 +1497,17 @@ Room.prototype.getAccountData = function(type) {
    return this.accountData[type];
 };

+
+/**
+ * Returns wheter the syncing user has permission to send a message in the room
+ * @return {boolean} true if the user should be permitted to send
+ *                   message events into the room.
+ */
+Room.prototype.maySendMessage = function() {
+    return this.getMyMembership() === 'join' &&
+        this.currentState.maySendEvent('m.room.message', this.myUserId);
+};
+
 /**
 * This is an internal method. Calculates the name of the room from the current
 * room state.
@@ -19,7 +19,7 @@ import Promise from 'bluebird';
 import SyncAccumulator from "../sync-accumulator";
 import utils from "../utils";

-const VERSION = 2;
+const VERSION = 3;

 function createDatabase(db) {
    // Make user store, clobber based on user ID. (userId property of User objects)
@@ -41,6 +41,12 @@ function upgradeSchemaV2(db) {
    oobMembersStore.createIndex("room", "room_id");
 }

+function upgradeSchemaV3(db) {
+    db.createObjectStore("client_options",
+        { keyPath: ["clobber"]});
+}
+
+
 /**
 * Helper method to collect results from a Cursor and promiseify it.
 * @param {ObjectStore|Index} store The store to perform openCursor on.
@@ -71,28 +77,39 @@ function selectQuery(store, keyRange, resultMapper) {
    });
 }

-function promiseifyTxn(txn) {
+function txnAsPromise(txn) {
    return new Promise((resolve, reject) => {
        txn.oncomplete = function(event) {
            resolve(event);
        };
        txn.onerror = function(event) {
-            reject(event);
+            reject(event.target.error);
        };
    });
 }

-function promiseifyRequest(req) {
+function reqAsEventPromise(req) {
    return new Promise((resolve, reject) => {
        req.onsuccess = function(event) {
            resolve(event);
        };
        req.onerror = function(event) {
-            reject(event);
+            reject(event.target.error);
        };
    });
 }

+function reqAsPromise(req) {
+    return new Promise((resolve, reject) => {
+        req.onsuccess = () => resolve(req);
+        req.onerror = (err) => reject(err);
+    });
+}
+
+function reqAsCursorPromise(req) {
+    return reqAsEventPromise(req).then((event) => event.target.result);
+}
+
 /**
 * Does the actual reading from and writing to the indexeddb
 *
@@ -112,6 +129,7 @@ const LocalIndexedDBStoreBackend = function LocalIndexedDBStoreBackend(
    this.db = null;
    this._disconnected = true;
    this._syncAccumulator = new SyncAccumulator();
+    this._isNewlyCreated = false;
 };


@@ -142,11 +160,15 @@ LocalIndexedDBStoreBackend.prototype = {
                `LocalIndexedDBStoreBackend.connect: upgrading from ${oldVersion}`,
            );
            if (oldVersion < 1) { // The database did not previously exist.
+                this._isNewlyCreated = true;
                createDatabase(db);
            }
            if (oldVersion < 2) {
                upgradeSchemaV2(db);
            }
+            if (oldVersion < 3) {
+                upgradeSchemaV3(db);
+            }
            // Expand as needed.
        };

@@ -159,7 +181,7 @@ LocalIndexedDBStoreBackend.prototype = {
        console.log(
            `LocalIndexedDBStoreBackend.connect: awaiting connection...`,
        );
-        return promiseifyRequest(req).then((ev) => {
+        return reqAsEventPromise(req).then((ev) => {
            console.log(
                `LocalIndexedDBStoreBackend.connect: connected`,
            );
@@ -174,6 +196,10 @@ LocalIndexedDBStoreBackend.prototype = {
            return this._init();
        });
    },
+    /** @return {bool} whether or not the database was newly created in this session. */
+    isNewlyCreated: function() {
+        return Promise.resolve(this._isNewlyCreated);
+    },

    /**
     * Having connected, load initial data from the database and prepare for use
@@ -254,42 +280,66 @@ LocalIndexedDBStoreBackend.prototype = {
     * marked as fetched, and getOutOfBandMembers will return an empty array instead of null
     * @param {string} roomId
     * @param {event[]} membershipEvents the membership events to store
-     * @returns {Promise} when all members have been stored
     */
-    setOutOfBandMembers: function(roomId, membershipEvents) {
+    setOutOfBandMembers: async function(roomId, membershipEvents) {
        console.log(`LL: backend about to store ${membershipEvents.length}` +
            ` members for ${roomId}`);
-        function ignoreResult() {}
-        // run everything in a promise so anything that throws will reject
-        return new Promise((resolve) =>{
-            const tx = this.db.transaction(["oob_membership_events"], "readwrite");
-            const store = tx.objectStore("oob_membership_events");
-            const eventPuts = membershipEvents.map((e) => {
-                const putPromise = promiseifyRequest(store.put(e));
-                // ignoring the result makes sure we discard the IDB success event
-                // ASAP, and not create a potentially big array containing them
-                // unneccesarily later on by calling Promise.all.
-                return putPromise.then(ignoreResult);
-            });
-            // aside from all the events, we also write a marker object to the store
-            // to mark the fact that OOB members have been written for this room.
-            // It's possible that 0 members need to be written as all where previously know
-            // but we still need to know whether to return null or [] from getOutOfBandMembers
-            // where null means out of band members haven't been stored yet for this room
-            const markerObject = {
-                room_id: roomId,
-                oob_written: true,
-                state_key: 0,
-            };
-            const markerPut = promiseifyRequest(store.put(markerObject));
-            const allPuts = eventPuts.concat(markerPut);
-            // ignore the empty array Promise.all creates
-            // as this method should just resolve
-            // to undefined on success
-            resolve(Promise.all(allPuts).then(ignoreResult));
-        }).then(() => {
-            console.log(`LL: backend done storing for ${roomId}!`);
+        const tx = this.db.transaction(["oob_membership_events"], "readwrite");
+        const store = tx.objectStore("oob_membership_events");
+        membershipEvents.forEach((e) => {
+            store.put(e);
        });
+        // aside from all the events, we also write a marker object to the store
+        // to mark the fact that OOB members have been written for this room.
+        // It's possible that 0 members need to be written as all where previously know
+        // but we still need to know whether to return null or [] from getOutOfBandMembers
+        // where null means out of band members haven't been stored yet for this room
+        const markerObject = {
+            room_id: roomId,
+            oob_written: true,
+            state_key: 0,
+        };
+        store.put(markerObject);
+        await txnAsPromise(tx);
+        console.log(`LL: backend done storing for ${roomId}!`);
+    },
+
+    clearOutOfBandMembers: async function(roomId) {
+        // the approach to delete all members for a room
+        // is to get the min and max state key from the index
+        // for that room, and then delete between those
+        // keys in the store.
+        // this should be way faster than deleting every member
+        // individually for a large room.
+        const readTx = this.db.transaction(
+            ["oob_membership_events"],
+            "readonly");
+        const store = readTx.objectStore("oob_membership_events");
+        const roomIndex = store.index("room");
+        const roomRange = IDBKeyRange.only(roomId);
+
+        const minStateKeyProm = reqAsCursorPromise(
+                roomIndex.openKeyCursor(roomRange, "next"),
+            ).then((cursor) => cursor && cursor.primaryKey[1]);
+        const maxStateKeyProm = reqAsCursorPromise(
+                roomIndex.openKeyCursor(roomRange, "prev"),
+            ).then((cursor) => cursor && cursor.primaryKey[1]);
+        const [minStateKey, maxStateKey] = await Promise.all(
+            [minStateKeyProm, maxStateKeyProm]);
+
+        const writeTx = this.db.transaction(
+            ["oob_membership_events"],
+            "readwrite");
+        const writeStore = writeTx.objectStore("oob_membership_events");
+        const membersKeyRange = IDBKeyRange.bound(
+            [roomId, minStateKey],
+            [roomId, maxStateKey],
+        );
+
+        console.log(`LL: Deleting all users + marker in storage for ` +
+            `room ${roomId}, with key range:`,
+            [roomId, minStateKey], [roomId, maxStateKey]);
+        await reqAsPromise(writeStore.delete(membersKeyRange));
    },

    /**
@@ -389,7 +439,7 @@ LocalIndexedDBStoreBackend.prototype = {
                roomsData: roomsData,
                groupsData: groupsData,
            }); // put == UPSERT
-            return promiseifyTxn(txn);
+            return txnAsPromise(txn);
        });
    },

@@ -406,7 +456,7 @@ LocalIndexedDBStoreBackend.prototype = {
            for (let i = 0; i < accountData.length; i++) {
                store.put(accountData[i]); // put == UPSERT
            }
-            return promiseifyTxn(txn);
+            return txnAsPromise(txn);
        });
    },

@@ -428,7 +478,7 @@ LocalIndexedDBStoreBackend.prototype = {
                    event: tuple[1],
                }); // put == UPSERT
            }
-            return promiseifyTxn(txn);
+            return txnAsPromise(txn);
        });
    },

@@ -494,6 +544,28 @@ LocalIndexedDBStoreBackend.prototype = {
            });
        });
    },
+
+    getClientOptions: function() {
+        return Promise.resolve().then(() => {
+            const txn = this.db.transaction(["client_options"], "readonly");
+            const store = txn.objectStore("client_options");
+            return selectQuery(store, undefined, (cursor) => {
+                if (cursor.value && cursor.value && cursor.value.options) {
+                    return cursor.value.options;
+                }
+            }).then((results) => results[0]);
+        });
+    },
+
+    storeClientOptions: async function(options) {
+        const txn = this.db.transaction(["client_options"], "readwrite");
+        const store = txn.objectStore("client_options");
+        store.put({
+            clobber: "-", // constant key so will always clobber
+            options: options,
+        }); // put == UPSERT
+        await txnAsPromise(txn);
+    },
 };

 export default LocalIndexedDBStoreBackend;
@@ -65,7 +65,10 @@ RemoteIndexedDBStoreBackend.prototype = {
    clearDatabase: function() {
        return this._ensureStarted().then(() => this._doCmd('clearDatabase'));
    },
-
+    /** @return {Promise<bool>} whether or not the database was newly created in this session. */
+    isNewlyCreated: function() {
+        return this._doCmd('isNewlyCreated');
+    },
    /**
     * @return {Promise} Resolves with a sync response to restore the
     * client state to where it was at the last save, or null if there
@@ -110,6 +113,18 @@ RemoteIndexedDBStoreBackend.prototype = {
        return this._doCmd('setOutOfBandMembers', [roomId, membershipEvents]);
    },

+    clearOutOfBandMembers: function(roomId) {
+        return this._doCmd('clearOutOfBandMembers', [roomId]);
+    },
+
+    getClientOptions: function() {
+        return this._doCmd('getClientOptions');
+    },
+
+    storeClientOptions: function(options) {
+        return this._doCmd('storeClientOptions', [options]);
+    },
+
    /**
     * Load all user presence events from the database. This is not cached.
     * @return {Promise<Object[]>} A list of presence events in their raw form.
@@ -169,7 +184,9 @@ RemoteIndexedDBStoreBackend.prototype = {
            if (msg.command == 'cmd_success') {
                def.resolve(msg.result);
            } else {
-                def.reject(msg.error);
+                const error = new Error(msg.error.message);
+                error.name = msg.error.name;
+                def.reject(error);
            }
        } else {
            console.warn("Unrecognised message from worker: " + msg);
@@ -67,6 +67,9 @@ class IndexedDBStoreWorker {
            case 'connect':
                prom = this.backend.connect();
                break;
+            case 'isNewlyCreated':
+                prom = this.backend.isNewlyCreated();
+                break;
            case 'clearDatabase':
                prom = this.backend.clearDatabase().then((result) => {
                    // This returns special classes which can't be cloned
@@ -95,13 +98,22 @@ class IndexedDBStoreWorker {
            case 'getOutOfBandMembers':
                prom = this.backend.getOutOfBandMembers(msg.args[0]);
                break;
+            case 'clearOutOfBandMembers':
+                prom = this.backend.clearOutOfBandMembers(msg.args[0]);
+                break;
            case 'setOutOfBandMembers':
                prom = this.backend.setOutOfBandMembers(msg.args[0], msg.args[1]);
                break;
+            case 'getClientOptions':
+                prom = this.backend.getClientOptions();
+                break;
+            case 'storeClientOptions':
+                prom = this.backend.storeClientOptions(msg.args[0]);
+                break;
        }

        if (prom === undefined) {
-            postMessage({
+            this.postMessage({
                command: 'cmd_fail',
                seq: msg.seq,
                // Can't be an Error because they're not structured cloneable
@@ -123,7 +135,10 @@ class IndexedDBStoreWorker {
                command: 'cmd_fail',
                seq: msg.seq,
                // Just send a string because Error objects aren't cloneable
-                error: "Error running command",
+                error: {
+                    message: err.message,
+                    name: err.name,
+                },
            });
        });
    }
@@ -146,6 +146,11 @@ IndexedDBStore.prototype.getSavedSync = function() {
    return this.backend.getSavedSync();
 };

+/** @return {Promise<bool>} whether or not the database was newly created in this session. */
+IndexedDBStore.prototype.isNewlyCreated = function() {
+    return this.backend.isNewlyCreated();
+};
+
 /**
 * @return {Promise} If there is a saved sync, the nextBatch token
 * for this sync, otherwise null.
@@ -242,4 +247,16 @@ IndexedDBStore.prototype.setOutOfBandMembers = function(roomId, membershipEvents
    return this.backend.setOutOfBandMembers(roomId, membershipEvents);
 };

+IndexedDBStore.prototype.clearOutOfBandMembers = function(roomId) {
+    return this.backend.clearOutOfBandMembers(roomId);
+};
+
+IndexedDBStore.prototype.getClientOptions = function() {
+    return this.backend.getClientOptions();
+};
+
+IndexedDBStore.prototype.storeClientOptions = function(options) {
+    return this.backend.storeClientOptions(options);
+};
+
 module.exports.IndexedDBStore = IndexedDBStore;
@@ -55,6 +55,7 @@ module.exports.MatrixInMemoryStore = function MatrixInMemoryStore(opts) {
    this._oobMembers = {
        // roomId: [member events]
    };
+    this._clientOptions = {};
 };

 module.exports.MatrixInMemoryStore.prototype = {
@@ -67,6 +68,10 @@ module.exports.MatrixInMemoryStore.prototype = {
        return this.syncToken;
    },

+    /** @return {Promise<bool>} whether or not the database was newly created in this session. */
+    isNewlyCreated: function() {
+        return Promise.resolve(true);
+    },

    /**
     * Set the token to stream from.
@@ -402,4 +407,13 @@ module.exports.MatrixInMemoryStore.prototype = {
        this._oobMembers[roomId] = membershipEvents;
        return Promise.resolve();
    },
+
+    getClientOptions: function() {
+        return Promise.resolve(this._clientOptions);
+    },
+
+    storeClientOptions: function(options) {
+        this._clientOptions = Object.assign({}, options);
+        return Promise.resolve();
+    },
 };
@@ -32,6 +32,11 @@ function StubStore() {

 StubStore.prototype = {

+    /** @return {Promise<bool>} whether or not the database was newly created in this session. */
+    isNewlyCreated: function() {
+        return Promise.resolve(true);
+    },
+
    /**
     * Get the sync token.
     * @return {string}
@@ -272,6 +277,18 @@ StubStore.prototype = {
    setOutOfBandMembers: function() {
        return Promise.resolve();
    },
+
+    clearOutOfBandMembers: function() {
+        return Promise.resolve();
+    },
+
+    getClientOptions: function() {
+        return Promise.resolve();
+    },
+
+    storeClientOptions: function() {
+        return Promise.resolve();
+    },
 };

 /** Stub Store class. */
@@ -33,6 +33,8 @@ const utils = require("./utils");
 const Filter = require("./filter");
 const EventTimeline = require("./models/event-timeline");

+import {InvalidStoreError} from './errors';
+
 const DEBUG = true;

 // /sync requests allow you to set a timeout= but the request may continue
@@ -100,6 +102,7 @@ function SyncApi(client, opts) {
    this._connectionReturnedDefer = null;
    this._notifEvents = []; // accumulator of sync events in the current sync response
    this._failedSyncCount = 0; // Number of consecutive failed /sync requests
+    this._storeIsInvalid = false; // flag set if the store needs to be cleared before we can start

    if (client.getNotifTimelineSet()) {
        client.reEmitter.reEmit(client.getNotifTimelineSet(),
@@ -123,6 +126,7 @@ SyncApi.prototype.createRoom = function(roomId) {
                          "Room.timelineReset",
                          "Room.localEchoUpdated",
                          "Room.accountData",
+                          "Room.myMembership",
                         ]);
    this._registerStateListeners(room);
    return room;
@@ -421,6 +425,26 @@ SyncApi.prototype.recoverFromSyncStartupError = async function(savedSyncPromise,
    await keepaliveProm;
 };

+/**
+ * Is the lazy loading option different than in previous session?
+ * @param {bool} lazyLoadMembers current options for lazy loading
+ * @return {bool} whether or not the option has changed compared to the previous session */
+SyncApi.prototype._wasLazyLoadingToggled = async function(lazyLoadMembers) {
+    lazyLoadMembers = !!lazyLoadMembers;
+    // assume it was turned off before
+    // if we don't know any better
+    let lazyLoadMembersBefore = false;
+    const isStoreNewlyCreated = await this.client.store.isNewlyCreated();
+    if (!isStoreNewlyCreated) {
+        const prevClientOptions = await this.client.store.getClientOptions();
+        if (prevClientOptions) {
+            lazyLoadMembersBefore = !!prevClientOptions.lazyLoadMembers;
+        }
+        return lazyLoadMembersBefore !== lazyLoadMembers;
+    }
+    return false;
+};
+
 /**
 * Main entry point
 */
@@ -443,6 +467,8 @@ SyncApi.prototype.sync = function() {
    //   1) We need to get push rules so we can check if events should bing as we get
    //      them from /sync.
    //   2) We need to get/create a filter which we can use for /sync.
+    //   3) We need to check the lazy loading option matches what was used in the
+    //       stored sync. If it doesn't, we can't use the stored sync.

    async function getPushRules() {
        try {
@@ -457,9 +483,47 @@ SyncApi.prototype.sync = function() {
            getPushRules();
            return;
        }
-        getFilter(); // Now get the filter and start syncing
+        checkLazyLoadStatus(); // advance to the next stage
    }

+    const checkLazyLoadStatus = async () => {
+        if (this.opts.lazyLoadMembers && client.isGuest()) {
+            this.opts.lazyLoadMembers = false;
+        }
+        if (this.opts.lazyLoadMembers) {
+            const supported = await client.doesServerSupportLazyLoading();
+            if (supported) {
+                this.opts.filter = await client.createFilter(
+                    Filter.LAZY_LOADING_SYNC_FILTER,
+                );
+            } else {
+                console.log("LL: lazy loading requested but not supported " +
+                    "by server, so disabling");
+                this.opts.lazyLoadMembers = false;
+            }
+        }
+        // need to vape the store when enabling LL and wasn't enabled before
+        const shouldClear = await this._wasLazyLoadingToggled(this.opts.lazyLoadMembers);
+        if (shouldClear) {
+            this._storeIsInvalid = true;
+            const reason = InvalidStoreError.TOGGLED_LAZY_LOADING;
+            const error = new InvalidStoreError(reason, !!this.opts.lazyLoadMembers);
+            this._updateSyncState("ERROR", { error });
+            // bail out of the sync loop now: the app needs to respond to this error.
+            // we leave the state as 'ERROR' which isn't great since this normally means
+            // we're retrying. The client must be stopped before clearing the stores anyway
+            // so the app should stop the client, clear the store and start it again.
+            console.warn("InvalidStoreError: store is not usable: stopping sync.");
+            return;
+        }
+        if (this.opts.lazyLoadMembers && this._crypto) {
+            this.opts.crypto.enableLazyLoading();
+        }
+        await this.client._storeClientOptions();
+
+        getFilter(); // Now get the filter and start syncing
+    };
+
    async function getFilter() {
        let filter;
        if (self.opts.filter) {
@@ -587,7 +651,12 @@ SyncApi.prototype._syncFromCache = async function(savedSync) {
        console.error("Error processing cached sync", e.stack || e);
    }

-    this._updateSyncState("PREPARED", syncEventData);
+    // Don't emit a prepared if we've bailed because the store is invalid:
+    // in this case the client will not be usable until stopped & restarted
+    // so this would be useless and misleading.
+    if (!this._storeIsInvalid) {
+        this._updateSyncState("PREPARED", syncEventData);
+    }
 };

 /**
@@ -779,7 +848,18 @@ SyncApi.prototype._onSyncError = function(err, syncOptions) {
    // erroneous. We set the state to 'reconnecting'
    // instead, so that clients can observe this state
    // if they wish.
-    this._startKeepAlives().then(() => {
+    this._startKeepAlives().then((connDidFail) => {
+        // Only emit CATCHUP if we detected a connectivity error: if we didn't,
+        // it's quite likely the sync will fail again for the same reason and we
+        // want to stay in ERROR rather than keep flip-flopping between ERROR
+        // and CATCHUP.
+        if (connDidFail && this.getSyncState() === 'ERROR') {
+            this._updateSyncState("CATCHUP", {
+                oldSyncToken: null,
+                nextSyncToken: null,
+                catchingUp: true,
+            });
+        }
        this._sync(syncOptions);
    });

@@ -965,9 +1045,10 @@ SyncApi.prototype._processSyncResponse = async function(
    // Handle invites
    inviteRooms.forEach(function(inviteObj) {
        const room = inviteObj.room;
-        room.setSyncedMembership("invite");
        const stateEvents =
            self._mapSyncEventsFormat(inviteObj.invite_state, room);
+
+        room.updateMyMembership("invite");
        self._processRoomEvents(room, stateEvents);
        if (inviteObj.isBrandNewRoom) {
            room.recalculate();
@@ -982,7 +1063,6 @@ SyncApi.prototype._processSyncResponse = async function(
    // Handle joins
    await Promise.mapSeries(joinRooms, async function(joinObj) {
        const room = joinObj.room;
-        room.setSyncedMembership("join");
        const stateEvents = self._mapSyncEventsFormat(joinObj.state, room);
        const timelineEvents = self._mapSyncEventsFormat(joinObj.timeline, room);
        const ephemeralEvents = self._mapSyncEventsFormat(joinObj.ephemeral);
@@ -998,6 +1078,8 @@ SyncApi.prototype._processSyncResponse = async function(
            );
        }

+        room.updateMyMembership("join");
+
        joinObj.timeline = joinObj.timeline || {};

        if (joinObj.isBrandNewRoom) {
@@ -1105,8 +1187,6 @@ SyncApi.prototype._processSyncResponse = async function(
    // Handle leaves (e.g. kicked rooms)
    leaveRooms.forEach(function(leaveObj) {
        const room = leaveObj.room;
-        room.setSyncedMembership("leave");
-
        const stateEvents =
            self._mapSyncEventsFormat(leaveObj.state, room);
        const timelineEvents =
@@ -1114,6 +1194,8 @@ SyncApi.prototype._processSyncResponse = async function(
        const accountDataEvents =
            self._mapSyncEventsFormat(leaveObj.account_data);

+        room.updateMyMembership("leave");
+
        self._processRoomEvents(room, stateEvents, timelineEvents);
        room.addAccountData(accountDataEvents);

@@ -1206,13 +1288,16 @@ SyncApi.prototype._startKeepAlives = function(delay) {
 *
 * On failure, schedules a call back to itself. On success, resolves
 * this._connectionReturnedDefer.
+ *
+ * @param {bool} connDidFail True if a connectivity failure has been detected. Optional.
 */
-SyncApi.prototype._pokeKeepAlive = function() {
+SyncApi.prototype._pokeKeepAlive = function(connDidFail) {
+    if (connDidFail === undefined) connDidFail = false;
    const self = this;
    function success() {
        clearTimeout(self._keepAliveTimer);
        if (self._connectionReturnedDefer) {
-            self._connectionReturnedDefer.resolve();
+            self._connectionReturnedDefer.resolve(connDidFail);
            self._connectionReturnedDefer = null;
        }
    }
@@ -1229,7 +1314,7 @@ SyncApi.prototype._pokeKeepAlive = function() {
    ).done(function() {
        success();
    }, function(err) {
-        if (err.httpStatus == 400) {
+        if (err.httpStatus == 400 || err.httpStatus == 404) {
            // treat this as a success because the server probably just doesn't
            // support /versions: point is, we're getting a response.
            // We wait a short time though, just in case somehow the server
@@ -1237,8 +1322,9 @@ SyncApi.prototype._pokeKeepAlive = function() {
            // responses fail, this will mean we don't hammer in a loop.
            self._keepAliveTimer = setTimeout(success, 2000);
        } else {
+            connDidFail = true;
            self._keepAliveTimer = setTimeout(
-                self._pokeKeepAlive.bind(self),
+                self._pokeKeepAlive.bind(self, connDidFail),
                5000 + Math.floor(Math.random() * 5000),
            );
            // A keepalive has failed, so we emit the
@@ -1246,7 +1332,7 @@ SyncApi.prototype._pokeKeepAlive = function() {
            // first failure).
            // Note we do this after setting the timer:
            // this lets the unit tests advance the mock
-            // clock when the get the error.
+            // clock when they get the error.
            self._updateSyncState("ERROR", { error: err });
        }
    });
Author	SHA1	Message	Date
David Baker	874bdea634	v0.12.0	2018-10-16 10:47:33 +01:00
David Baker	68497d3a1f	Prepare changelog for v0.12.0	2018-10-16 10:47:32 +01:00
David Baker	874029dff0	oops - fix changelog format in retrospect	2018-10-11 15:38:34 +01:00
David Baker	6aff3ed407	v0.12.0-rc.1	2018-10-11 14:42:29 +01:00
David Baker	c0ae78ae82	Prepare changelog for v0.12.0-rc.1	2018-10-11 14:42:29 +01:00
David Baker	8b22f01ecd	Merge pull request #760 from matrix-org/bwindels/fixleavesinmemberlist never replace /sync'ed memberships with OOB ones	2018-10-11 14:01:38 +01:00
Bruno Windels	2ed694b041	remove supersedes OOB logic	2018-10-11 14:32:03 +02:00
Bruno Windels	a0ef6ab811	typo	2018-10-11 14:14:42 +02:00
Bruno Windels	d098b39024	never replace /sync'ed memberships with OOB ones	2018-10-11 14:11:40 +02:00
David Baker	3cf23f8a5c	Document breaking change	2018-10-11 12:13:25 +01:00
Bruno Windels	72e77d237a	Merge pull request #759 from matrix-org/dbkr/lazy_load_check_before_sync Don't fail to start up if lazy load check fails	2018-10-10 18:28:47 +02:00
David Baker	ecc3e18e85	typo	2018-10-10 17:27:06 +01:00
David Baker	dea70af889	remove debug logging	2018-10-10 17:06:26 +01:00
David Baker	30362091e5	Don't fail to start up if lazy load check fails Do the lazy loading check in the batch of things we do before starting a sync rather than at client start time, so we don't fail to start the client if we can't hit the HS to determine LL support. Fixes https://github.com/vector-im/riot-web/issues/7455	2018-10-10 16:59:36 +01:00
David Baker	0aa3362671	Merge pull request #754 from matrix-org/dbkr/e2e_on_interslice Make e2e work on Edge	2018-10-09 10:48:24 +01:00
David Baker	92fbf58b13	Merge pull request #758 from matrix-org/bwindels/betteridberrors throw error with same name and message over idb worker boundary	2018-10-09 09:59:55 +01:00
Bruno Windels	a4b2cc84c7	Merge pull request #755 from matrix-org/travis/fix-vuln-warning Default to a room version of 1 when there is no room create event	2018-10-08 09:55:10 +02:00
David Baker	1395da694e	Merge pull request #757 from matrix-org/dbkr/gone_away_is_the_bluebird Silence bluebird warnings	2018-10-05 13:25:16 +01:00
David Baker	264b20535e	Silence bluebird warnings	2018-10-05 12:13:05 +01:00
Bruno Windels	caba350b33	throw error with same name and message over idb worker boundary instead of string currently thrown. This allows handling error from the main thread.	2018-10-05 12:48:12 +02:00
Travis Ralston	d9fe194111	Default to a room version of 1 when there is no room create event Fixes https://github.com/vector-im/riot-web/issues/7331 There is something to be worried about when there is no room create event, however. This should always be available, although due to cache problems or servers that don't provide the event we can't be sure of this.	2018-10-04 13:42:07 -06:00
David Baker	40dc13b2e2	lint try 2	2018-10-04 15:38:08 +01:00
David Baker	4cda54ca1c	lint	2018-10-04 15:15:30 +01:00
David Baker	8116c5b3f7	Make e2e work on Edge We were sucessfully opening indexeddb but any queries using compound indicies were failing because Edge doesn't support them, so messages were failing to decrypt with 'DataError'. Try a dummy query at startup, so if it fails we fall back to a different store (ie. end up using localstorage on Edge).	2018-10-04 13:49:32 +01:00
Bruno Windels	1dcc5127d0	Merge pull request #750 from matrix-org/bwindels/allownonffmerge allow non-ff merge from release branch into master	2018-10-04 10:22:02 +02:00
David Baker	85e3d7083c	Merge pull request #751 from matrix-org/dbkr/indexeddb_errors Reject with the actual error on indexeddb error	2018-10-03 16:22:26 +01:00
David Baker	bd2da08c4e	Reject with the actual error on indexeddb error Rather than the event	2018-10-02 16:48:27 +01:00
Bruno Windels	0a88d419c6	allow non-ff merge from release branch into master	2018-10-01 17:21:01 +02:00
Bruno Windels	80c190db36	Merge branch 'master' into develop	2018-10-01 15:40:16 +02:00
Bruno Windels	550cf00ee7	Merge branch 'release-v0.11.1'	2018-10-01 15:36:37 +02:00
Bruno Windels	1e1358fcef	v0.11.1	2018-10-01 15:23:37 +02:00
Bruno Windels	fd1b3329f5	Prepare changelog for v0.11.1	2018-10-01 15:23:37 +02:00
Bruno Windels	9c9d8468a5	Merge pull request #744 from matrix-org/dbkr/update_mocha Update mocha to v5	2018-09-28 12:57:49 +01:00
David Baker	6f59d62e5c	Merge pull request #748 from matrix-org/bwindels/nollforguests disable lazy loading for guests as they cant create filters	2018-09-27 19:29:54 +01:00
Bruno Windels	1c348f0cdb	disable lazy loading for guests as they cant create filters	2018-09-27 18:55:21 +01:00
Bruno Windels	634596257d	v0.11.1-rc.1	2018-09-27 11:47:28 +01:00
Bruno Windels	5e4973a1dc	changelog	2018-09-27 11:45:33 +01:00
Bruno Windels	19f023e0ee	Revert "v0.11.1-rc.1" This reverts commit `090c15fe19`.	2018-09-27 11:42:19 +01:00
Bruno Windels	090c15fe19	v0.11.1-rc.1	2018-09-27 11:38:25 +01:00
Bruno Windels	e8b307dc4f	Prepare changelog for v0.11.1-rc.1	2018-09-27 11:38:25 +01:00
Bruno Windels	056479d450	Revert "v0.11.1-rc.1" This reverts commit `847d40e567`.	2018-09-27 11:35:52 +01:00
Bruno Windels	e5ebe2f888	Merge pull request #747 from matrix-org/bwindels/releasehubversion make usage of hub compatible with latest version (2.5)	2018-09-27 11:30:17 +01:00
Bruno Windels	e8e1b431ad	make usage of hub compatible with latest version (2.5)	2018-09-27 11:28:01 +01:00
Bruno Windels	847d40e567	v0.11.1-rc.1	2018-09-27 11:20:21 +01:00
Bruno Windels	cf6c555e6a	Prepare changelog for v0.11.1-rc.1	2018-09-27 11:20:21 +01:00
Bruno Windels	b508aa9ebc	Merge pull request #746 from matrix-org/bwindels/resynconlltoggle Detect when lazy loading has been toggled in client.startClient	2018-09-27 09:49:01 +01:00
Bruno Windels	5e7634506e	Merge branch 'develop' into bwindels/resynconlltoggle	2018-09-26 18:23:07 +01:00
Bruno Windels	ba39b64ced	re-enable test	2018-09-26 18:00:40 +01:00
Travis Ralston	fcebe89f6c	Merge branch 'hs/upload-limits' into develop	2018-09-26 09:14:12 -06:00
Bruno Windels	2d5eb920b8	pass lazy loading flag into error, to format message based on it	2018-09-26 16:12:30 +01:00
Travis Ralston	26de2c86ed	Merge pull request #745 from matrix-org/revert-644-hs/upload-limits Revert "Add getMediaLimits to client"	2018-09-26 09:12:03 -06:00
Travis Ralston	cba1e95d0a	Revert "Add getMediaLimits to client"	2018-09-26 09:11:28 -06:00
Bruno Windels	78a5a88638	fix jsdoc lint and better naming	2018-09-26 12:49:33 +01:00
Bruno Windels	b7b9c67259	fix lint	2018-09-26 12:49:26 +01:00
Bruno Windels	54bff81470	clear sync data on toggling LL,also throw spec. error and delegate clear the sync data needs to be cleared toggling in both directions: not LL -> LL: you want to get rid of all the excess state in the sync data to get the RAM benefits LL -> not LL: you want to fill the sync data state again because getOutOfBandMembers won't be called	2018-09-26 12:39:56 +01:00
David Baker	fe21972f4a	Update mocha to v5 Mostly to get the non-vulnerable version of node-growl	2018-09-26 11:36:06 +01:00
Bruno Windels	58e3c72446	only store serializable options (string, boolean, number)	2018-09-26 11:34:58 +01:00
Bruno Windels	6dd5c6c317	fix existing missing this	2018-09-26 11:33:19 +01:00
Bruno Windels	4e0af3eafe	don't return the IDBEvent from storeClientOptions as it's not needed and not cloneable	2018-09-26 11:32:43 +01:00
Bruno Windels	1d0791142c	all store methods should return a promise	2018-09-26 11:32:11 +01:00
Bruno Windels	2560ba2980	dont clear the store if its a brand new one	2018-09-26 10:37:52 +01:00
Bruno Windels	19be3dd852	fix lint	2018-09-26 10:13:40 +01:00
Travis Ralston	40f2a6978b	Merge pull request #644 from Half-Shot/hs/upload-limits Add getMediaLimits to client	2018-09-25 18:12:55 -06:00
Bruno Windels	1fd8c43d94	fix tests	2018-09-25 18:50:09 +01:00
Matthew Hodgson	3e32f47903	Merge pull request #742 from matrix-org/travis/build-process Split npm start into an init and watch script	2018-09-25 17:31:23 +01:00
Travis Ralston	8f2824186a	Split npm start into an init and watch script This is to better support riot-web's build process without losing the functionality supplied by `npm start`. The watch script no longer performs an initial build and thus `start:init` has been created for this purpose.	2018-09-25 10:09:58 -06:00
Bruno Windels	b0dbb20e22	fixup of in memory stores	2018-09-25 15:53:40 +01:00
Bruno Windels	0519c4c6b1	await startClient and use promises also so error gets shown	2018-09-25 15:53:14 +01:00
Bruno Windels	28184b4a29	check if lazy loading was enabled before in startClient	2018-09-25 15:32:10 +01:00
Bruno Windels	76175abea2	allow storing client options in indexeddb so we can tell what options the sync data was created with	2018-09-25 15:30:35 +01:00
Bruno Windels	184c3dce15	Merge pull request #738 from matrix-org/revert-733-bwindels/roomnamealias Revert "room name should only take canonical alias into account"	2018-09-20 12:23:38 +02:00
David Baker	a08a3078da	Revert "room name should only take canonical alias into account"	2018-09-20 11:20:49 +01:00
David Baker	c2100d7622	Merge pull request #737 from matrix-org/bwindels/fixnamedisambiguation fix display name disambiguation with LL	2018-09-20 10:22:38 +01:00
Bruno Windels	a91fa59174	fix display name disambiguation with LL	2018-09-19 18:14:18 +02:00
Bruno Windels	574a6b68ae	Merge pull request #735 from matrix-org/bwindels/fixstalerr Introduce Room.myMembership event	2018-09-19 13:35:11 +02:00
Bruno Windels	fc3a00054f	add test for new event	2018-09-17 19:33:36 +02:00
Bruno Windels	84e41c2ade	fix tests	2018-09-17 18:28:07 +02:00
Bruno Windels	4630733b55	don't fall back anymore to member, as this is more reliable	2018-09-17 18:23:48 +02:00
Bruno Windels	eb690e14e4	introduce Room.myMembership event As you don't always have your own member with lazy loading of members enabled, looking at the sync response section where a room appears is the most reliable way of determining the syncing user's membership in a room. Before we already used this to read the current room membership with `room.getMyMembership()`, but we were still using the `RoomMember.membership` event to detect when the syncing user's membership changed. This event will help make those checks work well with LL enabled.	2018-09-17 18:20:49 +02:00
David Baker	833002f846	Merge pull request #733 from matrix-org/bwindels/roomnamealias room name should only take canonical alias into account	2018-09-14 11:38:26 +01:00
Bruno Windels	907e9fc476	fix lint	2018-09-13 10:04:31 +02:00
Bruno Windels	b829a39cd2	fix tests	2018-09-13 09:59:20 +02:00
Bruno Windels	daa7af0605	room name should only take canonical alias into account	2018-09-13 09:52:21 +02:00
David Baker	47a1adc864	Merge pull request #732 from matrix-org/bwindels/fixcontextstatenotwrapped state events from context response were not wrapped in a MatrixEvent	2018-09-11 13:27:03 +01:00
Bruno Windels	98e448acdd	state events from context response were not wrapped in a MatrixEvent	2018-09-11 14:13:35 +02:00
David Baker	cd5cda916f	Merge branch 'master' into develop	2018-09-10 11:35:25 +01:00
David Baker	33a1139772	v0.11.0	2018-09-10 11:33:51 +01:00
David Baker	c5b62903f3	Prepare changelog for v0.11.0	2018-09-10 11:33:50 +01:00
David Baker	387fd16b40	v0.11.0-rc.1	2018-09-07 14:28:45 +01:00
David Baker	c91b67d370	Prepare changelog for v0.11.0-rc.1	2018-09-07 14:28:44 +01:00
David Baker	b809d1c263	Merge pull request #724 from matrix-org/bwindels/fastermemberinsert Reduce amount of promises created when inserting members	2018-09-07 13:57:44 +01:00
David Baker	7bb6675abf	Merge pull request #726 from matrix-org/bwindels/dontwaitforstorage dont wait for LL members to be stored to resolve the members	2018-09-07 13:57:20 +01:00
David Baker	b91bea94f4	Merge pull request #728 from matrix-org/bwindels/wrongemitargs RoomState.members emitted with wrong argument order for OOB members	2018-09-07 13:57:09 +01:00
David Baker	96e21700bd	Merge pull request #727 from matrix-org/dbkr/conn_did_fail Only emit CATCHUP if recovering from conn error	2018-09-07 13:31:17 +01:00
David Baker	7e8f25bce3	Include 404 in connectivity success error codes	2018-09-07 13:18:45 +01:00
Bruno Windels	9e02049b05	RoomState.members emitted with wrong argument order for OOB members	2018-09-07 14:11:27 +02:00
David Baker	affdfccd60	Only emit CATCHUP if recovering from conn error Have the keepalive promise return a boolean indicating whether it detected a connectivity failure or not. Use this to only emit CATCHUP if there was a connectivity error, to try & suppress the state flip-flopping back & forth between CATCHUP and ERROR in the case where we have connectivity but the sync is returning and error for whatever reason.	2018-09-07 11:29:44 +01:00
David Baker	402f8c27a9	Merge pull request #725 from matrix-org/dbkr/fix_sync_error_doc Fix docstring for sync data.error	2018-09-07 09:44:28 +01:00
Bruno Windels	ba4dc6c60a	dont wait for LL members to be stored to resolve the members this can easily add up to 100ms / 1000 of members	2018-09-07 10:44:06 +02:00
David Baker	6b8dd42547	Fox docstring for sync data.error It's 'error', not 'err'	2018-09-06 19:53:52 +01:00
Bruno Windels	1511a27f4c	update/remove comments	2018-09-06 18:09:46 +02:00
Bruno Windels	7ae6c147fa	lint doesnt like async	2018-09-06 18:02:19 +02:00
Bruno Windels	f51630eb07	dont create a promise for every inserted member but await the transaction instead	2018-09-06 18:01:20 +02:00
Bruno Windels	e3586411e0	Merge pull request #723 from matrix-org/revert-721-revert-717-bwindels/fixllroompermission Re-apply "Don't rely on members to query if syncing user can post to room"	2018-09-06 14:55:24 +02:00
David Baker	a0639a32c7	Revert "Revert "Don't rely on members to query if syncing user can post to room""	2018-09-06 11:47:40 +01:00
David Baker	759c6e77a7	Merge pull request #721 from matrix-org/revert-717-bwindels/fixllroompermission Revert "Don't rely on members to query if syncing user can post to room"	2018-09-05 18:09:59 +01:00
David Baker	04ad3d7c3c	Revert "Don't rely on members to query if syncing user can post to room"	2018-09-05 18:04:19 +01:00
Bruno Windels	49badd9a2f	Merge pull request #717 from matrix-org/bwindels/fixllroompermission Don't rely on members to query if syncing user can post to room	2018-09-05 13:07:29 +02:00
Bruno Windels	8b00083bca	check power levels without relying on membership as this might not be known for the syncing user. instead, add a method to room which always knows the syncing user's membership	2018-09-05 12:36:53 +02:00
Bruno Windels	3d98e324b5	Merge pull request #719 from matrix-org/bwindels/fixavatars-parttrois Fixes for room.guessDMUserId	2018-09-04 18:17:39 +02:00
Bruno Windels	768c66313f	remove unneeded async keywords	2018-09-04 18:09:47 +02:00
Bruno Windels	3561fd1c05	Merge pull request #716 from matrix-org/bwindels/fixfilepanel Fix filepanel also filtering main timeline with LL turned on.	2018-09-04 17:10:49 +02:00
Bruno Windels	a6c055b6d1	Merge pull request #711 from matrix-org/bwindels/clearidbmembersonleave Remove lazy loaded members when leaving room	2018-09-04 16:53:56 +02:00
Bruno Windels	0d24c18fed	add comment to explain fix	2018-09-04 16:51:36 +02:00
Bruno Windels	dd8b2a79fb	Merge pull request #702 from matrix-org/bwindels/fixreconnectspinner Fix: show spinner again while recovering from connection error	2018-09-04 16:39:38 +02:00
Bruno Windels	f0095611bc	add new CATCHUP state as breaking change	2018-09-04 16:38:46 +02:00
Bruno Windels	a3567f0918	some tests for room.guessDMUserId()	2018-09-04 13:01:45 +02:00
Bruno Windels	f0d3d0d74e	remove unneeded checks, we should always have enough heroes or members	2018-09-04 13:01:15 +02:00
Bruno Windels	3e32bc0d5d	check heroes is not falsy first, this would fail without LL	2018-09-04 13:00:34 +02:00
Bruno Windels	632e4aa120	pick the first member, dont need an array	2018-09-04 13:00:00 +02:00
Bruno Windels	2391ce198d	this method returns userId, not member	2018-09-04 12:59:37 +02:00
David Baker	e5e2bbd482	Merge branch 'master' into develop	2018-09-03 14:11:12 +01:00
Bruno Windels	ca1a1c4f28	Merge pull request #714 from matrix-org/bwindels/memberlist-spinner Add method to query LL state in client	2018-09-03 15:05:13 +02:00
Bruno Windels	3363cc4f1d	shallow-clone the filter, so the timeline filter doesnt get written into it later on	2018-09-03 15:02:38 +02:00
Bruno Windels	f84684982f	add method to query LL state in client	2018-09-03 11:14:23 +02:00
Bruno Windels	3bed5969bf	remove count logging, approach confirmed to work and be according to idb spec	2018-09-03 10:27:00 +02:00
Bruno Windels	ebc162e3d8	do onLeft (which clears the LL members) as late as possible to avoid chance that something might call loadMembersIfNeeded on the room and load them back again.	2018-08-31 16:13:34 +02:00
Bruno Windels	f30136dba3	only clear promise at the end to avoid race between load and clear members	2018-08-31 16:12:54 +02:00
Bruno Windels	9b1926f902	also clear out lazy loaded members from storage	2018-08-31 16:11:37 +02:00
David Baker	d29524ba3f	Merge pull request #707 from matrix-org/bwindels/llinvites Fix: also load invited members when lazy loading members	2018-08-31 14:09:42 +01:00
Bruno Windels	7258fe4e5c	clear out of band members in store when leaving room	2018-08-31 14:42:15 +02:00
Bruno Windels	f8ea1702f8	store support for removing out of band members for a room	2018-08-31 14:42:15 +02:00
David Baker	b8744a79ae	Merge pull request #708 from matrix-org/bwindels/avatarfixbis Fix for "otherMember.getAvatarUrl is not a function"	2018-08-31 10:34:29 +01:00
David Baker	9b54c9b807	Merge pull request #704 from matrix-org/dbkr/discardsession Pass through function to discard megolm session	2018-08-30 18:25:50 +01:00
Bruno Windels	1bb608cdb6	remove filter for LL members so invite members are also sent	2018-08-30 17:29:16 +02:00
Bruno Windels	1239485b30	fix test	2018-08-30 15:42:15 +02:00
Bruno Windels	0d23d047fc	use CATCHUP state after ERROR before going back to SYNCING	2018-08-30 15:37:05 +02:00
Bruno Windels	d837ae64ac	triple =	2018-08-30 15:37:05 +02:00
Bruno Windels	d72a70396a	Pass through PREPARED state after error, when keepalive returns succes. This is according to the state diagram in client.js. This will show a spinner at the bottom of a room again while the catchup sync is in progress, which seems to have broken at some point.	2018-08-30 15:37:05 +02:00
David Baker	8f98504183	jsdoc	2018-08-29 18:11:53 +01:00
David Baker	1b77ee0ef4	Pass through function to discard megolm session To make debugging crypto slightly faster	2018-08-29 18:06:45 +01:00
Will Hunt	fde00b1c62	getMediaLimits -> getMediaConfig	2018-06-23 12:57:29 +01:00
Will Hunt	8dd425f8ff	Media/limits => /config	2018-06-20 17:24:45 +01:00
Will Hunt	9596087959	Remove extra return	2018-05-03 17:57:57 +01:00
Will Hunt	6570402b95	Add getMediaLimits to client	2018-05-03 13:43:37 +01:00