v0.9.2-cryptowraning.1

Prepare changelog for v0.9.2-cryptowraning.1
Merge pull request #631 from matrix-org/dbkr/cryptowarning
2018-03-26 14:11:29 +01:00 · 2018-03-26 14:11:29 +01:00 · 2018-03-26 14:04:42 +01:00 · 2018-03-26 13:57:37 +01:00 · 2018-03-23 17:29:35 +00:00
43 changed files with 1115 additions and 3212 deletions
@@ -1,122 +1,9 @@
-Changes in [0.10.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.2) (2018-04-30)
-==================================================================================================
-[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.2-rc.1...v0.10.2)
+Changes in [0.9.2-cryptowraning.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.9.2-cryptowraning.1) (2018-03-26)
+================================================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.9.2...v0.9.2-cryptowraning.1)

- * No changes from rc.1
-
-Changes in [0.10.2-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.2-rc.1) (2018-04-25)
-============================================================================================================
-[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.1...v0.10.2-rc.1)
-
- * Ignore inserts of dup inbound group sessions, pt 2
-   [\#641](https://github.com/matrix-org/matrix-js-sdk/pull/641)
- * Ignore inserts of duplicate inbound group sessions
-   [\#639](https://github.com/matrix-org/matrix-js-sdk/pull/639)
- * Log IDB errors
-   [\#638](https://github.com/matrix-org/matrix-js-sdk/pull/638)
- * Remove not very useful but veryv spammy log line
-   [\#632](https://github.com/matrix-org/matrix-js-sdk/pull/632)
- * Switch event type to m.sticker.
-   [\#628](https://github.com/matrix-org/matrix-js-sdk/pull/628)
-
-Changes in [0.10.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.1) (2018-04-12)
-==================================================================================================
-[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.0...v0.10.1)
-
- * Log IDB errors
-   [\#638](https://github.com/matrix-org/matrix-js-sdk/pull/638)
- * Ignore inserts of duplicate inbound group sessions
-   [\#639](https://github.com/matrix-org/matrix-js-sdk/pull/639)
- * Ignore inserts of dup inbound group sessions, pt 2
-   [\#641](https://github.com/matrix-org/matrix-js-sdk/pull/641)
-
-Changes in [0.10.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.0) (2018-04-11)
-==================================================================================================
-[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.0-rc.2...v0.10.0)
-
- * No changes
-
-Changes in [0.10.0-rc.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.0-rc.2) (2018-04-09)
-============================================================================================================
-[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.0-rc.1...v0.10.0-rc.2)
-
- * Add wrapper for group join API
- * Add wrapped API to set group join\_policy
-
-Changes in [0.10.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.0-rc.1) (2018-03-19)
-============================================================================================================
-[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.9.2...v0.10.0-rc.1)
-
- * Fix duplicated state events in timeline from peek
-   [\#630](https://github.com/matrix-org/matrix-js-sdk/pull/630)
- * Create indexeddb worker when starting the store
-   [\#627](https://github.com/matrix-org/matrix-js-sdk/pull/627)
- * Fix indexeddb logging
-   [\#626](https://github.com/matrix-org/matrix-js-sdk/pull/626)
- * Don't do /keys/changes on incremental sync
-   [\#625](https://github.com/matrix-org/matrix-js-sdk/pull/625)
- * Don't mark devicelist dirty unnecessarily
-   [\#623](https://github.com/matrix-org/matrix-js-sdk/pull/623)
- * Cache the joined member count for a room state
-   [\#619](https://github.com/matrix-org/matrix-js-sdk/pull/619)
- * Fix JS doc
-   [\#618](https://github.com/matrix-org/matrix-js-sdk/pull/618)
- * Precompute push actions for state events
-   [\#617](https://github.com/matrix-org/matrix-js-sdk/pull/617)
- * Fix bug where global "Never send to unverified..." is ignored
-   [\#616](https://github.com/matrix-org/matrix-js-sdk/pull/616)
- * Intern legacy top-level 'membership' field
-   [\#615](https://github.com/matrix-org/matrix-js-sdk/pull/615)
- * Don't synthesize RR for m.room.redaction as causes the RR to go missing.
-   [\#598](https://github.com/matrix-org/matrix-js-sdk/pull/598)
- * Make Events create Dates on demand
-   [\#613](https://github.com/matrix-org/matrix-js-sdk/pull/613)
- * Stop cloning events when adding to state
-   [\#612](https://github.com/matrix-org/matrix-js-sdk/pull/612)
- * De-dup code: use the initialiseState function
-   [\#611](https://github.com/matrix-org/matrix-js-sdk/pull/611)
- * Create sentinel members on-demand
-   [\#610](https://github.com/matrix-org/matrix-js-sdk/pull/610)
- * Some more doc on how sentinels work
-   [\#609](https://github.com/matrix-org/matrix-js-sdk/pull/609)
- * Migrate room encryption store to crypto store
-   [\#597](https://github.com/matrix-org/matrix-js-sdk/pull/597)
- * add parameter to getIdentityServerUrl to strip the protocol for invites
-   [\#600](https://github.com/matrix-org/matrix-js-sdk/pull/600)
- * Move Device Tracking Data to Crypto Store
-   [\#594](https://github.com/matrix-org/matrix-js-sdk/pull/594)
- * Optimise pushprocessor
-   [\#591](https://github.com/matrix-org/matrix-js-sdk/pull/591)
- * Set event error before emitting
-   [\#592](https://github.com/matrix-org/matrix-js-sdk/pull/592)
- * Add event type for stickers [WIP]
-   [\#590](https://github.com/matrix-org/matrix-js-sdk/pull/590)
- * Migrate inbound sessions to cryptostore
-   [\#587](https://github.com/matrix-org/matrix-js-sdk/pull/587)
- * Disambiguate names if they contain an mxid
-   [\#588](https://github.com/matrix-org/matrix-js-sdk/pull/588)
- * Check for sessions in indexeddb before migrating
-   [\#585](https://github.com/matrix-org/matrix-js-sdk/pull/585)
- * Emit an event for crypto store migration
-   [\#586](https://github.com/matrix-org/matrix-js-sdk/pull/586)
- * Supporting fixes For making UnknownDeviceDialog not pop up automatically
-   [\#575](https://github.com/matrix-org/matrix-js-sdk/pull/575)
- * Move sessions to the crypto store
-   [\#584](https://github.com/matrix-org/matrix-js-sdk/pull/584)
- * Change crypto store transaction API
-   [\#582](https://github.com/matrix-org/matrix-js-sdk/pull/582)
- * Add some missed copyright notices
-   [\#581](https://github.com/matrix-org/matrix-js-sdk/pull/581)
- * Move Olm account to IndexedDB
-   [\#579](https://github.com/matrix-org/matrix-js-sdk/pull/579)
- * Fix logging of DecryptionErrors to be more useful
-   [\#580](https://github.com/matrix-org/matrix-js-sdk/pull/580)
- * [BREAKING] Change the behaviour of the unverfied devices blacklist flag
-   [\#568](https://github.com/matrix-org/matrix-js-sdk/pull/568)
- * Support set_presence=offline for syncing
-   [\#557](https://github.com/matrix-org/matrix-js-sdk/pull/557)
- * Consider cases where the sender may not redact their own event
-   [\#556](https://github.com/matrix-org/matrix-js-sdk/pull/556)
+ * Disable crypto if indexeddb version too new
+   [\#631](https://github.com/matrix-org/matrix-js-sdk/pull/631)

 Changes in [0.9.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.9.2) (2017-12-04)
 ================================================================================================
@@ -1,6 +1,6 @@
 {
  "name": "matrix-js-sdk",
-  "version": "0.10.2",
+  "version": "0.9.2-cryptowraning.1",
  "description": "Matrix Client-Server SDK for Javascript",
  "main": "index.js",
  "scripts": {
@@ -15,7 +15,7 @@
    "build": "babel -s -d lib src && rimraf dist && mkdir dist && browserify -d browser-index.js | exorcist dist/browser-matrix.js.map > dist/browser-matrix.js && uglifyjs -c -m -o dist/browser-matrix.min.js --source-map dist/browser-matrix.min.js.map --in-source-map dist/browser-matrix.js.map dist/browser-matrix.js",
    "dist": "npm run build",
    "watch": "watchify -d browser-index.js -o 'exorcist dist/browser-matrix.js.map > dist/browser-matrix.js' -v",
-    "lint": "eslint --max-warnings 102 src spec",
+    "lint": "eslint --max-warnings 109 src spec",
    "prepublish": "npm run clean && npm run build && git rev-parse HEAD > git-revision.txt"
  },
  "repository": {
@@ -1,7 +1,6 @@
 /*
 Copyright 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -26,7 +25,6 @@ import testUtils from './test-utils';
 import MockHttpBackend from 'matrix-mock-request';
 import expect from 'expect';
 import Promise from 'bluebird';
-import LocalStorageCryptoStore from '../lib/crypto/store/localStorage-crypto-store';

 /**
 * Wrapper for a MockStorageApi, MockHttpBackend and MatrixClient
@@ -48,19 +46,14 @@ export default function TestClient(
    if (sessionStoreBackend === undefined) {
        sessionStoreBackend = new testUtils.MockStorageApi();
    }
-    const sessionStore = new sdk.WebStorageSessionStore(sessionStoreBackend);
-
-    // expose this so the tests can get to it
-    this.cryptoStore = new LocalStorageCryptoStore(sessionStoreBackend);
-
+    this.storage = new sdk.WebStorageSessionStore(sessionStoreBackend);
    this.httpBackend = new MockHttpBackend();
    this.client = sdk.createClient({
        baseUrl: "http://" + userId + ".test.server",
        userId: userId,
        accessToken: accessToken,
        deviceId: deviceId,
-        sessionStore: sessionStore,
-        cryptoStore: this.cryptoStore,
+        sessionStore: this.storage,
        request: this.httpBackend.requestFn,
    });

@@ -1,20 +1,3 @@
-/*
-Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
 import expect from 'expect';
 import Promise from 'bluebird';

@@ -168,12 +151,9 @@ describe("DeviceList management:", function() {
                   aliceTestClient.httpBackend.flush('/keys/query', 1).then(
                       () => aliceTestClient.httpBackend.flush('/send/', 1),
                   ),
-                   aliceTestClient.client._crypto._deviceList.saveIfDirty(),
               ]);
           }).then(() => {
-               aliceTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-                   expect(data.syncToken).toEqual(1);
-               });
+               expect(aliceTestClient.storage.getEndToEndDeviceSyncToken()).toEqual(1);

               // invalidate bob's and chris's device lists in separate syncs
               aliceTestClient.httpBackend.when('GET', '/sync').respond(200, {
@@ -205,21 +185,19 @@ describe("DeviceList management:", function() {
               return aliceTestClient.httpBackend.flush('/keys/query', 1);
           }).then((flushed) => {
               expect(flushed).toEqual(0);
-               return aliceTestClient.client._crypto._deviceList.saveIfDirty();
-           }).then(() => {
-               aliceTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-                   const bobStat = data.trackingStatus['@bob:xyz'];
-                   if (bobStat != 1 && bobStat != 2) {
-                       throw new Error('Unexpected status for bob: wanted 1 or 2, got ' +
-                                       bobStat);
-                   }
-                   const chrisStat = data.trackingStatus['@chris:abc'];
-                   if (chrisStat != 1 && chrisStat != 2) {
-                       throw new Error(
-                           'Unexpected status for chris: wanted 1 or 2, got ' + chrisStat,
-                       );
-                   }
-               });
+               const bobStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+               if (bobStat != 1 && bobStat != 2) {
+                   throw new Error('Unexpected status for bob: wanted 1 or 2, got ' +
+                                   bobStat);
+               }
+
+               const chrisStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];
+               if (chrisStat != 1 && chrisStat != 2) {
+                   throw new Error('Unexpected status for chris: wanted 1 or 2, got ' +
+                                   chrisStat);
+               }

               // now add an expectation for a query for bob's devices, and let
               // it complete.
@@ -238,18 +216,15 @@ describe("DeviceList management:", function() {
               // wait for the client to stop processing the response
               return aliceTestClient.client.downloadKeys(['@bob:xyz']);
           }).then(() => {
-               return aliceTestClient.client._crypto._deviceList.saveIfDirty();
-           }).then(() => {
-               aliceTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-                   const bobStat = data.trackingStatus['@bob:xyz'];
-                   expect(bobStat).toEqual(3);
-                   const chrisStat = data.trackingStatus['@chris:abc'];
-                   if (chrisStat != 1 && chrisStat != 2) {
-                       throw new Error(
-                           'Unexpected status for chris: wanted 1 or 2, got ' + bobStat,
-                       );
-                   }
-               });
+               const bobStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+               expect(bobStat).toEqual(3);
+               const chrisStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];
+               if (chrisStat != 1 && chrisStat != 2) {
+                   throw new Error('Unexpected status for chris: wanted 1 or 2, got ' +
+                                   bobStat);
+               }

               // now let the query for chris's devices complete.
               return aliceTestClient.httpBackend.flush('/keys/query', 1);
@@ -259,18 +234,16 @@ describe("DeviceList management:", function() {
               // wait for the client to stop processing the response
               return aliceTestClient.client.downloadKeys(['@chris:abc']);
           }).then(() => {
-               return aliceTestClient.client._crypto._deviceList.saveIfDirty();
-           }).then(() => {
-               aliceTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-                   const bobStat = data.trackingStatus['@bob:xyz'];
-                   const chrisStat = data.trackingStatus['@bob:xyz'];
+               const bobStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+               const chrisStat = aliceTestClient.storage
+                     .getEndToEndDeviceTrackingStatus()['@chris:abc'];

-                   expect(bobStat).toEqual(3);
-                   expect(chrisStat).toEqual(3);
-                   expect(data.syncToken).toEqual(3);
-               });
+               expect(bobStat).toEqual(3);
+               expect(chrisStat).toEqual(3);
+               expect(aliceTestClient.storage.getEndToEndDeviceSyncToken()).toEqual(3);
           });
-       }).timeout(3000);
+       });

    // https://github.com/vector-im/riot-web/issues/4983
    describe("Alice should know she has stale device lists", () => {
@@ -289,15 +262,13 @@ describe("DeviceList management:", function() {
                },
            );
            await aliceTestClient.httpBackend.flush('/keys/query', 1);
-            await aliceTestClient.client._crypto._deviceList.saveIfDirty();

-            aliceTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-                const bobStat = data.trackingStatus['@bob:xyz'];
+            const bobStat = aliceTestClient.storage
+                      .getEndToEndDeviceTrackingStatus()['@bob:xyz'];

-                expect(bobStat).toBeGreaterThan(
-                    0, "Alice should be tracking bob's device list",
-                );
-            });
+            expect(bobStat).toBeGreaterThan(
+                0, "Alice should be tracking bob's device list",
+            );
        });

        it("when Bob leaves", async function() {
@@ -326,15 +297,12 @@ describe("DeviceList management:", function() {


            await aliceTestClient.flushSync();
-            await aliceTestClient.client._crypto._deviceList.saveIfDirty();

-            aliceTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-                const bobStat = data.trackingStatus['@bob:xyz'];
-
-                expect(bobStat).toEqual(
-                    0, "Alice should have marked bob's device list as untracked",
-                );
-            });
+            const bobStat = aliceTestClient.storage
+                      .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+            expect(bobStat).toEqual(
+                0, "Alice should have marked bob's device list as untracked",
+            );
        });

        it("when Alice leaves", async function() {
@@ -362,15 +330,12 @@ describe("DeviceList management:", function() {
            );

            await aliceTestClient.flushSync();
-            await aliceTestClient.client._crypto._deviceList.saveIfDirty();

-            aliceTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-                const bobStat = data.trackingStatus['@bob:xyz'];
-
-                expect(bobStat).toEqual(
-                    0, "Alice should have marked bob's device list as untracked",
-                );
-            });
+            const bobStat = aliceTestClient.storage
+                      .getEndToEndDeviceTrackingStatus()['@bob:xyz'];
+            expect(bobStat).toEqual(
+                0, "Alice should have marked bob's device list as untracked",
+            );
        });

        it("when Bob leaves whilst Alice is offline", async function() {
@@ -379,19 +344,23 @@ describe("DeviceList management:", function() {
            const anotherTestClient = await createTestClient();

            try {
+                anotherTestClient.httpBackend.when('GET', '/keys/changes').respond(
+                    200, {
+                        changed: [],
+                        left: ['@bob:xyz'],
+                    },
+                );
                await anotherTestClient.start();
                anotherTestClient.httpBackend.when('GET', '/sync').respond(
                    200, getSyncResponse([]));
                await anotherTestClient.flushSync();
-                await anotherTestClient.client._crypto._deviceList.saveIfDirty();

-                anotherTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-                    const bobStat = data.trackingStatus['@bob:xyz'];
+                const bobStat = anotherTestClient.storage
+                      .getEndToEndDeviceTrackingStatus()['@bob:xyz'];

-                    expect(bobStat).toEqual(
-                        0, "Alice should have marked bob's device list as untracked",
-                    );
-                });
+                expect(bobStat).toEqual(
+                    0, "Alice should have marked bob's device list as untracked",
+                );
            } finally {
                anotherTestClient.stop();
            }
@@ -1,7 +1,6 @@
 /*
 Copyright 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -155,15 +154,11 @@ function aliDownloadsKeys() {

    // check that the localStorage is updated as we expect (not sure this is
    // an integration test, but meh)
-    return Promise.all([p1, p2]).then(() => {
-        return aliTestClient.client._crypto._deviceList.saveIfDirty();
-    }).then(() => {
-        aliTestClient.cryptoStore.getEndToEndDeviceData(null, (data) => {
-            const devices = data.devices[bobUserId];
-            expect(devices[bobDeviceId].keys).toEqual(bobTestClient.deviceKeys.keys);
-            expect(devices[bobDeviceId].verified).
-                toBe(0); // DeviceVerification.UNVERIFIED
-        });
+    return Promise.all([p1, p2]).then(function() {
+        const devices = aliTestClient.storage.getEndToEndDevicesForUser(bobUserId);
+        expect(devices[bobDeviceId].keys).toEqual(bobTestClient.deviceKeys.keys);
+        expect(devices[bobDeviceId].verified).
+            toBe(0); // DeviceVerification.UNVERIFIED
    });
 }

@@ -437,34 +437,6 @@ describe("MatrixClient syncing", function() {
            });
        });

-        // XXX: This test asserts that the js-sdk obeys the spec and treats state
-        // events that arrive in the incremental sync as if they preceeded the
-        // timeline events, however this breaks peeking, so it's disabled
-        // (see sync.js)
-        xit("should correctly interpret state in incremental sync.", function() {
-            httpBackend.when("GET", "/sync").respond(200, syncData);
-            httpBackend.when("GET", "/sync").respond(200, nextSyncData);
-
-            client.startClient();
-            return Promise.all([
-                httpBackend.flushAllExpected(),
-                awaitSyncEvent(2),
-            ]).then(function() {
-                const room = client.getRoom(roomOne);
-                const stateAtStart = room.getLiveTimeline().getState(
-                    EventTimeline.BACKWARDS,
-                );
-                const startRoomNameEvent = stateAtStart.getStateEvents('m.room.name', '');
-                expect(startRoomNameEvent.getContent().name).toEqual('Old room name');
-
-                const stateAtEnd = room.getLiveTimeline().getState(
-                    EventTimeline.FORWARDS,
-                );
-                const endRoomNameEvent = stateAtEnd.getStateEvents('m.room.name', '');
-                expect(endRoomNameEvent.getContent().name).toEqual('A new room name');
-            });
-        });
-
        xit("should update power levels for users in a room", function() {

        });
@@ -1,24 +1,6 @@
-/*
-Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
 import DeviceList from '../../../lib/crypto/DeviceList';
 import MockStorageApi from '../../MockStorageApi';
 import WebStorageSessionStore from '../../../lib/store/session/webstorage';
-import MemoryCryptoStore from '../../../lib/crypto/store/memory-crypto-store.js';
 import testUtils from '../../test-utils';
 import utils from '../../../lib/utils';

@@ -58,7 +40,6 @@ const signedDeviceList = {
 describe('DeviceList', function() {
    let downloadSpy;
    let sessionStore;
-    let cryptoStore;

    beforeEach(function() {
        testUtils.beforeEach(this); // eslint-disable-line no-invalid-this
@@ -66,7 +47,6 @@ describe('DeviceList', function() {
        downloadSpy = expect.createSpy();
        const mockStorage = new MockStorageApi();
        sessionStore = new WebStorageSessionStore(mockStorage);
-        cryptoStore = new MemoryCryptoStore();
    });

    function createTestDeviceList() {
@@ -76,7 +56,7 @@ describe('DeviceList', function() {
        const mockOlm = {
            verifySignature: function(key, message, signature) {},
        };
-        return new DeviceList(baseApis, cryptoStore, sessionStore, mockOlm);
+        return new DeviceList(baseApis, sessionStore, mockOlm);
    }

    it("should successfully download and store device keys", function() {
@@ -92,7 +72,7 @@ describe('DeviceList', function() {
        queryDefer1.resolve(utils.deepCopy(signedDeviceList));

        return prom1.then(() => {
-            const storedKeys = dl.getRawStoredDevicesForUser('@test1:sw1v.org');
+            const storedKeys = sessionStore.getEndToEndDevicesForUser('@test1:sw1v.org');
            expect(Object.keys(storedKeys)).toEqual(['HGKAWHRVJQ']);
        });
    });
@@ -117,15 +97,14 @@ describe('DeviceList', function() {
        dl.invalidateUserDeviceList('@test1:sw1v.org');
        dl.refreshOutdatedDeviceLists();

-        dl.saveIfDirty().then(() => {
-            // the first request completes
-            queryDefer1.resolve({
-                device_keys: {
-                    '@test1:sw1v.org': {},
-                },
-            });
-            return prom1;
-        }).then(() => {
+        // the first request completes
+        queryDefer1.resolve({
+            device_keys: {
+                '@test1:sw1v.org': {},
+            },
+        });
+
+        return prom1.then(() => {
            // uh-oh; user restarts before second request completes. The new instance
            // should know we never got a complete device list.
            console.log("Creating new devicelist to simulate app reload");
@@ -142,7 +121,7 @@ describe('DeviceList', function() {
            // allow promise chain to complete
            return prom3;
        }).then(() => {
-            const storedKeys = dl.getRawStoredDevicesForUser('@test1:sw1v.org');
+            const storedKeys = sessionStore.getEndToEndDevicesForUser('@test1:sw1v.org');
            expect(Object.keys(storedKeys)).toEqual(['HGKAWHRVJQ']);
        });
    });
@@ -10,7 +10,6 @@ import Promise from 'bluebird';
 import sdk from '../../../..';
 import algorithms from '../../../../lib/crypto/algorithms';
 import WebStorageSessionStore from '../../../../lib/store/session/webstorage';
-import MemoryCryptoStore from '../../../../lib/crypto/store/memory-crypto-store.js';
 import MockStorageApi from '../../../MockStorageApi';
 import testUtils from '../../../test-utils';

@@ -46,9 +45,8 @@ describe("MegolmDecryption", function() {

        const mockStorage = new MockStorageApi();
        const sessionStore = new WebStorageSessionStore(mockStorage);
-        const cryptoStore = new MemoryCryptoStore(mockStorage);

-        const olmDevice = new OlmDevice(sessionStore, cryptoStore);
+        const olmDevice = new OlmDevice(sessionStore);

        megolmDecryption = new MegolmDecryption({
            userId: '@user:id',
@@ -132,12 +132,11 @@ describe("MatrixClient", function() {
        ].reduce((r, k) => { r[k] = expect.createSpy(); return r; }, {});
        store = [
            "getRoom", "getRooms", "getUser", "getSyncToken", "scrollback",
-            "save", "wantsSave", "setSyncToken", "storeEvents", "storeRoom", "storeUser",
+            "save", "setSyncToken", "storeEvents", "storeRoom", "storeUser",
            "getFilterIdByName", "setFilterIdByName", "getFilter", "storeFilter",
            "getSyncAccumulator", "startup", "deleteAllData",
        ].reduce((r, k) => { r[k] = expect.createSpy(); return r; }, {});
        store.getSavedSync = expect.createSpy().andReturn(Promise.resolve(null));
-        store.getSavedSyncToken = expect.createSpy().andReturn(Promise.resolve(null));
        store.setSyncData = expect.createSpy().andReturn(Promise.resolve(null));
        client = new MatrixClient({
            baseUrl: "https://my.home.server",
@@ -24,9 +24,6 @@ describe('NotificationService', function() {
                            name: testDisplayName,
                        };
                    },
-                    getJoinedMemberCount: function() {
-                        return 0;
-                    },
                    members: {},
                },
            };
@@ -383,25 +383,22 @@ describe("Room", function() {
    });

    const resetTimelineTests = function(timelineSupport) {
-        let events = null;
+        const events = [
+            utils.mkMessage({
+                room: roomId, user: userA, msg: "A message", event: true,
+            }),
+            utils.mkEvent({
+                type: "m.room.name", room: roomId, user: userA, event: true,
+                content: { name: "New Room Name" },
+            }),
+            utils.mkEvent({
+                type: "m.room.name", room: roomId, user: userA, event: true,
+                content: { name: "Another New Name" },
+            }),
+        ];

        beforeEach(function() {
            room = new Room(roomId, {timelineSupport: timelineSupport});
-            // set events each time to avoid resusing Event objects (which
-            // doesn't work because they get frozen)
-            events = [
-                utils.mkMessage({
-                    room: roomId, user: userA, msg: "A message", event: true,
-                }),
-                utils.mkEvent({
-                    type: "m.room.name", room: roomId, user: userA, event: true,
-                    content: { name: "New Room Name" },
-                }),
-                utils.mkEvent({
-                    type: "m.room.name", room: roomId, user: userA, event: true,
-                    content: { name: "Another New Name" },
-                }),
-            ];
        });

        it("should copy state from previous timeline", function() {
@@ -89,14 +89,9 @@ MatrixBaseApis.prototype.getHomeserverUrl = function() {

 /**
 * Get the Identity Server URL of this client
- * @param {boolean} stripProto whether or not to strip the protocol from the URL
 * @return {string} Identity Server URL of this client
 */
-MatrixBaseApis.prototype.getIdentityServerUrl = function(stripProto=false) {
-    if (stripProto && (this.idBaseUrl.startsWith("http://") ||
-            this.idBaseUrl.startsWith("https://"))) {
-        return this.idBaseUrl.split("://")[1];
-    }
+MatrixBaseApis.prototype.getIdentityServerUrl = function() {
    return this.idBaseUrl;
 };

@@ -443,27 +438,6 @@ MatrixBaseApis.prototype.setGroupProfile = function(groupId, profile) {
    );
 };

-/**
- * @param {string} groupId
- * @param {object} policy The join policy for the group. Must include at
- *     least a 'type' field which is 'open' if anyone can join the group
- *     the group without prior approval, or 'invite' if an invite is
- *     required to join.
- * @return {module:client.Promise} Resolves: Empty object
- * @return {module:http-api.MatrixError} Rejects: with an error response.
- */
-MatrixBaseApis.prototype.setGroupJoinPolicy = function(groupId, policy) {
-    const path = utils.encodeUri(
-        "/groups/$groupId/settings/m.join_policy",
-        {$groupId: groupId},
-    );
-    return this._http.authedRequest(
-        undefined, "PUT", path, undefined, {
-            'm.join_policy': policy,
-        },
-    );
-};
-
 /**
 * @param {string} groupId
 * @return {module:client.Promise} Resolves: Group users list object
@@ -642,28 +616,14 @@ MatrixBaseApis.prototype.removeRoomFromGroup = function(groupId, roomId) {

 /**
 * @param {string} groupId
- * @param {Object} opts Additional options to send alongside the acceptance.
 * @return {module:client.Promise} Resolves: Empty object
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
-MatrixBaseApis.prototype.acceptGroupInvite = function(groupId, opts = null) {
+MatrixBaseApis.prototype.acceptGroupInvite = function(groupId) {
    const path = utils.encodeUri(
        "/groups/$groupId/self/accept_invite",
        {$groupId: groupId},
    );
-    return this._http.authedRequest(undefined, "PUT", path, undefined, opts || {});
-};
-
-/**
- * @param {string} groupId
- * @return {module:client.Promise} Resolves: Empty object
- * @return {module:http-api.MatrixError} Rejects: with an error response.
- */
-MatrixBaseApis.prototype.joinGroup = function(groupId) {
-    const path = utils.encodeUri(
-        "/groups/$groupId/self/join",
-        {$groupId: groupId},
-    );
    return this._http.authedRequest(undefined, "PUT", path, undefined, {});
 };

@@ -1269,27 +1229,6 @@ MatrixBaseApis.prototype.deleteDevice = function(device_id, auth) {
    );
 };

-/**
- * Delete multiple device
- *
- * @param {string[]} devices IDs of the devices to delete
- * @param {object} auth Optional. Auth data to supply for User-Interactive auth.
- * @return {module:client.Promise} Resolves: result object
- * @return {module:http-api.MatrixError} Rejects: with an error response.
- */
-MatrixBaseApis.prototype.deleteMultipleDevices = function(devices, auth) {
-    const body = {devices};
-
-    if (auth) {
-        body.auth = auth;
-    }
-
-    return this._http.authedRequestWithPrefix(
-        undefined, "POST", "/delete_devices", undefined, body,
-        httpApi.PREFIX_UNSTABLE,
-    );
-};
-

 // Push operations
 // ===============
@@ -1,7 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -40,10 +39,8 @@ const Filter = require("./filter");
 const SyncApi = require("./sync");
 const MatrixBaseApis = require("./base-apis");
 const MatrixError = httpApi.MatrixError;
-const ContentHelpers = require("./content-helpers");

 import ReEmitter from './ReEmitter';
-import RoomList from './crypto/RoomList';

 const SCROLLBACK_DELAY_MS = 3000;
 let CRYPTO_ENABLED = false;
@@ -183,14 +180,6 @@ function MatrixClient(opts) {
    if (CRYPTO_ENABLED) {
        this.olmVersion = Crypto.getOlmVersion();
    }
-
-    // List of which rooms have encryption enabled: separate from crypto because
-    // we still want to know which rooms are encrypted even if crypto is disabled:
-    // we don't want to start sending unencrypted events to them.
-    this._roomList = new RoomList(this._cryptoStore, this._sessionStore);
-
-    // The pushprocessor caches useful things, so keep one and re-use it
-    this._pushProcessor = new PushProcessor(this);
 }
 utils.inherits(MatrixClient, EventEmitter);
 utils.extend(MatrixClient.prototype, MatrixBaseApis.prototype);
@@ -361,6 +350,13 @@ MatrixClient.prototype.initCrypto = async function() {
        return;
    }

+    if (!CRYPTO_ENABLED) {
+        throw new Error(
+            `End-to-end encryption not supported in this js-sdk build: did ` +
+                `you remember to load the olm library?`,
+        );
+    }
+
    if (!this._sessionStore) {
        // this is temporary, the sessionstore is supposed to be going away
        throw new Error(`Cannot enable encryption: no sessionStore provided`);
@@ -370,16 +366,6 @@ MatrixClient.prototype.initCrypto = async function() {
        throw new Error(`Cannot enable encryption: no cryptoStore provided`);
    }

-    // initialise the list of encrypted rooms (whether or not crypto is enabled)
-    await this._roomList.init();
-
-    if (!CRYPTO_ENABLED) {
-        throw new Error(
-            `End-to-end encryption not supported in this js-sdk build: did ` +
-                `you remember to load the olm library?`,
-        );
-    }
-
    const userId = this.getUserId();
    if (userId === null) {
        throw new Error(
@@ -400,13 +386,11 @@ MatrixClient.prototype.initCrypto = async function() {
        userId, this.deviceId,
        this.store,
        this._cryptoStore,
-        this._roomList,
    );

    this.reEmitter.reEmit(crypto, [
        "crypto.roomKeyRequest",
        "crypto.roomKeyRequestCancellation",
-        "crypto.warning",
    ]);

    await crypto.init();
@@ -571,10 +555,11 @@ async function _setDeviceVerification(

 /**
 * Set the global override for whether the client should ever send encrypted
- * messages to unverified devices.  This provides the default for rooms which
- * do not specify a value.
+ * messages to unverified devices.  If false, it can still be overridden
+ * per-room.  If true, it overrides the per-room settings.
 *
- * @param {boolean} value whether to blacklist all unverified devices by default
+ * @param {boolean} value whether to unilaterally blacklist all
+ * unverified devices
 */
 MatrixClient.prototype.setGlobalBlacklistUnverifiedDevices = function(value) {
    if (this._crypto === null) {
@@ -584,7 +569,8 @@ MatrixClient.prototype.setGlobalBlacklistUnverifiedDevices = function(value) {
 };

 /**
- * @return {boolean} whether to blacklist all unverified devices by default
+ * @return {boolean} whether to unilaterally blacklist all
+ * unverified devices
 */
 MatrixClient.prototype.getGlobalBlacklistUnverifiedDevices = function() {
    if (this._crypto === null) {
@@ -624,16 +610,6 @@ MatrixClient.prototype.isEventSenderVerified = async function(event) {
    return device.isVerified();
 };

-/**
- * Cancel a room key request for this event if one is ongoing and resend the
- * request.
- * @param  {MatrxEvent} event event of which to cancel and resend the room
- *                            key request.
- */
-MatrixClient.prototype.cancelAndResendEventRoomKeyRequest = function(event) {
-    event.cancelAndResendKeyRequest(this._crypto);
-};
-
 /**
 * Enable end-to-end encryption for a room.
 * @param {string} roomId The room ID to enable encryption in.
@@ -670,7 +646,11 @@ MatrixClient.prototype.isRoomEncrypted = function(roomId) {
    // we don't have an m.room.encrypted event, but that might be because
    // the server is hiding it from us. Check the store to see if it was
    // previously encrypted.
-    return this._roomList.isRoomEncrypted(roomId);
+    if (!this._sessionStore) {
+        return false;
+    }
+
+    return Boolean(this._sessionStore.getEndToEndRoom(roomId));
 };

 /**
@@ -792,6 +772,7 @@ MatrixClient.prototype.setAccountData = function(eventType, contents, callback)
 /**
 * Get account data event of given type for the current user.
 * @param {string} eventType The event type
+ * @param {module:client.callback} callback Optional.
 * @return {?object} The contents of the given account data event
 */
 MatrixClient.prototype.getAccountData = function(eventType) {
@@ -1156,14 +1137,8 @@ function _sendEvent(client, room, event, callback) {
        console.error("Error sending event", err.stack || err);

        try {
-            // set the error on the event before we update the status:
-            // updating the status emits the event, so the state should be
-            // consistent at that point.
-            event.error = err;
            _updatePendingEventStatus(room, event, EventStatus.NOT_SENT);
-            // also put the event object on the error: the caller will need this
-            // to resend or cancel the event
-            err.event = event;
+            event.error = err;

            if (callback) {
                callback(err);
@@ -1280,7 +1255,10 @@ MatrixClient.prototype.sendMessage = function(roomId, content, txnId, callback)
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixClient.prototype.sendTextMessage = function(roomId, body, txnId, callback) {
-    const content = ContentHelpers.makeTextMessage(body);
+    const content = {
+         msgtype: "m.text",
+         body: body,
+    };
    return this.sendMessage(roomId, content, txnId, callback);
 };

@@ -1293,7 +1271,10 @@ MatrixClient.prototype.sendTextMessage = function(roomId, body, txnId, callback)
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixClient.prototype.sendNotice = function(roomId, body, txnId, callback) {
-    const content = ContentHelpers.makeNotice(body);
+    const content = {
+         msgtype: "m.notice",
+         body: body,
+    };
    return this.sendMessage(roomId, content, txnId, callback);
 };

@@ -1306,7 +1287,10 @@ MatrixClient.prototype.sendNotice = function(roomId, body, txnId, callback) {
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixClient.prototype.sendEmoteMessage = function(roomId, body, txnId, callback) {
-    const content = ContentHelpers.makeEmoteMessage(body);
+    const content = {
+         msgtype: "m.emote",
+         body: body,
+    };
    return this.sendMessage(roomId, content, txnId, callback);
 };

@@ -1335,32 +1319,6 @@ MatrixClient.prototype.sendImageMessage = function(roomId, url, info, text, call
    return this.sendMessage(roomId, content, callback);
 };

-/**
- * @param {string} roomId
- * @param {string} url
- * @param {Object} info
- * @param {string} text
- * @param {module:client.callback} callback Optional.
- * @return {module:client.Promise} Resolves: TODO
- * @return {module:http-api.MatrixError} Rejects: with an error response.
- */
-MatrixClient.prototype.sendStickerMessage = function(roomId, url, info, text, callback) {
-    if (utils.isFunction(text)) {
-        callback = text; text = undefined;
-    }
-    if (!text) {
-        text = "Sticker";
-    }
-    const content = {
-         url: url,
-         info: info,
-         body: text,
-    };
-    return this.sendEvent(
-        roomId, "m.sticker", content, callback, undefined,
-    );
-};
-
 /**
 * @param {string} roomId
 * @param {string} body
@@ -1370,7 +1328,12 @@ MatrixClient.prototype.sendStickerMessage = function(roomId, url, info, text, ca
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixClient.prototype.sendHtmlMessage = function(roomId, body, htmlBody, callback) {
-    const content = ContentHelpers.makeHtmlMessage(body, htmlBody);
+    const content = {
+        msgtype: "m.text",
+        format: "org.matrix.custom.html",
+        body: body,
+        formatted_body: htmlBody,
+    };
    return this.sendMessage(roomId, content, callback);
 };

@@ -1383,7 +1346,12 @@ MatrixClient.prototype.sendHtmlMessage = function(roomId, body, htmlBody, callba
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixClient.prototype.sendHtmlNotice = function(roomId, body, htmlBody, callback) {
-    const content = ContentHelpers.makeHtmlNotice(body, htmlBody);
+    const content = {
+        msgtype: "m.notice",
+        format: "org.matrix.custom.html",
+        body: body,
+        formatted_body: htmlBody,
+    };
    return this.sendMessage(roomId, content, callback);
 };

@@ -1396,7 +1364,12 @@ MatrixClient.prototype.sendHtmlNotice = function(roomId, body, htmlBody, callbac
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixClient.prototype.sendHtmlEmote = function(roomId, body, htmlBody, callback) {
-    const content = ContentHelpers.makeHtmlEmote(body, htmlBody);
+    const content = {
+        msgtype: "m.emote",
+        format: "org.matrix.custom.html",
+        body: body,
+        formatted_body: htmlBody,
+    };
    return this.sendMessage(roomId, content, callback);
 };

@@ -1572,13 +1545,18 @@ MatrixClient.prototype.inviteByThreePid = function(roomId, medium, address, call
        { $roomId: roomId },
    );

-    const identityServerUrl = this.getIdentityServerUrl(true);
+    let identityServerUrl = this.getIdentityServerUrl();
    if (!identityServerUrl) {
        return Promise.reject(new MatrixError({
            error: "No supplied identity server URL",
            errcode: "ORG.MATRIX.JSSDK_MISSING_PARAM",
        }));
    }
+    if (identityServerUrl.indexOf("http://") === 0 ||
+            identityServerUrl.indexOf("https://") === 0) {
+        // this request must not have the protocol part because reasons
+        identityServerUrl = identityServerUrl.split("://")[1];
+    }

    return this._http.authedRequest(callback, "POST", path, undefined, {
        id_server: identityServerUrl,
@@ -1738,7 +1716,8 @@ function _membershipChange(client, roomId, userId, membership, reason, callback)
 */
 MatrixClient.prototype.getPushActionsForEvent = function(event) {
    if (!event.getPushActions()) {
-        event.setPushActions(this._pushProcessor.actionsForEvent(event));
+        const pushProcessor = new PushProcessor(this);
+        event.setPushActions(pushProcessor.actionsForEvent(event));
    }
    return event.getPushActions();
 };
@@ -3016,9 +2995,6 @@ MatrixClient.prototype.getTurnServers = function() {
 *
 * @param {Filter=} opts.filter The filter to apply to /sync calls. This will override
 * the opts.initialSyncLimit, which would normally result in a timeline limit filter.
- *
- * @param {Boolean=} opts.disablePresence True to perform syncing without automatically
- * updating presence.
 */
 MatrixClient.prototype.startClient = function(opts) {
    if (this.clientRunning) {
@@ -3133,7 +3109,10 @@ function setupCallEventHandler(client) {
            // now loop through the buffer chronologically and inject them
            callEventBuffer.forEach(function(e) {
                if (ignoreCallIds[e.getContent().call_id]) {
-                    // This call has previously been ansered or hung up: ignore it
+                    console.log(
+                        'Ignoring previously answered/hungup call ' +
+                            e.getContent().call_id,
+                    );
                    return;
                }
                callEventHandler(e);
@@ -1,100 +0,0 @@
-/*
-Copyright 2018 New Vector Ltd
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-"use strict";
-
-/** @module ContentHelpers */
-module.exports = {
-    /**
-     * Generates the content for a HTML Message event
-     * @param {string} body the plaintext body of the message
-     * @param {string} htmlBody the HTML representation of the message
-     * @returns {{msgtype: string, format: string, body: string, formatted_body: string}}
-     */
-    makeHtmlMessage: function(body, htmlBody) {
-        return {
-            msgtype: "m.text",
-            format: "org.matrix.custom.html",
-            body: body,
-            formatted_body: htmlBody,
-        };
-    },
-
-    /**
-     * Generates the content for a HTML Notice event
-     * @param {string} body the plaintext body of the notice
-     * @param {string} htmlBody the HTML representation of the notice
-     * @returns {{msgtype: string, format: string, body: string, formatted_body: string}}
-     */
-    makeHtmlNotice: function(body, htmlBody) {
-        return {
-            msgtype: "m.notice",
-            format: "org.matrix.custom.html",
-            body: body,
-            formatted_body: htmlBody,
-        };
-    },
-
-    /**
-     * Generates the content for a HTML Emote event
-     * @param {string} body the plaintext body of the emote
-     * @param {string} htmlBody the HTML representation of the emote
-     * @returns {{msgtype: string, format: string, body: string, formatted_body: string}}
-     */
-    makeHtmlEmote: function(body, htmlBody) {
-        return {
-            msgtype: "m.emote",
-            format: "org.matrix.custom.html",
-            body: body,
-            formatted_body: htmlBody,
-        };
-    },
-
-    /**
-     * Generates the content for a Plaintext Message event
-     * @param {string} body the plaintext body of the emote
-     * @returns {{msgtype: string, body: string}}
-     */
-    makeTextMessage: function(body) {
-        return {
-            msgtype: "m.text",
-            body: body,
-        };
-    },
-
-    /**
-     * Generates the content for a Plaintext Notice event
-     * @param {string} body the plaintext body of the notice
-     * @returns {{msgtype: string, body: string}}
-     */
-    makeNotice: function(body) {
-        return {
-            msgtype: "m.notice",
-            body: body,
-        };
-    },
-
-    /**
-     * Generates the content for a Plaintext Emote event
-     * @param {string} body the plaintext body of the emote
-     * @returns {{msgtype: string, body: string}}
-     */
-    makeEmoteMessage: function(body) {
-        return {
-            msgtype: "m.emote",
-            body: body,
-        };
-    },
-};
@@ -1,6 +1,5 @@
 /*
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -26,7 +25,6 @@ import Promise from 'bluebird';

 import DeviceInfo from './deviceinfo';
 import olmlib from './olmlib';
-import IndexedDBCryptoStore from './store/indexeddb-crypto-store';


 /* State transition diagram for DeviceList._deviceTrackingStatus
@@ -60,189 +58,31 @@ const TRACKING_STATUS_UP_TO_DATE = 3;
 * @alias module:crypto/DeviceList
 */
 export default class DeviceList {
-    constructor(baseApis, cryptoStore, sessionStore, olmDevice) {
-        this._cryptoStore = cryptoStore;
+    constructor(baseApis, sessionStore, olmDevice) {
        this._sessionStore = sessionStore;
-
-        // userId -> {
-        //     deviceId -> {
-        //         [device info]
-        //     }
-        // }
-        this._devices = {};
+        this._serialiser = new DeviceListUpdateSerialiser(
+            baseApis, sessionStore, olmDevice,
+        );

        // which users we are tracking device status for.
        // userId -> TRACKING_STATUS_*
-        this._deviceTrackingStatus = {}; // loaded from storage in load()
-
-        // The 'next_batch' sync token at the point the data was writen,
-        // ie. a token representing the point immediately after the
-        // moment represented by the snapshot in the db.
-        this._syncToken = null;
-
-        this._serialiser = new DeviceListUpdateSerialiser(
-            baseApis, olmDevice, this,
-        );
-
-        // userId -> promise
-        this._keyDownloadsInProgressByUser = {};
-
-        // Set whenever changes are made other than setting the sync token
-        this._dirty = false;
-
-        // Promise resolved when device data is saved
-        this._savePromise = null;
-        // Function that resolves the save promise
-        this._resolveSavePromise = null;
-        // The time the save is scheduled for
-        this._savePromiseTime = null;
-        // The timer used to delay the save
-        this._saveTimer = null;
-    }
-
-    /**
-     * Load the device tracking state from storage
-     */
-    async load() {
-        let shouldDeleteSessionStore = false;
-        await this._cryptoStore.doTxn(
-            // migrate from session store if there's data there and not here
-            'readwrite', [IndexedDBCryptoStore.STORE_DEVICE_DATA], (txn) => {
-                this._cryptoStore.getEndToEndDeviceData(txn, (deviceData) => {
-                    if (deviceData === null) {
-                        console.log("Migrating e2e device data...");
-                        this._devices = this._sessionStore.getAllEndToEndDevices() || {};
-                        this._deviceTrackingStatus = (
-                            this._sessionStore.getEndToEndDeviceTrackingStatus() || {}
-                        );
-                        this._syncToken = this._sessionStore.getEndToEndDeviceSyncToken();
-                        this._cryptoStore.storeEndToEndDeviceData({
-                            devices: this._devices,
-                            trackingStatus: this._deviceTrackingStatus,
-                            syncToken: this._syncToken,
-                        }, txn);
-                        shouldDeleteSessionStore = true;
-                    } else {
-                        this._devices = deviceData ? deviceData.devices : {},
-                        this._deviceTrackingStatus = deviceData ?
-                            deviceData.trackingStatus : {};
-                        this._syncToken = deviceData ? deviceData.syncToken : null;
-                    }
-                });
-            },
-        );
-
-        if (shouldDeleteSessionStore) {
-            // migrated data is now safely persisted: remove from old store
-            this._sessionStore.removeEndToEndDeviceData();
-        }
-
+        this._deviceTrackingStatus = sessionStore.getEndToEndDeviceTrackingStatus() || {};
        for (const u of Object.keys(this._deviceTrackingStatus)) {
            // if a download was in progress when we got shut down, it isn't any more.
            if (this._deviceTrackingStatus[u] == TRACKING_STATUS_DOWNLOAD_IN_PROGRESS) {
                this._deviceTrackingStatus[u] = TRACKING_STATUS_PENDING_DOWNLOAD;
            }
        }
+
+        // userId -> promise
+        this._keyDownloadsInProgressByUser = {};
+
+        this.lastKnownSyncToken = null;
    }

    /**
-     * Save the device tracking state to storage, if any changes are
-     * pending other than updating the sync token
-     *
-     * The actual save will be delayed by a short amount of time to
-     * aggregate multiple writes to the database.
-     *
-     * @param {integer} delay Time in ms before which the save actually happens.
-     *     By default, the save is delayed for a short period in order to batch
-     *     multiple writes, but this behaviour can be disabled by passing 0.
-     *
-     * @return {Promise<bool>} true if the data was saved, false if
-     *     it was not (eg. because no changes were pending). The promise
-     *     will only resolve once the data is saved, so may take some time
-     *     to resolve.
-     */
-    async saveIfDirty(delay) {
-        if (!this._dirty) return Promise.resolve(false);
-        // Delay saves for a bit so we can aggregate multiple saves that happen
-        // in quick succession (eg. when a whole room's devices are marked as known)
-        if (delay === undefined) delay = 500;
-
-        const targetTime = Date.now + delay;
-        if (this._savePromiseTime && targetTime < this._savePromiseTime) {
-            // There's a save scheduled but for after we would like: cancel
-            // it & schedule one for the time we want
-            clearTimeout(this._saveTimer);
-            this._saveTimer = null;
-            this._savePromiseTime = null;
-            // (but keep the save promise since whatever called save before
-            // will still want to know when the save is done)
-        }
-
-        let savePromise = this._savePromise;
-        if (savePromise === null) {
-            savePromise = new Promise((resolve, reject) => {
-                this._resolveSavePromise = resolve;
-            });
-            this._savePromise = savePromise;
-        }
-
-        if (this._saveTimer === null) {
-            const resolveSavePromise = this._resolveSavePromise;
-            this._savePromiseTime = targetTime;
-            this._saveTimer = setTimeout(() => {
-                console.log('Saving device tracking data at token ' + this._syncToken);
-                // null out savePromise now (after the delay but before the write),
-                // otherwise we could return the existing promise when the save has
-                // actually already happened. Likewise for the dirty flag.
-                this._savePromiseTime = null;
-                this._saveTimer = null;
-                this._savePromise = null;
-                this._resolveSavePromise = null;
-
-                this._dirty = false;
-                this._cryptoStore.doTxn(
-                    'readwrite', [IndexedDBCryptoStore.STORE_DEVICE_DATA], (txn) => {
-                        this._cryptoStore.storeEndToEndDeviceData({
-                            devices: this._devices,
-                            trackingStatus: this._deviceTrackingStatus,
-                            syncToken: this._syncToken,
-                        }, txn);
-                    },
-                ).then(() => {
-                    resolveSavePromise();
-                });
-            }, delay);
-        }
-        return savePromise;
-    }
-
-    /**
-     * Gets the sync token last set with setSyncToken
-     *
-     * @return {string} The sync token
-     */
-    getSyncToken() {
-        return this._syncToken;
-    }
-
-    /**
-     * Sets the sync token that the app will pass as the 'since' to the /sync
-     * endpoint next time it syncs.
-     * The sync token must always be set after any changes made as a result of
-     * data in that sync since setting the sync token to a newer one will mean
-     * those changed will not be synced from the server if a new client starts
-     * up with that data.
-     *
-     * @param {string} st The sync token
-     */
-    setSyncToken(st) {
-        this._syncToken = st;
-    }
-
-    /**
-     * Ensures up to date keys for a list of users are stored in the session store,
-     * downloading and storing them if they're not (or if forceDownload is
-     * true).
+     * Download the keys for a list of users and stores the keys in the session
+     * store.
     * @param {Array} userIds The users to fetch.
     * @param {bool} forceDownload Always download the keys even if cached.
     *
@@ -312,7 +152,7 @@ export default class DeviceList {
     * managed to get a list of devices for this user yet.
     */
    getStoredDevicesForUser(userId) {
-        const devs = this._devices[userId];
+        const devs = this._sessionStore.getEndToEndDevicesForUser(userId);
        if (!devs) {
            return null;
        }
@@ -325,18 +165,6 @@ export default class DeviceList {
        return res;
    }

-    /**
-     * Get the stored device data for a user, in raw object form
-     *
-     * @param {string} userId the user to get data for
-     *
-     * @return {Object} deviceId->{object} devices, or undefined if
-     * there is no data for this user.
-     */
-    getRawStoredDevicesForUser(userId) {
-        return this._devices[userId];
-    }
-
    /**
     * Get the stored keys for a single device
     *
@@ -347,7 +175,7 @@ export default class DeviceList {
     * if we don't know about this device
     */
    getStoredDevice(userId, deviceId) {
-        const devs = this._devices[userId];
+        const devs = this._sessionStore.getEndToEndDevicesForUser(userId);
        if (!devs || !devs[deviceId]) {
            return undefined;
        }
@@ -372,7 +200,7 @@ export default class DeviceList {
            return null;
        }

-        const devices = this._devices[userId];
+        const devices = this._sessionStore.getEndToEndDevicesForUser(userId);
        if (!devices) {
            return null;
        }
@@ -401,17 +229,6 @@ export default class DeviceList {
        return null;
    }

-    /**
-     * Replaces the list of devices for a user with the given device list
-     *
-     * @param {string} u The user ID
-     * @param {Object} devs New device info for user
-     */
-    storeDevicesForUser(u, devs) {
-        this._devices[u] = devs;
-        this._dirty = true;
-    }
-
    /**
     * flag the given user for device-list tracking, if they are not already.
     *
@@ -437,8 +254,8 @@ export default class DeviceList {
            this._deviceTrackingStatus[userId] = TRACKING_STATUS_PENDING_DOWNLOAD;
        }
        // we don't yet persist the tracking status, since there may be a lot
-        // of calls; we save all data together once the sync is done
-        this._dirty = true;
+        // of calls; instead we wait for the forthcoming
+        // refreshOutdatedDeviceLists.
    }

    /**
@@ -454,25 +271,12 @@ export default class DeviceList {
        if (this._deviceTrackingStatus[userId]) {
            console.log('No longer tracking device list for ' + userId);
            this._deviceTrackingStatus[userId] = TRACKING_STATUS_NOT_TRACKED;
-
-            // we don't yet persist the tracking status, since there may be a lot
-            // of calls; we save all data together once the sync is done
-            this._dirty = true;
        }
+        // we don't yet persist the tracking status, since there may be a lot
+        // of calls; instead we wait for the forthcoming
+        // refreshOutdatedDeviceLists.
    }

-    /**
-     * Set all users we're currently tracking to untracked
-     *
-     * This will flag each user whose devices we are tracking as in need of an
-     * update.
-     */
-    stopTrackingAllDeviceLists() {
-        for (const userId of Object.keys(this._deviceTrackingStatus)) {
-            this._deviceTrackingStatus[userId] = TRACKING_STATUS_NOT_TRACKED;
-        }
-        this._dirty = true;
-    }

    /**
     * Mark the cached device list for the given user outdated.
@@ -489,10 +293,21 @@ export default class DeviceList {
        if (this._deviceTrackingStatus[userId]) {
            console.log("Marking device list outdated for", userId);
            this._deviceTrackingStatus[userId] = TRACKING_STATUS_PENDING_DOWNLOAD;
+        }
+        // we don't yet persist the tracking status, since there may be a lot
+        // of calls; instead we wait for the forthcoming
+        // refreshOutdatedDeviceLists.
+    }

-            // we don't yet persist the tracking status, since there may be a lot
-            // of calls; we save all data together once the sync is done
-            this._dirty = true;
+    /**
+     * Mark all tracked device lists as outdated.
+     *
+     * This will flag each user whose devices we are tracking as in need of an
+     * update.
+     */
+    invalidateAllDeviceLists() {
+        for (const userId of Object.keys(this._deviceTrackingStatus)) {
+            this.invalidateUserDeviceList(userId);
        }
    }

@@ -503,8 +318,6 @@ export default class DeviceList {
     *    is no need to wait for this (it's mostly for the unit tests).
     */
    refreshOutdatedDeviceLists() {
-        this.saveIfDirty();
-
        const usersToDownload = [];
        for (const userId of Object.keys(this._deviceTrackingStatus)) {
            const stat = this._deviceTrackingStatus[userId];
@@ -513,20 +326,13 @@ export default class DeviceList {
            }
        }

+        // we didn't persist the tracking status during
+        // invalidateUserDeviceList, so do it now.
+        this._persistDeviceTrackingStatus();
+
        return this._doKeyDownload(usersToDownload);
    }

-    /**
-     * Set the stored device data for a user, in raw object form
-     * Used only by internal class DeviceListUpdateSerialiser
-     *
-     * @param {string} userId the user to get data for
-     *
-     * @param {Object} devices deviceId->{object} the new devices
-     */
-    _setRawStoredDevicesForUser(userId, devices) {
-        this._devices[userId] = devices;
-    }

    /**
     * Fire off download update requests for the given users, and update the
@@ -546,7 +352,7 @@ export default class DeviceList {
        }

        const prom = this._serialiser.updateDevicesForUsers(
-            users, this._syncToken,
+            users, this.lastKnownSyncToken,
        ).then(() => {
            finished(true);
        }, (e) => {
@@ -567,8 +373,6 @@ export default class DeviceList {

        const finished = (success) => {
            users.forEach((u) => {
-                this._dirty = true;
-
                // we may have queued up another download request for this user
                // since we started this request. If that happens, we should
                // ignore the completion of the first one.
@@ -590,11 +394,15 @@ export default class DeviceList {
                    }
                }
            });
-            this.saveIfDirty();
+            this._persistDeviceTrackingStatus();
        };

        return prom;
    }
+
+    _persistDeviceTrackingStatus() {
+        this._sessionStore.storeEndToEndDeviceTrackingStatus(this._deviceTrackingStatus);
+    }
 }

 /**
@@ -607,15 +415,10 @@ export default class DeviceList {
 * time (and queuing other requests up).
 */
 class DeviceListUpdateSerialiser {
-    /*
-     * @param {object} baseApis Base API object
-     * @param {object} olmDevice The Olm Device
-     * @param {object} deviceList The device list object
-     */
-    constructor(baseApis, olmDevice, deviceList) {
+    constructor(baseApis, sessionStore, olmDevice) {
        this._baseApis = baseApis;
+        this._sessionStore = sessionStore;
        this._olmDevice = olmDevice;
-        this._deviceList = deviceList; // the device list to be updated

        this._downloadInProgress = false;

@@ -628,7 +431,9 @@ class DeviceListUpdateSerialiser {
        // non-null indicates that we have users queued for download.
        this._queuedQueryDeferred = null;

-        this._syncToken = null; // The sync token we send with the requests
+        // sync token to be used for the next query: essentially the
+        // most recent one we know about
+        this._nextSyncToken = null;
    }

    /**
@@ -647,16 +452,12 @@ class DeviceListUpdateSerialiser {
        users.forEach((u) => {
            this._keyDownloadsQueuedByUser[u] = true;
        });
+        this._nextSyncToken = syncToken;

        if (!this._queuedQueryDeferred) {
            this._queuedQueryDeferred = Promise.defer();
        }

-        // We always take the new sync token and just use the latest one we've
-        // been given, since it just needs to be at least as recent as the
-        // sync response the device invalidation message arrived in
-        this._syncToken = syncToken;
-
        if (this._downloadInProgress) {
            // just queue up these users
            console.log('Queued key download for', users);
@@ -683,8 +484,8 @@ class DeviceListUpdateSerialiser {
        this._downloadInProgress = true;

        const opts = {};
-        if (this._syncToken) {
-            opts.token = this._syncToken;
+        if (this._nextSyncToken) {
+            opts.token = this._nextSyncToken;
        }

        this._baseApis.downloadKeysForUsers(
@@ -729,7 +530,7 @@ class DeviceListUpdateSerialiser {

        // map from deviceid -> deviceinfo for this user
        const userStore = {};
-        const devs = this._deviceList.getRawStoredDevicesForUser(userId);
+        const devs = this._sessionStore.getEndToEndDevicesForUser(userId);
        if (devs) {
            Object.keys(devs).forEach((deviceId) => {
                const d = DeviceInfo.fromStorage(devs[deviceId], deviceId);
@@ -741,13 +542,15 @@ class DeviceListUpdateSerialiser {
            this._olmDevice, userId, userStore, response || {},
        );

-        // put the updates into thr object that will be returned as our results
+        // update the session store
        const storage = {};
        Object.keys(userStore).forEach((deviceId) => {
            storage[deviceId] = userStore[deviceId].toStorage();
        });

-        this._deviceList._setRawStoredDevicesForUser(userId, storage);
+        this._sessionStore.storeEndToEndDevicesForUser(
+            userId, storage,
+        );
    }
 }

@@ -35,19 +35,13 @@ const SEND_KEY_REQUESTS_DELAY_MS = 500;
 *
 * The state machine looks like:
 *
- *     |         (cancellation sent)
- *     | .-------------------------------------------------.
- *     | |                                                 |
- *     V V       (cancellation requested)                  |
- *   UNSENT  -----------------------------+                |
- *     |                                  |                |
- *     |                                  |                |
- *     | (send successful)                |  CANCELLATION_PENDING_AND_WILL_RESEND
- *     V                                  |                Λ
- *    SENT                                |                |
- *     |--------------------------------  |  --------------'
- *     |                                  |  (cancellation requested with intent
- *     |                                  |   to resend the original request)
+ *     |
+ *     V         (cancellation requested)
+ *   UNSENT  -----------------------------+
+ *     |                                  |
+ *     | (send successful)                |
+ *     V                                  |
+ *    SENT                                |
 *     |                                  |
 *     | (cancellation requested)         |
 *     V                                  |
@@ -68,12 +62,6 @@ const ROOM_KEY_REQUEST_STATES = {

    /** reply received, cancellation not yet sent */
    CANCELLATION_PENDING: 2,
-
-    /**
-     * Cancellation not yet sent and will transition to UNSENT instead of
-     * being deleted once the cancellation has been sent.
-     */
-    CANCELLATION_PENDING_AND_WILL_RESEND: 3,
 };

 export default class OutgoingRoomKeyRequestManager {
@@ -142,16 +130,14 @@ export default class OutgoingRoomKeyRequestManager {
    }

    /**
-     * Cancel room key requests, if any match the given requestBody
+     * Cancel room key requests, if any match the given details
     *
     * @param {module:crypto~RoomKeyRequestBody} requestBody
-     * @param {boolean} andResend if true, transition to UNSENT instead of
-     *                            deleting after sending cancellation.
     *
     * @returns {Promise} resolves when the request has been updated in our
     *    pending list.
     */
-    cancelRoomKeyRequest(requestBody, andResend=false) {
+    cancelRoomKeyRequest(requestBody) {
        return this._cryptoStore.getOutgoingRoomKeyRequest(
            requestBody,
        ).then((req) => {
@@ -161,7 +147,6 @@ export default class OutgoingRoomKeyRequestManager {
            }
            switch (req.state) {
                case ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING:
-                case ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING_AND_WILL_RESEND:
                    // nothing to do here
                    return;

@@ -181,16 +166,11 @@ export default class OutgoingRoomKeyRequestManager {
                        req.requestId, ROOM_KEY_REQUEST_STATES.UNSENT,
                    );

-                case ROOM_KEY_REQUEST_STATES.SENT: {
-                    // If `andResend` is set, transition to UNSENT once the
-                    // cancellation has successfully been sent.
-                    const state = andResend ?
-                        ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING_AND_WILL_RESEND :
-                        ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING;
+                case ROOM_KEY_REQUEST_STATES.SENT:
                    // send a cancellation.
                    return this._cryptoStore.updateOutgoingRoomKeyRequest(
                        req.requestId, ROOM_KEY_REQUEST_STATES.SENT, {
-                            state,
+                            state: ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING,
                            cancellationTxnId: this._baseApis.makeTxnId(),
                        },
                    ).then((updatedReq) => {
@@ -220,23 +200,15 @@ export default class OutgoingRoomKeyRequestManager {
                        // do.)
                        this._sendOutgoingRoomKeyRequestCancellation(
                            updatedReq,
-                            andResend,
                        ).catch((e) => {
                            console.error(
                                "Error sending room key request cancellation;"
                                + " will retry later.", e,
                            );
                            this._startTimer();
-                        }).then(() => {
-                            if (!andResend) return;
-                            // The request has transitioned from
-                            // CANCELLATION_PENDING_AND_WILL_RESEND to UNSENT. We
-                            // still need to resend the request which is now UNSENT, so
-                            // start the timer if it isn't already started.
-                            this._startTimer();
-                        });
+                        }).done();
                    });
-                }
+
                default:
                    throw new Error('unhandled state: ' + req.state);
            }
@@ -286,7 +258,6 @@ export default class OutgoingRoomKeyRequestManager {

        return this._cryptoStore.getOutgoingRoomKeyRequestByState([
            ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING,
-            ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING_AND_WILL_RESEND,
            ROOM_KEY_REQUEST_STATES.UNSENT,
        ]).then((req) => {
            if (!req) {
@@ -296,16 +267,10 @@ export default class OutgoingRoomKeyRequestManager {
            }

            let prom;
-            switch (req.state) {
-                case ROOM_KEY_REQUEST_STATES.UNSENT:
-                    prom = this._sendOutgoingRoomKeyRequest(req);
-                    break;
-                case ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING:
-                    prom = this._sendOutgoingRoomKeyRequestCancellation(req);
-                    break;
-                case ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING_AND_WILL_RESEND:
-                    prom = this._sendOutgoingRoomKeyRequestCancellation(req, true);
-                    break;
+            if (req.state === ROOM_KEY_REQUEST_STATES.UNSENT) {
+                prom = this._sendOutgoingRoomKeyRequest(req);
+            } else { // must be a cancellation
+                prom = this._sendOutgoingRoomKeyRequestCancellation(req);
            }

            return prom.then(() => {
@@ -344,9 +309,8 @@ export default class OutgoingRoomKeyRequestManager {
        });
    }

-    // Given a RoomKeyRequest, cancel it and delete the request record unless
-    // andResend is set, in which case transition to UNSENT.
-    _sendOutgoingRoomKeyRequestCancellation(req, andResend) {
+    // given a RoomKeyRequest, cancel it and delete the request record
+    _sendOutgoingRoomKeyRequestCancellation(req) {
        console.log(
            `Sending cancellation for key request for ` +
            `${stringifyRequestBody(req.requestBody)} to ` +
@@ -363,14 +327,6 @@ export default class OutgoingRoomKeyRequestManager {
        return this._sendMessageToDevices(
            requestMessage, req.recipients, req.cancellationTxnId,
        ).then(() => {
-            if (andResend) {
-                // We want to resend, so transition to UNSENT
-                return this._cryptoStore.updateOutgoingRoomKeyRequest(
-                    req.requestId,
-                    ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING_AND_WILL_RESEND,
-                    { state: ROOM_KEY_REQUEST_STATES.UNSENT },
-                );
-            }
            return this._cryptoStore.deleteOutgoingRoomKeyRequest(
                req.requestId, ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING,
            );
@@ -1,81 +0,0 @@
-/*
-Copyright 2018 New Vector Ltd
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-/**
- * @module crypto/RoomList
- *
- * Manages the list of encrypted rooms
- */
-
-import IndexedDBCryptoStore from './store/indexeddb-crypto-store';
-
-/**
- * @alias module:crypto/RoomList
- */
-export default class RoomList {
-    constructor(cryptoStore, sessionStore) {
-        this._cryptoStore = cryptoStore;
-        this._sessionStore = sessionStore;
-
-        // Object of roomId -> room e2e info object (body of the m.room.encryption event)
-        this._roomEncryption = {};
-    }
-
-    async init() {
-        let removeSessionStoreRooms = false;
-        await this._cryptoStore.doTxn(
-            'readwrite', [IndexedDBCryptoStore.STORE_ROOMS], (txn) => {
-                this._cryptoStore.getEndToEndRooms(txn, (result) => {
-                    if (result === null || Object.keys(result).length === 0) {
-                        // migrate from session store, if there's data there
-                        const sessStoreRooms = this._sessionStore.getAllEndToEndRooms();
-                        if (sessStoreRooms !== null) {
-                            for (const roomId of Object.keys(sessStoreRooms)) {
-                                this._cryptoStore.storeEndToEndRoom(
-                                    roomId, sessStoreRooms[roomId], txn,
-                                );
-                            }
-                        }
-                        this._roomEncryption = sessStoreRooms;
-                        removeSessionStoreRooms = true;
-                    } else {
-                        this._roomEncryption = result;
-                    }
-                });
-            },
-        );
-        if (removeSessionStoreRooms) {
-            this._sessionStore.removeAllEndToEndRooms();
-        }
-    }
-
-    getRoomEncryption(roomId) {
-        return this._roomEncryption[roomId] || null;
-    }
-
-    isRoomEncrypted(roomId) {
-        return Boolean(this.getRoomEncryption(roomId));
-    }
-
-    async setRoomEncryption(roomId, roomInfo) {
-        this._roomEncryption[roomId] = roomInfo;
-        await this._cryptoStore.doTxn(
-            'readwrite', [IndexedDBCryptoStore.STORE_ROOMS], (txn) => {
-                this._cryptoStore.storeEndToEndRoom(roomId, roomInfo, txn);
-            },
-        );
-    }
-}
@@ -179,26 +179,31 @@ class DecryptionError extends Error {
    constructor(msg, details) {
        super(msg);
        this.name = 'DecryptionError';
-        this.detailedString = _detailedStringForDecryptionError(this, details);
+        this.details = details;
+    }
+
+    /**
+     * override the string used when logging
+     *
+     * @returns {String}
+     */
+    toString() {
+        let result = this.name + '[msg: ' + this.message;
+
+        if (this.details) {
+            result += ', ' +
+                Object.keys(this.details).map(
+                    (k) => k + ': ' + this.details[k],
+                ).join(', ');
+        }
+
+        result += ']';
+
+        return result;
    }
 }
 export {DecryptionError}; // https://github.com/jsdoc3/jsdoc/issues/1272

-function _detailedStringForDecryptionError(err, details) {
-    let result = err.name + '[msg: ' + err.message;
-
-    if (details) {
-        result += ', ' +
-            Object.keys(details).map(
-                (k) => k + ': ' + details[k],
-            ).join(', ');
-    }
-
-    result += ']';
-
-    return result;
-}
-
 /**
 * Exception thrown specifically when we want to warn the user to consider
 * the security of their conversation before continuing
@@ -541,12 +541,6 @@ MegolmEncryption.prototype._getDevicesInRoom = function(room) {
        return u.userId;
    });

-    // The global value is treated as a default for when rooms don't specify a value.
-    let isBlacklisting = this._crypto.getGlobalBlacklistUnverifiedDevices();
-    if (typeof room.getBlacklistUnverifiedDevices() === 'boolean') {
-        isBlacklisting = room.getBlacklistUnverifiedDevices();
-    }
-
    // We are happy to use a cached version here: we assume that if we already
    // have a list of the user's devices, then we already share an e2e room
    // with them, which means that they will have announced any new devices via
@@ -570,7 +564,9 @@ MegolmEncryption.prototype._getDevicesInRoom = function(room) {
                }

                if (userDevices[deviceId].isBlocked() ||
-                    (userDevices[deviceId].isUnverified() && isBlacklisting)
+                    (userDevices[deviceId].isUnverified() &&
+                     (room.getBlacklistUnverifiedDevices() ||
+                      this._crypto.getGlobalBlacklistUnverifiedDevices()))
                   ) {
                    delete userDevices[deviceId];
                }
@@ -927,18 +923,10 @@ MegolmDecryption.prototype._buildKeyForwardingMessage = async function(
 * @param {module:crypto/OlmDevice.MegolmSessionData} session
 */
 MegolmDecryption.prototype.importRoomKey = function(session) {
-    return this._olmDevice.addInboundGroupSession(
-        session.room_id,
-        session.sender_key,
-        session.forwarding_curve25519_key_chain,
-        session.session_id,
-        session.session_key,
-        session.sender_claimed_keys,
-        true,
-    ).then(() => {
-        // have another go at decrypting events sent with this session.
-        this._retryDecryption(session.sender_key, session.session_id);
-    });
+    this._olmDevice.importInboundGroupSession(session);
+
+    // have another go at decrypting events sent with this session.
+    this._retryDecryption(session.sender_key, session.session_id);
 };

 /**
@@ -1,7 +1,6 @@
 /*
 Copyright 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -34,7 +33,6 @@ const DeviceVerification = DeviceInfo.DeviceVerification;
 const DeviceList = require('./DeviceList').default;

 import OutgoingRoomKeyRequestManager from './OutgoingRoomKeyRequestManager';
-import IndexedDBCryptoStore from './store/indexeddb-crypto-store';

 /**
 * Cryptography bits
@@ -59,23 +57,18 @@ import IndexedDBCryptoStore from './store/indexeddb-crypto-store';
 *
 * @param {module:crypto/store/base~CryptoStore} cryptoStore
 *    storage for the crypto layer.
- *
- * @param {RoomList} roomList An initialised RoomList object
 */
 function Crypto(baseApis, sessionStore, userId, deviceId,
-                clientStore, cryptoStore, roomList) {
+                clientStore, cryptoStore) {
    this._baseApis = baseApis;
    this._sessionStore = sessionStore;
    this._userId = userId;
    this._deviceId = deviceId;
    this._clientStore = clientStore;
    this._cryptoStore = cryptoStore;
-    this._roomList = roomList;

-    this._olmDevice = new OlmDevice(sessionStore, cryptoStore);
-    this._deviceList = new DeviceList(
-        baseApis, cryptoStore, sessionStore, this._olmDevice,
-    );
+    this._olmDevice = new OlmDevice(sessionStore);
+    this._deviceList = new DeviceList(baseApis, sessionStore, this._olmDevice);

    // the last time we did a check for the number of one-time-keys on the
    // server.
@@ -115,26 +108,13 @@ utils.inherits(Crypto, EventEmitter);
 * Returns a promise which resolves once the crypto module is ready for use.
 */
 Crypto.prototype.init = async function() {
-    const sessionStoreHasAccount = Boolean(this._sessionStore.getEndToEndAccount());
-    let cryptoStoreHasAccount;
-    await this._cryptoStore.doTxn(
-        'readonly', [IndexedDBCryptoStore.STORE_ACCOUNT], (txn) => {
-            this._cryptoStore.getAccount(txn, (pickledAccount) => {
-                cryptoStoreHasAccount = Boolean(pickledAccount);
-            });
-        },
-    );
-    if (sessionStoreHasAccount && !cryptoStoreHasAccount) {
-        // we're about to migrate to the crypto store
-        this.emit("crypto.warning", 'CRYPTO_WARNING_ACCOUNT_MIGRATED');
-    } else if (sessionStoreHasAccount && cryptoStoreHasAccount) {
-        // There's an account in both stores: an old version of
-        // the code has been run against this store.
-        this.emit("crypto.warning", 'CRYPTO_WARNING_OLD_VERSION_DETECTED');
+    // XXX: Connect to the backend now so we can fail faster if the
+    // version is wrong
+    if (this._cryptoStore._connect) {
+        await this._cryptoStore._connect();
    }

    await this._olmDevice.init();
-    await this._deviceList.load();

    // build our device keys: these will later be uploaded
    this._deviceKeys["ed25519:" + this._deviceId] =
@@ -142,7 +122,7 @@ Crypto.prototype.init = async function() {
    this._deviceKeys["curve25519:" + this._deviceId] =
        this._olmDevice.deviceCurve25519Key;

-    let myDevices = this._deviceList.getRawStoredDevicesForUser(
+    let myDevices = this._sessionStore.getEndToEndDevicesForUser(
        this._userId,
    );

@@ -160,10 +140,9 @@ Crypto.prototype.init = async function() {
        };

        myDevices[this._deviceId] = deviceInfo;
-        this._deviceList.storeDevicesForUser(
+        this._sessionStore.storeEndToEndDevicesForUser(
            this._userId, myDevices,
        );
-        this._deviceList.saveIfDirty();
    }
 };

@@ -219,17 +198,18 @@ Crypto.prototype.getDeviceEd25519Key = function() {

 /**
 * Set the global override for whether the client should ever send encrypted
- * messages to unverified devices.  This provides the default for rooms which
- * do not specify a value.
+ * messages to unverified devices.  If false, it can still be overridden
+ * per-room.  If true, it overrides the per-room settings.
 *
- * @param {boolean} value whether to blacklist all unverified devices by default
+ * @param {boolean} value whether to unilaterally blacklist all
+ * unverified devices
 */
 Crypto.prototype.setGlobalBlacklistUnverifiedDevices = function(value) {
    this._globalBlacklistUnverifiedDevices = value;
 };

 /**
- * @return {boolean} whether to blacklist all unverified devices by default
+ * @return {boolean} whether to unilaterally blacklist all unverified devices
 */
 Crypto.prototype.getGlobalBlacklistUnverifiedDevices = function() {
    return this._globalBlacklistUnverifiedDevices;
@@ -444,22 +424,6 @@ Crypto.prototype.getStoredDevice = function(userId, deviceId) {
    return this._deviceList.getStoredDevice(userId, deviceId);
 };

-/**
- * Save the device list, if necessary
- *
- * @param {integer} delay Time in ms before which the save actually happens.
- *     By default, the save is delayed for a short period in order to batch
- *     multiple writes, but this behaviour can be disabled by passing 0.
- *
- * @return {Promise<bool>} true if the data was saved, false if
- *     it was not (eg. because no changes were pending). The promise
- *     will only resolve once the data is saved, so may take some time
- *     to resolve.
- */
-Crypto.prototype.saveDeviceList = function(delay) {
-    return this._deviceList.saveIfDirty(delay);
-};
-
 /**
 * Update the blocked/verified state of the given device
 *
@@ -480,7 +444,7 @@ Crypto.prototype.saveDeviceList = function(delay) {
 Crypto.prototype.setDeviceVerification = async function(
    userId, deviceId, verified, blocked, known,
 ) {
-    const devices = this._deviceList.getRawStoredDevicesForUser(userId);
+    const devices = this._sessionStore.getEndToEndDevicesForUser(userId);
    if (!devices || !devices[deviceId]) {
        throw new Error("Unknown device " + userId + ":" + deviceId);
    }
@@ -508,8 +472,7 @@ Crypto.prototype.setDeviceVerification = async function(
    if (dev.verified !== verificationStatus || dev.known !== knownStatus) {
        dev.verified = verificationStatus;
        dev.known = knownStatus;
-        this._deviceList.storeDevicesForUser(userId, devices);
-        this._deviceList.saveIfDirty();
+        this._sessionStore.storeEndToEndDevicesForUser(userId, devices);
    }
    return DeviceInfo.fromStorage(dev, deviceId);
 };
@@ -606,6 +569,7 @@ Crypto.prototype.getEventSenderDeviceInfo = function(event) {
    return device;
 };

+
 /**
 * Configure a room to use encryption (ie, save a flag in the sessionstore).
 *
@@ -619,11 +583,13 @@ Crypto.prototype.getEventSenderDeviceInfo = function(event) {
 Crypto.prototype.setRoomEncryption = async function(roomId, config, inhibitDeviceQuery) {
    // if we already have encryption in this room, we should ignore this event
    // (for now at least. maybe we should alert the user somehow?)
-    const existingConfig = this._roomList.getRoomEncryption(roomId);
-    if (existingConfig && JSON.stringify(existingConfig) != JSON.stringify(config)) {
-        console.error("Ignoring m.room.encryption event which requests " +
-                      "a change of config in " + roomId);
-        return;
+    const existingConfig = this._sessionStore.getEndToEndRoom(roomId);
+    if (existingConfig) {
+        if (JSON.stringify(existingConfig) != JSON.stringify(config)) {
+            console.error("Ignoring m.room.encryption event which requests " +
+                          "a change of config in " + roomId);
+            return;
+        }
    }

    const AlgClass = algorithms.ENCRYPTION_CLASSES[config.algorithm];
@@ -631,7 +597,7 @@ Crypto.prototype.setRoomEncryption = async function(roomId, config, inhibitDevic
        throw new Error("Unable to encrypt with " + config.algorithm);
    }

-    await this._roomList.setRoomEncryption(roomId, config);
+    this._sessionStore.storeEndToEndRoom(roomId, config);

    const alg = new AlgClass({
        userId: this._userId,
@@ -709,28 +675,34 @@ Crypto.prototype.ensureOlmSessionsForUsers = function(users) {
    );
 };

+/**
+ * Whether encryption is enabled for a room.
+ * @param {string} roomId the room id to query.
+ * @return {bool} whether encryption is enabled.
+ */
+Crypto.prototype.isRoomEncrypted = function(roomId) {
+    return Boolean(this._roomEncryptors[roomId]);
+};
+
+
 /**
 * Get a list containing all of the room keys
 *
- * @return {module:crypto/OlmDevice.MegolmSessionData[]} a list of session export objects
+ * @return {module:client.Promise} a promise which resolves to a list of
+ *    session export objects
 */
-Crypto.prototype.exportRoomKeys = async function() {
-    const exportedSessions = [];
-    await this._cryptoStore.doTxn(
-        'readonly', [IndexedDBCryptoStore.STORE_INBOUND_GROUP_SESSIONS], (txn) => {
-            this._cryptoStore.getAllEndToEndInboundGroupSessions(txn, (s) => {
-                if (s === null) return;
-
-                const sess = this._olmDevice.exportInboundGroupSession(
-                    s.senderKey, s.sessionId, s.sessionData,
-                );
+Crypto.prototype.exportRoomKeys = function() {
+    return Promise.map(
+        this._sessionStore.getAllEndToEndInboundGroupSessionKeys(),
+        (s) => {
+            return this._olmDevice.exportInboundGroupSession(
+                s.senderKey, s.sessionId,
+            ).then((sess) => {
                sess.algorithm = olmlib.MEGOLM_ALGORITHM;
-                exportedSessions.push(sess);
+                return sess;
            });
        },
    );
-
-    return exportedSessions;
 };

 /**
@@ -781,23 +753,9 @@ Crypto.prototype.encryptEvent = function(event, room) {
        );
    }

-    let content = event.getContent();
-    // If event has an m.relates_to then we need
-    // to put this on the wrapping event instead
-    const mRelatesTo = content['m.relates_to'];
-    if (mRelatesTo) {
-        // Clone content here so we don't remove `m.relates_to` from the local-echo
-        content = Object.assign({}, content);
-        delete content['m.relates_to'];
-    }
-
    return alg.encryptMessage(
-        room, event.getType(), content,
+        room, event.getType(), event.getContent(),
    ).then((encryptedContent) => {
-        if (mRelatesTo) {
-            encryptedContent['m.relates_to'] = mRelatesTo;
-        }
-
        event.makeEncrypted(
            "m.room.encrypted",
            encryptedContent,
@@ -835,24 +793,24 @@ Crypto.prototype.decryptEvent = function(event) {
 * Handle the notification from /sync or /keys/changes that device lists have
 * been changed.
 *
- * @param {Object} syncData Object containing sync tokens associated with this sync
- * @param {Object} syncDeviceLists device_lists field from /sync, or response from
+ * @param {Object} deviceLists device_lists field from /sync, or response from
 * /keys/changes
 */
-Crypto.prototype.handleDeviceListChanges = async function(syncData, syncDeviceLists) {
-    // Initial syncs don't have device change lists. We'll either get the complete list
-    // of changes for the interval or will have invalidated everything in willProcessSync
-    if (!syncData.oldSyncToken) return;
+Crypto.prototype.handleDeviceListChanges = async function(deviceLists) {
+    if (deviceLists.changed && Array.isArray(deviceLists.changed)) {
+        deviceLists.changed.forEach((u) => {
+            this._deviceList.invalidateUserDeviceList(u);
+        });
+    }

-    // Here, we're relying on the fact that we only ever save the sync data after
-    // sucessfully saving the device list data, so we're guaranteed that the device
-    // list store is at least as fresh as the sync token from the sync store, ie.
-    // any device changes received in sync tokens prior to the 'next' token here
-    // have been processed and are reflected in the current device list.
-    // If we didn't make this assumption, we'd have to use the /keys/changes API
-    // to get key changes between the sync token in the device list and the 'old'
-    // sync token used here to make sure we didn't miss any.
-    this._evalDeviceListChanges(syncDeviceLists);
+    if (deviceLists.left && Array.isArray(deviceLists.left)) {
+        deviceLists.left.forEach((u) => {
+            this._deviceList.stopTrackingDeviceList(u);
+        });
+    }
+
+    // don't flush the outdated device list yet - we do it once we finish
+    // processing the sync.
 };

 /**
@@ -877,11 +835,9 @@ Crypto.prototype.requestRoomKey = function(requestBody, recipients) {
 *
 * @param {module:crypto~RoomKeyRequestBody} requestBody
 *    parameters to match for cancellation
- * @param {boolean} andResend
- *    if true, resend the key request after cancelling.
 */
-Crypto.prototype.cancelRoomKeyRequest = function(requestBody, andResend) {
-    this._outgoingRoomKeyRequestManager.cancelRoomKeyRequest(requestBody, andResend)
+Crypto.prototype.cancelRoomKeyRequest = function(requestBody) {
+    this._outgoingRoomKeyRequestManager.cancelRoomKeyRequest(requestBody)
    .catch((e) => {
        console.warn("Error clearing pending room key requests", e);
    }).done();
@@ -906,22 +862,6 @@ Crypto.prototype.onCryptoEvent = async function(event) {
    }
 };

-/**
- * Called before the result of a sync is procesed
- *
- * @param {Object} syncData  the data from the 'MatrixClient.sync' event
- */
-Crypto.prototype.onSyncWillProcess = async function(syncData) {
-    if (!syncData.oldSyncToken) {
-        // If there is no old sync token, we start all our tracking from
-        // scratch, so mark everything as untracked. onCryptoEvent will
-        // be called for all e2e rooms during the processing of the sync,
-        // at which point we'll start tracking all the users of that room.
-        console.log("Initial sync performed - resetting device tracking state");
-        this._deviceList.stopTrackingAllDeviceLists();
-    }
-};
-
 /**
 * handle the completion of a /sync
 *
@@ -933,8 +873,37 @@ Crypto.prototype.onSyncWillProcess = async function(syncData) {
 Crypto.prototype.onSyncCompleted = async function(syncData) {
    const nextSyncToken = syncData.nextSyncToken;

-    this._deviceList.setSyncToken(syncData.nextSyncToken);
-    this._deviceList.saveIfDirty();
+    if (!syncData.oldSyncToken) {
+        console.log("Completed initial sync");
+
+        // if we have a deviceSyncToken, we can tell the deviceList to
+        // invalidate devices which have changed since then.
+        const oldSyncToken = this._sessionStore.getEndToEndDeviceSyncToken();
+        if (oldSyncToken !== null) {
+            try {
+                await this._invalidateDeviceListsSince(
+                    oldSyncToken, nextSyncToken,
+                );
+            } catch (e) {
+                // if that failed, we fall back to invalidating everyone.
+                console.warn("Error fetching changed device list", e);
+                this._deviceList.invalidateAllDeviceLists();
+            }
+        } else {
+            // otherwise, we have to invalidate all devices for all users we
+            // are tracking.
+            console.log("Completed first initialsync; invalidating all " +
+                        "device list caches");
+            this._deviceList.invalidateAllDeviceLists();
+        }
+    }
+
+    // we can now store our sync token so that we can get an update on
+    // restart rather than having to invalidate everyone.
+    //
+    // (we don't really need to do this on every sync - we could just
+    // do it periodically)
+    this._sessionStore.storeEndToEndDeviceSyncToken(nextSyncToken);

    // catch up on any new devices we got told about during the sync.
    this._deviceList.lastKnownSyncToken = nextSyncToken;
@@ -951,47 +920,25 @@ Crypto.prototype.onSyncCompleted = async function(syncData) {
 };

 /**
- * Trigger the appropriate invalidations and removes for a given
- * device list
+ * Ask the server which users have new devices since a given token,
+ * and invalidate them
 *
- * @param {Object} deviceLists device_lists field from /sync, or response from
- * /keys/changes
- */
-Crypto.prototype._evalDeviceListChanges = async function(deviceLists) {
-    if (deviceLists.changed && Array.isArray(deviceLists.changed)) {
-        deviceLists.changed.forEach((u) => {
-            this._deviceList.invalidateUserDeviceList(u);
-        });
-    }
-
-    if (deviceLists.left && Array.isArray(deviceLists.left)) {
-        // Check we really don't share any rooms with these users
-        // any more: the server isn't required to give us the
-        // exact correct set.
-        const e2eUserIds = new Set(this._getE2eUsers());
-
-        deviceLists.left.forEach((u) => {
-            if (!e2eUserIds.has(u)) {
-                this._deviceList.stopTrackingDeviceList(u);
-            }
-        });
-    }
-};
-
-/**
- * Get a list of all the IDs of users we share an e2e room with
+ * @param {String} oldSyncToken
+ * @param {String} lastKnownSyncToken
 *
- * @returns {string[]} List of user IDs
+ * Returns a Promise which resolves once the query is complete. Rejects if the
+ *   keyChange query fails.
 */
-Crypto.prototype._getE2eUsers = function() {
-    const e2eUserIds = [];
-    for (const room of this._getE2eRooms()) {
-        const members = room.getJoinedMembers();
-        for (const member of members) {
-            e2eUserIds.push(member.userId);
-        }
-    }
-    return e2eUserIds;
+Crypto.prototype._invalidateDeviceListsSince = async function(
+    oldSyncToken, lastKnownSyncToken,
+) {
+    const r = await this._baseApis.getKeyChanges(
+        oldSyncToken, lastKnownSyncToken,
+    );
+
+    console.log("got key changes since", oldSyncToken, ":", r);
+
+    await this.handleDeviceListChanges(r);
 };

 /**
@@ -1392,24 +1339,5 @@ class IncomingRoomKeyRequestCancellation {
 * @param {module:crypto~IncomingRoomKeyRequestCancellation} req
 */

-/**
- * Fires when the app may wish to warn the user about something related
- * the end-to-end crypto.
- *
- * Comes with a type which is one of:
- * * CRYPTO_WARNING_ACCOUNT_MIGRATED: Account data has been migrated from an older
- *       version of the store in such a way that older clients will no longer be
- *       able to read it. The app may wish to warn the user against going back to
- *       an older version of the app.
- * * CRYPTO_WARNING_OLD_VERSION_DETECTED: js-sdk has detected that an older version
- *       of js-sdk has been run against the same store after a migration has been
- *       performed. This is likely have caused unexpected behaviour in the old
- *       version. For example, the old version and the new version may have two
- *       different identity keys.
- *
- * @event module:client~MatrixClient#"crypto.warning"
- * @param {string} type One of the strings listed above
- */
-
 /** */
 module.exports = Crypto;
@@ -1,24 +1,7 @@
-/*
-Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
 import Promise from 'bluebird';
 import utils from '../../utils';

-export const VERSION = 6;
+export const VERSION = 1;

 /**
 * Implementation of a CryptoStore which is backed by an existing
@@ -79,9 +62,9 @@ export class Backend {
                `enqueueing key request for ${requestBody.room_id} / ` +
                    requestBody.session_id,
            );
-            txn.oncomplete = () => { deferred.resolve(request); };
            const store = txn.objectStore("outgoingRoomKeyRequests");
            store.add(request);
+            txn.onsuccess = () => { deferred.resolve(request); };
        });

        return deferred.promise;
@@ -274,200 +257,6 @@ export class Backend {
        };
        return promiseifyTxn(txn);
    }
-
-    // Olm Account
-
-    getAccount(txn, func) {
-        const objectStore = txn.objectStore("account");
-        const getReq = objectStore.get("-");
-        getReq.onsuccess = function() {
-            try {
-                func(getReq.result || null);
-            } catch (e) {
-                abortWithException(txn, e);
-            }
-        };
-    }
-
-    storeAccount(txn, newData) {
-        const objectStore = txn.objectStore("account");
-        objectStore.put(newData, "-");
-    }
-
-    // Olm Sessions
-
-    countEndToEndSessions(txn, func) {
-        const objectStore = txn.objectStore("sessions");
-        const countReq = objectStore.count();
-        countReq.onsuccess = function() {
-            func(countReq.result);
-        };
-    }
-
-    getEndToEndSessions(deviceKey, txn, func) {
-        const objectStore = txn.objectStore("sessions");
-        const idx = objectStore.index("deviceKey");
-        const getReq = idx.openCursor(deviceKey);
-        const results = {};
-        getReq.onsuccess = function() {
-            const cursor = getReq.result;
-            if (cursor) {
-                results[cursor.value.sessionId] = cursor.value.session;
-                cursor.continue();
-            } else {
-                try {
-                    func(results);
-                } catch (e) {
-                    abortWithException(txn, e);
-                }
-            }
-        };
-    }
-
-    getEndToEndSession(deviceKey, sessionId, txn, func) {
-        const objectStore = txn.objectStore("sessions");
-        const getReq = objectStore.get([deviceKey, sessionId]);
-        getReq.onsuccess = function() {
-            try {
-                if (getReq.result) {
-                    func(getReq.result.session);
-                } else {
-                    func(null);
-                }
-            } catch (e) {
-                abortWithException(txn, e);
-            }
-        };
-    }
-
-    storeEndToEndSession(deviceKey, sessionId, session, txn) {
-        const objectStore = txn.objectStore("sessions");
-        objectStore.put({deviceKey, sessionId, session});
-    }
-
-    // Inbound group sessions
-
-    getEndToEndInboundGroupSession(senderCurve25519Key, sessionId, txn, func) {
-        const objectStore = txn.objectStore("inbound_group_sessions");
-        const getReq = objectStore.get([senderCurve25519Key, sessionId]);
-        getReq.onsuccess = function() {
-            try {
-                if (getReq.result) {
-                    func(getReq.result.session);
-                } else {
-                    func(null);
-                }
-            } catch (e) {
-                abortWithException(txn, e);
-            }
-        };
-    }
-
-    getAllEndToEndInboundGroupSessions(txn, func) {
-        const objectStore = txn.objectStore("inbound_group_sessions");
-        const getReq = objectStore.openCursor();
-        getReq.onsuccess = function() {
-            const cursor = getReq.result;
-            if (cursor) {
-                try {
-                    func({
-                        senderKey: cursor.value.senderCurve25519Key,
-                        sessionId: cursor.value.sessionId,
-                        sessionData: cursor.value.session,
-                    });
-                } catch (e) {
-                    abortWithException(txn, e);
-                }
-                cursor.continue();
-            } else {
-                try {
-                    func(null);
-                } catch (e) {
-                    abortWithException(txn, e);
-                }
-            }
-        };
-    }
-
-    addEndToEndInboundGroupSession(senderCurve25519Key, sessionId, sessionData, txn) {
-        const objectStore = txn.objectStore("inbound_group_sessions");
-        const addReq = objectStore.add({
-            senderCurve25519Key, sessionId, session: sessionData,
-        });
-        addReq.onerror = (ev) => {
-            if (addReq.error.name === 'ConstraintError') {
-                // This stops the error from triggering the txn's onerror
-                ev.stopPropagation();
-                // ...and this stops it from aborting the transaction
-                ev.preventDefault();
-                console.log(
-                    "Ignoring duplicate inbound group session: " +
-                    senderCurve25519Key + " / " + sessionId,
-                );
-            } else {
-                abortWithException(txn, new Error(
-                    "Failed to add inbound group session: " + addReq.error,
-                ));
-            }
-        };
-    }
-
-    storeEndToEndInboundGroupSession(senderCurve25519Key, sessionId, sessionData, txn) {
-        const objectStore = txn.objectStore("inbound_group_sessions");
-        objectStore.put({
-            senderCurve25519Key, sessionId, session: sessionData,
-        });
-    }
-
-    getEndToEndDeviceData(txn, func) {
-        const objectStore = txn.objectStore("device_data");
-        const getReq = objectStore.get("-");
-        getReq.onsuccess = function() {
-            try {
-                func(getReq.result || null);
-            } catch (e) {
-                abortWithException(txn, e);
-            }
-        };
-    }
-
-    storeEndToEndDeviceData(deviceData, txn) {
-        const objectStore = txn.objectStore("device_data");
-        objectStore.put(deviceData, "-");
-    }
-
-    storeEndToEndRoom(roomId, roomInfo, txn) {
-        const objectStore = txn.objectStore("rooms");
-        objectStore.put(roomInfo, roomId);
-    }
-
-    getEndToEndRooms(txn, func) {
-        const rooms = {};
-        const objectStore = txn.objectStore("rooms");
-        const getReq = objectStore.openCursor();
-        getReq.onsuccess = function() {
-            const cursor = getReq.result;
-            if (cursor) {
-                rooms[cursor.key] = cursor.value;
-                cursor.continue();
-            } else {
-                try {
-                    func(rooms);
-                } catch (e) {
-                    abortWithException(txn, e);
-                }
-            }
-        };
-    }
-
-    doTxn(mode, stores, func) {
-        const txn = this._db.transaction(stores, mode);
-        const promise = promiseifyTxn(txn);
-        const result = func(txn);
-        return promise.then(() => {
-            return result;
-        });
-    }
 }

 export function upgradeDatabase(db, oldVersion) {
@@ -478,26 +267,6 @@ export function upgradeDatabase(db, oldVersion) {
    if (oldVersion < 1) { // The database did not previously exist.
        createDatabase(db);
    }
-    if (oldVersion < 2) {
-        db.createObjectStore("account");
-    }
-    if (oldVersion < 3) {
-        const sessionsStore = db.createObjectStore("sessions", {
-            keyPath: ["deviceKey", "sessionId"],
-        });
-        sessionsStore.createIndex("deviceKey", "deviceKey");
-    }
-    if (oldVersion < 4) {
-        db.createObjectStore("inbound_group_sessions", {
-            keyPath: ["senderCurve25519Key", "sessionId"],
-        });
-    }
-    if (oldVersion < 5) {
-        db.createObjectStore("device_data");
-    }
-    if (oldVersion < 6) {
-        db.createObjectStore("rooms");
-    }
    // Expand as needed.
 }

@@ -514,37 +283,9 @@ function createDatabase(db) {
    outgoingRoomKeyRequestsStore.createIndex("state", "state");
 }

-/*
- * Aborts a transaction with a given exception
- * The transaction promise will be rejected with this exception.
- */
-function abortWithException(txn, e) {
-    // We cheekily stick our exception onto the transaction object here
-    // We could alternatively make the thing we pass back to the app
-    // an object containing the transaction and exception.
-    txn._mx_abortexception = e;
-    try {
-        txn.abort();
-    } catch (e) {
-        // sometimes we won't be able to abort the transaction
-        // (ie. if it's aborted or completed)
-    }
-}
-
 function promiseifyTxn(txn) {
    return new Promise((resolve, reject) => {
-        txn.oncomplete = () => {
-            if (txn._mx_abortexception !== undefined) {
-                reject(txn._mx_abortexception);
-            }
-            resolve();
-        };
-        txn.onerror = () => {
-            if (txn._mx_abortexception !== undefined) {
-                reject(txn._mx_abortexception);
-            }
-            reject();
-        };
-        txn.onabort = () => reject(txn._mx_abortexception);
+        txn.oncomplete = resolve;
+        txn.onerror = reject;
    });
 }
@@ -1,6 +1,5 @@
 /*
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -17,7 +16,6 @@ limitations under the License.

 import Promise from 'bluebird';

-import LocalStorageCryptoStore from './localStorage-crypto-store';
 import MemoryCryptoStore from './memory-crypto-store';
 import * as IndexedDBCryptoStoreBackend from './indexeddb-crypto-store-backend';

@@ -48,7 +46,7 @@ export default class IndexedDBCryptoStore {

    /**
     * Ensure the database exists and is up-to-date, or fall back to
-     * a local storage or in-memory store.
+     * an in-memory store.
     *
     * @return {Promise} resolves to either an IndexedDBCryptoStoreBackend.Backend,
     * or a MemoryCryptoStore
@@ -93,14 +91,15 @@ export default class IndexedDBCryptoStore {
                resolve(new IndexedDBCryptoStoreBackend.Backend(db));
            };
        }).catch((e) => {
+            // Don't fall back to memory in this case: we'd end up recreating
+            // a new Olm account in memory and advertising new keys for the
+            // same device.
+            if (e.name == 'VersionError') {
+                throw e;
+            }
            console.warn(
                `unable to connect to indexeddb ${this._dbName}` +
-                    `: falling back to localStorage store: ${e}`,
-            );
-            return new LocalStorageCryptoStore(global.localStorage);
-        }).catch((e) => {
-            console.warn(
-                `unable to open localStorage: falling back to in-memory store: ${e}`,
+                    `: falling back to in-memory store: ${e}`,
            );
            return new MemoryCryptoStore();
        });
@@ -227,216 +226,4 @@ export default class IndexedDBCryptoStore {
            return backend.deleteOutgoingRoomKeyRequest(requestId, expectedState);
        });
    }
-
-    // Olm Account
-
-    /*
-     * Get the account pickle from the store.
-     * This requires an active transaction. See doTxn().
-     *
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {function(string)} func Called with the account pickle
-     */
-    getAccount(txn, func) {
-        this._backendPromise.value().getAccount(txn, func);
-    }
-
-    /*
-     * Write the account pickle to the store.
-     * This requires an active transaction. See doTxn().
-     *
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {string} newData The new account pickle to store.
-     */
-    storeAccount(txn, newData) {
-        this._backendPromise.value().storeAccount(txn, newData);
-    }
-
-    // Olm sessions
-
-    /**
-     * Returns the number of end-to-end sessions in the store
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {function(int)} func Called with the count of sessions
-     */
-    countEndToEndSessions(txn, func) {
-        this._backendPromise.value().countEndToEndSessions(txn, func);
-    }
-
-    /**
-     * Retrieve a specific end-to-end session between the logged-in user
-     * and another device.
-     * @param {string} deviceKey The public key of the other device.
-     * @param {string} sessionId The ID of the session to retrieve
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {function(object)} func Called with A map from sessionId
-     *     to Base64 end-to-end session.
-     */
-    getEndToEndSession(deviceKey, sessionId, txn, func) {
-        this._backendPromise.value().getEndToEndSession(deviceKey, sessionId, txn, func);
-    }
-
-    /**
-     * Retrieve the end-to-end sessions between the logged-in user and another
-     * device.
-     * @param {string} deviceKey The public key of the other device.
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {function(object)} func Called with A map from sessionId
-     *     to Base64 end-to-end session.
-     */
-    getEndToEndSessions(deviceKey, txn, func) {
-        this._backendPromise.value().getEndToEndSessions(deviceKey, txn, func);
-    }
-
-    /**
-     * Store a session between the logged-in user and another device
-     * @param {string} deviceKey The public key of the other device.
-     * @param {string} sessionId The ID for this end-to-end session.
-     * @param {string} session Base64 encoded end-to-end session.
-     * @param {*} txn An active transaction. See doTxn().
-     */
-    storeEndToEndSession(deviceKey, sessionId, session, txn) {
-        this._backendPromise.value().storeEndToEndSession(
-            deviceKey, sessionId, session, txn,
-        );
-    }
-
-    // Inbound group saessions
-
-    /**
-     * Retrieve the end-to-end inbound group session for a given
-     * server key and session ID
-     * @param {string} senderCurve25519Key The sender's curve 25519 key
-     * @param {string} sessionId The ID of the session
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {function(object)} func Called with A map from sessionId
-     *     to Base64 end-to-end session.
-     */
-    getEndToEndInboundGroupSession(senderCurve25519Key, sessionId, txn, func) {
-        this._backendPromise.value().getEndToEndInboundGroupSession(
-            senderCurve25519Key, sessionId, txn, func,
-        );
-    }
-
-    /**
-     * Fetches all inbound group sessions in the store
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {function(object)} func Called once for each group session
-     *     in the store with an object having keys {senderKey, sessionId,
-     *     sessionData}, then once with null to indicate the end of the list.
-     */
-    getAllEndToEndInboundGroupSessions(txn, func) {
-        this._backendPromise.value().getAllEndToEndInboundGroupSessions(txn, func);
-    }
-
-    /**
-     * Adds an end-to-end inbound group session to the store.
-     * If there already exists an inbound group session with the same
-     * senderCurve25519Key and sessionID, the session will not be added.
-     * @param {string} senderCurve25519Key The sender's curve 25519 key
-     * @param {string} sessionId The ID of the session
-     * @param {object} sessionData The session data structure
-     * @param {*} txn An active transaction. See doTxn().
-     */
-    addEndToEndInboundGroupSession(senderCurve25519Key, sessionId, sessionData, txn) {
-        this._backendPromise.value().addEndToEndInboundGroupSession(
-            senderCurve25519Key, sessionId, sessionData, txn,
-        );
-    }
-
-    /**
-     * Writes an end-to-end inbound group session to the store.
-     * If there already exists an inbound group session with the same
-     * senderCurve25519Key and sessionID, it will be overwritten.
-     * @param {string} senderCurve25519Key The sender's curve 25519 key
-     * @param {string} sessionId The ID of the session
-     * @param {object} sessionData The session data structure
-     * @param {*} txn An active transaction. See doTxn().
-     */
-    storeEndToEndInboundGroupSession(senderCurve25519Key, sessionId, sessionData, txn) {
-        this._backendPromise.value().storeEndToEndInboundGroupSession(
-            senderCurve25519Key, sessionId, sessionData, txn,
-        );
-    }
-
-    // End-to-end device tracking
-
-    /**
-     * Store the state of all tracked devices
-     * This contains devices for each user, a tracking state for each user
-     * and a sync token matching the point in time the snapshot represents.
-     * These all need to be written out in full each time such that the snapshot
-     * is always consistent, so they are stored in one object.
-     *
-     * @param {Object} deviceData
-     * @param {*} txn An active transaction. See doTxn().
-     */
-    storeEndToEndDeviceData(deviceData, txn) {
-        this._backendPromise.value().storeEndToEndDeviceData(deviceData, txn);
-    }
-
-    /**
-     * Get the state of all tracked devices
-     *
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {function(Object)} func Function called with the
-     *     device data
-     */
-    getEndToEndDeviceData(txn, func) {
-        this._backendPromise.value().getEndToEndDeviceData(txn, func);
-    }
-
-    // End to End Rooms
-
-    /**
-     * Store the end-to-end state for a room.
-     * @param {string} roomId The room's ID.
-     * @param {object} roomInfo The end-to-end info for the room.
-     * @param {*} txn An active transaction. See doTxn().
-     */
-    storeEndToEndRoom(roomId, roomInfo, txn) {
-        this._backendPromise.value().storeEndToEndRoom(roomId, roomInfo, txn);
-    }
-
-    /**
-     * Get an object of roomId->roomInfo for all e2e rooms in the store
-     * @param {*} txn An active transaction. See doTxn().
-     * @param {function(Object)} func Function called with the end to end encrypted rooms
-     */
-    getEndToEndRooms(txn, func) {
-        this._backendPromise.value().getEndToEndRooms(txn, func);
-    }
-
-    /**
-     * Perform a transaction on the crypto store. Any store methods
-     * that require a transaction (txn) object to be passed in may
-     * only be called within a callback of either this function or
-     * one of the store functions operating on the same transaction.
-     *
-     * @param {string} mode 'readwrite' if you need to call setter
-     *     functions with this transaction. Otherwise, 'readonly'.
-     * @param {string[]} stores List IndexedDBCryptoStore.STORE_*
-     *     options representing all types of object that will be
-     *     accessed or written to with this transaction.
-     * @param {function(*)} func Function called with the
-     *     transaction object: an opaque object that should be passed
-     *     to store functions.
-     * @return {Promise} Promise that resolves with the result of the `func`
-     *     when the transaction is complete. If the backend is
-     *     async (ie. the indexeddb backend) any of the callback
-     *     functions throwing an exception will cause this promise to
-     *     reject with that exception. On synchronous backends, the
-     *     exception will propagate to the caller of the getFoo method.
-     */
-    doTxn(mode, stores, func) {
-        return this._connect().then((backend) => {
-            return backend.doTxn(mode, stores, func);
-        });
-    }
 }
-
-IndexedDBCryptoStore.STORE_ACCOUNT = 'account';
-IndexedDBCryptoStore.STORE_SESSIONS = 'sessions';
-IndexedDBCryptoStore.STORE_INBOUND_GROUP_SESSIONS = 'inbound_group_sessions';
-IndexedDBCryptoStore.STORE_DEVICE_DATA = 'device_data';
-IndexedDBCryptoStore.STORE_ROOMS = 'rooms';
@@ -1,210 +0,0 @@
-/*
-Copyright 2017, 2018 New Vector Ltd
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-import Promise from 'bluebird';
-import MemoryCryptoStore from './memory-crypto-store.js';
-
-/**
- * Internal module. Partial localStorage backed storage for e2e.
- * This is not a full crypto store, just the in-memory store with
- * some things backed by localStorage. It exists because indexedDB
- * is broken in Firefox private mode or set to, "will not remember
- * history".
- *
- * @module
- */
-
-const E2E_PREFIX = "crypto.";
-const KEY_END_TO_END_ACCOUNT = E2E_PREFIX + "account";
-const KEY_DEVICE_DATA = E2E_PREFIX + "device_data";
-const KEY_INBOUND_SESSION_PREFIX = E2E_PREFIX + "inboundgroupsessions/";
-const KEY_ROOMS_PREFIX = E2E_PREFIX + "rooms/";
-
-function keyEndToEndSessions(deviceKey) {
-    return E2E_PREFIX + "sessions/" + deviceKey;
-}
-
-function keyEndToEndInboundGroupSession(senderKey, sessionId) {
-    return KEY_INBOUND_SESSION_PREFIX + senderKey + "/" + sessionId;
-}
-
-function keyEndToEndRoomsPrefix(roomId) {
-    return KEY_ROOMS_PREFIX + roomId;
-}
-
-/**
- * @implements {module:crypto/store/base~CryptoStore}
- */
-export default class LocalStorageCryptoStore extends MemoryCryptoStore {
-    constructor(webStore) {
-        super();
-        this.store = webStore;
-    }
-
-    // Olm Sessions
-
-    countEndToEndSessions(txn, func) {
-        let count = 0;
-        for (let i = 0; i < this.store.length; ++i) {
-            if (this.store.key(i).startsWith(keyEndToEndSessions(''))) ++count;
-        }
-        func(count);
-    }
-
-    _getEndToEndSessions(deviceKey, txn, func) {
-        return getJsonItem(this.store, keyEndToEndSessions(deviceKey));
-    }
-
-    getEndToEndSession(deviceKey, sessionId, txn, func) {
-        const sessions = this._getEndToEndSessions(deviceKey);
-        func(sessions[sessionId] || {});
-    }
-
-    getEndToEndSessions(deviceKey, txn, func) {
-        func(this._getEndToEndSessions(deviceKey) || {});
-    }
-
-    storeEndToEndSession(deviceKey, sessionId, session, txn) {
-        const sessions = this._getEndToEndSessions(deviceKey) || {};
-        sessions[sessionId] = session;
-        setJsonItem(
-            this.store, keyEndToEndSessions(deviceKey), sessions,
-        );
-    }
-
-    // Inbound Group Sessions
-
-    getEndToEndInboundGroupSession(senderCurve25519Key, sessionId, txn, func) {
-        func(getJsonItem(
-            this.store,
-            keyEndToEndInboundGroupSession(senderCurve25519Key, sessionId),
-        ));
-    }
-
-    getAllEndToEndInboundGroupSessions(txn, func) {
-        for (let i = 0; i < this.store.length; ++i) {
-            const key = this.store.key(i);
-            if (key.startsWith(KEY_INBOUND_SESSION_PREFIX)) {
-                // we can't use split, as the components we are trying to split out
-                // might themselves contain '/' characters. We rely on the
-                // senderKey being a (32-byte) curve25519 key, base64-encoded
-                // (hence 43 characters long).
-
-                func({
-                    senderKey: key.substr(KEY_INBOUND_SESSION_PREFIX.length, 43),
-                    sessionId: key.substr(KEY_INBOUND_SESSION_PREFIX.length + 44),
-                    sessionData: getJsonItem(this.store, key),
-                });
-            }
-        }
-        func(null);
-    }
-
-    addEndToEndInboundGroupSession(senderCurve25519Key, sessionId, sessionData, txn) {
-        const existing = getJsonItem(
-            this.store,
-            keyEndToEndInboundGroupSession(senderCurve25519Key, sessionId),
-        );
-        if (!existing) {
-            this.storeEndToEndInboundGroupSession(
-                senderCurve25519Key, sessionId, sessionData, txn,
-            );
-        }
-    }
-
-    storeEndToEndInboundGroupSession(senderCurve25519Key, sessionId, sessionData, txn) {
-        setJsonItem(
-            this.store,
-            keyEndToEndInboundGroupSession(senderCurve25519Key, sessionId),
-            sessionData,
-        );
-    }
-
-    getEndToEndDeviceData(txn, func) {
-        func(getJsonItem(
-            this.store, KEY_DEVICE_DATA,
-        ));
-    }
-
-    storeEndToEndDeviceData(deviceData, txn) {
-        setJsonItem(
-            this.store, KEY_DEVICE_DATA, deviceData,
-        );
-    }
-
-    storeEndToEndRoom(roomId, roomInfo, txn) {
-        setJsonItem(
-            this.store, keyEndToEndRoomsPrefix(roomId), roomInfo,
-        );
-    }
-
-    getEndToEndRooms(txn, func) {
-        const result = {};
-        const prefix = keyEndToEndRoomsPrefix('');
-
-        for (let i = 0; i < this.store.length; ++i) {
-            const key = this.store.key(i);
-            if (key.startsWith(prefix)) {
-                const roomId = key.substr(prefix.length);
-                result[roomId] = getJsonItem(this.store, key);
-            }
-        }
-        func(result);
-    }
-
-    /**
-     * Delete all data from this store.
-     *
-     * @returns {Promise} Promise which resolves when the store has been cleared.
-     */
-    deleteAllData() {
-        this.store.removeItem(KEY_END_TO_END_ACCOUNT);
-        return Promise.resolve();
-    }
-
-    // Olm account
-
-    getAccount(txn, func) {
-        const account = getJsonItem(this.store, KEY_END_TO_END_ACCOUNT);
-        func(account);
-    }
-
-    storeAccount(txn, newData) {
-        setJsonItem(
-            this.store, KEY_END_TO_END_ACCOUNT, newData,
-        );
-    }
-
-    doTxn(mode, stores, func) {
-        return Promise.resolve(func(null));
-    }
-}
-
-function getJsonItem(store, key) {
-    try {
-        // if the key is absent, store.getItem() returns null, and
-        // JSON.parse(null) === null, so this returns null.
-        return JSON.parse(store.getItem(key));
-    } catch (e) {
-        console.log("Error: Failed to get key %s: %s", key, e.stack || e);
-        console.log(e.stack);
-    }
-    return null;
-}
-
-function setJsonItem(store, key, val) {
-    store.setItem(key, JSON.stringify(val));
-}
@@ -1,6 +1,5 @@
 /*
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -31,16 +30,6 @@ import utils from '../../utils';
 export default class MemoryCryptoStore {
    constructor() {
        this._outgoingRoomKeyRequests = [];
-        this._account = null;
-
-        // Map of {devicekey -> {sessionId -> session pickle}}
-        this._sessions = {};
-        // Map of {senderCurve25519Key+'/'+sessionId -> session data object}
-        this._inboundGroupSessions = {};
-        // Opaque device data object
-        this._deviceData = null;
-        // roomId -> Opaque roomInfo object
-        this._rooms = {};
    }

    /**
@@ -207,95 +196,4 @@ export default class MemoryCryptoStore {

        return Promise.resolve(null);
    }
-
-    // Olm Account
-
-    getAccount(txn, func) {
-        func(this._account);
-    }
-
-    storeAccount(txn, newData) {
-        this._account = newData;
-    }
-
-    // Olm Sessions
-
-    countEndToEndSessions(txn, func) {
-        return Object.keys(this._sessions).length;
-    }
-
-    getEndToEndSession(deviceKey, sessionId, txn, func) {
-        const deviceSessions = this._sessions[deviceKey] || {};
-        func(deviceSessions[sessionId] || null);
-    }
-
-    getEndToEndSessions(deviceKey, txn, func) {
-        func(this._sessions[deviceKey] || {});
-    }
-
-    storeEndToEndSession(deviceKey, sessionId, session, txn) {
-        let deviceSessions = this._sessions[deviceKey];
-        if (deviceSessions === undefined) {
-            deviceSessions = {};
-            this._sessions[deviceKey] = deviceSessions;
-        }
-        deviceSessions[sessionId] = session;
-    }
-
-    // Inbound Group Sessions
-
-    getEndToEndInboundGroupSession(senderCurve25519Key, sessionId, txn, func) {
-        func(this._inboundGroupSessions[senderCurve25519Key+'/'+sessionId] || null);
-    }
-
-    getAllEndToEndInboundGroupSessions(txn, func) {
-        for (const key of Object.keys(this._inboundGroupSessions)) {
-            // we can't use split, as the components we are trying to split out
-            // might themselves contain '/' characters. We rely on the
-            // senderKey being a (32-byte) curve25519 key, base64-encoded
-            // (hence 43 characters long).
-
-            func({
-                senderKey: key.substr(0, 43),
-                sessionId: key.substr(44),
-                sessionData: this._inboundGroupSessions[key],
-            });
-        }
-        func(null);
-    }
-
-    addEndToEndInboundGroupSession(senderCurve25519Key, sessionId, sessionData, txn) {
-        const k = senderCurve25519Key+'/'+sessionId;
-        if (this._inboundGroupSessions[k] === undefined) {
-            this._inboundGroupSessions[k] = sessionData;
-        }
-    }
-
-    storeEndToEndInboundGroupSession(senderCurve25519Key, sessionId, sessionData, txn) {
-        this._inboundGroupSessions[senderCurve25519Key+'/'+sessionId] = sessionData;
-    }
-
-    // Device Data
-
-    getEndToEndDeviceData(txn, func) {
-        func(this._deviceData);
-    }
-
-    storeEndToEndDeviceData(deviceData, txn) {
-        this._deviceData = deviceData;
-    }
-
-    // E2E rooms
-
-    storeEndToEndRoom(roomId, roomInfo, txn) {
-        this._rooms[roomId] = roomInfo;
-    }
-
-    getEndToEndRooms(txn, func) {
-        func(this._rooms);
-    }
-
-    doTxn(mode, stores, func) {
-        return Promise.resolve(func(null));
-    }
 }
@@ -97,12 +97,11 @@ FilterComponent.prototype._checkFields =
    };

    const self = this;
-    for (let n=0; n < Object.keys(literal_keys).length; n++) {
-        const name = Object.keys(literal_keys)[n];
+    Object.keys(literal_keys).forEach(function(name) {
        const match_func = literal_keys[name];
        const not_name = "not_" + name;
        const disallowed_values = self[not_name];
-        if (disallowed_values.filter(match_func).length > 0) {
+        if (disallowed_values.map(match_func)) {
            return false;
        }

@@ -112,7 +111,7 @@ FilterComponent.prototype._checkFields =
                return false;
            }
        }
-    }
+    });

    const contains_url_filter = this.filter_json.contains_url;
    if (contains_url_filter !== undefined) {
@@ -16,8 +16,6 @@ limitations under the License.
 */
 "use strict";

-/** The {@link module:ContentHelpers} object */
-module.exports.ContentHelpers = require("./content-helpers");
 /** The {@link module:models/event.MatrixEvent|MatrixEvent} class. */
 module.exports.MatrixEvent = require("./models/event").MatrixEvent;
 /** The {@link module:models/event.EventStatus|EventStatus} enum. */
@@ -20,6 +20,8 @@ limitations under the License.
 */

 const RoomState = require("./room-state");
+const utils = require("../utils");
+const MatrixEvent = require("./event").MatrixEvent;

 /**
 * Construct a new EventTimeline
@@ -86,23 +88,19 @@ EventTimeline.prototype.initialiseState = function(stateEvents) {
        throw new Error("Cannot initialise state after events are added");
    }

-    // We previously deep copied events here and used different copies in
-    // the oldState and state events: this decision seems to date back
-    // quite a way and was apparently made to fix a bug where modifications
-    // made to the start state leaked through to the end state.
-    // This really shouldn't be possible though: the events themselves should
-    // not change. Duplicating the events uses a lot of extra memory,
-    // so we now no longer do it. To assert that they really do never change,
-    // freeze them! Note that we can't do this for events in general:
-    // although it looks like the only things preventing us are the
-    // 'status' flag, forwardLooking (which is only set once when adding to the
-    // timeline) and possibly the sender (which seems like it should never be
-    // reset but in practice causes a lot of the tests to break).
-    for (const e of stateEvents) {
-        Object.freeze(e);
-    }
+    // we deep-copy the events here, in case they get changed later - we don't
+    // want changes to the start state leaking through to the end state.
+    const oldStateEvents = utils.map(
+        utils.deepCopy(
+            stateEvents.map(function(mxEvent) {
+                return mxEvent.event;
+            }),
+        ),
+    function(ev) {
+        return new MatrixEvent(ev);
+    });

-    this._startState.setStateEvents(stateEvents);
+    this._startState.setStateEvents(oldStateEvents);
    this._endState.setStateEvents(stateEvents);
 };

@@ -80,10 +80,7 @@ module.exports.MatrixEvent = function MatrixEvent(
    // intern the values of matrix events to force share strings and reduce the
    // amount of needless string duplication. This can save moderate amounts of
    // memory (~10% on a 350MB heap).
-    // 'membership' at the event level (rather than the content level) is a legacy
-    // field that Riot never otherwise looks at, but it will still take up a lot
-    // of space if we don't intern it.
-    ["state_key", "type", "sender", "room_id", "membership"].forEach((prop) => {
+    ["state_key", "type", "sender", "room_id"].forEach((prop) => {
        if (!event[prop]) {
            return;
        }
@@ -111,6 +108,8 @@ module.exports.MatrixEvent = function MatrixEvent(
    this.error = null;
    this.forwardLooking = true;
    this._pushActions = null;
+    this._date = this.event.origin_server_ts ?
+        new Date(this.event.origin_server_ts) : null;

    this._clearEvent = {};

@@ -205,7 +204,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {
     * @return {Date} The event date, e.g. <code>new Date(1433502692297)</code>
     */
    getDate: function() {
-        return this.event.origin_server_ts ? new Date(this.event.origin_server_ts) : null;
+        return this._date;
    },

    /**
@@ -378,21 +377,6 @@ utils.extend(module.exports.MatrixEvent.prototype, {
        return this._decryptionPromise;
    },

-    /**
-     * Cancel any room key request for this event and resend another.
-     *
-     * @param {module:crypto} crypto crypto module
-     */
-    cancelAndResendKeyRequest: function(crypto) {
-        const wireContent = this.getWireContent();
-        crypto.cancelRoomKeyRequest({
-            algorithm: wireContent.algorithm,
-            room_id: this.getRoomId(),
-            session_id: wireContent.session_id,
-            sender_key: wireContent.sender_key,
-        }, true);
-    },
-
    _decryptionLoop: async function(crypto) {
        // make sure that this method never runs completely synchronously.
        // (doing so would mean that we would clear _decryptionPromise *before*
@@ -440,7 +424,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {
                    // decryption error, but we have a retry queued.
                    console.log(
                        `Got error decrypting event (id=${this.getId()}: ` +
-                        `${e}), but retrying`,
+                        `${e.message}), but retrying`,
                    );
                    continue;
                }
@@ -448,7 +432,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {
                // decryption error, no retries queued. Warn about the error and
                // set it to m.bad.encrypted.
                console.warn(
-                    `Error decrypting event (id=${this.getId()}): ${e.detailedString}`,
+                    `Error decrypting event (id=${this.getId()}): ${e}`,
                );

                res = this._badEncryptedMessage(e.message);
@@ -668,6 +652,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {
        this.event = event;
        // successfully sent.
        this.status = null;
+        this._date = new Date(this.event.origin_server_ts);
     },
 });

@@ -229,21 +229,11 @@ function calculateDisplayName(member, event, roomState) {
        return displayName;
    }

-
-    // Check if the name contains something that look like a mxid
-    // If it does, it may be someone trying to impersonate someone else
-    // Show full mxid in this case
-    // Also show mxid if there are other people with the same displayname
-    let disambiguate = /@.+:.+/.test(displayName);
-    if (!disambiguate) {
-        const userIds = roomState.getUserIdsWithDisplayName(displayName);
-        const otherUsers = userIds.filter(function(u) {
-            return u !== selfUserId;
-        });
-        disambiguate = otherUsers.length > 0;
-    }
-
-    if (disambiguate) {
+    const userIds = roomState.getUserIdsWithDisplayName(displayName);
+    const otherUsers = userIds.filter(function(u) {
+        return u !== selfUserId;
+    });
+    if (otherUsers.length > 0) {
        return displayName + " (" + selfUserId + ")";
    }
    return displayName;
@@ -24,25 +24,6 @@ const RoomMember = require("./room-member");

 /**
 * Construct room state.
- *
- * Room State represents the state of the room at a given point.
- * It can be mutated by adding state events to it.
- * There are two types of room member associated with a state event:
- * normal member objects (accessed via getMember/getMembers) which mutate
- * with the state to represent the current state of that room/user, eg.
- * the object returned by getMember('@bob:example.com') will mutate to
- * get a different display name if Bob later changes his display name
- * in the room.
- * There are also 'sentinel' members (accessed via getSentinelMember).
- * These also represent the state of room members at the point in time
- * represented by the RoomState object, but unlike objects from getMember,
- * sentinel objects will always represent the room state as at the time
- * getSentinelMember was called, so if Bob subsequently changes his display
- * name, a room member object previously acquired with getSentinelMember
- * will still have his old display name. Calling getSentinelMember again
- * after the display name change will return a new RoomMember object
- * with Bob's new display name.
- *
 * @constructor
 * @param {?string} roomId Optional. The ID of the room which has this state.
 * If none is specified it just tracks paginationTokens, useful for notifTimelineSet
@@ -69,24 +50,9 @@ function RoomState(roomId) {
    this._displayNameToUserIds = {};
    this._userIdsToDisplayNames = {};
    this._tokenToInvite = {}; // 3pid invite state_key to m.room.member invite
-    this._joinedMemberCount = null; // cache of the number of joined members
 }
 utils.inherits(RoomState, EventEmitter);

-/**
- * Returns the number of joined members in this room
- * This method caches the result.
- * @return {integer} The number of members in this room whose membership is 'join'
- */
-RoomState.prototype.getJoinedMemberCount = function() {
-    if (this._joinedMemberCount === null) {
-        this._joinedMemberCount = this.getMembers().filter((m) => {
-            return m.membership === 'join';
-        }).length;
-    }
-    return this._joinedMemberCount;
-};
-
 /**
 * Get all RoomMembers in this room.
 * @return {Array<RoomMember>} A list of RoomMembers.
@@ -114,21 +80,7 @@ RoomState.prototype.getMember = function(userId) {
 * @return {RoomMember} The member or null if they do not exist.
 */
 RoomState.prototype.getSentinelMember = function(userId) {
-    if (!userId) return null;
-    let sentinel = this._sentinels[userId];
-
-    if (sentinel === undefined) {
-        sentinel = new RoomMember(this.roomId, userId);
-        const membershipEvent = this.getStateEvents("m.room.member", userId);
-        if (!membershipEvent) return null;
-        sentinel.setMembershipEvent(membershipEvent, this);
-        const pwrLvlEvent = this.getStateEvents("m.room.power_levels", "");
-        if (pwrLvlEvent) {
-            sentinel.setPowerLevelEvent(pwrLvlEvent);
-        }
-        this._sentinels[userId] = sentinel;
-    }
-    return sentinel;
+    return this._sentinels[userId] || null;
 };

 /**
@@ -221,19 +173,23 @@ RoomState.prototype.setStateEvents = function(stateEvents) {
                member = new RoomMember(event.getRoomId(), userId);
                self.emit("RoomState.newMember", event, self, member);
            }
+            // Add a new sentinel for this change. We apply the same
+            // operations to both sentinel and member rather than deep copying
+            // so we don't make assumptions about the properties of RoomMember
+            // (e.g. and manage to break it because deep copying doesn't do
+            // everything).
+            const sentinel = new RoomMember(event.getRoomId(), userId);
+            utils.forEach([member, sentinel], function(roomMember) {
+                roomMember.setMembershipEvent(event, self);
+                // this member may have a power level already, so set it.
+                const pwrLvlEvent = self.getStateEvents("m.room.power_levels", "");
+                if (pwrLvlEvent) {
+                    roomMember.setPowerLevelEvent(pwrLvlEvent);
+                }
+            });

-            member.setMembershipEvent(event, self);
-            // this member may have a power level already, so set it.
-            const pwrLvlEvent = self.getStateEvents("m.room.power_levels", "");
-            if (pwrLvlEvent) {
-                member.setPowerLevelEvent(pwrLvlEvent);
-            }
-
-            // blow away the sentinel which is now outdated
-            delete self._sentinels[userId];
-
+            self._sentinels[userId] = sentinel;
            self.members[userId] = member;
-            self._joinedMemberCount = null;
            self.emit("RoomState.members", event, self, member);
        } else if (event.getType() === "m.room.power_levels") {
            const members = utils.values(self.members);
@@ -242,8 +198,15 @@ RoomState.prototype.setStateEvents = function(stateEvents) {
                self.emit("RoomState.members", event, self, member);
            });

-            // assume all our sentinels are now out-of-date
-            self._sentinels = {};
+            // Go through the sentinel members and see if any of them would be
+            // affected by the new power levels. If so, replace the sentinel.
+            for (const userId of Object.keys(self._sentinels)) {
+                const oldSentinel = self._sentinels[userId];
+                const newSentinel = new RoomMember(event.getRoomId(), userId);
+                newSentinel.setMembershipEvent(oldSentinel.events.member, self);
+                newSentinel.setPowerLevelEvent(event);
+                self._sentinels[userId] = newSentinel;
+            }
        }
    });
 };
@@ -305,11 +268,7 @@ RoomState.prototype.maySendRedactionForEvent = function(mxEvent, userId) {
    if (!member || member.membership === 'leave') return false;

    if (mxEvent.status || mxEvent.isRedacted()) return false;
-
-    // The user may have been the sender, but they can't redact their own message
-    // if redactions are blocked.
-    const canRedact = this.maySendEvent("m.room.redaction", userId);
-    if (mxEvent.getSender() === userId) return canRedact;
+    if (mxEvent.getSender() === userId) return true;

    return this._hasSufficientPowerLevelFor('redact', member.powerLevel);
 };
@@ -169,8 +169,7 @@ function Room(roomId, opts) {
        this._pendingEventList = [];
    }

-    // read by megolm; boolean value - null indicates "use global value"
-    this._blacklistUnverifiedDevices = null;
+    this._blacklistUnverifiedDevices = false; // read by megolm
 }
 utils.inherits(Room, EventEmitter);

@@ -308,8 +307,8 @@ Room.prototype.setUnreadNotificationCount = function(type, count) {

 /**
 * Whether to send encrypted messages to devices within this room.
- * @param {Boolean} value true to blacklist unverified devices, null
- * to use the global value for this room.
+ * Will be ignored if MatrixClient's blacklistUnverifiedDevices setting is true.
+ * @param {boolean} value if true, blacklist unverified devices.
 */
 Room.prototype.setBlacklistUnverifiedDevices = function(value) {
    this._blacklistUnverifiedDevices = value;
@@ -317,8 +316,8 @@ Room.prototype.setBlacklistUnverifiedDevices = function(value) {

 /**
 * Whether to send encrypted messages to devices within this room.
- * @return {Boolean} true if blacklisting unverified devices, null
- * if the global value should be used for this room.
+ * Will be ignored if MatrixClient's blacklistUnverifiedDevices setting is true.
+ * @return {boolean} true if blacklisting unverified devices.
 */
 Room.prototype.getBlacklistUnverifiedDevices = function() {
    return this._blacklistUnverifiedDevices;
@@ -597,8 +596,7 @@ Room.prototype._addLiveEvent = function(event, duplicateStrategy) {
    // synthesize and inject implicit read receipts
    // Done after adding the event because otherwise the app would get a read receipt
    // pointing to an event that wasn't yet in the timeline
-    // Don't synthesize RR for m.room.redaction as this causes the RR to go missing.
-    if (event.sender && event.getType() !== "m.room.redaction") {
+    if (event.sender) {
        this.addReceipt(synthesizeReceipt(
            event.sender.userId, event, "m.read",
        ), true);
@@ -1221,9 +1219,6 @@ function calculateRoomName(room, userId, ignoreRoomNameEvent) {


    if (otherMembers.length === 0) {
-        const leftMembers = utils.filter(room.currentState.getMembers(), function(m) {
-            return m.userId !== userId && m.membership === "leave";
-        });
        if (allMembers.length === 1) {
            // self-chat, peeked room with 1 participant,
            // or inbound invite, or outbound 3PID invite.
@@ -1243,10 +1238,6 @@ function calculateRoomName(room, userId, ignoreRoomNameEvent) {
                        }
                    }
                    return name;
-                } else if (leftMembers.length === 1) {
-                    // if it was a chat with one person who's now left, it's still
-                    // notionally a chat with them
-                    return leftMembers[0].name;
                } else {
                    return "Empty room";
                }
@@ -30,10 +30,6 @@ function PushProcessor(client) {
        return string.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
    };

-    const cachedGlobToRegex = {
-        // $glob: RegExp,
-    };
-
    const matchingRuleFromKindSet = (ev, kindset, device) => {
        for (let ruleKindIndex = 0;
                ruleKindIndex < RULEKINDS_IN_ORDER.length;
@@ -79,7 +75,7 @@ function PushProcessor(client) {
                rawrule.conditions.push({
                    'kind': 'event_match',
                    'key': 'room_id',
-                    'value': tprule.rule_id,
+                    'pattern': tprule.rule_id,
                });
                break;
            case 'sender':
@@ -89,7 +85,7 @@ function PushProcessor(client) {
                rawrule.conditions.push({
                    'kind': 'event_match',
                    'key': 'user_id',
-                    'value': tprule.rule_id,
+                    'pattern': tprule.rule_id,
                });
                break;
            case 'content':
@@ -156,7 +152,9 @@ function PushProcessor(client) {
            return false;
        }

-        const memberCount = room.currentState.getJoinedMemberCount();
+        const memberCount = Object.keys(room.currentState.members).filter(function(m) {
+            return room.currentState.members[m].membership == 'join';
+        }).length;

        const m = cond.is.match(/^([=<>]*)([0-9]*)$/);
        if (!m) {
@@ -209,41 +207,21 @@ function PushProcessor(client) {
    };

    const eventFulfillsEventMatchCondition = function(cond, ev) {
-        if (!cond.key) {
-            return false;
-        }
-
        const val = valueForDottedKey(cond.key, ev);
        if (!val || typeof val != 'string') {
            return false;
        }

-        if (cond.value) {
-            return cond.value === val;
-        }
-
-        let regex;
-
+        let pat;
        if (cond.key == 'content.body') {
-            regex = createCachedRegex('(^|\\W)', cond.pattern, '(\\W|$)');
+            pat = '(^|\\W)' + globToRegexp(cond.pattern) + '(\\W|$)';
        } else {
-            regex = createCachedRegex('^', cond.pattern, '$');
+            pat = '^' + globToRegexp(cond.pattern) + '$';
        }
-
+        const regex = new RegExp(pat, 'i');
        return !!val.match(regex);
    };

-    const createCachedRegex = function(prefix, glob, suffix) {
-        if (cachedGlobToRegex[glob]) {
-            return cachedGlobToRegex[glob];
-        }
-        cachedGlobToRegex[glob] = new RegExp(
-            prefix + globToRegexp(glob) + suffix,
-            'i', // Case insensitive
-        );
-        return cachedGlobToRegex[glob];
-    };
-
    const globToRegexp = function(glob) {
        // From
        // https://github.com/matrix-org/synapse/blob/abbee6b29be80a77e05730707602f3bbfc3f38cb/synapse/push/__init__.py#L132
@@ -1,6 +1,5 @@
 /*
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -102,7 +101,6 @@ const LocalIndexedDBStoreBackend = function LocalIndexedDBStoreBackend(
    this.indexedDB = indexedDBInterface;
    this._dbName = "matrix-js-sdk:" + (dbName || "default");
    this.db = null;
-    this._disconnected = true;
    this._syncAccumulator = new SyncAccumulator();
 };

@@ -114,17 +112,15 @@ LocalIndexedDBStoreBackend.prototype = {
     * @return {Promise} Resolves if successfully connected.
     */
    connect: function() {
-        if (!this._disconnected) {
+        if (this.db) {
            console.log(
-                `LocalIndexedDBStoreBackend.connect: already connected or connecting`,
+                `LocalIndexedDBStoreBackend.connect: already connected`,
            );
            return Promise.resolve();
        }

-        this._disconnected = false;
-
        console.log(
-            `LocalIndexedDBStoreBackend.connect: connecting...`,
+            `LocalIndexedDBStoreBackend.connect: connecting`,
        );
        const req = this.indexedDB.open(this._dbName, VERSION);
        req.onupgradeneeded = (ev) => {
@@ -146,7 +142,7 @@ LocalIndexedDBStoreBackend.prototype = {
        };

        console.log(
-            `LocalIndexedDBStoreBackend.connect: awaiting connection...`,
+            `LocalIndexedDBStoreBackend.connect: awaiting connection`,
        );
        return promiseifyRequest(req).then((ev) => {
            console.log(
@@ -223,7 +219,7 @@ LocalIndexedDBStoreBackend.prototype = {

    /**
     * @param {boolean=} copy If false, the data returned is from internal
-     * buffers and must not be mutated. Otherwise, a copy is made before
+     * buffers and must not be muated. Otherwise, a copy is made before
     * returning such that the data can be safely mutated. Default: true.
     *
     * @return {Promise} Resolves with a sync response to restore the
@@ -244,10 +240,6 @@ LocalIndexedDBStoreBackend.prototype = {
        }
    },

-    getNextBatchToken: function() {
-        return Promise.resolve(this._syncAccumulator.getNextBatchToken());
-    },
-
    setSyncData: function(syncData) {
        return Promise.resolve().then(() => {
            this._syncAccumulator.accumulate(syncData);
@@ -349,18 +341,16 @@ LocalIndexedDBStoreBackend.prototype = {
     */
    _loadAccountData: function() {
        console.log(
-            `LocalIndexedDBStoreBackend: loading account data...`,
+            `LocalIndexedDBStoreBackend: loading account data`,
        );
        return Promise.try(() => {
+            console.log(
+                `LocalIndexedDBStoreBackend: loaded account data`,
+            );
            const txn = this.db.transaction(["accountData"], "readonly");
            const store = txn.objectStore("accountData");
            return selectQuery(store, undefined, (cursor) => {
                return cursor.value;
-            }).then((result) => {
-                console.log(
-                    `LocalIndexedDBStoreBackend: loaded account data`,
-                );
-                return result;
            });
        });
    },
@@ -371,17 +361,17 @@ LocalIndexedDBStoreBackend.prototype = {
     */
    _loadSyncData: function() {
        console.log(
-            `LocalIndexedDBStoreBackend: loading sync data...`,
+            `LocalIndexedDBStoreBackend: loaded sync data`,
        );
        return Promise.try(() => {
+            console.log(
+                `LocalIndexedDBStoreBackend: loaded sync data`,
+            );
            const txn = this.db.transaction(["sync"], "readonly");
            const store = txn.objectStore("sync");
            return selectQuery(store, undefined, (cursor) => {
                return cursor.value;
            }).then((results) => {
-                console.log(
-                    `LocalIndexedDBStoreBackend: loaded sync data`,
-                );
                if (results.length > 1) {
                    console.warn("loadSyncData: More than 1 sync row found.");
                }
@@ -1,6 +1,5 @@
 /*
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -27,23 +26,25 @@ import Promise from 'bluebird';
 * @param {string} workerScript URL to the worker script
 * @param {string=} dbName Optional database name. The same name must be used
 * to open the same database.
- * @param {Object} workerApi The web worker compatible interface object
+ * @param {Object} WorkerApi The web worker compatible interface object
 */
 const RemoteIndexedDBStoreBackend = function RemoteIndexedDBStoreBackend(
-    workerScript, dbName, workerApi,
+    workerScript, dbName, WorkerApi,
 ) {
-    this._workerScript = workerScript;
    this._dbName = dbName;
-    this._workerApi = workerApi;
-    this._worker = null;
+    this._worker = new WorkerApi(workerScript);
    this._nextSeq = 0;
    // The currently in-flight requests to the actual backend
    this._inFlight = {
        // seq: promise,
    };
-    // Once we start connecting, we keep the promise and re-use it
-    // if we try to connect again
-    this._startPromise = null;
+
+    this._worker.onmessage = this._onWorkerMessage.bind(this);
+
+    // tell the worker the db name.
+    this._startPromise = this._doCmd('_setupWorker', [this._dbName]).then(() => {
+        console.log("IndexedDB worker is ready");
+    });
 };


@@ -54,7 +55,7 @@ RemoteIndexedDBStoreBackend.prototype = {
     * @return {Promise} Resolves if successfully connected.
     */
    connect: function() {
-        return this._ensureStarted().then(() => this._doCmd('connect'));
+        return this._startPromise.then(() => this._doCmd('connect'));
    },

    /**
@@ -63,7 +64,7 @@ RemoteIndexedDBStoreBackend.prototype = {
     * @return {Promise} Resolved when the database is cleared.
     */
    clearDatabase: function() {
-        return this._ensureStarted().then(() => this._doCmd('clearDatabase'));
+        return this._startPromise.then(() => this._doCmd('clearDatabase'));
    },

    /**
@@ -75,10 +76,6 @@ RemoteIndexedDBStoreBackend.prototype = {
        return this._doCmd('getSavedSync');
    },

-    getNextBatchToken: function() {
-        return this._doCmd('getNextBatchToken');
-    },
-
    setSyncData: function(syncData) {
        return this._doCmd('setSyncData', [syncData]);
    },
@@ -96,19 +93,6 @@ RemoteIndexedDBStoreBackend.prototype = {
        return this._doCmd('getUserPresenceEvents');
    },

-    _ensureStarted: function() {
-        if (this._startPromise === null) {
-            this._worker = new this._workerApi(this._workerScript);
-            this._worker.onmessage = this._onWorkerMessage.bind(this);
-
-            // tell the worker the db name.
-            this._startPromise = this._doCmd('_setupWorker', [this._dbName]).then(() => {
-                console.log("IndexedDB worker is ready");
-            });
-        }
-        return this._startPromise;
-    },
-
    _doCmd: function(cmd, args) {
        // wrap in a q so if the postMessage throws,
        // the promise automatically gets rejected
@@ -1,6 +1,5 @@
 /*
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -89,9 +88,6 @@ class IndexedDBStoreWorker {
            case 'getUserPresenceEvents':
                prom = this.backend.getUserPresenceEvents();
                break;
-            case 'getNextBatchToken':
-                prom = this.backend.getNextBatchToken();
-                break;
        }

        if (prom === undefined) {
@@ -1,6 +1,5 @@
 /*
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -146,14 +145,6 @@ IndexedDBStore.prototype.getSavedSync = function() {
    return this.backend.getSavedSync();
 };

-/**
- * @return {Promise} If there is a saved sync, the nextBatch token
- * for this sync, otherwise null.
- */
-IndexedDBStore.prototype.getSavedSyncToken = function() {
-    return this.backend.getNextBatchToken();
-},
-
 /**
 * Delete all data from this store.
 * @return {Promise} Resolves if the data was deleted from the database.
@@ -168,27 +159,13 @@ IndexedDBStore.prototype.deleteAllData = function() {
    });
 };

-/**
- * Whether this store would like to save its data
- * Note that obviously whether the store wants to save or
- * not could change between calling this function and calling
- * save().
- *
- * @return {boolean} True if calling save() will actually save
- *     (at the time this function is called).
- */
-IndexedDBStore.prototype.wantsSave = function() {
-    const now = Date.now();
-    return now - this._syncTs > WRITE_DELAY_MS;
-};
-
 /**
 * Possibly write data to the database.
- * @return {Promise} Promise resolves after the write completes
- *     (or immediately if no write is performed)
+ * @return {Promise} Promise resolves after the write completes.
 */
 IndexedDBStore.prototype.save = function() {
-    if (this.wantsSave()) {
+    const now = Date.now();
+    if (now - this._syncTs > WRITE_DELAY_MS) {
        return this._reallySave();
    }
    return Promise.resolve();
@@ -1,7 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -316,15 +315,6 @@ module.exports.MatrixInMemoryStore.prototype = {
        return Promise.resolve();
    },

-    /**
-     * We never want to save becase we have nothing to save to.
-     *
-     * @return {boolean} If the store wants to save
-     */
-    wantsSave: function() {
-        return false;
-    },
-
    /**
     * Save does nothing as there is no backing data store.
     */
@@ -347,14 +337,6 @@ module.exports.MatrixInMemoryStore.prototype = {
        return Promise.resolve(null);
    },

-    /**
-     * @return {Promise} If there is a saved sync, the nextBatch token
-     * for this sync, otherwise null.
-     */
-    getSavedSyncToken: function() {
-        return Promise.resolve(null);
-    },
-
    /**
     * Delete all data from this store.
     * @return {Promise} An immediately resolved promise.
@@ -1,7 +1,5 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
-Copyright 2017 New Vector Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -50,18 +48,17 @@ function WebStorageSessionStore(webStore) {
 }

 WebStorageSessionStore.prototype = {
+
    /**
-     * Remove the stored end to end account for the logged-in user.
+     * Store the end to end account for the logged-in user.
+     * @param {string} account Base64 encoded account.
     */
-    removeEndToEndAccount: function() {
-        this.store.removeItem(KEY_END_TO_END_ACCOUNT);
+    storeEndToEndAccount: function(account) {
+        this.store.setItem(KEY_END_TO_END_ACCOUNT, account);
    },

    /**
     * Load the end to end account for the logged-in user.
-     * Note that the end-to-end account is now stored in the
-     * crypto store rather than here: this remains here so
-     * old sessions can be migrated out of the session store.
     * @return {?string} Base64 encoded account.
     */
    getEndToEndAccount: function() {
@@ -69,24 +66,43 @@ WebStorageSessionStore.prototype = {
    },

    /**
-     * Retrieves the known devices for all users.
-     * @return {object} A map from user ID to map of device ID to keys for the device.
+     * Stores the known devices for a user.
+     * @param {string} userId The user's ID.
+     * @param {object} devices A map from device ID to keys for the device.
     */
-    getAllEndToEndDevices: function() {
-        const prefix = keyEndToEndDevicesForUser('');
-        const devices = {};
-        for (let i = 0; i < this.store.length; ++i) {
-            const key = this.store.key(i);
-            const userId = key.substr(prefix.length);
-            if (key.startsWith(prefix)) devices[userId] = getJsonItem(this.store, key);
-        }
-        return devices;
+    storeEndToEndDevicesForUser: function(userId, devices) {
+        setJsonItem(this.store, keyEndToEndDevicesForUser(userId), devices);
+    },
+
+    /**
+     * Retrieves the known devices for a user.
+     * @param {string} userId The user's ID.
+     * @return {object} A map from device ID to keys for the device.
+     */
+    getEndToEndDevicesForUser: function(userId) {
+        return getJsonItem(this.store, keyEndToEndDevicesForUser(userId));
+    },
+
+    storeEndToEndDeviceTrackingStatus: function(statusMap) {
+        setJsonItem(this.store, KEY_END_TO_END_DEVICE_LIST_TRACKING_STATUS, statusMap);
    },

    getEndToEndDeviceTrackingStatus: function() {
        return getJsonItem(this.store, KEY_END_TO_END_DEVICE_LIST_TRACKING_STATUS);
    },

+    /**
+     * Store the sync token corresponding to the device list.
+     *
+     * This is used when starting the client, to get a list of the users who
+     * have changed their device list since the list time we were running.
+     *
+     * @param {String?} token
+     */
+    storeEndToEndDeviceSyncToken: function(token) {
+        setJsonItem(this.store, KEY_END_TO_END_DEVICE_SYNC_TOKEN, token);
+    },
+
    /**
     * Get the sync token corresponding to the device list.
     *
@@ -97,12 +113,17 @@ WebStorageSessionStore.prototype = {
    },

    /**
-     * Removes all end to end device data from the store
+     * Store a session between the logged-in user and another device
+     * @param {string} deviceKey The public key of the other device.
+     * @param {string} sessionId The ID for this end-to-end session.
+     * @param {string} session Base64 encoded end-to-end session.
     */
-    removeEndToEndDeviceData: function() {
-        removeByPrefix(this.store, keyEndToEndDevicesForUser(''));
-        removeByPrefix(this.store, KEY_END_TO_END_DEVICE_LIST_TRACKING_STATUS);
-        removeByPrefix(this.store, KEY_END_TO_END_DEVICE_SYNC_TOKEN);
+    storeEndToEndSession: function(deviceKey, sessionId, session) {
+        const sessions = this.getEndToEndSessions(deviceKey) || {};
+        sessions[sessionId] = session;
+        setJsonItem(
+            this.store, keyEndToEndSessions(deviceKey), sessions,
+        );
    },

    /**
@@ -115,29 +136,6 @@ WebStorageSessionStore.prototype = {
        return getJsonItem(this.store, keyEndToEndSessions(deviceKey));
    },

-    /**
-     * Retrieve all end-to-end sessions between the logged-in user and other
-     * devices.
-     * @return {object} A map of {deviceKey -> {sessionId -> session pickle}}
-     */
-    getAllEndToEndSessions: function() {
-        const deviceKeys = getKeysWithPrefix(this.store, keyEndToEndSessions(''));
-        const results = {};
-        for (const k of deviceKeys) {
-            const unprefixedKey = k.substr(keyEndToEndSessions('').length);
-            results[unprefixedKey] = getJsonItem(this.store, k);
-        }
-        return results;
-    },
-
-    /**
-     * Remove all end-to-end sessions from the store
-     * This is used after migrating sessions awat from the sessions store.
-     */
-    removeAllEndToEndSessions: function() {
-        removeByPrefix(this.store, keyEndToEndSessions(''));
-    },
-
    /**
     * Retrieve a list of all known inbound group sessions
     *
@@ -169,26 +167,27 @@ WebStorageSessionStore.prototype = {
        return this.store.getItem(key);
    },

-    removeAllEndToEndInboundGroupSessions: function() {
-        removeByPrefix(this.store, E2E_PREFIX + 'inboundgroupsessions/');
+    storeEndToEndInboundGroupSession: function(senderKey, sessionId, pickledSession) {
+        const key = keyEndToEndInboundGroupSession(senderKey, sessionId);
+        return this.store.setItem(key, pickledSession);
    },

    /**
-     * Get the end-to-end state for all rooms
-     * @return {object} roomId -> object with the end-to-end info for the room.
+     * Store the end-to-end state for a room.
+     * @param {string} roomId The room's ID.
+     * @param {object} roomInfo The end-to-end info for the room.
     */
-    getAllEndToEndRooms: function() {
-        const roomKeys = getKeysWithPrefix(this.store, keyEndToEndRoom(''));
-        const results = {};
-        for (const k of roomKeys) {
-            const unprefixedKey = k.substr(keyEndToEndRoom('').length);
-            results[unprefixedKey] = getJsonItem(this.store, k);
-        }
-        return results;
+    storeEndToEndRoom: function(roomId, roomInfo) {
+        setJsonItem(this.store, keyEndToEndRoom(roomId), roomInfo);
    },

-    removeAllEndToEndRooms: function() {
-        removeByPrefix(this.store, keyEndToEndRoom(''));
+    /**
+     * Get the end-to-end state for a room
+     * @param {string} roomId The room's ID.
+     * @return {object} The end-to-end info for the room.
+     */
+    getEndToEndRoom: function(roomId) {
+        return getJsonItem(this.store, keyEndToEndRoom(roomId));
    },
 };

@@ -224,24 +223,8 @@ function getJsonItem(store, key) {
    return null;
 }

-function getKeysWithPrefix(store, prefix) {
-    const results = [];
-    for (let i = 0; i < store.length; ++i) {
-        const key = store.key(i);
-        if (key.startsWith(prefix)) results.push(key);
-    }
-    return results;
-}
-
-function removeByPrefix(store, prefix) {
-    const toRemove = [];
-    for (let i = 0; i < store.length; ++i) {
-        const key = store.key(i);
-        if (key.startsWith(prefix)) toRemove.push(key);
-    }
-    for (const key of toRemove) {
-        store.removeItem(key);
-    }
+function setJsonItem(store, key, val) {
+    store.setItem(key, JSON.stringify(val));
 }

 function debuglog() {
@@ -1,7 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -217,15 +216,6 @@ StubStore.prototype = {
        return Promise.resolve();
    },

-    /**
-     * We never want to save becase we have nothing to save to.
-     *
-     * @return {boolean} If the store wants to save
-     */
-    wantsSave: function() {
-        return false;
-    },
-
    /**
     * Save does nothing as there is no backing data store.
     */
@@ -248,14 +238,6 @@ StubStore.prototype = {
        return Promise.resolve(null);
    },

-    /**
-     * @return {Promise} If there is a saved sync, the nextBatch token
-     * for this sync, otherwise null.
-     */
-    getSavedSyncToken: function() {
-        return Promise.resolve(null);
-    },
-
    /**
     * Delete all data from this store. Does nothing since this store
     * doesn't store anything.
@@ -1,6 +1,5 @@
 /*
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -521,10 +520,6 @@ class SyncAccumulator {
            accountData: accData,
        };
    }
-
-    getNextBatchToken() {
-        return this.nextBatch;
-    }
 }

 function setState(eventMap, event) {
@@ -1,7 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
-Copyright 2018 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -72,8 +71,6 @@ function debuglog(...params) {
 * SAFELY remove events from this room. It may not be safe to remove events if
 * there are other references to the timelines for this room.
 * Default: returns false.
- * @param {Boolean=} opts.disablePresence True to perform syncing without automatically
- * updating presence.
 */
 function SyncApi(client, opts) {
    this.client = client;
@@ -396,17 +393,6 @@ SyncApi.prototype.getSyncState = function() {
    return this._syncState;
 };

-SyncApi.prototype.recoverFromSyncStartupError = async function(savedSyncPromise, err) {
-    // Wait for the saved sync to complete - we send the pushrules and filter requests
-    // before the saved sync has finished so they can run in parallel, but only process
-    // the results after the saved sync is done. Equivalently, we wait for it to finish
-    // before reporting failures from these functions.
-    await savedSyncPromise;
-    const keepaliveProm = this._startKeepAlives();
-    this._updateSyncState("ERROR", { error: err });
-    await keepaliveProm;
-};
-
 /**
 * Main entry point
 */
@@ -421,32 +407,26 @@ SyncApi.prototype.sync = function() {
        global.document.addEventListener("online", this._onOnlineBound, false);
    }

-    let savedSyncPromise = Promise.resolve();
-    let savedSyncToken = null;
-
    // We need to do one-off checks before we can begin the /sync loop.
    // These are:
    //   1) We need to get push rules so we can check if events should bing as we get
    //      them from /sync.
    //   2) We need to get/create a filter which we can use for /sync.

-    async function getPushRules() {
-        try {
-            const result = await client.getPushRules();
+    function getPushRules() {
+        client.getPushRules().done(function(result) {
            debuglog("Got push rules");
-
            client.pushRules = result;
-        } catch (err) {
-            // wait for saved sync to complete before doing anything else,
-            // otherwise the sync state will end up being incorrect
-            await self.recoverFromSyncStartupError(savedSyncPromise, err);
-            getPushRules();
-            return;
-        }
-        getFilter(); // Now get the filter and start syncing
+            getFilter(); // Now get the filter
+        }, function(err) {
+            self._startKeepAlives().done(function() {
+                getPushRules();
+            });
+            self._updateSyncState("ERROR", { error: err });
+        });
    }

-    async function getFilter() {
+    function getFilter() {
        let filter;
        if (self.opts.filter) {
            filter = self.opts.filter;
@@ -455,54 +435,28 @@ SyncApi.prototype.sync = function() {
            filter.setTimelineLimit(self.opts.initialSyncLimit);
        }

-        let filterId;
-        try {
-            filterId = await client.getOrCreateFilter(
-                getFilterName(client.credentials.userId), filter,
-            );
-        } catch (err) {
-            // wait for saved sync to complete before doing anything else,
-            // otherwise the sync state will end up being incorrect
-            await self.recoverFromSyncStartupError(savedSyncPromise, err);
-            getFilter();
-            return;
-        }
-        // reset the notifications timeline to prepare it to paginate from
-        // the current point in time.
-        // The right solution would be to tie /sync pagination tokens into
-        // /notifications API somehow.
-        client.resetNotifTimelineSet();
+        client.getOrCreateFilter(
+            getFilterName(client.credentials.userId), filter,
+        ).done(function(filterId) {
+            // reset the notifications timeline to prepare it to paginate from
+            // the current point in time.
+            // The right solution would be to tie /sync pagination tokens into
+            // /notifications API somehow.
+            client.resetNotifTimelineSet();

-        if (self._currentSyncRequest === null) {
-            // Send this first sync request here so we can then wait for the saved
-            // sync data to finish processing before we process the results of this one.
-            console.log("Sending first sync request...");
-            self._currentSyncRequest = self._doSyncRequest({ filterId }, savedSyncToken);
-        }
-
-        // Now wait for the saved sync to finish...
-        await savedSyncPromise;
-        self._sync({ filterId });
+            self._sync({ filterId: filterId });
+        }, function(err) {
+            self._startKeepAlives().done(function() {
+                getFilter();
+            });
+            self._updateSyncState("ERROR", { error: err });
+        });
    }

    if (client.isGuest()) {
        // no push rules for guests, no access to POST filter for guests.
        self._sync({});
    } else {
-        // Pull the saved sync token out first, before the worker starts sending
-        // all the sync data which could take a while. This will let us send our
-        // first incremental sync request before we've processed our saved data.
-        savedSyncPromise = client.store.getSavedSyncToken().then((tok) => {
-            savedSyncToken = tok;
-            return client.store.getSavedSync();
-        }).then((savedSync) => {
-            if (savedSync) {
-                return self._syncFromCache(savedSync);
-            }
-        });
-        // Now start the first incremental sync request: this can also
-        // take a while so if we set it going now, we can wait for it
-        // to finish while we process our saved sync data.
        getPushRules();
    }
 };
@@ -538,43 +492,6 @@ SyncApi.prototype.retryImmediately = function() {
    this._startKeepAlives(0);
    return true;
 };
-/**
- * Process a single set of cached sync data.
- * @param {Object} savedSync a saved sync that was persisted by a store. This
- * should have been acquired via client.store.getSavedSync().
- */
-SyncApi.prototype._syncFromCache = async function(savedSync) {
-    debuglog("sync(): not doing HTTP hit, instead returning stored /sync data");
-
-    const nextSyncToken = savedSync.nextBatch;
-
-    // Set sync token for future incremental syncing
-    this.client.store.setSyncToken(nextSyncToken);
-
-    // No previous sync, set old token to null
-    const syncEventData = {
-        oldSyncToken: null,
-        nextSyncToken,
-        catchingUp: false,
-    };
-
-    const data = {
-        next_batch: nextSyncToken,
-        rooms: savedSync.roomsData,
-        groups: savedSync.groupsData,
-        account_data: {
-            events: savedSync.accountData,
-        },
-    };
-
-    try {
-        await this._processSyncResponse(syncEventData, data);
-    } catch(e) {
-        console.error("Error processing cached sync", e.stack || e);
-    }
-
-    this._updateSyncState("PREPARED", syncEventData);
-};

 /**
 * Invoke me to do /sync calls
@@ -595,99 +512,13 @@ SyncApi.prototype._sync = async function(syncOptions) {
        return;
    }

+    let filterId = syncOptions.filterId;
+    if (client.isGuest() && !filterId) {
+        filterId = this._getGuestFilter();
+    }
+
    const syncToken = client.store.getSyncToken();

-    let data;
-    try {
-        //debuglog('Starting sync since=' + syncToken);
-        if (this._currentSyncRequest === null) {
-            this._currentSyncRequest = this._doSyncRequest(syncOptions, syncToken);
-        }
-        data = await this._currentSyncRequest;
-    } catch (e) {
-        this._onSyncError(e, syncOptions);
-        return;
-    } finally {
-        this._currentSyncRequest = null;
-    }
-
-    //debuglog('Completed sync, next_batch=' + data.next_batch);
-
-    // set the sync token NOW *before* processing the events. We do this so
-    // if something barfs on an event we can skip it rather than constantly
-    // polling with the same token.
-    client.store.setSyncToken(data.next_batch);
-
-    // Reset after a successful sync
-    this._failedSyncCount = 0;
-
-    await client.store.setSyncData(data);
-
-    const syncEventData = {
-        oldSyncToken: syncToken,
-        nextSyncToken: data.next_batch,
-        catchingUp: this._catchingUp,
-    };
-
-    if (this.opts.crypto) {
-        // tell the crypto module we're about to process a sync
-        // response
-        await this.opts.crypto.onSyncWillProcess(syncEventData);
-    }
-
-    try {
-        await this._processSyncResponse(syncEventData, data);
-    } catch(e) {
-        // log the exception with stack if we have it, else fall back
-        // to the plain description
-        console.error("Caught /sync error", e.stack || e);
-    }
-
-    // update this as it may have changed
-    syncEventData.catchingUp = this._catchingUp;
-
-    // emit synced events
-    if (!syncOptions.hasSyncedBefore) {
-        this._updateSyncState("PREPARED", syncEventData);
-        syncOptions.hasSyncedBefore = true;
-    }
-
-    // tell the crypto module to do its processing. It may block (to do a
-    // /keys/changes request).
-    if (this.opts.crypto) {
-        await this.opts.crypto.onSyncCompleted(syncEventData);
-    }
-
-    // keep emitting SYNCING -> SYNCING for clients who want to do bulk updates
-    this._updateSyncState("SYNCING", syncEventData);
-
-    if (client.store.wantsSave()) {
-        // We always save the device list (if it's dirty) before saving the sync data:
-        // this means we know the saved device list data is at least as fresh as the
-        // stored sync data which means we don't have to worry that we may have missed
-        // device changes. We can also skip the delay since we're not calling this very
-        // frequently (and we don't really want to delay the sync for it).
-        if (this.opts.crypto) {
-            await this.opts.crypto.saveDeviceList(0);
-        }
-
-        // tell databases that everything is now in a consistent state and can be saved.
-        client.store.save();
-    }
-
-    // Begin next sync
-    this._sync(syncOptions);
-};
-
-SyncApi.prototype._doSyncRequest = function(syncOptions, syncToken) {
-    const qps = this._getSyncParams(syncOptions, syncToken);
-    return this.client._http.authedRequest(
-        undefined, "GET", "/sync", qps, undefined,
-        qps.timeout + BUFFER_PERIOD_MS,
-    );
-};
-
-SyncApi.prototype._getSyncParams = function(syncOptions, syncToken) {
    let pollTimeout = this.opts.pollTimeout;

    if (this.getSyncState() !== 'SYNCING' || this._catchingUp) {
@@ -706,20 +537,14 @@ SyncApi.prototype._getSyncParams = function(syncOptions, syncToken) {
        pollTimeout = 0;
    }

-    let filterId = syncOptions.filterId;
-    if (this.client.isGuest() && !filterId) {
-        filterId = this._getGuestFilter();
-    }
+    // normal timeout= plus buffer time
+    const clientSideTimeoutMs = pollTimeout + BUFFER_PERIOD_MS;

    const qps = {
        filter: filterId,
        timeout: pollTimeout,
    };

-    if (this.opts.disablePresence) {
-        qps.set_presence = "offline";
-    }
-
    if (syncToken) {
        qps.since = syncToken;
    } else {
@@ -737,7 +562,97 @@ SyncApi.prototype._getSyncParams = function(syncOptions, syncToken) {
        qps.timeout = 0;
    }

-    return qps;
+    let savedSync;
+    if (!syncOptions.hasSyncedBefore) {
+        // Don't do an HTTP hit to /sync. Instead, load up the persisted /sync data,
+        // if there is data there.
+        savedSync = await client.store.getSavedSync();
+    }
+
+    let isCachedResponse = false;
+    let data;
+
+    if (savedSync) {
+        debuglog("sync(): not doing HTTP hit, instead returning stored /sync data");
+        isCachedResponse = true;
+        data = {
+            next_batch: savedSync.nextBatch,
+            rooms: savedSync.roomsData,
+            groups: savedSync.groupsData,
+            account_data: {
+                events: savedSync.accountData,
+            },
+        };
+    } else {
+        try {
+            //debuglog('Starting sync since=' + syncToken);
+            this._currentSyncRequest = client._http.authedRequest(
+                undefined, "GET", "/sync", qps, undefined, clientSideTimeoutMs,
+            );
+            data = await this._currentSyncRequest;
+        } catch (e) {
+            this._onSyncError(e, syncOptions);
+            return;
+        }
+    }
+
+    //debuglog('Completed sync, next_batch=' + data.next_batch);
+
+    // set the sync token NOW *before* processing the events. We do this so
+    // if something barfs on an event we can skip it rather than constantly
+    // polling with the same token.
+    client.store.setSyncToken(data.next_batch);
+
+    // Reset after a successful sync
+    this._failedSyncCount = 0;
+
+    // We need to wait until the sync data has been sent to the backend
+    // because it appears that the sync data gets modified somewhere in
+    // processing it in such a way as to make it no longer cloneable.
+    // XXX: Find out what is modifying it!
+    if (!isCachedResponse) {
+        // Don't give the store back its own cached data
+        await client.store.setSyncData(data);
+    }
+
+    try {
+        await this._processSyncResponse(syncToken, data, isCachedResponse);
+    } catch(e) {
+        // log the exception with stack if we have it, else fall back
+        // to the plain description
+        console.error("Caught /sync error", e.stack || e);
+    }
+
+    // emit synced events
+    const syncEventData = {
+        oldSyncToken: syncToken,
+        nextSyncToken: data.next_batch,
+        catchingUp: this._catchingUp,
+    };
+
+    if (!syncOptions.hasSyncedBefore) {
+        this._updateSyncState("PREPARED", syncEventData);
+        syncOptions.hasSyncedBefore = true;
+    }
+
+    if (!isCachedResponse) {
+        // tell the crypto module to do its processing. It may block (to do a
+        // /keys/changes request).
+        if (this.opts.crypto) {
+            await this.opts.crypto.onSyncCompleted(syncEventData);
+        }
+
+        // keep emitting SYNCING -> SYNCING for clients who want to do bulk updates
+        this._updateSyncState("SYNCING", syncEventData);
+
+        // tell databases that everything is now in a consistent state and can be
+        // saved (no point doing so if we only have the data we just got out of the
+        // store).
+        client.store.save();
+    }
+
+    // Begin next sync
+    this._sync(syncOptions);
 };

 SyncApi.prototype._onSyncError = function(err, syncOptions) {
@@ -781,11 +696,13 @@ SyncApi.prototype._onSyncError = function(err, syncOptions) {
 * Process data returned from a sync response and propagate it
 * into the model objects
 *
- * @param {Object} syncEventData Object containing sync tokens associated with this sync
+ * @param {string} syncToken the old next_batch token sent to this
+ *    sync request.
 * @param {Object} data The response from /sync
+ * @param {bool} isCachedResponse True if this response is from our local cache
 */
 SyncApi.prototype._processSyncResponse = async function(
-    syncEventData, data,
+    syncToken, data, isCachedResponse,
 ) {
    const client = this.client;
    const self = this;
@@ -865,11 +782,13 @@ SyncApi.prototype._processSyncResponse = async function(
        client.store.storeAccountDataEvents(events);
        events.forEach(
            function(accountDataEvent) {
-                // Honour push rules that come down the sync stream but also
-                // honour push rules that were previously cached. Base rules
-                // will be updated when we recieve push rules via getPushRules
-                // (see SyncApi.prototype.sync) before syncing over the network.
-                if (accountDataEvent.getType() == 'm.push_rules') {
+                // XXX: This is awful: ignore push rules from our cached sync. We fetch the
+                // push rules before syncing so we actually have up-to-date ones. We do want
+                // to honour new push rules that come down the sync but synapse doesn't
+                // put new push rules in the sync stream when the base rules change, so
+                // if the base rules change, we do need to refresh. We therefore ignore
+                // the push rules in our cached sync response.
+                if (accountDataEvent.getType() == 'm.push_rules' && !isCachedResponse) {
                    client.pushRules = accountDataEvent.getContent();
                }
                client.emit("accountData", accountDataEvent);
@@ -1025,8 +944,7 @@ SyncApi.prototype._processSyncResponse = async function(
                self._deregisterStateListeners(room);
                room.resetLiveTimeline(
                    joinObj.timeline.prev_batch,
-                    self.opts.canResetEntireTimeline(room.roomId) ?
-                        null : syncEventData.oldSyncToken,
+                    self.opts.canResetEntireTimeline(room.roomId) ? null : syncToken,
                );

                // We have to assume any gap in any timeline is
@@ -1110,7 +1028,7 @@ SyncApi.prototype._processSyncResponse = async function(
    // in the timeline relative to ones paginated in by /notifications.
    // XXX: we could fix this by making EventTimeline support chronological
    // ordering... but it doesn't, right now.
-    if (syncEventData.oldSyncToken && this._notifEvents.length) {
+    if (syncToken && this._notifEvents.length) {
        this._notifEvents.sort(function(a, b) {
            return a.getTs() - b.getTs();
        });
@@ -1122,9 +1040,7 @@ SyncApi.prototype._processSyncResponse = async function(
    // Handle device list updates
    if (data.device_lists) {
        if (this.opts.crypto) {
-            await this.opts.crypto.handleDeviceListChanges(
-                syncEventData, data.device_lists,
-            );
+            await this.opts.crypto.handleDeviceListChanges(data.device_lists);
        } else {
            // FIXME if we *don't* have a crypto module, we still need to
            // invalidate the device lists. But that would require a
@@ -1346,60 +1262,41 @@ SyncApi.prototype._resolveInvites = function(room) {
 */
 SyncApi.prototype._processRoomEvents = function(room, stateEventList,
                                                timelineEventList) {
-    // If there are no events in the timeline yet, initialise it with
-    // the given state events
-    const liveTimeline = room.getLiveTimeline();
-    const timelineWasEmpty = liveTimeline.getEvents().length == 0;
-    if (timelineWasEmpty) {
-        // Passing these events into initialiseState will freeze them, so we need
-        // to compute and cache the push actions for them now, otherwise sync dies
-        // with an attempt to assign to read only property.
-        // XXX: This is pretty horrible and is assuming all sorts of behaviour from
-        // these functions that it shouldn't be. We should probably either store the
-        // push actions cache elsewhere so we can freeze MatrixEvents, or otherwise
-        // find some solution where MatrixEvents are immutable but allow for a cache
-        // field.
-        for (const ev of stateEventList) {
-            this.client.getPushActionsForEvent(ev);
-        }
-        liveTimeline.initialiseState(stateEventList);
-    }
+    timelineEventList = timelineEventList || [];
+    const client = this.client;
+    // "old" and "current" state are the same initially; they
+    // start diverging if the user paginates.
+    // We must deep copy otherwise membership changes in old state
+    // will leak through to current state!
+    const oldStateEvents = utils.map(
+        utils.deepCopy(
+            stateEventList.map(function(mxEvent) {
+                return mxEvent.event;
+            }),
+        ), client.getEventMapper(),
+    );
+    const stateEvents = stateEventList;
+
+    // set the state of the room to as it was before the timeline executes
+    //
+    // XXX: what if we've already seen (some of) the events in the timeline,
+    // and they modify some of the state set in stateEvents? In that case we'll
+    // end up with the state from stateEvents, instead of the more recent state
+    // from the timeline.
+    room.oldState.setStateEvents(oldStateEvents);
+    room.currentState.setStateEvents(stateEvents);

    this._resolveInvites(room);

    // recalculate the room name at this point as adding events to the timeline
    // may make notifications appear which should have the right name.
-    // XXX: This looks suspect: we'll end up recalculating the room once here
-    // and then again after adding events (_processSyncResponse calls it after
-    // calling us) even if no state events were added. It also means that if
-    // one of the room events in timelineEventList is something that needs
-    // a recalculation (like m.room.name) we won't recalculate until we've
-    // finished adding all the events, which will cause the notification to have
-    // the old room name rather than the new one.
    room.recalculate(this.client.credentials.userId);

-    // If the timeline wasn't empty, we process the state events here: they're
-    // defined as updates to the state before the start of the timeline, so this
-    // starts to roll the state forward.
-    // XXX: That's what we *should* do, but this can happen if we were previously
-    // peeking in a room, in which case we obviously do *not* want to add the
-    // state events here onto the end of the timeline. Historically, the js-sdk
-    // has just set these new state events on the old and new state. This seems
-    // very wrong because there could be events in the timeline that diverge the
-    // state, in which case this is going to leave things out of sync. However,
-    // for now I think it;s best to behave the same as the code has done previously.
-    if (!timelineWasEmpty) {
-        // XXX: As above, don't do this...
-        //room.addLiveEvents(stateEventList || []);
-        // Do this instead...
-        room.oldState.setStateEvents(stateEventList || []);
-        room.currentState.setStateEvents(stateEventList || []);
-    }
-    // execute the timeline events. This will continue to diverge the current state
+    // execute the timeline events, this will begin to diverge the current state
    // if the timeline has any state events in it.
    // This also needs to be done before running push rules on the events as they need
    // to be decorated with sender etc.
-    room.addLiveEvents(timelineEventList || []);
+    room.addLiveEvents(timelineEventList);
 };

 /**
@@ -1,6 +1,5 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
-Copyright 2017 New Vector Ltd

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -25,6 +24,31 @@ const DEBUG = true;  // set true to enable console logging.

 // events: hangup, error(err), replaced(call), state(state, oldState)

+/**
+ * Fires when the MatrixCall encounters an error when sending a Matrix event.
+ * <p>
+ * This is required to allow errors, which occur during sending of events, to bubble up.
+ * (This is because call.js does a hangup when it encounters a normal `error`, which in
+ * turn could lead to an UnknownDeviceError.)
+ * <p>
+ * To deal with an UnknownDeviceError when trying to send events, the application should let
+ * users know that there are new devices in the encrypted room (into which the event was
+ * sent) and give the user the options to resend unsent events or cancel them. Resending
+ * is done using {@link module:client~MatrixClient#resendEvent} and cancelling can be done by using
+ * {@link module:client~MatrixClient#cancelPendingEvent}.
+ * <p>
+ * MatrixCall will not do anything in response to an error that causes `send_event_error`
+ * to be emitted with the exception of sending `m.call.candidates`, which is retried upon
+ * failure when ICE candidates are being sent. This happens during call setup.
+ *
+ * @event module:webrtc/call~MatrixCall#"send_event_error"
+ * @param {Error} err The error caught from calling client.sendEvent in call.js.
+ * @example
+ * matrixCall.on("send_event_error", function(err){
+ *   console.error(err);
+ * });
+ */
+
 /**
 * Fires whenever an error occurs when call.js encounters an issue with setting up the call.
 * <p>
@@ -86,8 +110,6 @@ function MatrixCall(opts) {
    this.mediaPromises = Object.create(null);

    this.screenSharingStream = null;
-
-    this._answerContent = null;
 }
 /** The length of time a call can be ringing for. */
 MatrixCall.CALL_TIMEOUT_MS = 60000;
@@ -101,24 +123,6 @@ MatrixCall.ERR_LOCAL_OFFER_FAILED = "local_offer_failed";
 */
 MatrixCall.ERR_NO_USER_MEDIA = "no_user_media";

-/*
- * Error code used when a call event failed to send
- * because unknown devices were present in the room
- */
-MatrixCall.ERR_UNKNOWN_DEVICES = "unknown_devices";
-
-/*
- * Error code usewd when we fail to send the invite
- * for some reason other than there being unknown devices
- */
-MatrixCall.ERR_SEND_INVITE = "send_invite";
-
-/*
- * Error code usewd when we fail to send the answer
- * for some reason other than there being unknown devices
- */
-MatrixCall.ERR_SEND_ANSWER = "send_answer";
-
 utils.inherits(MatrixCall, EventEmitter);

 /**
@@ -405,11 +409,6 @@ MatrixCall.prototype.answer = function() {
    debuglog("Answering call %s of type %s", this.callId, this.type);
    const self = this;

-    if (self._answerContent) {
-        self._sendAnswer();
-        return;
-    }
-
    if (!this.localAVStream && !this.waitForLocalAVStream) {
        this.webRtc.getUserMedia(
            _getUserMediaVideoContraints(this.type),
@@ -458,8 +457,6 @@ MatrixCall.prototype._replacedBy = function(newCall) {
 * @param {boolean} suppressEvent True to suppress emitting an event.
 */
 MatrixCall.prototype.hangup = function(reason, suppressEvent) {
-    if (this.state == 'ended') return;
-
    debuglog("Ending call " + this.callId);
    terminate(this, "local", reason, !suppressEvent);
    const content = {
@@ -596,28 +593,6 @@ MatrixCall.prototype._maybeGotUserMediaForInvite = function(stream) {
    setState(self, 'create_offer');
 };

-MatrixCall.prototype._sendAnswer = function(stream) {
-    sendEvent(this, 'm.call.answer', this._answerContent).then(() => {
-        setState(this, 'connecting');
-        // If this isn't the first time we've tried to send the answer,
-        // we may have candidates queued up, so send them now.
-        _sendCandidateQueue(this);
-    }).catch((error) => {
-        // We've failed to answer: back to the ringing state
-        setState(this, 'ringing');
-        this.client.cancelPendingEvent(error.event);
-
-        let code = MatrixCall.ERR_SEND_ANSWER;
-        let message = "Failed to send answer";
-        if (error.name == 'UnknownDeviceError') {
-            code = MatrixCall.ERR_UNKNOWN_DEVICES;
-            message = "Unknown devices present in the room";
-        }
-        this.emit("error", callError(code, message));
-        throw error;
-    });
-};
-
 /**
 * Internal
 * @private
@@ -666,7 +641,7 @@ MatrixCall.prototype._maybeGotUserMediaForAnswer = function(stream) {
    self.peerConn.createAnswer(function(description) {
        debuglog("Created answer: " + description);
        self.peerConn.setLocalDescription(description, function() {
-            self._answerContent = {
+            const content = {
                version: 0,
                call_id: self.callId,
                answer: {
@@ -674,7 +649,8 @@ MatrixCall.prototype._maybeGotUserMediaForAnswer = function(stream) {
                    type: self.peerConn.localDescription.type,
                },
            };
-            self._sendAnswer();
+            sendEvent(self, 'm.call.answer', content);
+            setState(self, 'connecting');
        }, function() {
            debuglog("Error setting local description!");
        }, constraints);
@@ -695,9 +671,6 @@ MatrixCall.prototype._gotLocalIceCandidate = function(event) {
            "Got local ICE " + event.candidate.sdpMid + " candidate: " +
            event.candidate.candidate,
        );
-
-        if (this.state == 'ended') return;
-
        // As with the offer, note we need to make a copy of this object, not
        // pass the original: that broke in Chrome ~m43.
        const c = {
@@ -780,26 +753,14 @@ MatrixCall.prototype._gotLocalOffer = function(description) {
            },
            lifetime: MatrixCall.CALL_TIMEOUT_MS,
        };
-        sendEvent(self, 'm.call.invite', content).then(() => {
-            setState(self, 'invite_sent');
-            setTimeout(function() {
-                if (self.state == 'invite_sent') {
-                    self.hangup('invite_timeout');
-                }
-            }, MatrixCall.CALL_TIMEOUT_MS);
-        }).catch((error) => {
-            let code = MatrixCall.ERR_SEND_INVITE;
-            let message = "Failed to send invite";
-            if (error.name == 'UnknownDeviceError') {
-                code = MatrixCall.ERR_UNKNOWN_DEVICES;
-                message = "Unknown devices present in the room";
-            }
+        sendEvent(self, 'm.call.invite', content);

-            self.client.cancelPendingEvent(error.event);
-            terminate(self, "local", code, false);
-            self.emit("error", callError(code, message));
-            throw error;
-        });
+        setTimeout(function() {
+            if (self.state == 'invite_sent') {
+                self.hangup('invite_timeout');
+            }
+        }, MatrixCall.CALL_TIMEOUT_MS);
+        setState(self, 'invite_sent');
    }, function() {
        debuglog("Error setting local description!");
    });
@@ -823,7 +784,6 @@ MatrixCall.prototype._getLocalOfferFailed = function(error) {
 * @param {Object} error
 */
 MatrixCall.prototype._getUserMediaFailed = function(error) {
-    terminate(this, "local", 'user_media_failed', false);
    this.emit(
        "error",
        callError(
@@ -832,6 +792,7 @@ MatrixCall.prototype._getUserMediaFailed = function(error) {
            "does this app have permission?",
        ),
    );
+    this.hangup("user_media_failed");
 };

 /**
@@ -1011,20 +972,17 @@ const setState = function(self, state) {
 * @return {Promise}
 */
 const sendEvent = function(self, eventType, content) {
-    return self.client.sendEvent(self.roomId, eventType, content);
+    return self.client.sendEvent(self.roomId, eventType, content).catch(
+        (err) => {
+            self.emit('send_event_error', err);
+        },
+    );
 };

 const sendCandidate = function(self, content) {
    // Sends candidates with are sent in a special way because we try to amalgamate
    // them into one message
    self.candidateSendQueue.push(content);
-
-    // Don't send the ICE candidates yet if the call is in the ringing state: this
-    // means we tried to pick (ie. started generating candidates) and then failed to
-    // send the answer and went back to the ringing state. Queue up the candidates
-    // to send if we sucessfully send the answer.
-    if (self.state == 'ringing') return;
-
    if (self.candidateSendTries === 0) {
        setTimeout(function() {
            _sendCandidateQueue(self);
Author	SHA1	Message	Date
David Baker	d764abc1d1	v0.9.2-cryptowraning.1	2018-03-26 14:11:29 +01:00
David Baker	ee0f8abe71	Prepare changelog for v0.9.2-cryptowraning.1	2018-03-26 14:11:29 +01:00
David Baker	b45e231968	Merge pull request #631 from matrix-org/dbkr/cryptowarning Disable crypto if indexeddb version too new	2018-03-26 14:04:42 +01:00
David Baker	c37db5ab9e	Remove added whitespace	2018-03-26 13:57:37 +01:00
David Baker	77abc45489	Disable crypto if indexeddb version too new	2018-03-23 17:29:35 +00:00