v0.14.0

Prevent messages from being sent if other messages have failed to send

CHANGELOG.md

@@ -1,3 +1,216 @@
+Changes in [0.14.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.14.0) (2018-11-19)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.14.0-rc.1...v0.14.0)
+
+ * No changes since rc.1
+
+Changes in [0.14.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.14.0-rc.1) (2018-11-15)
+============================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.13.1...v0.14.0-rc.1)
+
+BREAKING CHANGE
+----------------
+ 
+ * js-sdk now uses Olm 3.0. Apps using Olm must update to 3.0 to
+   continue using Olm with the js-sdk. The js-sdk will call Olm's
+   init() method when the client is started.
+
+All Changes
+-----------
+
+ * Prevent messages from being sent if other messages have failed to send
+   [\#781](https://github.com/matrix-org/matrix-js-sdk/pull/781)
+ * A unit test for olm
+   [\#777](https://github.com/matrix-org/matrix-js-sdk/pull/777)
+ * Set access_token and user_id after login in with username and password.
+   [\#778](https://github.com/matrix-org/matrix-js-sdk/pull/778)
+ * Add function to get currently joined rooms.
+   [\#779](https://github.com/matrix-org/matrix-js-sdk/pull/779)
+ * Remove the request-only stuff we don't need anymore
+   [\#775](https://github.com/matrix-org/matrix-js-sdk/pull/775)
+ * Manually construct query strings for browser-request instances
+   [\#770](https://github.com/matrix-org/matrix-js-sdk/pull/770)
+ * Fix: correctly check for crypto being present
+   [\#769](https://github.com/matrix-org/matrix-js-sdk/pull/769)
+ * Update babel-eslint to 8.1.1
+   [\#768](https://github.com/matrix-org/matrix-js-sdk/pull/768)
+ * Support `request` in the browser and support supplying servers to try in
+   joinRoom()
+   [\#764](https://github.com/matrix-org/matrix-js-sdk/pull/764)
+ * loglevel should be a normal dependency
+   [\#767](https://github.com/matrix-org/matrix-js-sdk/pull/767)
+ * Stop devicelist when client is stopped
+   [\#766](https://github.com/matrix-org/matrix-js-sdk/pull/766)
+ * Update to WebAssembly-powered Olm
+   [\#743](https://github.com/matrix-org/matrix-js-sdk/pull/743)
+ * Logging lib. Fixes #332
+   [\#763](https://github.com/matrix-org/matrix-js-sdk/pull/763)
+ * Use new stop() method on matrix-mock-request
+   [\#765](https://github.com/matrix-org/matrix-js-sdk/pull/765)
+
+Changes in [0.13.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.13.1) (2018-11-14)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.13.0...v0.13.1)
+
+ * Add function to get currently joined rooms.
+   [\#779](https://github.com/matrix-org/matrix-js-sdk/pull/779)
+
+Changes in [0.13.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.13.0) (2018-11-15)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.12.1...v0.13.0)
+
+BREAKING CHANGE
+----------------
+ * `MatrixClient::login` now sets client `access_token` and `user_id` following successful login with username and password.
+
+Changes in [0.12.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.12.1) (2018-10-29)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.12.1-rc.1...v0.12.1)
+
+ * No changes since rc.1
+
+Changes in [0.12.1-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.12.1-rc.1) (2018-10-24)
+============================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.12.0...v0.12.1-rc.1)
+
+ * Add repository type to package.json to make it valid
+   [\#762](https://github.com/matrix-org/matrix-js-sdk/pull/762)
+ * Add getMediaConfig()
+   [\#761](https://github.com/matrix-org/matrix-js-sdk/pull/761)
+ * add new examples, to be expanded into a post
+   [\#739](https://github.com/matrix-org/matrix-js-sdk/pull/739)
+
+Changes in [0.12.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.12.0) (2018-10-16)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.12.0-rc.1...v0.12.0)
+
+ * No changes since rc.1
+
+Changes in [0.12.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.12.0-rc.1) (2018-10-11)
+============================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.11.1...v0.12.0-rc.1)
+
+BREAKING CHANGES
+----------------
+ * If js-sdk finds data in the store that is incompatible with the options currently being used,
+   it will emit sync state ERROR with an error of type InvalidStoreError. It will also stop trying
+   to sync in this situation: the app must stop the client and then either clear the store or
+   change the options (in this case, enable or disable lazy loading of members) and then start
+   the client again.
+
+All Changes
+-----------
+
+ * never replace /sync'ed memberships with OOB ones
+   [\#760](https://github.com/matrix-org/matrix-js-sdk/pull/760)
+ * Don't fail to start up if lazy load check fails
+   [\#759](https://github.com/matrix-org/matrix-js-sdk/pull/759)
+ * Make e2e work on Edge
+   [\#754](https://github.com/matrix-org/matrix-js-sdk/pull/754)
+ * throw error with same name and message over idb worker boundary
+   [\#758](https://github.com/matrix-org/matrix-js-sdk/pull/758)
+ * Default to a room version of 1 when there is no room create event
+   [\#755](https://github.com/matrix-org/matrix-js-sdk/pull/755)
+ * Silence bluebird warnings
+   [\#757](https://github.com/matrix-org/matrix-js-sdk/pull/757)
+ * allow non-ff merge from release branch into master
+   [\#750](https://github.com/matrix-org/matrix-js-sdk/pull/750)
+ * Reject with the actual error on indexeddb error
+   [\#751](https://github.com/matrix-org/matrix-js-sdk/pull/751)
+ * Update mocha to v5
+   [\#744](https://github.com/matrix-org/matrix-js-sdk/pull/744)
+ * disable lazy loading for guests as they cant create filters
+   [\#748](https://github.com/matrix-org/matrix-js-sdk/pull/748)
+ * Revert "Add getMediaLimits to client"
+   [\#745](https://github.com/matrix-org/matrix-js-sdk/pull/745)
+
+Changes in [0.11.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.1) (2018-10-01)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.11.1-rc.1...v0.11.1)
+
+ * No changes since rc.1
+
+Changes in [0.11.1-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.1-rc.1) (2018-09-27)
+============================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.11.0...v0.11.1-rc.1)
+
+ * make usage of hub compatible with latest version (2.5)
+   [\#747](https://github.com/matrix-org/matrix-js-sdk/pull/747)
+ * Detect when lazy loading has been toggled in client.startClient
+   [\#746](https://github.com/matrix-org/matrix-js-sdk/pull/746)
+ * Add getMediaLimits to client
+   [\#644](https://github.com/matrix-org/matrix-js-sdk/pull/644)
+ * Split npm start into an init and watch script
+   [\#742](https://github.com/matrix-org/matrix-js-sdk/pull/742)
+ * Revert "room name should only take canonical alias into account"
+   [\#738](https://github.com/matrix-org/matrix-js-sdk/pull/738)
+ * fix display name disambiguation with LL
+   [\#737](https://github.com/matrix-org/matrix-js-sdk/pull/737)
+ * Introduce Room.myMembership event
+   [\#735](https://github.com/matrix-org/matrix-js-sdk/pull/735)
+ * room name should only take canonical alias into account
+   [\#733](https://github.com/matrix-org/matrix-js-sdk/pull/733)
+ * state events from context response were not wrapped in a MatrixEvent
+   [\#732](https://github.com/matrix-org/matrix-js-sdk/pull/732)
+ * Reduce amount of promises created when inserting members
+   [\#724](https://github.com/matrix-org/matrix-js-sdk/pull/724)
+ * dont wait for LL members to be stored to resolve the members
+   [\#726](https://github.com/matrix-org/matrix-js-sdk/pull/726)
+ * RoomState.members emitted with wrong argument order for OOB members
+   [\#728](https://github.com/matrix-org/matrix-js-sdk/pull/728)
+
+Changes in [0.11.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.0) (2018-09-10)
+==================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.11.0-rc.1...v0.11.0)
+
+BREAKING CHANGES
+----------------
+ * v0.11.0-rc.1 introduced some breaking changes - see the respective release notes.
+
+No changes since rc.1
+
+Changes in [0.11.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.0-rc.1) (2018-09-07)
+============================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.9...v0.11.0-rc.1)
+
+ * Support for lazy loading members. This should improve performance for
+   users who joined big rooms a lot. Pass to `lazyLoadMembers = true` option when calling `startClient`.
+
+BREAKING CHANGES
+----------------
+
+ * `MatrixClient::startClient` now returns a Promise. No method should be called on the client before that promise resolves. Before this method didn't return anything.
+ * A new `CATCHUP` sync state, emitted by `MatrixClient#"sync"` and returned by `MatrixClient::getSyncState()`, when doing initial sync after the `ERROR` state. See `MatrixClient` documentation for details.
+ * `RoomState::maySendEvent('m.room.message', userId)` & `RoomState::maySendMessage(userId)` do not check the membership of the user anymore, only the power level. To check if the syncing user is allowed to write in a room, use `Room::maySendMessage()` as `RoomState` is not always aware of the syncing user's membership anymore, in case lazy loading of members is enabled.
+
+All Changes
+-----------
+
+ * Only emit CATCHUP if recovering from conn error
+   [\#727](https://github.com/matrix-org/matrix-js-sdk/pull/727)
+ * Fix docstring for sync data.error
+   [\#725](https://github.com/matrix-org/matrix-js-sdk/pull/725)
+ * Re-apply "Don't rely on members to query if syncing user can post to room"
+   [\#723](https://github.com/matrix-org/matrix-js-sdk/pull/723)
+ * Revert "Don't rely on members to query if syncing user can post to room"
+   [\#721](https://github.com/matrix-org/matrix-js-sdk/pull/721)
+ * Don't rely on members to query if syncing user can post to room
+   [\#717](https://github.com/matrix-org/matrix-js-sdk/pull/717)
+ * Fixes for room.guessDMUserId
+   [\#719](https://github.com/matrix-org/matrix-js-sdk/pull/719)
+ * Fix filepanel also filtering main timeline with LL turned on.
+   [\#716](https://github.com/matrix-org/matrix-js-sdk/pull/716)
+ * Remove lazy loaded members when leaving room
+   [\#711](https://github.com/matrix-org/matrix-js-sdk/pull/711)
+ * Fix: show spinner again while recovering from connection error
+   [\#702](https://github.com/matrix-org/matrix-js-sdk/pull/702)
+ * Add method to query LL state in client
+   [\#714](https://github.com/matrix-org/matrix-js-sdk/pull/714)
+ * Fix: also load invited members when lazy loading members
+   [\#707](https://github.com/matrix-org/matrix-js-sdk/pull/707)
+ * Pass through function to discard megolm session
+   [\#704](https://github.com/matrix-org/matrix-js-sdk/pull/704)
+
 Changes in [0.10.9](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.9) (2018-09-03)
 ==================================================================================================
 [Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.9-rc.2...v0.10.9)
@@ -36,18 +249,6 @@ Changes in [0.10.9-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/ta
  * Lazy loading of room members
    [\#691](https://github.com/matrix-org/matrix-js-sdk/pull/691)
 
-BREAKING CHANGE
----------------
-
- * `MatrixClient::startClient` now returns a Promise. No method should be called on the client before that promise resolves. Before this method didn't return anything.
-
-Changes in [0.11.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.11.0) (TDB)
-==================================================================================================
-[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.6...v0.11.0)
-
- * Support for lazy loading members. This should improve performance for
-   users who joined big rooms a lot. Pass to `lazyLoadMembers = true` option when calling `startClient`.
-
 Changes in [0.10.8](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v0.10.8) (2018-08-20)
 ==================================================================================================
 [Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v0.10.8-rc.1...v0.10.8)

README.md

@@ -30,9 +30,61 @@ In Node.js
     console.log("Public Rooms: %s", JSON.stringify(data));
   });
 ```
+
 See below for how to include libolm to enable end-to-end-encryption. Please check
 [the Node.js terminal app](examples/node) for a more complex example.
 
+To start the client:
+
+```javascript
+await client.startClient({initialSyncLimit: 10});
+```
+
+You can perform a call to `/sync` to get the current state of the client:
+
+```javascript
+client.once('sync', function(state, prevState, res) {
+    if(state === 'PREPARED') {
+        console.log("prepared");
+    } else {
+        console.log(state);
+        process.exit(1);
+    }
+});
+```
+
+To send a message:
+
+```javascript
+var content = {
+    "body": "message text",
+    "msgtype": "m.text"
+};
+client.sendEvent("roomId", "m.room.message", content, "", (err, res) => {
+    console.log(err);
+});
+```
+
+To listen for message events:
+
+```javascript
+client.on("Room.timeline", function(event, room, toStartOfTimeline) {
+  if (event.getType() !== "m.room.message") {
+    return; // only use messages
+  }
+  console.log(event.event.content.body);
+});
+```
+
+By default, the `matrix-js-sdk` client uses the `MatrixInMemoryStore` to store events as they are received. For example to iterate through the currently stored timeline for a room:
+
+```javascript
+Object.keys(client.store.rooms).forEach((roomId) => {
+  client.getRoom(roomId).timeline.forEach(t => {
+      console.log(t.event);
+  });
+});
+```
 
 What does this SDK do?
 ----------------------
@@ -267,13 +319,13 @@ To provide the Olm library in a browser application:
  
 To provide the Olm library in a node.js application:
 
- * ``npm install https://matrix.org/packages/npm/olm/olm-2.2.2.tgz``
+ * ``npm install https://matrix.org/packages/npm/olm/olm-3.0.0.tgz``
    (replace the URL with the latest version you want to use from
     https://matrix.org/packages/npm/olm/)
  * ``global.Olm = require('olm');`` *before* loading ``matrix-js-sdk``.
 
 If you want to package Olm as dependency for your node.js application, you
-can use ``npm install https://matrix.org/packages/npm/olm/olm-2.2.2.tgz 
+can use ``npm install https://matrix.org/packages/npm/olm/olm-3.0.0.tgz
 --save-optional`` (if your application also works without e2e crypto enabled)
 or ``--save`` (if it doesn't) to do so.
 

browser-index.js

@@ -1,5 +1,17 @@
 var matrixcs = require("./lib/matrix");
-matrixcs.request(require("browser-request"));
+const request = require('browser-request');
+const queryString = require('qs');
+
+matrixcs.request(function(opts, fn) {
+    // We manually fix the query string for browser-request because
+    // it doesn't correctly handle cases like ?via=one&via=two. Instead
+    // we mimic `request`'s query string interface to make it all work
+    // as expected.
+    // browser-request will happily take the constructed string as the
+    // query string without trying to modify it further.
+    opts.qs = queryString.stringify(opts.qs || {}, opts.qsStringifyOptions);
+    return request(opts, fn);
+});
 
 // just *accessing* indexedDB throws an exception in firefox with
 // indexeddb disabled.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "matrix-js-sdk",
-  "version": "0.10.9",
+  "version": "0.14.0",
   "description": "Matrix Client-Server SDK for Javascript",
   "main": "index.js",
   "scripts": {
@@ -10,7 +10,9 @@
     "test": "npm run test:build && npm run test:run",
     "check": "npm run test:build && _mocha --recursive specbuild --colors",
     "gendoc": "babel --no-babelrc -d .jsdocbuild src && jsdoc -r .jsdocbuild -P package.json -R README.md -d .jsdoc",
-    "start": "babel -s -w -d lib src",
+    "start": "npm run start:init && npm run start:watch",
+    "start:watch": "babel -s -w --skip-initial-build -d lib src",
+    "start:init": "babel -s -d lib src",
     "clean": "rimraf lib dist",
     "build": "babel -s -d lib src && rimraf dist && mkdir dist && browserify -d browser-index.js | exorcist dist/browser-matrix.js.map > dist/browser-matrix.js && uglifyjs -c -m -o dist/browser-matrix.min.js --source-map dist/browser-matrix.min.js.map --in-source-map dist/browser-matrix.js.map dist/browser-matrix.js",
     "dist": "npm run build",
@@ -19,6 +21,7 @@
     "prepublish": "npm run clean && npm run build && git rev-parse HEAD > git-revision.txt"
   },
   "repository": {
+    "type": "git",
     "url": "https://github.com/matrix-org/matrix-js-sdk"
   },
   "keywords": [
@@ -54,15 +57,17 @@
     "bluebird": "^3.5.0",
     "browser-request": "^0.3.3",
     "content-type": "^1.0.2",
-    "request": "^2.53.0"
+    "loglevel": "1.6.1",
+    "qs": "^6.5.2",
+    "request": "^2.88.0"
   },
   "devDependencies": {
     "babel-cli": "^6.18.0",
-    "babel-eslint": "^7.1.1",
+    "babel-eslint": "^8.1.1",
     "babel-plugin-transform-async-to-bluebird": "^1.1.1",
     "babel-plugin-transform-runtime": "^6.23.0",
     "babel-preset-es2015": "^6.18.0",
-    "browserify": "^14.0.0",
+    "browserify": "^16.2.3",
     "browserify-shim": "^3.8.13",
     "eslint": "^3.13.1",
     "eslint-config-google": "^0.7.1",
@@ -71,14 +76,14 @@
     "istanbul": "^0.4.5",
     "jsdoc": "^3.5.5",
     "lolex": "^1.5.2",
-    "matrix-mock-request": "^1.2.0",
-    "mocha": "^3.2.0",
-    "mocha-jenkins-reporter": "^0.3.6",
+    "matrix-mock-request": "^1.2.2",
+    "mocha": "^5.2.0",
+    "mocha-jenkins-reporter": "^0.4.0",
     "rimraf": "^2.5.4",
     "source-map-support": "^0.4.11",
     "sourceify": "^0.1.0",
     "uglify-js": "^2.8.26",
-    "watchify": "^3.2.1"
+    "watchify": "^3.11.0"
   },
   "browserify": {
     "transform": [

release.sh

@@ -245,7 +245,7 @@ release_text=`mktemp`
 echo "$tag" > "${release_text}"
 echo >> "${release_text}"
 cat "${latest_changes}" >> "${release_text}"
-hub release create $hubflags $assets -f "${release_text}" "$tag"
+hub release create $hubflags $assets -F "${release_text}" "$tag"
 
 if [ $dodist -eq 0 ]; then
     rm -rf "$builddir"
@@ -281,7 +281,7 @@ fi
 echo "updating master branch"
 git checkout master
 git pull
-git merge --ff-only "$rel_branch"
+git merge "$rel_branch"
 
 # push master  and docs (if generated) to github
 git push origin master

spec/TestClient.js

@@ -102,9 +102,11 @@ TestClient.prototype.start = function() {
 
 /**
  * stop the client
+ * @return {Promise} Resolves once the mock http backend has finished all pending flushes
  */
 TestClient.prototype.stop = function() {
     this.client.stopClient();
+    return this.httpBackend.stop();
 };
 
 /**

spec/integ/devicelist-integ-spec.js

@@ -97,7 +97,7 @@ describe("DeviceList management:", function() {
     });
 
     afterEach(function() {
-        aliceTestClient.stop();
+        return aliceTestClient.stop();
     });
 
     it("Alice shouldn't do a second /query for non-e2e-capable devices", function() {

spec/integ/matrix-client-crypto.spec.js

@@ -410,10 +410,10 @@ describe("MatrixClient crypto", function() {
     });
 
     afterEach(function() {
-        aliTestClient.stop();
         aliTestClient.httpBackend.verifyNoOutstandingExpectation();
-        bobTestClient.stop();
         bobTestClient.httpBackend.verifyNoOutstandingExpectation();
+
+        return Promise.all([aliTestClient.stop(), bobTestClient.stop()]);
     });
 
     it("Bob uploads device keys", function() {

spec/integ/matrix-client-event-emitter.spec.js

@@ -30,6 +30,7 @@ describe("MatrixClient events", function() {
     afterEach(function() {
         httpBackend.verifyNoOutstandingExpectation();
         client.stopClient();
+        return httpBackend.stop();
     });
 
     describe("emissions", function() {

spec/integ/matrix-client-event-timeline.spec.js

@@ -111,6 +111,7 @@ describe("getEventTimeline support", function() {
         if (client) {
             client.stopClient();
         }
+        return httpBackend.stop();
     });
 
     it("timeline support must be enabled to work", function(done) {

spec/integ/matrix-client-methods.spec.js

@@ -41,6 +41,7 @@ describe("MatrixClient", function() {
 
     afterEach(function() {
         httpBackend.verifyNoOutstandingExpectation();
+        return httpBackend.stop();
     });
 
     describe("uploadContent", function() {

spec/integ/matrix-client-opts.spec.js

@@ -64,6 +64,7 @@ describe("MatrixClient opts", function() {
 
     afterEach(function() {
         httpBackend.verifyNoOutstandingExpectation();
+        return httpBackend.stop();
     });
 
     describe("without opts.store", function() {
@@ -94,7 +95,7 @@ describe("MatrixClient opts", function() {
             httpBackend.flush("/txn1", 1);
         });
 
-        it("should be able to sync / get new events", function(done) {
+        it("should be able to sync / get new events", async function() {
             const expectedEventTypes = [ // from /initialSync
                 "m.room.message", "m.room.name", "m.room.member", "m.room.member",
                 "m.room.create",
@@ -110,20 +111,16 @@ describe("MatrixClient opts", function() {
             httpBackend.when("GET", "/pushrules").respond(200, {});
             httpBackend.when("POST", "/filter").respond(200, { filter_id: "foo" });
             httpBackend.when("GET", "/sync").respond(200, syncData);
-            client.startClient();
-            httpBackend.flush("/pushrules", 1).then(function() {
-                return httpBackend.flush("/filter", 1);
-            }).then(function() {
-                return Promise.all([
-                    httpBackend.flush("/sync", 1),
-                    utils.syncPromise(client),
-                ]);
-            }).done(function() {
-                expect(expectedEventTypes.length).toEqual(
-                    0, "Expected to see event types: " + expectedEventTypes,
-                );
-                done();
-            });
+            await client.startClient();
+            await httpBackend.flush("/pushrules", 1);
+            await httpBackend.flush("/filter", 1);
+            await Promise.all([
+                httpBackend.flush("/sync", 1),
+                utils.syncPromise(client),
+            ]);
+            expect(expectedEventTypes.length).toEqual(
+                0, "Expected to see event types: " + expectedEventTypes,
+            );
         });
     });
 

spec/integ/matrix-client-retrying.spec.js

@@ -36,6 +36,7 @@ describe("MatrixClient retrying", function() {
 
     afterEach(function() {
         httpBackend.verifyNoOutstandingExpectation();
+        return httpBackend.stop();
     });
 
     xit("should retry according to MatrixScheduler.retryFn", function() {

spec/integ/matrix-client-room-timeline.spec.js

@@ -130,6 +130,7 @@ describe("MatrixClient room timelines", function() {
     afterEach(function() {
         httpBackend.verifyNoOutstandingExpectation();
         client.stopClient();
+        return httpBackend.stop();
     });
 
     describe("local echo events", function() {

spec/integ/matrix-client-syncing.spec.js

@@ -38,6 +38,7 @@ describe("MatrixClient syncing", function() {
     afterEach(function() {
         httpBackend.verifyNoOutstandingExpectation();
         client.stopClient();
+        return httpBackend.stop();
     });
 
     describe("startClient", function() {

spec/integ/megolm-integ.spec.js

@@ -296,7 +296,7 @@ describe("megolm", function() {
     });
 
     afterEach(function() {
-        aliceTestClient.stop();
+        return aliceTestClient.stop();
     });
 
     it("Alice receives a megolm message", function() {
@@ -817,8 +817,14 @@ describe("megolm", function() {
                 };
             });
 
+            // Grab the event that we'll need to resend
+            const room = aliceTestClient.client.getRoom(ROOM_ID);
+            const pendingEvents = room.getPendingEvents();
+            expect(pendingEvents.length).toEqual(1);
+            const unsentEvent = pendingEvents[0];
+
             return Promise.all([
-                aliceTestClient.client.sendTextMessage(ROOM_ID, 'test'),
+                aliceTestClient.client.resendEvent(unsentEvent, room),
 
                 // the crypto stuff can take a while, so give the requests a whole second.
                 aliceTestClient.httpBackend.flushAllExpected({

spec/unit/crypto.spec.js

@@ -1,20 +1,23 @@
 
 "use strict";
 import 'source-map-support/register';
+import Crypto from '../../lib/crypto';
+import expect from 'expect';
 
 const sdk = require("../..");
-let Crypto;
-if (sdk.CRYPTO_ENABLED) {
-    Crypto = require("../../lib/crypto");
-}
 
-import expect from 'expect';
+const Olm = global.Olm;
 
 describe("Crypto", function() {
     if (!sdk.CRYPTO_ENABLED) {
         return;
     }
+
+    beforeEach(function(done) {
+        Olm.init().then(done);
+    });
+
     it("Crypto exposes the correct olm library version", function() {
-        expect(Crypto.getOlmVersion()[0]).toEqual(2);
+        expect(Crypto.getOlmVersion()[0]).toEqual(3);
     });
 });

spec/unit/crypto/DeviceList.spec.js

@@ -59,16 +59,25 @@ describe('DeviceList', function() {
     let downloadSpy;
     let sessionStore;
     let cryptoStore;
+    let deviceLists = [];
 
     beforeEach(function() {
         testUtils.beforeEach(this); // eslint-disable-line no-invalid-this
 
+        deviceLists = [];
+
         downloadSpy = expect.createSpy();
         const mockStorage = new MockStorageApi();
         sessionStore = new WebStorageSessionStore(mockStorage);
         cryptoStore = new MemoryCryptoStore();
     });
 
+    afterEach(function() {
+        for (const dl of deviceLists) {
+            dl.stop();
+        }
+    });
+
     function createTestDeviceList() {
         const baseApis = {
             downloadKeysForUsers: downloadSpy,
@@ -76,7 +85,9 @@ describe('DeviceList', function() {
         const mockOlm = {
             verifySignature: function(key, message, signature) {},
         };
-        return new DeviceList(baseApis, cryptoStore, sessionStore, mockOlm);
+        const dl = new DeviceList(baseApis, cryptoStore, sessionStore, mockOlm);
+        deviceLists.push(dl);
+        return dl;
     }
 
     it("should successfully download and store device keys", function() {

spec/unit/crypto/algorithms/megolm.spec.js

@@ -13,20 +13,16 @@ import WebStorageSessionStore from '../../../../lib/store/session/webstorage';
 import MemoryCryptoStore from '../../../../lib/crypto/store/memory-crypto-store.js';
 import MockStorageApi from '../../../MockStorageApi';
 import testUtils from '../../../test-utils';
-
-// Crypto and OlmDevice won't import unless we have global.Olm
-let OlmDevice;
-let Crypto;
-if (global.Olm) {
-    OlmDevice = require('../../../../lib/crypto/OlmDevice');
-    Crypto = require('../../../../lib/crypto');
-}
+import OlmDevice from '../../../../lib/crypto/OlmDevice';
+import Crypto from '../../../../lib/crypto';
 
 const MatrixEvent = sdk.MatrixEvent;
 const MegolmDecryption = algorithms.DECRYPTION_CLASSES['m.megolm.v1.aes-sha2'];
 
 const ROOM_ID = '!ROOM:ID';
 
+const Olm = global.Olm;
+
 describe("MegolmDecryption", function() {
     if (!global.Olm) {
         console.warn('Not running megolm unit tests: libolm not present');
@@ -69,7 +65,8 @@ describe("MegolmDecryption", function() {
 
     describe('receives some keys:', function() {
         let groupSession;
-        beforeEach(function() {
+        beforeEach(async function() {
+            await Olm.init();
             groupSession = new global.Olm.OutboundGroupSession();
             groupSession.create();
 
@@ -98,7 +95,7 @@ describe("MegolmDecryption", function() {
                 },
             };
 
-            return event.attemptDecryption(mockCrypto).then(() => {
+            await event.attemptDecryption(mockCrypto).then(() => {
                 megolmDecryption.onRoomKeyEvent(event);
             });
         });

spec/unit/crypto/algorithms/olm.spec.js

@@ -0,0 +1,90 @@
+/*
+Copyright 2018 New Vector Ltd
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+try {
+    global.Olm = require('olm');
+} catch (e) {
+    console.warn("unable to run megolm tests: libolm not available");
+}
+
+import expect from 'expect';
+import WebStorageSessionStore from '../../../../lib/store/session/webstorage';
+import MemoryCryptoStore from '../../../../lib/crypto/store/memory-crypto-store.js';
+import MockStorageApi from '../../../MockStorageApi';
+import testUtils from '../../../test-utils';
+
+import OlmDevice from '../../../../lib/crypto/OlmDevice';
+
+function makeOlmDevice() {
+    const mockStorage = new MockStorageApi();
+    const sessionStore = new WebStorageSessionStore(mockStorage);
+    const cryptoStore = new MemoryCryptoStore(mockStorage);
+    const olmDevice = new OlmDevice(sessionStore, cryptoStore);
+    return olmDevice;
+}
+
+async function setupSession(initiator, opponent) {
+    await opponent.generateOneTimeKeys(1);
+    const keys = await opponent.getOneTimeKeys();
+    const firstKey = Object.values(keys['curve25519'])[0];
+
+    const sid = await initiator.createOutboundSession(
+        opponent.deviceCurve25519Key, firstKey,
+    );
+    return sid;
+}
+
+describe("OlmDecryption", function() {
+    if (!global.Olm) {
+        console.warn('Not running megolm unit tests: libolm not present');
+        return;
+    }
+
+    let aliceOlmDevice;
+    let bobOlmDevice;
+
+    beforeEach(async function() {
+        testUtils.beforeEach(this); // eslint-disable-line no-invalid-this
+
+        await global.Olm.init();
+
+        aliceOlmDevice = makeOlmDevice();
+        bobOlmDevice = makeOlmDevice();
+        await aliceOlmDevice.init();
+        await bobOlmDevice.init();
+    });
+
+    describe('olm', function() {
+        it("can decrypt messages", async function() {
+            const sid = await setupSession(aliceOlmDevice, bobOlmDevice);
+
+            const ciphertext = await aliceOlmDevice.encryptMessage(
+                bobOlmDevice.deviceCurve25519Key,
+                sid,
+                "The olm or proteus is an aquatic salamander in the family Proteidae",
+            );
+
+            const result = await bobOlmDevice.createInboundSession(
+                aliceOlmDevice.deviceCurve25519Key,
+                ciphertext.type,
+                ciphertext.body,
+            );
+            expect(result.payload).toEqual(
+                "The olm or proteus is an aquatic salamander in the family Proteidae",
+            );
+        });
+    });
+});

spec/unit/matrix-client.spec.js

@@ -139,6 +139,9 @@ describe("MatrixClient", function() {
         store.getSavedSync = expect.createSpy().andReturn(Promise.resolve(null));
         store.getSavedSyncToken = expect.createSpy().andReturn(Promise.resolve(null));
         store.setSyncData = expect.createSpy().andReturn(Promise.resolve(null));
+        store.getClientOptions = expect.createSpy().andReturn(Promise.resolve(null));
+        store.storeClientOptions = expect.createSpy().andReturn(Promise.resolve(null));
+        store.isNewlyCreated = expect.createSpy().andReturn(Promise.resolve(true));
         client = new MatrixClient({
             baseUrl: "https://my.home.server",
             idBaseUrl: identityServerUrl,
@@ -182,7 +185,7 @@ describe("MatrixClient", function() {
         });
     });
 
-    it("should not POST /filter if a matching filter already exists", function(done) {
+    it("should not POST /filter if a matching filter already exists", async function() {
         httpLookups = [];
         httpLookups.push(PUSH_RULES_RESPONSE);
         httpLookups.push(SYNC_RESPONSE);
@@ -191,15 +194,19 @@ describe("MatrixClient", function() {
         const filter = new sdk.Filter(0, filterId);
         filter.setDefinition({"room": {"timeline": {"limit": 8}}});
         store.getFilter.andReturn(filter);
-        client.startClient();
-
-        client.on("sync", function syncListener(state) {
-            if (state === "SYNCING") {
-                expect(httpLookups.length).toEqual(0);
-                client.removeListener("sync", syncListener);
-                done();
-            }
+        const syncPromise = new Promise((resolve, reject) => {
+            client.on("sync", function syncListener(state) {
+                if (state === "SYNCING") {
+                    expect(httpLookups.length).toEqual(0);
+                    client.removeListener("sync", syncListener);
+                    resolve();
+                } else if (state === "ERROR") {
+                    reject(new Error("sync error"));
+                }
+            });
         });
+        await client.startClient();
+        await syncPromise;
     });
 
     describe("getSyncState", function() {
@@ -207,15 +214,18 @@ describe("MatrixClient", function() {
             expect(client.getSyncState()).toBe(null);
         });
 
-        it("should return the same sync state as emitted sync events", function(done) {
-            client.on("sync", function syncListener(state) {
-                expect(state).toEqual(client.getSyncState());
-                if (state === "SYNCING") {
-                    client.removeListener("sync", syncListener);
-                    done();
-                }
+        it("should return the same sync state as emitted sync events", async function() {
+            const syncingPromise = new Promise((resolve) => {
+                client.on("sync", function syncListener(state) {
+                    expect(state).toEqual(client.getSyncState());
+                    if (state === "SYNCING") {
+                        client.removeListener("sync", syncListener);
+                        resolve();
+                    }
+                });
             });
-            client.startClient();
+            await client.startClient();
+            await syncingPromise;
         });
     });
 
@@ -258,8 +268,8 @@ describe("MatrixClient", function() {
     });
 
     describe("retryImmediately", function() {
-        it("should return false if there is no request waiting", function() {
-            client.startClient();
+        it("should return false if there is no request waiting", async function() {
+            await client.startClient();
             expect(client.retryImmediately()).toBe(false);
         });
 
@@ -380,7 +390,7 @@ describe("MatrixClient", function() {
             client.startClient();
         });
 
-        it("should transition ERROR -> PREPARED after /sync if prev failed",
+        it("should transition ERROR -> CATCHUP after /sync if prev failed",
         function(done) {
             const expectedStates = [];
             acceptKeepalives = false;
@@ -403,7 +413,7 @@ describe("MatrixClient", function() {
 
             expectedStates.push(["RECONNECTING", null]);
             expectedStates.push(["ERROR", "RECONNECTING"]);
-            expectedStates.push(["PREPARED", "ERROR"]);
+            expectedStates.push(["CATCHUP", "ERROR"]);
             client.on("sync", syncChecker(expectedStates, done));
             client.startClient();
         });

spec/unit/room-member.spec.js

@@ -201,22 +201,6 @@ describe("RoomMember", function() {
         });
     });
 
-    describe("supersedesOutOfBand", function() {
-        it("should be set by markSupersedesOutOfBand", function() {
-            const member = new RoomMember();
-            expect(member.supersedesOutOfBand()).toEqual(false);
-            member.markSupersedesOutOfBand();
-            expect(member.supersedesOutOfBand()).toEqual(true);
-        });
-        it("should be cleared by clearSupersedesOutOfBand", function() {
-            const member = new RoomMember();
-            member.markSupersedesOutOfBand();
-            expect(member.supersedesOutOfBand()).toEqual(true);
-            member.clearSupersedesOutOfBand();
-            expect(member.supersedesOutOfBand()).toEqual(false);
-        });
-    });
-
     describe("setMembershipEvent", function() {
         const joinEvent = utils.mkMembership({
             event: true,

spec/unit/room-state.spec.js

@@ -299,41 +299,25 @@ describe("RoomState", function() {
             expect(eventReceived).toEqual(true);
         });
 
-        it("should overwrite existing members", function() {
+        it("should never overwrite existing members", function() {
             const oobMemberEvent = utils.mkMembership({
                 user: userA, mship: "join", room: roomId, event: true,
             });
             state.markOutOfBandMembersStarted();
             state.setOutOfBandMembers([oobMemberEvent]);
             const memberA = state.getMember(userA);
-            expect(memberA.events.member.getId()).toEqual(oobMemberEvent.getId());
-            expect(memberA.isOutOfBand()).toEqual(true);
-        });
-
-        it("should allow later state events to overwrite", function() {
-            const oobMemberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            });
-            const memberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            });
-
-            state.markOutOfBandMembersStarted();
-            state.setOutOfBandMembers([oobMemberEvent]);
-            state.setStateEvents([memberEvent]);
-
-            const memberA = state.getMember(userA);
-            expect(memberA.events.member.getId()).toEqual(memberEvent.getId());
+            expect(memberA.events.member.getId()).toNotEqual(oobMemberEvent.getId());
             expect(memberA.isOutOfBand()).toEqual(false);
         });
 
         it("should emit members when updating a member", function() {
+            const doesntExistYetUserId = "@doesntexistyet:hs";
             const oobMemberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
+                user: doesntExistYetUserId, mship: "join", room: roomId, event: true,
             });
             let eventReceived = false;
             state.once('RoomState.members', (_, __, member) => {
-                expect(member.userId).toEqual(userA);
+                expect(member.userId).toEqual(doesntExistYetUserId);
                 eventReceived = true;
             });
 
@@ -341,28 +325,6 @@ describe("RoomState", function() {
             state.setOutOfBandMembers([oobMemberEvent]);
             expect(eventReceived).toEqual(true);
         });
-
-
-        it("should not overwrite members updated since starting loading oob",
-        function() {
-            const oobMemberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            });
-
-            const existingMemberEvent = utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            });
-
-            state.markOutOfBandMembersStarted();
-            state.setStateEvents([existingMemberEvent]);
-            expect(state.getMember(userA).supersedesOutOfBand()).toEqual(true);
-            state.setOutOfBandMembers([oobMemberEvent]);
-
-            const memberA = state.getMember(userA);
-            expect(memberA.events.member.getId()).toEqual(existingMemberEvent.getId());
-            expect(memberA.isOutOfBand()).toEqual(false);
-            expect(memberA.supersedesOutOfBand()).toEqual(false);
-        });
     });
 
     describe("clone", function() {
@@ -386,20 +348,6 @@ describe("RoomState", function() {
             expect(state.getJoinedMemberCount()).toEqual(copy.getJoinedMemberCount());
         });
 
-        it("should copy supersedes flag when OOB loading is progress",
-        function() {
-            // include OOB members in copy
-            state.markOutOfBandMembersStarted();
-            state.setStateEvents([utils.mkMembership({
-                user: userA, mship: "join", room: roomId, event: true,
-            })]);
-            const copy = state.clone();
-            const memberA = state.getMember(userA);
-            const memberACopy = copy.getMember(userA);
-            expect(memberA.supersedesOutOfBand()).toEqual(true);
-            expect(memberACopy.supersedesOutOfBand()).toEqual(true);
-        });
-
         it("should mark old copy as not waiting for out of band anymore", function() {
             state.markOutOfBandMembersStarted();
             const copy = state.clone();
@@ -447,13 +395,6 @@ describe("RoomState", function() {
     });
 
     describe("maySendStateEvent", function() {
-        it("should say non-joined members may not send state",
-        function() {
-            expect(state.maySendStateEvent(
-                'm.room.name', "@nobody:nowhere",
-            )).toEqual(false);
-        });
-
         it("should say any member may send state with no power level event",
         function() {
             expect(state.maySendStateEvent('m.room.name', userA)).toEqual(true);
@@ -640,14 +581,6 @@ describe("RoomState", function() {
     });
 
     describe("maySendEvent", function() {
-        it("should say non-joined members may not send events",
-        function() {
-            expect(state.maySendEvent(
-                'm.room.message', "@nobody:nowhere",
-            )).toEqual(false);
-            expect(state.maySendMessage("@nobody:nowhere")).toEqual(false);
-        });
-
         it("should say any member may send events with no power level event",
         function() {
             expect(state.maySendEvent('m.room.message', userA)).toEqual(true);

spec/unit/room.spec.js

@@ -1400,15 +1400,63 @@ describe("Room", function() {
 
     describe("getMyMembership", function() {
         it("should return synced membership if membership isn't available yet",
-        async function() {
+        function() {
             const room = new Room(roomId, null, userA);
-            room.setSyncedMembership("invite");
+            room.updateMyMembership("invite");
             expect(room.getMyMembership()).toEqual("invite");
+        });
+        it("should emit a Room.myMembership event on a change",
+        function() {
+            const room = new Room(roomId, null, userA);
+            const events = [];
+            room.on("Room.myMembership", (_room, membership, oldMembership) => {
+                events.push({membership, oldMembership});
+            });
+            room.updateMyMembership("invite");
+            expect(room.getMyMembership()).toEqual("invite");
+            expect(events[0]).toEqual({membership: "invite", oldMembership: null});
+            events.splice(0);   //clear
+            room.updateMyMembership("invite");
+            expect(events.length).toEqual(0);
+            room.updateMyMembership("join");
+            expect(room.getMyMembership()).toEqual("join");
+            expect(events[0]).toEqual({membership: "join", oldMembership: "invite"});
+        });
+    });
+
+    describe("guessDMUserId", function() {
+        it("should return first hero id",
+        function() {
+            const room = new Room(roomId, null, userA);
+            room.setSummary({'m.heroes': [userB]});
+            expect(room.guessDMUserId()).toEqual(userB);
+        });
+        it("should return first member that isn't self",
+        function() {
+            const room = new Room(roomId, null, userA);
             room.addLiveEvents([utils.mkMembership({
-                user: userA, mship: "join",
+                user: userB, mship: "join",
                 room: roomId, event: true,
             })]);
-            expect(room.getMyMembership()).toEqual("join");
+            expect(room.guessDMUserId()).toEqual(userB);
+        });
+        it("should return self if only member present",
+        function() {
+            const room = new Room(roomId, null, userA);
+            expect(room.guessDMUserId()).toEqual(userA);
+        });
+    });
+
+    describe("maySendMessage", function() {
+        it("should return false if synced membership not join",
+        function() {
+            const room = new Room(roomId, null, userA);
+            room.updateMyMembership("invite");
+            expect(room.maySendMessage()).toEqual(false);
+            room.updateMyMembership("leave");
+            expect(room.maySendMessage()).toEqual(false);
+            room.updateMyMembership("join");
+            expect(room.maySendMessage()).toEqual(true);
         });
     });
 });

src/base-apis.js

@@ -262,7 +262,19 @@ MatrixBaseApis.prototype.login = function(loginType, data, callback) {
     utils.extend(login_data, data);
 
     return this._http.authedRequest(
-        callback, "POST", "/login", undefined, login_data,
+        (error, response) => {
+            if (loginType === "m.login.password" && response &&
+                response.access_token && response.user_id) {
+                this._http.opts.accessToken = response.access_token;
+                this.credentials = {
+                    userId: response.user_id,
+                };
+            }
+
+            if (callback) {
+                callback(error, response);
+            }
+        }, "POST", "/login", undefined, login_data,
     );
 };
 
@@ -927,6 +939,28 @@ MatrixBaseApis.prototype.setRoomReadMarkersHttpRequest =
     );
 };
 
+/**
+ * @return {module:client.Promise} Resolves: A list of the user's current rooms
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.getJoinedRooms = function() {
+    const path = utils.encodeUri("/joined_rooms");
+    return this._http.authedRequest(undefined, "GET", path);
+};
+
+/**
+ * Retrieve membership info. for a room.
+ * @param {string} roomId ID of the room to get membership for
+ * @return {module:client.Promise} Resolves: A list of currently joined users
+ *                                 and their profile data.
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.getJoinedRoomMembers = function(roomId) {
+    const path = utils.encodeUri("/rooms/$roomId/joined_members", {
+        $roomId: roomId,
+    });
+    return this._http.authedRequest(undefined, "GET", path);
+};
 
 // Room Directory operations
 // =========================

src/client.js

@@ -45,27 +45,16 @@ const ContentHelpers = require("./content-helpers");
 import ReEmitter from './ReEmitter';
 import RoomList from './crypto/RoomList';
 
+import Crypto from './crypto';
+import { isCryptoAvailable } from './crypto';
 
-const LAZY_LOADING_MESSAGES_FILTER = {
-    lazy_load_members: true,
-};
-
-const LAZY_LOADING_SYNC_FILTER = {
-    room: {
-        state: LAZY_LOADING_MESSAGES_FILTER,
-    },
-};
+// Disable warnings for now: we use deprecated bluebird functions
+// and need to migrate, but they spam the console with warnings.
+Promise.config({warnings: false});
 
 
 const SCROLLBACK_DELAY_MS = 3000;
-let CRYPTO_ENABLED = false;
-
-try {
-    var Crypto = require("./crypto");
-    CRYPTO_ENABLED = true;
-} catch (e) {
-    console.warn("Unable to load crypto module: crypto will be disabled: " + e);
-}
+const CRYPTO_ENABLED = isCryptoAvailable();
 
 /**
  * Construct a Matrix Client. Only directly construct this if you want to use
@@ -140,6 +129,8 @@ function MatrixClient(opts) {
 
     MatrixBaseApis.call(this, opts);
 
+    this.olmVersion = null; // Populated after initCrypto is done
+
     this.reEmitter = new ReEmitter(this);
 
     this.store = opts.store || new StubStore();
@@ -192,10 +183,6 @@ function MatrixClient(opts) {
 
     this._forceTURN = opts.forceTURN || false;
 
-    if (CRYPTO_ENABLED) {
-        this.olmVersion = Crypto.getOlmVersion();
-    }
-
     // List of which rooms have encryption enabled: separate from crypto because
     // we still want to know which rooms are encrypted even if crypto is disabled:
     // we don't want to start sending unencrypted events to them.
@@ -385,6 +372,13 @@ MatrixClient.prototype.setNotifTimelineSet = function(notifTimelineSet) {
  * successfully initialised.
  */
 MatrixClient.prototype.initCrypto = async function() {
+    if (!isCryptoAvailable()) {
+        throw new Error(
+            `End-to-end encryption not supported in this js-sdk build: did ` +
+                `you remember to load the olm library?`,
+        );
+    }
+
     if (this._crypto) {
         console.warn("Attempt to re-initialise e2e encryption on MatrixClient");
         return;
@@ -402,13 +396,6 @@ MatrixClient.prototype.initCrypto = async function() {
     // initialise the list of encrypted rooms (whether or not crypto is enabled)
     await this._roomList.init();
 
-    if (!CRYPTO_ENABLED) {
-        throw new Error(
-            `End-to-end encryption not supported in this js-sdk build: did ` +
-                `you remember to load the olm library?`,
-        );
-    }
-
     const userId = this.getUserId();
     if (userId === null) {
         throw new Error(
@@ -440,6 +427,9 @@ MatrixClient.prototype.initCrypto = async function() {
 
     await crypto.init();
 
+    this.olmVersion = Crypto.getOlmVersion();
+
+
     // if crypto initialisation was successful, tell it to attach its event
     // handlers.
     crypto.registerEventHandlers(this);
@@ -702,6 +692,21 @@ MatrixClient.prototype.isRoomEncrypted = function(roomId) {
     return this._roomList.isRoomEncrypted(roomId);
 };
 
+/**
+ * Forces the current outbound group session to be discarded such
+ * that another one will be created next time an event is sent.
+ *
+ * @param {string} roomId The ID of the room to discard the session for
+ *
+ * This should not normally be necessary.
+ */
+MatrixClient.prototype.forceDiscardSession = function(roomId) {
+    if (!this._crypto) {
+        throw new Error("End-to-End encryption disabled");
+    }
+    this._crypto.forceDiscardSession(roomId);
+};
+
 /**
  * Get a list containing all of the room keys
  *
@@ -756,6 +761,17 @@ MatrixClient.prototype.getGroups = function() {
     return this.store.getGroups();
 };
 
+/**
+ * Get the config for the media repository.
+ * @param {module:client.callback} callback Optional.
+ * @return {module:client.Promise} Resolves with an object containing the config.
+ */
+MatrixClient.prototype.getMediaConfig = function(callback) {
+    return this._http.requestWithPrefix(
+        callback, "GET", "/config", undefined, undefined, httpApi.PREFIX_MEDIA_R0,
+    );
+};
+
 // Room ops
 // ========
 
@@ -902,6 +918,8 @@ MatrixClient.prototype.isUserIgnored = function(userId) {
  * </strong> Default: true.
  * @param {boolean} opts.inviteSignUrl If the caller has a keypair 3pid invite,
  *                                     the signing URL is passed in this parameter.
+ * @param {string[]} opts.viaServers The server names to try and join through in
+ *                                   addition to those that are automatically chosen.
  * @param {module:client.callback} callback Optional.
  * @return {module:client.Promise} Resolves: Room object.
  * @return {module:http-api.MatrixError} Rejects: with an error response.
@@ -930,6 +948,13 @@ MatrixClient.prototype.joinRoom = function(roomIdOrAlias, opts, callback) {
         );
     }
 
+    const queryString = {};
+    if (opts.viaServers) {
+        queryString["server_name"] = opts.viaServers;
+    }
+
+    const reqOpts = {qsStringifyOptions: {arrayFormat: 'repeat'}};
+
     const defer = Promise.defer();
 
     const self = this;
@@ -940,7 +965,8 @@ MatrixClient.prototype.joinRoom = function(roomIdOrAlias, opts, callback) {
         }
 
         const path = utils.encodeUri("/join/$roomid", { $roomid: roomIdOrAlias});
-        return self._http.authedRequest(undefined, "POST", path, undefined, data);
+        return self._http.authedRequest(
+            undefined, "POST", path, queryString, data, reqOpts);
     }).then(function(res) {
         const roomId = res.room_id;
         const syncApi = new SyncApi(self, self._clientOpts);
@@ -1160,6 +1186,13 @@ MatrixClient.prototype.sendEvent = function(roomId, eventType, content, txnId,
         room.addPendingEvent(localEvent, txnId);
     }
 
+    // addPendingEvent can change the state to NOT_SENT if it believes
+    // that there's other events that have failed. We won't bother to
+    // try sending the event if the state has changed as such.
+    if (localEvent.status === EventStatus.NOT_SENT) {
+        return Promise.reject(new Error("Event blocked by other events not yet sent"));
+    }
+
     return _sendEvent(this, room, localEvent, callback);
 };
 
@@ -2052,7 +2085,7 @@ MatrixClient.prototype.getEventTimeline = function(timelineSet, eventId) {
 
     let params = undefined;
     if (this._clientOpts.lazyLoadMembers) {
-        params = {filter: JSON.stringify(LAZY_LOADING_MESSAGES_FILTER)};
+        params = {filter: JSON.stringify(Filter.LAZY_LOADING_MESSAGES_FILTER)};
     }
 
     // TODO: we should implement a backoff (as per scrollback()) to deal more
@@ -2087,7 +2120,8 @@ MatrixClient.prototype.getEventTimeline = function(timelineSet, eventId) {
                                                self.getEventMapper()));
             timeline.getState(EventTimeline.FORWARDS).paginationToken = res.end;
         } else {
-            timeline.getState(EventTimeline.BACKWARDS).setUnknownStateEvents(res.state);
+            const stateEvents = utils.map(res.state, self.getEventMapper());
+            timeline.getState(EventTimeline.BACKWARDS).setUnknownStateEvents(stateEvents);
         }
         timelineSet.addEventsToTimeline(matrixEvents, true, timeline, res.start);
 
@@ -2130,7 +2164,9 @@ function(roomId, fromToken, limit, dir, timelineFilter = undefined) {
 
     let filter = null;
     if (this._clientOpts.lazyLoadMembers) {
-        filter = LAZY_LOADING_MESSAGES_FILTER;
+        // create a shallow copy of LAZY_LOADING_MESSAGES_FILTER,
+        // so the timelineFilter doesn't get written into it below
+        filter = Object.assign({}, Filter.LAZY_LOADING_MESSAGES_FILTER);
     }
     if (timelineFilter) {
         // XXX: it's horrific that /messages' filter parameter doesn't match
@@ -3087,21 +3123,6 @@ MatrixClient.prototype.startClient = async function(opts) {
     // shallow-copy the opts dict before modifying and storing it
     opts = Object.assign({}, opts);
 
-    if (opts.lazyLoadMembers) {
-        const supported = await this.doesServerSupportLazyLoading();
-        if (supported) {
-            opts.filter = await this.createFilter(LAZY_LOADING_SYNC_FILTER);
-        } else {
-            console.log("LL: lazy loading requested but not supported " +
-                "by server, so disabling");
-            opts.lazyLoadMembers = false;
-        }
-    }
-
-    if (opts.lazyLoadMembers && this._crypto) {
-        this._crypto.enableLazyLoading();
-    }
-
     opts.crypto = this._crypto;
     opts.canResetEntireTimeline = (roomId) => {
         if (!this._canResetTimelineCallback) {
@@ -3110,11 +3131,29 @@ MatrixClient.prototype.startClient = async function(opts) {
         return this._canResetTimelineCallback(roomId);
     };
     this._clientOpts = opts;
-
     this._syncApi = new SyncApi(this, opts);
     this._syncApi.sync();
 };
 
+/**
+ * store client options with boolean/string/numeric values
+ * to know in the next session what flags the sync data was
+ * created with (e.g. lazy loading)
+ * @param {object} opts the complete set of client options
+ * @return {Promise} for store operation */
+MatrixClient.prototype._storeClientOptions = function() {
+    const primTypes = ["boolean", "string", "number"];
+    const serializableOpts = Object.entries(this._clientOpts)
+        .filter(([key, value]) => {
+            return primTypes.includes(typeof value);
+        })
+        .reduce((obj, [key, value]) => {
+            obj[key] = value;
+            return obj;
+        }, {});
+    return this.store.storeClientOptions(serializableOpts);
+};
+
 /**
  * High level helper method to stop the client from polling and allow a
  * clean shutdown.
@@ -3159,6 +3198,14 @@ MatrixClient.prototype.doesServerSupportLazyLoading = async function() {
     return this._serverSupportsLazyLoading;
 };
 
+/*
+ * Get if lazy loading members is being used.
+ * @return {boolean} Whether or not members are lazy loaded by this client
+ */
+MatrixClient.prototype.hasLazyLoadMembersEnabled = function() {
+    return !!this._clientOpts.lazyLoadMembers;
+};
+
 /*
  * Set a function which is called when /sync returns a 'limited' response.
  * It is called with a room ID and returns a boolean. It should return 'true' if the SDK
@@ -3504,6 +3551,12 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED;
  * a state of SYNCING. <i>This is the equivalent of "syncComplete" in the
  * previous API.</i></li>
  *
+ * <li>CATCHUP: The client has detected the connection to the server might be
+ * available again and will now try to do a sync again. As this sync might take
+ * a long time (depending how long ago was last synced, and general server
+ * performance) the client is put in this mode so the UI can reflect trying
+ * to catch up with the server after losing connection.</li>
+ *
  * <li>SYNCING : The client is currently polling for new events from the server.
  * This will be called <i>after</i> processing latest events from a sync.</li>
  *
@@ -3527,11 +3580,11 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED;
  *                                          +---->STOPPED
  *                                          |
  *              +----->PREPARED -------> SYNCING <--+
- *              |        ^                |  ^      |
- *              |        |                |  |      |
- *              |        |                V  |      |
- *   null ------+        |  +--------RECONNECTING   |
- *              |        |  V                       |
+ *              |                        ^  |  ^    |
+ *              |      CATCHUP ----------+  |  |    |
+ *              |        ^                  V  |    |
+ *   null ------+        |  +------- RECONNECTING   |
+ *              |        V  V                       |
  *              +------->ERROR ---------------------+
  *
  * NB: 'null' will never be emitted by this event.
@@ -3581,7 +3634,7 @@ module.exports.CRYPTO_ENABLED = CRYPTO_ENABLED;
  *
  * @param {?Object} data Data about this transition.
  *
- * @param {MatrixError} data.err The matrix error if <code>state=ERROR</code>.
+ * @param {MatrixError} data.error The matrix error if <code>state=ERROR</code>.
  *
  * @param {String} data.oldSyncToken The 'since' token passed to /sync.
  *    <code>null</code> for the first successful sync since this client was

src/crypto/DeviceList.js

@@ -24,6 +24,7 @@ limitations under the License.
 
 import Promise from 'bluebird';
 
+import logger from '../logger';
 import DeviceInfo from './deviceinfo';
 import olmlib from './olmlib';
 import IndexedDBCryptoStore from './store/indexeddb-crypto-store';
@@ -110,7 +111,7 @@ export default class DeviceList {
             'readwrite', [IndexedDBCryptoStore.STORE_DEVICE_DATA], (txn) => {
                 this._cryptoStore.getEndToEndDeviceData(txn, (deviceData) => {
                     if (deviceData === null) {
-                        console.log("Migrating e2e device data...");
+                        logger.log("Migrating e2e device data...");
                         this._devices = this._sessionStore.getAllEndToEndDevices() || {};
                         this._deviceTrackingStatus = (
                             this._sessionStore.getEndToEndDeviceTrackingStatus() || {}
@@ -145,6 +146,12 @@ export default class DeviceList {
         }
     }
 
+    stop() {
+        if (this._saveTimer !== null) {
+            clearTimeout(this._saveTimer);
+        }
+    }
+
     /**
      * Save the device tracking state to storage, if any changes are
      * pending other than updating the sync token
@@ -190,7 +197,7 @@ export default class DeviceList {
             const resolveSavePromise = this._resolveSavePromise;
             this._savePromiseTime = targetTime;
             this._saveTimer = setTimeout(() => {
-                console.log('Saving device tracking data at token ' + this._syncToken);
+                logger.log('Saving device tracking data at token ' + this._syncToken);
                 // null out savePromise now (after the delay but before the write),
                 // otherwise we could return the existing promise when the save has
                 // actually already happened. Likewise for the dirty flag.
@@ -258,7 +265,7 @@ export default class DeviceList {
             if (this._keyDownloadsInProgressByUser[u]) {
                 // already a key download in progress/queued for this user; its results
                 // will be good enough for us.
-                console.log(
+                logger.log(
                     `downloadKeys: already have a download in progress for ` +
                     `${u}: awaiting its result`,
                 );
@@ -269,13 +276,13 @@ export default class DeviceList {
         });
 
         if (usersToDownload.length != 0) {
-            console.log("downloadKeys: downloading for", usersToDownload);
+            logger.log("downloadKeys: downloading for", usersToDownload);
             const downloadPromise = this._doKeyDownload(usersToDownload);
             promises.push(downloadPromise);
         }
 
         if (promises.length === 0) {
-            console.log("downloadKeys: already have all necessary keys");
+            logger.log("downloadKeys: already have all necessary keys");
         }
 
         return Promise.all(promises).then(() => {
@@ -433,7 +440,7 @@ export default class DeviceList {
             throw new Error('userId must be a string; was '+userId);
         }
         if (!this._deviceTrackingStatus[userId]) {
-            console.log('Now tracking device list for ' + userId);
+            logger.log('Now tracking device list for ' + userId);
             this._deviceTrackingStatus[userId] = TRACKING_STATUS_PENDING_DOWNLOAD;
         }
         // we don't yet persist the tracking status, since there may be a lot
@@ -452,7 +459,7 @@ export default class DeviceList {
      */
     stopTrackingDeviceList(userId) {
         if (this._deviceTrackingStatus[userId]) {
-            console.log('No longer tracking device list for ' + userId);
+            logger.log('No longer tracking device list for ' + userId);
             this._deviceTrackingStatus[userId] = TRACKING_STATUS_NOT_TRACKED;
 
             // we don't yet persist the tracking status, since there may be a lot
@@ -487,7 +494,7 @@ export default class DeviceList {
      */
     invalidateUserDeviceList(userId) {
         if (this._deviceTrackingStatus[userId]) {
-            console.log("Marking device list outdated for", userId);
+            logger.log("Marking device list outdated for", userId);
             this._deviceTrackingStatus[userId] = TRACKING_STATUS_PENDING_DOWNLOAD;
 
             // we don't yet persist the tracking status, since there may be a lot
@@ -550,7 +557,7 @@ export default class DeviceList {
         ).then(() => {
             finished(true);
         }, (e) => {
-            console.error(
+            logger.error(
                 'Error downloading keys for ' + users + ":", e,
             );
             finished(false);
@@ -573,7 +580,7 @@ export default class DeviceList {
                 // since we started this request. If that happens, we should
                 // ignore the completion of the first one.
                 if (this._keyDownloadsInProgressByUser[u] !== prom) {
-                    console.log('Another update in the queue for', u,
+                    logger.log('Another update in the queue for', u,
                                 '- not marking up-to-date');
                     return;
                 }
@@ -584,7 +591,7 @@ export default class DeviceList {
                         // we didn't get any new invalidations since this download started:
                         // this user's device list is now up to date.
                         this._deviceTrackingStatus[u] = TRACKING_STATUS_UP_TO_DATE;
-                        console.log("Device list for", u, "now up to date");
+                        logger.log("Device list for", u, "now up to date");
                     } else {
                         this._deviceTrackingStatus[u] = TRACKING_STATUS_PENDING_DOWNLOAD;
                     }
@@ -659,7 +666,7 @@ class DeviceListUpdateSerialiser {
 
         if (this._downloadInProgress) {
             // just queue up these users
-            console.log('Queued key download for', users);
+            logger.log('Queued key download for', users);
             return this._queuedQueryDeferred.promise;
         }
 
@@ -679,7 +686,7 @@ class DeviceListUpdateSerialiser {
         const deferred = this._queuedQueryDeferred;
         this._queuedQueryDeferred = null;
 
-        console.log('Starting key download for', downloadUsers);
+        logger.log('Starting key download for', downloadUsers);
         this._downloadInProgress = true;
 
         const opts = {};
@@ -706,7 +713,7 @@ class DeviceListUpdateSerialiser {
 
             return prom;
         }).done(() => {
-            console.log('Completed key download for ' + downloadUsers);
+            logger.log('Completed key download for ' + downloadUsers);
 
             this._downloadInProgress = false;
             deferred.resolve();
@@ -716,7 +723,7 @@ class DeviceListUpdateSerialiser {
                 this._doQueuedQueries();
             }
         }, (e) => {
-            console.warn('Error downloading keys for ' + downloadUsers + ':', e);
+            logger.warn('Error downloading keys for ' + downloadUsers + ':', e);
             this._downloadInProgress = false;
             deferred.reject(e);
         });
@@ -725,7 +732,7 @@ class DeviceListUpdateSerialiser {
     }
 
     async _processQueryResponseForUser(userId, response) {
-        console.log('got keys for ' + userId + ':', response);
+        logger.log('got keys for ' + userId + ':', response);
 
         // map from deviceid -> deviceinfo for this user
         const userStore = {};
@@ -763,7 +770,7 @@ async function _updateStoredDeviceKeysForUser(_olmDevice, userId, userStore,
         }
 
         if (!(deviceId in userResult)) {
-            console.log("Device " + userId + ":" + deviceId +
+            logger.log("Device " + userId + ":" + deviceId +
                 " has been removed");
             delete userStore[deviceId];
             updated = true;
@@ -780,12 +787,12 @@ async function _updateStoredDeviceKeysForUser(_olmDevice, userId, userStore,
         // check that the user_id and device_id in the response object are
         // correct
         if (deviceResult.user_id !== userId) {
-            console.warn("Mismatched user_id " + deviceResult.user_id +
+            logger.warn("Mismatched user_id " + deviceResult.user_id +
                " in keys from " + userId + ":" + deviceId);
             continue;
         }
         if (deviceResult.device_id !== deviceId) {
-            console.warn("Mismatched device_id " + deviceResult.device_id +
+            logger.warn("Mismatched device_id " + deviceResult.device_id +
                " in keys from " + userId + ":" + deviceId);
             continue;
         }
@@ -815,7 +822,7 @@ async function _storeDeviceKeys(_olmDevice, userStore, deviceResult) {
     const signKeyId = "ed25519:" + deviceId;
     const signKey = deviceResult.keys[signKeyId];
     if (!signKey) {
-        console.warn("Device " + userId + ":" + deviceId +
+        logger.warn("Device " + userId + ":" + deviceId +
             " has no ed25519 key");
         return false;
     }
@@ -825,7 +832,7 @@ async function _storeDeviceKeys(_olmDevice, userStore, deviceResult) {
     try {
         await olmlib.verifySignature(_olmDevice, deviceResult, userId, deviceId, signKey);
     } catch (e) {
-        console.warn("Unable to verify signature on device " +
+        logger.warn("Unable to verify signature on device " +
             userId + ":" + deviceId + ":" + e);
         return false;
     }
@@ -842,7 +849,7 @@ async function _storeDeviceKeys(_olmDevice, userStore, deviceResult) {
             // best off sticking with the original keys.
             //
             // Should we warn the user about it somehow?
-            console.warn("Ed25519 key for device " + userId + ":" +
+            logger.warn("Ed25519 key for device " + userId + ":" +
                deviceId + " has changed");
             return false;
         }

src/crypto/OlmDevice.js

@@ -15,19 +15,9 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
+import logger from '../logger';
 import IndexedDBCryptoStore from './store/indexeddb-crypto-store';
 
-/**
- * olm.js wrapper
- *
- * @module crypto/OlmDevice
- */
-const Olm = global.Olm;
-if (!Olm) {
-    throw new Error("global.Olm is not defined");
-}
-
-
 // The maximum size of an event is 65K, and we base64 the content, so this is a
 // reasonable approximation to the biggest plaintext we can encrypt.
 const MAX_PLAINTEXT_LENGTH = 65536 * 3 / 4;
@@ -127,7 +117,7 @@ OlmDevice.prototype.init = async function() {
     await this._migrateFromSessionStore();
 
     let e2eKeys;
-    const account = new Olm.Account();
+    const account = new global.Olm.Account();
     try {
         await _initialiseAccount(
             this._sessionStore, this._cryptoStore, this._pickleKey, account,
@@ -161,7 +151,7 @@ async function _initialiseAccount(sessionStore, cryptoStore, pickleKey, account)
  * @return {array} The version of Olm.
  */
 OlmDevice.getOlmVersion = function() {
-    return Olm.get_library_version();
+    return global.Olm.get_library_version();
 };
 
 OlmDevice.prototype._migrateFromSessionStore = async function() {
@@ -173,7 +163,7 @@ OlmDevice.prototype._migrateFromSessionStore = async function() {
                     // Migrate from sessionStore
                     pickledAccount = this._sessionStore.getEndToEndAccount();
                     if (pickledAccount !== null) {
-                        console.log("Migrating account from session store");
+                        logger.log("Migrating account from session store");
                         this._cryptoStore.storeAccount(txn, pickledAccount);
                     }
                 }
@@ -195,7 +185,7 @@ OlmDevice.prototype._migrateFromSessionStore = async function() {
                 // has run against the same localstorage and created some spurious sessions.
                 this._cryptoStore.countEndToEndSessions(txn, (count) => {
                     if (count) {
-                        console.log("Crypto store already has sessions: not migrating");
+                        logger.log("Crypto store already has sessions: not migrating");
                         return;
                     }
                     let numSessions = 0;
@@ -207,7 +197,7 @@ OlmDevice.prototype._migrateFromSessionStore = async function() {
                             );
                         }
                     }
-                    console.log(
+                    logger.log(
                         "Migrating " + numSessions + " sessions from session store",
                     );
                 });
@@ -236,14 +226,14 @@ OlmDevice.prototype._migrateFromSessionStore = async function() {
                             ), txn,
                         );
                     } catch (e) {
-                        console.warn(
+                        logger.warn(
                             "Failed to migrate session " + s.senderKey + "/" +
                             s.sessionId + ": " + e.stack || e,
                         );
                     }
                     ++numIbSessions;
                 }
-                console.log(
+                logger.log(
                     "Migrated " + numIbSessions +
                     " inbound group sessions from session store",
                 );
@@ -268,7 +258,7 @@ OlmDevice.prototype._migrateFromSessionStore = async function() {
  */
 OlmDevice.prototype._getAccount = function(txn, func) {
     this._cryptoStore.getAccount(txn, (pickledAccount) => {
-        const account = new Olm.Account();
+        const account = new global.Olm.Account();
         try {
             account.unpickle(this._pickleKey, pickledAccount);
             func(account);
@@ -321,7 +311,7 @@ OlmDevice.prototype._getSession = function(deviceKey, sessionId, txn, func) {
  * @private
  */
 OlmDevice.prototype._unpickleSession = function(pickledSession, func) {
-    const session = new Olm.Session();
+    const session = new global.Olm.Session();
     try {
         session.unpickle(this._pickleKey, pickledSession);
         func(session);
@@ -354,7 +344,7 @@ OlmDevice.prototype._saveSession = function(deviceKey, session, txn) {
  * @private
  */
 OlmDevice.prototype._getUtility = function(func) {
-    const utility = new Olm.Utility();
+    const utility = new global.Olm.Utility();
     try {
         return func(utility);
     } finally {
@@ -466,7 +456,7 @@ OlmDevice.prototype.createOutboundSession = async function(
         ],
         (txn) => {
             this._getAccount(txn, (account) => {
-                const session = new Olm.Session();
+                const session = new global.Olm.Session();
                 try {
                     session.create_outbound(account, theirIdentityKey, theirOneTimeKey);
                     newSessionId = session.session_id();
@@ -510,7 +500,7 @@ OlmDevice.prototype.createInboundSession = async function(
         ],
         (txn) => {
             this._getAccount(txn, (account) => {
-                const session = new Olm.Session();
+                const session = new global.Olm.Session();
                 try {
                     session.create_inbound_from(
                         account, theirDeviceIdentityKey, ciphertext,
@@ -728,7 +718,7 @@ OlmDevice.prototype._getOutboundGroupSession = function(sessionId, func) {
         throw new Error("Unknown outbound group session " + sessionId);
     }
 
-    const session = new Olm.OutboundGroupSession();
+    const session = new global.Olm.OutboundGroupSession();
     try {
         session.unpickle(this._pickleKey, pickled);
         return func(session);
@@ -744,7 +734,7 @@ OlmDevice.prototype._getOutboundGroupSession = function(sessionId, func) {
  * @return {string} sessionId for the outbound session.
  */
 OlmDevice.prototype.createOutboundGroupSession = function() {
-    const session = new Olm.OutboundGroupSession();
+    const session = new global.Olm.OutboundGroupSession();
     try {
         session.create();
         this._saveOutboundGroupSession(session);
@@ -816,7 +806,7 @@ OlmDevice.prototype.getOutboundGroupSessionKey = function(sessionId) {
  * @return {*} result of func
  */
 OlmDevice.prototype._unpickleInboundGroupSession = function(sessionData, func) {
-    const session = new Olm.InboundGroupSession();
+    const session = new global.Olm.InboundGroupSession();
     try {
         session.unpickle(this._pickleKey, sessionData.session);
         return func(session);
@@ -889,7 +879,7 @@ OlmDevice.prototype.addInboundGroupSession = async function(
                 roomId, senderKey, sessionId, txn,
                 (existingSession, existingSessionData) => {
                     if (existingSession) {
-                        console.log(
+                        logger.log(
                             "Update for megolm session " + senderKey + "/" + sessionId,
                         );
                         // for now we just ignore updates. TODO: implement something here
@@ -897,7 +887,7 @@ OlmDevice.prototype.addInboundGroupSession = async function(
                     }
 
                     // new session.
-                    const session = new Olm.InboundGroupSession();
+                    const session = new global.Olm.InboundGroupSession();
                     try {
                         if (exportFormat) {
                             session.import_session(sessionKey);
@@ -1034,7 +1024,7 @@ OlmDevice.prototype.hasInboundSessionKeys = async function(roomId, senderKey, se
                     }
 
                     if (roomId !== sessionData.room_id) {
-                        console.warn(
+                        logger.warn(
                             `requested keys for inbound group session ${senderKey}|` +
                             `${sessionId}, with incorrect room_id ` +
                             `(expected ${sessionData.room_id}, ` +

src/crypto/OutgoingRoomKeyRequestManager.js

@@ -16,6 +16,7 @@ limitations under the License.
 
 import Promise from 'bluebird';
 
+import logger from '../logger';
 import utils from '../utils';
 
 /**
@@ -108,7 +109,7 @@ export default class OutgoingRoomKeyRequestManager {
      * Called when the client is stopped. Stops any running background processes.
      */
     stop() {
-        console.log('stopping OutgoingRoomKeyRequestManager');
+        logger.log('stopping OutgoingRoomKeyRequestManager');
         // stop the timer on the next run
         this._clientRunning = false;
     }
@@ -173,7 +174,7 @@ export default class OutgoingRoomKeyRequestManager {
                     // may have seen it, so we still need to send a cancellation
                     // in that case :/
 
-                    console.log(
+                    logger.log(
                         'deleting unnecessary room key request for ' +
                         stringifyRequestBody(requestBody),
                     );
@@ -201,7 +202,7 @@ export default class OutgoingRoomKeyRequestManager {
                             // the request cancelled. There is no point in
                             // sending another cancellation since the other tab
                             // will do it.
-                            console.log(
+                            logger.log(
                                 'Tried to cancel room key request for ' +
                                 stringifyRequestBody(requestBody) +
                                 ' but it was already cancelled in another tab',
@@ -222,7 +223,7 @@ export default class OutgoingRoomKeyRequestManager {
                             updatedReq,
                             andResend,
                         ).catch((e) => {
-                            console.error(
+                            logger.error(
                                 "Error sending room key request cancellation;"
                                 + " will retry later.", e,
                             );
@@ -261,7 +262,7 @@ export default class OutgoingRoomKeyRequestManager {
             }).catch((e) => {
                 // this should only happen if there is an indexeddb error,
                 // in which case we're a bit stuffed anyway.
-                console.warn(
+                logger.warn(
                     `error in OutgoingRoomKeyRequestManager: ${e}`,
                 );
             }).done();
@@ -282,7 +283,7 @@ export default class OutgoingRoomKeyRequestManager {
             return Promise.resolve();
         }
 
-        console.log("Looking for queued outgoing room key requests");
+        logger.log("Looking for queued outgoing room key requests");
 
         return this._cryptoStore.getOutgoingRoomKeyRequestByState([
             ROOM_KEY_REQUEST_STATES.CANCELLATION_PENDING,
@@ -290,7 +291,7 @@ export default class OutgoingRoomKeyRequestManager {
             ROOM_KEY_REQUEST_STATES.UNSENT,
         ]).then((req) => {
             if (!req) {
-                console.log("No more outgoing room key requests");
+                logger.log("No more outgoing room key requests");
                 this._sendOutgoingRoomKeyRequestsTimer = null;
                 return;
             }
@@ -312,7 +313,7 @@ export default class OutgoingRoomKeyRequestManager {
                 // go around the loop again
                 return this._sendOutgoingRoomKeyRequests();
             }).catch((e) => {
-                console.error("Error sending room key request; will retry later.", e);
+                logger.error("Error sending room key request; will retry later.", e);
                 this._sendOutgoingRoomKeyRequestsTimer = null;
                 this._startTimer();
             }).done();
@@ -321,7 +322,7 @@ export default class OutgoingRoomKeyRequestManager {
 
     // given a RoomKeyRequest, send it and update the request record
     _sendOutgoingRoomKeyRequest(req) {
-        console.log(
+        logger.log(
             `Requesting keys for ${stringifyRequestBody(req.requestBody)}` +
             ` from ${stringifyRecipientList(req.recipients)}` +
             `(id ${req.requestId})`,
@@ -347,7 +348,7 @@ export default class OutgoingRoomKeyRequestManager {
     // Given a RoomKeyRequest, cancel it and delete the request record unless
     // andResend is set, in which case transition to UNSENT.
     _sendOutgoingRoomKeyRequestCancellation(req, andResend) {
-        console.log(
+        logger.log(
             `Sending cancellation for key request for ` +
             `${stringifyRequestBody(req.requestBody)} to ` +
             `${stringifyRecipientList(req.recipients)} ` +

src/crypto/algorithms/megolm.js

@@ -1,5 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
+Copyright 2018 New Vector Ltd
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -23,6 +24,7 @@ limitations under the License.
 
 import Promise from 'bluebird';
 
+const logger = require("../../logger");
 const utils = require("../../utils");
 const olmlib = require("../olmlib");
 const base = require("./base");
@@ -64,7 +66,7 @@ OutboundSessionInfo.prototype.needsRotation = function(
     if (this.useCount >= rotationPeriodMsgs ||
         sessionLifetime >= rotationPeriodMs
        ) {
-        console.log(
+        logger.log(
             "Rotating megolm session after " + this.useCount +
                 " messages, " + sessionLifetime + "ms",
         );
@@ -102,7 +104,7 @@ OutboundSessionInfo.prototype.sharedWithTooManyDevices = function(
         }
 
         if (!devicesInRoom.hasOwnProperty(userId)) {
-            console.log("Starting new session because we shared with " + userId);
+            logger.log("Starting new session because we shared with " + userId);
             return true;
         }
 
@@ -112,7 +114,7 @@ OutboundSessionInfo.prototype.sharedWithTooManyDevices = function(
             }
 
             if (!devicesInRoom[userId].hasOwnProperty(deviceId)) {
-                console.log(
+                logger.log(
                     "Starting new session because we shared with " +
                         userId + ":" + deviceId,
                 );
@@ -181,7 +183,7 @@ MegolmEncryption.prototype._ensureOutboundSession = function(devicesInRoom) {
         if (session && session.needsRotation(self._sessionRotationPeriodMsgs,
                                              self._sessionRotationPeriodMs)
            ) {
-            console.log("Starting new megolm session because we need to rotate.");
+            logger.log("Starting new megolm session because we need to rotate.");
             session = null;
         }
 
@@ -191,7 +193,7 @@ MegolmEncryption.prototype._ensureOutboundSession = function(devicesInRoom) {
         }
 
         if (!session) {
-            console.log(`Starting new megolm session for room ${self._roomId}`);
+            logger.log(`Starting new megolm session for room ${self._roomId}`);
             session = await self._prepareNewSession();
         }
 
@@ -318,7 +320,7 @@ MegolmEncryption.prototype._splitUserDeviceMap = function(
                 continue;
             }
 
-            console.log(
+            logger.log(
                 "share keys with device " + userId + ":" + deviceId,
             );
 
@@ -440,10 +442,10 @@ MegolmEncryption.prototype._shareKeyWithDevices = async function(session, device
             await this._encryptAndSendKeysToDevices(
                 session, key.chain_index, userDeviceMaps[i], payload,
             );
-            console.log(`Completed megolm keyshare in ${this._roomId} `
+            logger.log(`Completed megolm keyshare in ${this._roomId} `
                 + `(slice ${i + 1}/${userDeviceMaps.length})`);
         } catch (e) {
-            console.log(`megolm keyshare in ${this._roomId} `
+            logger.log(`megolm keyshare in ${this._roomId} `
                 + `(slice ${i + 1}/${userDeviceMaps.length}) failed`);
 
             throw e;
@@ -462,7 +464,7 @@ MegolmEncryption.prototype._shareKeyWithDevices = async function(session, device
  */
 MegolmEncryption.prototype.encryptMessage = function(room, eventType, content) {
     const self = this;
-    console.log(`Starting to encrypt event for ${this._roomId}`);
+    logger.log(`Starting to encrypt event for ${this._roomId}`);
 
     return this._getDevicesInRoom(room).then(function(devicesInRoom) {
         // check if any of these devices are not yet known to the user.
@@ -498,6 +500,16 @@ MegolmEncryption.prototype.encryptMessage = function(room, eventType, content) {
     });
 };
 
+/**
+ * Forces the current outbound group session to be discarded such
+ * that another one will be created next time an event is sent.
+ *
+ * This should not normally be necessary.
+ */
+MegolmEncryption.prototype.forceDiscardSession = function() {
+    this._setupPromise = this._setupPromise.then(() => null);
+};
+
 /**
  * Checks the devices we're about to send to and see if any are entirely
  * unknown to the user.  If so, warn the user, and mark them as known to
@@ -771,12 +783,12 @@ MegolmDecryption.prototype.onRoomKeyEvent = function(event) {
         !sessionId ||
         !content.session_key
        ) {
-        console.error("key event is missing fields");
+        logger.error("key event is missing fields");
         return;
     }
 
     if (!senderKey) {
-        console.error("key event has no sender key (not encrypted?)");
+        logger.error("key event has no sender key (not encrypted?)");
         return;
     }
 
@@ -793,13 +805,13 @@ MegolmDecryption.prototype.onRoomKeyEvent = function(event) {
 
         senderKey = content.sender_key;
         if (!senderKey) {
-            console.error("forwarded_room_key event is missing sender_key field");
+            logger.error("forwarded_room_key event is missing sender_key field");
             return;
         }
 
         const ed25519Key = content.sender_claimed_ed25519_key;
         if (!ed25519Key) {
-            console.error(
+            logger.error(
                 `forwarded_room_key_event is missing sender_claimed_ed25519_key field`,
             );
             return;
@@ -812,7 +824,7 @@ MegolmDecryption.prototype.onRoomKeyEvent = function(event) {
         keysClaimed = event.getKeysClaimed();
     }
 
-    console.log(`Adding key for megolm session ${senderKey}|${sessionId}`);
+    logger.log(`Adding key for megolm session ${senderKey}|${sessionId}`);
     this._olmDevice.addInboundGroupSession(
         content.room_id, senderKey, forwardingKeyChain, sessionId,
         content.session_key, keysClaimed,
@@ -829,7 +841,7 @@ MegolmDecryption.prototype.onRoomKeyEvent = function(event) {
         // have another go at decrypting events sent with this session.
         this._retryDecryption(senderKey, sessionId);
     }).catch((e) => {
-        console.error(`Error handling m.room_key_event: ${e}`);
+        logger.error(`Error handling m.room_key_event: ${e}`);
     });
 };
 
@@ -871,7 +883,7 @@ MegolmDecryption.prototype.shareKeysWithDevice = function(keyRequest) {
             return null;
         }
 
-        console.log(
+        logger.log(
             "sharing keys for session " + body.sender_key + "|"
             + body.session_id + " with device "
             + userId + ":" + deviceId,

src/crypto/algorithms/olm.js

@@ -22,6 +22,7 @@ limitations under the License.
  */
 import Promise from 'bluebird';
 
+const logger = require("../../logger");
 const utils = require("../../utils");
 const olmlib = require("../olmlib");
 const DeviceInfo = require("../deviceinfo");
@@ -273,7 +274,7 @@ OlmDecryption.prototype._decryptMessage = async function(
             const payload = await this._olmDevice.decryptMessage(
                 theirDeviceIdentityKey, sessionId, message.type, message.body,
             );
-            console.log(
+            logger.log(
                 "Decrypted Olm message from " + theirDeviceIdentityKey +
                     " with session " + sessionId,
             );
@@ -328,7 +329,7 @@ OlmDecryption.prototype._decryptMessage = async function(
         );
     }
 
-    console.log(
+    logger.log(
         "created new inbound Olm session ID " +
             res.session_id + " with " + theirDeviceIdentityKey,
     );

src/crypto/index.js

@@ -25,6 +25,7 @@ const anotherjson = require('another-json');
 import Promise from 'bluebird';
 import {EventEmitter} from 'events';
 
+const logger = require("../logger");
 const utils = require("../utils");
 const OlmDevice = require("./OlmDevice");
 const olmlib = require("./olmlib");
@@ -36,6 +37,10 @@ const DeviceList = require('./DeviceList').default;
 import OutgoingRoomKeyRequestManager from './OutgoingRoomKeyRequestManager';
 import IndexedDBCryptoStore from './store/indexeddb-crypto-store';
 
+export function isCryptoAvailable() {
+    return Boolean(global.Olm);
+}
+
 /**
  * Cryptography bits
  *
@@ -62,7 +67,7 @@ import IndexedDBCryptoStore from './store/indexeddb-crypto-store';
  *
  * @param {RoomList} roomList An initialised RoomList object
  */
-function Crypto(baseApis, sessionStore, userId, deviceId,
+export default function Crypto(baseApis, sessionStore, userId, deviceId,
                 clientStore, cryptoStore, roomList) {
     this._baseApis = baseApis;
     this._sessionStore = sessionStore;
@@ -124,6 +129,8 @@ utils.inherits(Crypto, EventEmitter);
  * Returns a promise which resolves once the crypto module is ready for use.
  */
 Crypto.prototype.init = async function() {
+    await global.Olm.init();
+
     const sessionStoreHasAccount = Boolean(this._sessionStore.getEndToEndAccount());
     let cryptoStoreHasAccount;
     await this._cryptoStore.doTxn(
@@ -196,7 +203,7 @@ Crypto.prototype.registerEventHandlers = function(eventEmitter) {
         try {
             crypto._onRoomMembership(event, member, oldMembership);
         } catch (e) {
-             console.error("Error handling membership change:", e);
+             logger.error("Error handling membership change:", e);
         }
     });
 
@@ -214,6 +221,7 @@ Crypto.prototype.start = function() {
 /** Stop background processes related to crypto */
 Crypto.prototype.stop = function() {
     this._outgoingRoomKeyRequestManager.stop();
+    this._deviceList.stop();
 };
 
 /**
@@ -381,7 +389,7 @@ function _maybeUploadOneTimeKeys(crypto) {
         // create any more keys.
         return uploadLoop(keyCount);
     }).catch((e) => {
-        console.error("Error uploading one-time keys", e.stack || e);
+        logger.error("Error uploading one-time keys", e.stack || e);
     }).finally(() => {
         // reset _oneTimeKeyCount to prevent start uploading based on old data.
         // it will be set again on the next /sync-response
@@ -606,13 +614,13 @@ Crypto.prototype.getEventSenderDeviceInfo = function(event) {
 
     const claimedKey = event.getClaimedEd25519Key();
     if (!claimedKey) {
-        console.warn("Event " + event.getId() + " claims no ed25519 key: " +
+        logger.warn("Event " + event.getId() + " claims no ed25519 key: " +
                      "cannot verify sending device");
         return null;
     }
 
     if (claimedKey !== device.getFingerprint()) {
-        console.warn(
+        logger.warn(
             "Event " + event.getId() + " claims ed25519 key " + claimedKey +
                 "but sender device has key " + device.getFingerprint());
         return null;
@@ -621,6 +629,23 @@ Crypto.prototype.getEventSenderDeviceInfo = function(event) {
     return device;
 };
 
+/**
+ * Forces the current outbound group session to be discarded such
+ * that another one will be created next time an event is sent.
+ *
+ * @param {string} roomId The ID of the room to discard the session for
+ *
+ * This should not normally be necessary.
+ */
+Crypto.prototype.forceDiscardSession = function(roomId) {
+    const alg = this._roomEncryptors[roomId];
+    if (alg === undefined) throw new Error("Room not encrypted");
+    if (alg.forceDiscardSession === undefined) {
+        throw new Error("Room encryption algorithm doesn't support session discarding");
+    }
+    alg.forceDiscardSession();
+};
+
 /**
  * Configure a room to use encryption (ie, save a flag in the sessionstore).
  *
@@ -641,7 +666,7 @@ Crypto.prototype.setRoomEncryption = async function(roomId, config, inhibitDevic
     const existingConfig = this._roomList.getRoomEncryption(roomId);
     if (existingConfig) {
         if (JSON.stringify(existingConfig) != JSON.stringify(config)) {
-            console.error("Ignoring m.room.encryption event which requests " +
+            logger.error("Ignoring m.room.encryption event which requests " +
                           "a change of config in " + roomId);
             return;
         }
@@ -688,7 +713,7 @@ Crypto.prototype.setRoomEncryption = async function(roomId, config, inhibitDevic
     }
 
     if (!this._lazyLoadMembers) {
-        console.log("Enabling encryption in " + roomId + "; " +
+        logger.log("Enabling encryption in " + roomId + "; " +
             "starting to track device lists for all users therein");
 
         await this.trackRoomDevices(roomId);
@@ -700,7 +725,7 @@ Crypto.prototype.setRoomEncryption = async function(roomId, config, inhibitDevic
             this._deviceList.refreshOutdatedDeviceLists();
         }
     } else {
-        console.log("Enabling encryption in " + roomId);
+        logger.log("Enabling encryption in " + roomId);
     }
 };
 
@@ -721,7 +746,7 @@ Crypto.prototype.trackRoomDevices = function(roomId) {
         if (!room) {
             throw new Error(`Unable to start tracking devices in unknown room ${roomId}`);
         }
-        console.log(`Starting to track devices for room ${roomId} ...`);
+        logger.log(`Starting to track devices for room ${roomId} ...`);
         const members = await room.getEncryptionTargetMembers();
         members.forEach((m) => {
             this._deviceList.startTrackingDeviceList(m.userId);
@@ -817,7 +842,7 @@ Crypto.prototype.importRoomKeys = function(keys) {
     return Promise.map(
         keys, (key) => {
             if (!key.room_id || !key.algorithm) {
-                console.warn("ignoring room key entry with missing fields", key);
+                logger.warn("ignoring room key entry with missing fields", key);
                 return null;
             }
 
@@ -946,7 +971,7 @@ Crypto.prototype.requestRoomKey = function(requestBody, recipients) {
         requestBody, recipients,
     ).catch((e) => {
         // this normally means we couldn't talk to the store
-        console.error(
+        logger.error(
             'Error requesting key for event', e,
         );
     }).done();
@@ -963,7 +988,7 @@ Crypto.prototype.requestRoomKey = function(requestBody, recipients) {
 Crypto.prototype.cancelRoomKeyRequest = function(requestBody, andResend) {
     this._outgoingRoomKeyRequestManager.cancelRoomKeyRequest(requestBody, andResend)
     .catch((e) => {
-        console.warn("Error clearing pending room key requests", e);
+        logger.warn("Error clearing pending room key requests", e);
     }).done();
 };
 
@@ -981,7 +1006,7 @@ Crypto.prototype.onCryptoEvent = async function(event) {
         // finished processing the sync, in onSyncCompleted.
         await this.setRoomEncryption(roomId, content, true);
     } catch (e) {
-        console.error("Error configuring encryption in room " + roomId +
+        logger.error("Error configuring encryption in room " + roomId +
                       ":", e);
     }
 };
@@ -997,7 +1022,7 @@ Crypto.prototype.onSyncWillProcess = async function(syncData) {
         // scratch, so mark everything as untracked. onCryptoEvent will
         // be called for all e2e rooms during the processing of the sync,
         // at which point we'll start tracking all the users of that room.
-        console.log("Initial sync performed - resetting device tracking state");
+        logger.log("Initial sync performed - resetting device tracking state");
         this._deviceList.stopTrackingAllDeviceLists();
         this._roomDeviceTrackingState = {};
     }
@@ -1115,7 +1140,7 @@ Crypto.prototype._onToDeviceEvent = function(event) {
             });
         }
     } catch (e) {
-        console.error("Error handling toDeviceEvent:", e);
+        logger.error("Error handling toDeviceEvent:", e);
     }
 };
 
@@ -1129,7 +1154,7 @@ Crypto.prototype._onRoomKeyEvent = function(event) {
     const content = event.getContent();
 
     if (!content.room_id || !content.algorithm) {
-        console.error("key event is missing fields");
+        logger.error("key event is missing fields");
         return;
     }
 
@@ -1167,12 +1192,12 @@ Crypto.prototype._onRoomMembership = function(event, member, oldMembership) {
     // by calling _trackRoomDevices
     if (this._roomDeviceTrackingState[roomId]) {
         if (member.membership == 'join') {
-            console.log('Join event for ' + member.userId + ' in ' + roomId);
+            logger.log('Join event for ' + member.userId + ' in ' + roomId);
             // make sure we are tracking the deviceList for this user
             this._deviceList.startTrackingDeviceList(member.userId);
         } else if (member.membership == 'invite' &&
                  this._clientStore.getRoom(roomId).shouldEncryptForInvitedMembers()) {
-            console.log('Invite event for ' + member.userId + ' in ' + roomId);
+            logger.log('Invite event for ' + member.userId + ' in ' + roomId);
             this._deviceList.startTrackingDeviceList(member.userId);
         }
     }
@@ -1239,7 +1264,7 @@ Crypto.prototype._processReceivedRoomKeyRequests = async function() {
                 this._processReceivedRoomKeyRequestCancellation(cancellation),
         );
     } catch (e) {
-        console.error(`Error processing room key requsts: ${e}`);
+        logger.error(`Error processing room key requsts: ${e}`);
     } finally {
         this._processingRoomKeyRequests = false;
     }
@@ -1258,13 +1283,13 @@ Crypto.prototype._processReceivedRoomKeyRequest = async function(req) {
     const roomId = body.room_id;
     const alg = body.algorithm;
 
-    console.log(`m.room_key_request from ${userId}:${deviceId}` +
+    logger.log(`m.room_key_request from ${userId}:${deviceId}` +
                 ` for ${roomId} / ${body.session_id} (id ${req.requestId})`);
 
     if (userId !== this._userId) {
         // TODO: determine if we sent this device the keys already: in
         // which case we can do so again.
-        console.log("Ignoring room key request from other user for now");
+        logger.log("Ignoring room key request from other user for now");
         return;
     }
 
@@ -1274,18 +1299,18 @@ Crypto.prototype._processReceivedRoomKeyRequest = async function(req) {
     // if we don't have a decryptor for this room/alg, we don't have
     // the keys for the requested events, and can drop the requests.
     if (!this._roomDecryptors[roomId]) {
-        console.log(`room key request for unencrypted room ${roomId}`);
+        logger.log(`room key request for unencrypted room ${roomId}`);
         return;
     }
 
     const decryptor = this._roomDecryptors[roomId][alg];
     if (!decryptor) {
-        console.log(`room key request for unknown alg ${alg} in room ${roomId}`);
+        logger.log(`room key request for unknown alg ${alg} in room ${roomId}`);
         return;
     }
 
     if (!await decryptor.hasKeysForKeyRequest(req)) {
-        console.log(
+        logger.log(
             `room key request for unknown session ${roomId} / ` +
                 body.session_id,
         );
@@ -1299,7 +1324,7 @@ Crypto.prototype._processReceivedRoomKeyRequest = async function(req) {
     // if the device is is verified already, share the keys
     const device = this._deviceList.getStoredDevice(userId, deviceId);
     if (device && device.isVerified()) {
-        console.log('device is already verified: sharing keys');
+        logger.log('device is already verified: sharing keys');
         req.share();
         return;
     }
@@ -1316,7 +1341,7 @@ Crypto.prototype._processReceivedRoomKeyRequest = async function(req) {
 Crypto.prototype._processReceivedRoomKeyRequestCancellation = async function(
     cancellation,
 ) {
-    console.log(
+    logger.log(
         `m.room_key_request cancellation for ${cancellation.userId}:` +
             `${cancellation.deviceId} (id ${cancellation.requestId})`,
     );
@@ -1501,6 +1526,3 @@ class IncomingRoomKeyRequestCancellation {
  * @event module:client~MatrixClient#"crypto.warning"
  * @param {string} type One of the strings listed above
  */
-
-/** */
-module.exports = Crypto;

src/crypto/olmlib.js

@@ -23,6 +23,7 @@ limitations under the License.
 import Promise from 'bluebird';
 const anotherjson = require('another-json');
 
+const logger = require("../logger");
 const utils = require("../utils");
 
 /**
@@ -65,7 +66,7 @@ module.exports.encryptMessageForDevice = async function(
         return;
     }
 
-    console.log(
+    logger.log(
         "Using sessionid " + sessionId + " for device " +
             recipientUserId + ":" + recipientDevice.deviceId,
     );
@@ -190,7 +191,7 @@ module.exports.ensureOlmSessionsForDevices = async function(
             }
 
             if (!oneTimeKey) {
-                console.warn(
+                logger.warn(
                     "No one-time keys (alg=" + oneTimeKeyAlgorithm +
                         ") for device " + userId + ":" + deviceId,
                 );
@@ -219,7 +220,7 @@ async function _verifyKeyAndStartSession(olmDevice, oneTimeKey, userId, deviceIn
             deviceInfo.getFingerprint(),
         );
     } catch (e) {
-        console.error(
+        logger.error(
             "Unable to verify signature on one-time key for device " +
                 userId + ":" + deviceId + ":", e,
         );
@@ -233,12 +234,12 @@ async function _verifyKeyAndStartSession(olmDevice, oneTimeKey, userId, deviceIn
         );
     } catch (e) {
         // possibly a bad key
-        console.error("Error starting session with device " +
+        logger.error("Error starting session with device " +
                       userId + ":" + deviceId + ": " + e);
         return null;
     }
 
-    console.log("Started new sessionid " + sid +
+    logger.log("Started new sessionid " + sid +
                 " for device " + userId + ":" + deviceId);
     return sid;
 }

src/crypto/store/indexeddb-crypto-store-backend.js

@@ -16,6 +16,8 @@ limitations under the License.
 */
 
 import Promise from 'bluebird';
+
+import logger from '../../logger';
 import utils from '../../utils';
 
 export const VERSION = 6;
@@ -38,7 +40,7 @@ export class Backend {
         // attempts to delete the database will block (and subsequent
         // attempts to re-create it will also block).
         db.onversionchange = (ev) => {
-            console.log(`versionchange for indexeddb ${this._dbName}: closing`);
+            logger.log(`versionchange for indexeddb ${this._dbName}: closing`);
             db.close();
         };
     }
@@ -64,7 +66,7 @@ export class Backend {
         this._getOutgoingRoomKeyRequest(txn, requestBody, (existing) => {
             if (existing) {
                 // this entry matches the request - return it.
-                console.log(
+                logger.log(
                     `already have key request outstanding for ` +
                         `${requestBody.room_id} / ${requestBody.session_id}: ` +
                         `not sending another`,
@@ -75,7 +77,7 @@ export class Backend {
 
             // we got to the end of the list without finding a match
             // - add the new request.
-            console.log(
+            logger.log(
                 `enqueueing key request for ${requestBody.room_id} / ` +
                     requestBody.session_id,
             );
@@ -226,7 +228,7 @@ export class Backend {
             }
             const data = cursor.value;
             if (data.state != expectedState) {
-                console.warn(
+                logger.warn(
                     `Cannot update room key request from ${expectedState} ` +
                     `as it was already updated to ${data.state}`,
                 );
@@ -264,7 +266,7 @@ export class Backend {
             }
             const data = cursor.value;
             if (data.state != expectedState) {
-                console.warn(
+                logger.warn(
                     `Cannot delete room key request in state ${data.state} `
                         + `(expected ${expectedState})`,
                 );
@@ -400,7 +402,7 @@ export class Backend {
                 ev.stopPropagation();
                 // ...and this stops it from aborting the transaction
                 ev.preventDefault();
-                console.log(
+                logger.log(
                     "Ignoring duplicate inbound group session: " +
                     senderCurve25519Key + " / " + sessionId,
                 );
@@ -471,7 +473,7 @@ export class Backend {
 }
 
 export function upgradeDatabase(db, oldVersion) {
-    console.log(
+    logger.log(
         `Upgrading IndexedDBCryptoStore from version ${oldVersion}`
             + ` to ${VERSION}`,
     );

src/crypto/store/indexeddb-crypto-store.js

@@ -17,6 +17,7 @@ limitations under the License.
 
 import Promise from 'bluebird';
 
+import logger from '../../logger';
 import LocalStorageCryptoStore from './localStorage-crypto-store';
 import MemoryCryptoStore from './memory-crypto-store';
 import * as IndexedDBCryptoStoreBackend from './indexeddb-crypto-store-backend';
@@ -64,7 +65,7 @@ export default class IndexedDBCryptoStore {
                 return;
             }
 
-            console.log(`connecting to indexeddb ${this._dbName}`);
+            logger.log(`connecting to indexeddb ${this._dbName}`);
 
             const req = this._indexedDB.open(
                 this._dbName, IndexedDBCryptoStoreBackend.VERSION,
@@ -77,7 +78,7 @@ export default class IndexedDBCryptoStore {
             };
 
             req.onblocked = () => {
-                console.log(
+                logger.log(
                     `can't yet open IndexedDBCryptoStore because it is open elsewhere`,
                 );
             };
@@ -89,17 +90,30 @@ export default class IndexedDBCryptoStore {
             req.onsuccess = (r) => {
                 const db = r.target.result;
 
-                console.log(`connected to indexeddb ${this._dbName}`);
+                logger.log(`connected to indexeddb ${this._dbName}`);
                 resolve(new IndexedDBCryptoStoreBackend.Backend(db));
             };
+        }).then((backend) => {
+            // Edge has IndexedDB but doesn't support compund keys which we use fairly extensively.
+            // Try a dummy query which will fail if the browser doesn't support compund keys, so
+            // we can fall back to a different backend.
+            return backend.doTxn(
+                'readonly',
+                [IndexedDBCryptoStore.STORE_INBOUND_GROUP_SESSIONS],
+                (txn) => {
+                    backend.getEndToEndInboundGroupSession('', '', txn, () => {});
+                }).then(() => {
+                    return backend;
+                },
+            );
         }).catch((e) => {
-            console.warn(
+            logger.warn(
                 `unable to connect to indexeddb ${this._dbName}` +
                     `: falling back to localStorage store: ${e}`,
             );
             return new LocalStorageCryptoStore(global.localStorage);
         }).catch((e) => {
-            console.warn(
+            logger.warn(
                 `unable to open localStorage: falling back to in-memory store: ${e}`,
             );
             return new MemoryCryptoStore();
@@ -120,11 +134,11 @@ export default class IndexedDBCryptoStore {
                 return;
             }
 
-            console.log(`Removing indexeddb instance: ${this._dbName}`);
+            logger.log(`Removing indexeddb instance: ${this._dbName}`);
             const req = this._indexedDB.deleteDatabase(this._dbName);
 
             req.onblocked = () => {
-                console.log(
+                logger.log(
                     `can't yet delete IndexedDBCryptoStore because it is open elsewhere`,
                 );
             };
@@ -134,14 +148,14 @@ export default class IndexedDBCryptoStore {
             };
 
             req.onsuccess = () => {
-                console.log(`Removed indexeddb instance: ${this._dbName}`);
+                logger.log(`Removed indexeddb instance: ${this._dbName}`);
                 resolve();
             };
         }).catch((e) => {
             // in firefox, with indexedDB disabled, this fails with a
             // DOMError. We treat this as non-fatal, so that people can
             // still use the app.
-            console.warn(`unable to delete IndexedDBCryptoStore: ${e}`);
+            logger.warn(`unable to delete IndexedDBCryptoStore: ${e}`);
         });
     }
 

src/crypto/store/localStorage-crypto-store.js

@@ -15,6 +15,8 @@ limitations under the License.
 */
 
 import Promise from 'bluebird';
+
+import logger from '../../logger';
 import MemoryCryptoStore from './memory-crypto-store.js';
 
 /**
@@ -199,8 +201,8 @@ function getJsonItem(store, key) {
         // JSON.parse(null) === null, so this returns null.
         return JSON.parse(store.getItem(key));
     } catch (e) {
-        console.log("Error: Failed to get key %s: %s", key, e.stack || e);
-        console.log(e.stack);
+        logger.log("Error: Failed to get key %s: %s", key, e.stack || e);
+        logger.log(e.stack);
     }
     return null;
 }

src/crypto/store/memory-crypto-store.js

@@ -17,6 +17,7 @@ limitations under the License.
 
 import Promise from 'bluebird';
 
+import logger from '../../logger';
 import utils from '../../utils';
 
 /**
@@ -71,7 +72,7 @@ export default class MemoryCryptoStore {
 
             if (existing) {
                 // this entry matches the request - return it.
-                console.log(
+                logger.log(
                     `already have key request outstanding for ` +
                     `${requestBody.room_id} / ${requestBody.session_id}: ` +
                     `not sending another`,
@@ -81,7 +82,7 @@ export default class MemoryCryptoStore {
 
             // we got to the end of the list without finding a match
             // - add the new request.
-            console.log(
+            logger.log(
                 `enqueueing key request for ${requestBody.room_id} / ` +
                 requestBody.session_id,
             );
@@ -163,7 +164,7 @@ export default class MemoryCryptoStore {
             }
 
             if (req.state != expectedState) {
-                console.warn(
+                logger.warn(
                     `Cannot update room key request from ${expectedState} ` +
                     `as it was already updated to ${req.state}`,
                 );
@@ -194,7 +195,7 @@ export default class MemoryCryptoStore {
             }
 
             if (req.state != expectedState) {
-                console.warn(
+                logger.warn(
                     `Cannot delete room key request in state ${req.state} `
                     + `(expected ${expectedState})`,
                 );

src/errors.js

@@ -0,0 +1,25 @@
+// can't just do InvalidStoreError extends Error
+// because of http://babeljs.io/docs/usage/caveats/#classes
+function InvalidStoreError(reason, value) {
+    const message = `Store is invalid because ${reason}, ` +
+        `please stop the client, delete all data and start the client again`;
+    const instance = Reflect.construct(Error, [message]);
+    Reflect.setPrototypeOf(instance, Reflect.getPrototypeOf(this));
+    instance.reason = reason;
+    instance.value = value;
+    return instance;
+}
+
+InvalidStoreError.TOGGLED_LAZY_LOADING = "TOGGLED_LAZY_LOADING";
+
+InvalidStoreError.prototype = Object.create(Error.prototype, {
+  constructor: {
+    value: Error,
+    enumerable: false,
+    writable: true,
+    configurable: true,
+  },
+});
+Reflect.setPrototypeOf(InvalidStoreError, Error);
+
+module.exports.InvalidStoreError = InvalidStoreError;

src/filter.js

@@ -51,6 +51,17 @@ function Filter(userId, filterId) {
     this.definition = {};
 }
 
+Filter.LAZY_LOADING_MESSAGES_FILTER = {
+    lazy_load_members: true,
+};
+
+Filter.LAZY_LOADING_SYNC_FILTER = {
+    room: {
+        state: Filter.LAZY_LOADING_MESSAGES_FILTER,
+    },
+};
+
+
 /**
  * Get the ID of this filter on your homeserver (if known)
  * @return {?Number} The filter ID

src/http-api.js

@@ -752,6 +752,8 @@ module.exports.MatrixHttpApi.prototype = {
                     method: method,
                     withCredentials: false,
                     qs: queryParams,
+                    qsStringifyOptions: opts.qsStringifyOptions,
+                    useQuerystring: true,
                     body: data,
                     json: false,
                     timeout: localTimeoutMs,

src/logger.js

@@ -0,0 +1,36 @@
+/*
+Copyright 2018 André Jaenisch
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+/**
+ * @module logger
+ */
+const log = require("loglevel");
+
+// This is to demonstrate, that you can use any namespace you want.
+// Namespaces allow you to turn on/off the logging for specific parts of the
+// application.
+// An idea would be to control this via an environment variable (on Node.js).
+// See https://www.npmjs.com/package/debug to see how this could be implemented
+// Part of #332 is introducing a logging library in the first place.
+const DEFAULT_NAME_SPACE = "matrix";
+const logger = log.getLogger(DEFAULT_NAME_SPACE);
+logger.setLevel(log.levels.DEBUG);
+
+/**
+ * Drop-in replacement for <code>console</code> using {@link https://www.npmjs.com/package/loglevel|loglevel}.
+ * Can be tailored down to specific use cases if needed.
+*/
+module.exports = logger;

src/matrix.js

@@ -34,6 +34,8 @@ module.exports.SyncAccumulator = require("./sync-accumulator");
 module.exports.MatrixHttpApi = require("./http-api").MatrixHttpApi;
 /** The {@link module:http-api.MatrixError|MatrixError} class. */
 module.exports.MatrixError = require("./http-api").MatrixError;
+/** The {@link module:errors.InvalidStoreError|InvalidStoreError} class. */
+module.exports.InvalidStoreError = require("./errors").InvalidStoreError;
 /** The {@link module:client.MatrixClient|MatrixClient} class. */
 module.exports.MatrixClient = require("./client").MatrixClient;
 /** The {@link module:models/room|Room} class. */

src/models/room-member.js

@@ -59,7 +59,6 @@ function RoomMember(roomId, userId) {
         member: null,
     };
     this._isOutOfBand = false;
-    this._supersedesOutOfBand = false;
     this._updateModifiedTime();
 }
 utils.inherits(RoomMember, EventEmitter);
@@ -80,31 +79,6 @@ RoomMember.prototype.isOutOfBand = function() {
     return this._isOutOfBand;
 };
 
-/**
- * Does the member supersede an incoming out-of-band
- * member? If so the out-of-band member should be ignored.
- * @return {bool}
- */
-RoomMember.prototype.supersedesOutOfBand = function() {
-    return this._supersedesOutOfBand;
-};
-
-/**
- * Mark the member as superseding the future incoming
- * out-of-band members.
- */
-RoomMember.prototype.markSupersedesOutOfBand = function() {
-    this._supersedesOutOfBand = true;
-};
-
-/**
- * Clear the member superseding the future incoming
- * out-of-band members, as loading finished or failed.
- */
-RoomMember.prototype.clearSupersedesOutOfBand = function() {
-    this._supersedesOutOfBand = false;
-};
-
 /**
  * Update this room member's membership event. May fire "RoomMember.name" if
  * this event updates this member's name.

src/models/room-state.js

@@ -180,7 +180,7 @@ RoomState.prototype.getSentinelMember = function(userId) {
         sentinel = new RoomMember(this.roomId, userId);
         const member = this.members[userId];
         if (member) {
-            sentinel.setMembershipEvent(member.events.member);
+            sentinel.setMembershipEvent(member.events.member, this);
         }
         this._sentinels[userId] = sentinel;
     }
@@ -220,8 +220,7 @@ RoomState.prototype.clone = function() {
     // if loading is in progress (through _oobMemberFlags)
     // since these are not new members, we're merely copying them
     // set the status to not started
-    // after copying, we set back the status and
-    // copy the superseding flag from the current state
+    // after copying, we set back the status
     const status = this._oobMemberFlags.status;
     this._oobMemberFlags.status = OOB_STATUS_NOTSTARTED;
 
@@ -249,14 +248,6 @@ RoomState.prototype.clone = function() {
                 copyMember.markOutOfBand();
             }
         });
-    } else if (this._oobMemberFlags.status == OOB_STATUS_INPROGRESS) {
-        // copy markSupersedesOutOfBand flags
-        this.getMembers().forEach((member) => {
-            if (member.supersedesOutOfBand()) {
-                const copyMember = copy.getMember(member.userId);
-                copyMember.markSupersedesOutOfBand();
-            }
-        });
     }
 
     return copy;
@@ -341,11 +332,6 @@ RoomState.prototype.setStateEvents = function(stateEvents) {
 
             const member = self._getOrCreateMember(userId, event);
             member.setMembershipEvent(event, self);
-            // if out of band members are loading,
-            // mark the member as more recent
-            if (self._oobMemberFlags.status == OOB_STATUS_INPROGRESS) {
-                member.markSupersedesOutOfBand();
-            }
 
             self._updateMember(member);
             self.emit("RoomState.members", event, self, member);
@@ -434,12 +420,22 @@ RoomState.prototype.markOutOfBandMembersFailed = function() {
     if (this._oobMemberFlags.status !== OOB_STATUS_INPROGRESS) {
         return;
     }
-    // the request failed, there is nothing to supersede
-    // in case of a retry, these event would not supersede the
-    // retry anymore.
-    this.getMembers().forEach((m) => {
-        m.clearSupersedesOutOfBand();
+    this._oobMemberFlags.status = OOB_STATUS_NOTSTARTED;
+};
+
+/**
+ * Clears the loaded out-of-band members
+ */
+RoomState.prototype.clearOutOfBandMembers = function() {
+    let count = 0;
+    Object.keys(this.members).forEach((userId) => {
+        const member = this.members[userId];
+        if (member.isOutOfBand()) {
+            ++count;
+            delete this.members[userId];
+        }
     });
+    console.log(`LL: RoomState removed ${count} members...`);
     this._oobMemberFlags.status = OOB_STATUS_NOTSTARTED;
 };
 
@@ -467,27 +463,15 @@ RoomState.prototype._setOutOfBandMember = function(stateEvent) {
     }
     const userId = stateEvent.getStateKey();
     const existingMember = this.getMember(userId);
-    if (existingMember) {
-        const existingMemberEvent = existingMember.events.member;
-        // ignore out of band members with events we are
-        // already aware of.
-        if (existingMemberEvent.getId() === stateEvent.getId()) {
-            return;
-        }
-        // this member was updated since we started
-        // loading the out of band members.
-        // Ignore the out of band member and clear
-        // the "supersedes" flag as the out of members are now loaded
-        if (existingMember.supersedesOutOfBand()) {
-            existingMember.clearSupersedesOutOfBand();
-            return;
-        }
+    // never replace members received as part of the sync
+    if (existingMember && !existingMember.isOutOfBand()) {
+        return;
     }
 
     const member = this._getOrCreateMember(userId, stateEvent);
-    member.setMembershipEvent(stateEvent);
+    member.setMembershipEvent(stateEvent, this);
     // needed to know which members need to be stored seperately
-    // as the are not part of the sync accumulator
+    // as they are not part of the sync accumulator
     // this is cleared by setMembershipEvent so when it's updated through /sync
     member.markOutOfBand();
 
@@ -495,7 +479,7 @@ RoomState.prototype._setOutOfBandMember = function(stateEvent) {
 
     this._setStateEvent(stateEvent);
     this._updateMember(member);
-    this.emit("RoomState.members", {}, stateEvent, member);
+    this.emit("RoomState.members", stateEvent, this, member);
 };
 
 /**
@@ -651,11 +635,6 @@ RoomState.prototype.maySendStateEvent = function(stateEventType, userId) {
  *                        according to the room's state.
  */
 RoomState.prototype._maySendEventOfType = function(eventType, userId, state) {
-    const member = this.getMember(userId);
-    if (!member || member.membership == 'leave') {
-        return false;
-    }
-
     const power_levels_event = this.getStateEvents('m.room.power_levels', '');
 
     let power_levels;
@@ -663,25 +642,34 @@ RoomState.prototype._maySendEventOfType = function(eventType, userId, state) {
 
     let state_default = 0;
     let events_default = 0;
+    let powerLevel = 0;
     if (power_levels_event) {
         power_levels = power_levels_event.getContent();
         events_levels = power_levels.events || {};
 
-        if (utils.isNumber(power_levels.state_default)) {
+        if (Number.isFinite(power_levels.state_default)) {
             state_default = power_levels.state_default;
         } else {
             state_default = 50;
         }
-        if (utils.isNumber(power_levels.events_default)) {
+
+        const userPowerLevel = power_levels.users && power_levels.users[userId];
+        if (Number.isFinite(userPowerLevel)) {
+            powerLevel = userPowerLevel;
+        } else if(Number.isFinite(power_levels.users_default)) {
+            powerLevel = power_levels.users_default;
+        }
+
+        if (Number.isFinite(power_levels.events_default)) {
             events_default = power_levels.events_default;
         }
     }
 
     let required_level = state ? state_default : events_default;
-    if (utils.isNumber(events_levels[eventType])) {
+    if (Number.isFinite(events_levels[eventType])) {
         required_level = events_levels[eventType];
     }
-    return member.powerLevel >= required_level;
+    return powerLevel >= required_level;
 };
 
 /**

src/models/room.js

@@ -178,7 +178,7 @@ function Room(roomId, client, myUserId, opts) {
 
     // read by megolm; boolean value - null indicates "use global value"
     this._blacklistUnverifiedDevices = null;
-    this._syncedMembership = null;
+    this._selfMembership = null;
     this._summaryHeroes = null;
     // awaited by getEncryptionTargetMembers while room members are loading
 
@@ -198,7 +198,10 @@ utils.inherits(Room, EventEmitter);
  */
 Room.prototype.getVersion = function() {
     const createEvent = this.currentState.getStateEvents("m.room.create", "");
-    if (!createEvent) return null;
+    if (!createEvent) {
+        console.warn("Room " + this.room_id + " does not have an m.room.create event");
+        return '1';
+    }
     const ver = createEvent.getContent()['room_version'];
     if (ver === undefined) return '1';
     return ver;
@@ -257,13 +260,7 @@ Room.prototype.getLiveTimeline = function() {
  * @return {string} the membership type (join | leave | invite) for the logged in user
  */
 Room.prototype.getMyMembership = function() {
-    if (this.myUserId) {
-        const me = this.getMember(this.myUserId);
-        if (me) {
-            return me.membership;
-        }
-    }
-    return this._syncedMembership;
+    return this._selfMembership;
 };
 
 /**
@@ -278,7 +275,7 @@ Room.prototype.getDMInviter = function() {
             return me.getDMInviter();
         }
     }
-    if (this._syncedMembership === "invite") {
+    if (this._selfMembership === "invite") {
         // fall back to summary information
         const memberCount = this.getInvitedAndJoinedMemberCount();
         if (memberCount == 2 && this._summaryHeroes.length) {
@@ -299,27 +296,17 @@ Room.prototype.guessDMUserId = function() {
             return inviterId;
         }
     }
-    const fallbackMember = this.getAvatarFallbackMember();
-    if (fallbackMember) {
-        return fallbackMember.userId;
-    }
-    // now we're getting into sketchy territory,
-    // but we're assuming this room is marked as a DM
-    // so we're going to make a wild-ish guess with whom
-    if (this._summaryHeroes.length) {
+    // remember, we're assuming this room is a DM,
+    // so returning the first member we find should be fine
+    const hasHeroes = Array.isArray(this._summaryHeroes) &&
+        this._summaryHeroes.length;
+    if (hasHeroes) {
         return this._summaryHeroes[0];
     }
     const members = this.currentState.getMembers();
-    const anyMember = members.filter((m) => m.userId !== this.myUserId);
+    const anyMember = members.find((m) => m.userId !== this.myUserId);
     if (anyMember) {
-        return anyMember;
-    }
-    const createEvent = this.currentState.getStateEvents("m.room.create", "");
-    if (createEvent) {
-        const sender = createEvent.getSender();
-        if (sender !== this.myUserId) {
-            return sender;
-        }
+        return anyMember.userId;
     }
     // it really seems like I'm the only user in the room
     // so I probably created a room with just me in it
@@ -372,14 +359,20 @@ Room.prototype.getAvatarFallbackMember = function() {
  * Sets the membership this room was received as during sync
  * @param {string} membership join | leave | invite
  */
-Room.prototype.setSyncedMembership = function(membership) {
-    this._syncedMembership = membership;
+Room.prototype.updateMyMembership = function(membership) {
+    const prevMembership = this._selfMembership;
+    this._selfMembership = membership;
+    if (prevMembership !== membership) {
+        if (membership === "leave") {
+            this._cleanupAfterLeaving();
+        }
+        this.emit("Room.myMembership", this, membership, prevMembership);
+    }
 };
 
 Room.prototype._loadMembersFromServer = async function() {
     const lastSyncToken = this._client.store.getSyncToken();
     const queryString = utils.encodeParams({
-        membership: "join",
         not_membership: "leave",
         at: lastSyncToken,
     });
@@ -424,8 +417,21 @@ Room.prototype.loadMembersIfNeeded = function() {
     // the OOB members
     this.currentState.markOutOfBandMembersStarted();
 
-    const promise = this._loadMembers().then(({memberEvents, fromServer}) => {
-        this.currentState.setOutOfBandMembers(memberEvents);
+    const inMemoryUpdate = this._loadMembers().then((result) => {
+        this.currentState.setOutOfBandMembers(result.memberEvents);
+        // now the members are loaded, start to track the e2e devices if needed
+        if (this._client.isRoomEncrypted(this.roomId)) {
+            this._client._crypto.trackRoomDevices(this.roomId);
+        }
+        return result.fromServer;
+    }).catch((err) => {
+        // allow retries on fail
+        this._membersPromise = null;
+        this.currentState.markOutOfBandMembersFailed();
+        throw err;
+    });
+    // update members in storage, but don't wait for it
+    inMemoryUpdate.then((fromServer) => {
         if (fromServer) {
             const oobMembers = this.currentState.getMembers()
                 .filter((m) => m.isOutOfBand())
@@ -442,21 +448,40 @@ Room.prototype.loadMembersIfNeeded = function() {
                 });
         }
     }).catch((err) => {
-        // allow retries on fail
-        this._membersPromise = null;
-        this.currentState.markOutOfBandMembersFailed();
-        throw err;
+        // as this is not awaited anywhere,
+        // at least show the error in the console
+        console.error(err);
     });
 
-    this._membersPromise = promise;
-    // now the members are loaded, start to track the e2e devices if needed
-    if (this._client.isRoomEncrypted(this.roomId)) {
-        this._client._crypto.trackRoomDevices(this.roomId);
-    }
+    this._membersPromise = inMemoryUpdate;
 
     return this._membersPromise;
 };
 
+/**
+ * Removes the lazily loaded members from storage if needed
+ */
+Room.prototype.clearLoadedMembersIfNeeded = async function() {
+    if (this._opts.lazyLoadMembers && this._membersPromise) {
+        await this.loadMembersIfNeeded();
+        await this._client.store.clearOutOfBandMembers(this.roomId);
+        this.currentState.clearOutOfBandMembers();
+        this._membersPromise = null;
+    }
+};
+
+/**
+ * called when sync receives this room in the leave section
+ * to do cleanup after leaving a room. Possibly called multiple times.
+ */
+Room.prototype._cleanupAfterLeaving = function() {
+    this.clearLoadedMembersIfNeeded().catch((err) => {
+        console.error(`error after clearing loaded members from ` +
+            `room ${this.roomId} after leaving`);
+        console.dir(err);
+    });
+};
+
 /**
  * Reset the live timeline of all timelineSets, and start new ones.
  *
@@ -974,6 +999,10 @@ Room.prototype.addPendingEvent = function(event, txnId) {
     this._txnToEvent[txnId] = event;
 
     if (this._opts.pendingEventOrdering == "detached") {
+        if (this._pendingEventList.some((e) => e.status === EventStatus.NOT_SENT)) {
+            console.warn("Setting event as NOT_SENT due to messages in the same state");
+            event.status = EventStatus.NOT_SENT;
+        }
         this._pendingEventList.push(event);
     } else {
         for (let i = 0; i < this._timelineSets.length; i++) {
@@ -1472,6 +1501,17 @@ Room.prototype.getAccountData = function(type) {
     return this.accountData[type];
 };
 
+
+/**
+ * Returns wheter the syncing user has permission to send a message in the room
+ * @return {boolean} true if the user should be permitted to send
+ *                   message events into the room.
+ */
+Room.prototype.maySendMessage = function() {
+    return this.getMyMembership() === 'join' &&
+        this.currentState.maySendEvent('m.room.message', this.myUserId);
+};
+
 /**
  * This is an internal method. Calculates the name of the room from the current
  * room state.

src/store/indexeddb-local-backend.js

@@ -19,7 +19,7 @@ import Promise from 'bluebird';
 import SyncAccumulator from "../sync-accumulator";
 import utils from "../utils";
 
-const VERSION = 2;
+const VERSION = 3;
 
 function createDatabase(db) {
     // Make user store, clobber based on user ID. (userId property of User objects)
@@ -41,6 +41,12 @@ function upgradeSchemaV2(db) {
     oobMembersStore.createIndex("room", "room_id");
 }
 
+function upgradeSchemaV3(db) {
+    db.createObjectStore("client_options",
+        { keyPath: ["clobber"]});
+}
+
+
 /**
  * Helper method to collect results from a Cursor and promiseify it.
  * @param {ObjectStore|Index} store The store to perform openCursor on.
@@ -71,28 +77,39 @@ function selectQuery(store, keyRange, resultMapper) {
     });
 }
 
-function promiseifyTxn(txn) {
+function txnAsPromise(txn) {
     return new Promise((resolve, reject) => {
         txn.oncomplete = function(event) {
             resolve(event);
         };
         txn.onerror = function(event) {
-            reject(event);
+            reject(event.target.error);
         };
     });
 }
 
-function promiseifyRequest(req) {
+function reqAsEventPromise(req) {
     return new Promise((resolve, reject) => {
         req.onsuccess = function(event) {
             resolve(event);
         };
         req.onerror = function(event) {
-            reject(event);
+            reject(event.target.error);
         };
     });
 }
 
+function reqAsPromise(req) {
+    return new Promise((resolve, reject) => {
+        req.onsuccess = () => resolve(req);
+        req.onerror = (err) => reject(err);
+    });
+}
+
+function reqAsCursorPromise(req) {
+    return reqAsEventPromise(req).then((event) => event.target.result);
+}
+
 /**
  * Does the actual reading from and writing to the indexeddb
  *
@@ -112,6 +129,7 @@ const LocalIndexedDBStoreBackend = function LocalIndexedDBStoreBackend(
     this.db = null;
     this._disconnected = true;
     this._syncAccumulator = new SyncAccumulator();
+    this._isNewlyCreated = false;
 };
 
 
@@ -142,11 +160,15 @@ LocalIndexedDBStoreBackend.prototype = {
                 `LocalIndexedDBStoreBackend.connect: upgrading from ${oldVersion}`,
             );
             if (oldVersion < 1) { // The database did not previously exist.
+                this._isNewlyCreated = true;
                 createDatabase(db);
             }
             if (oldVersion < 2) {
                 upgradeSchemaV2(db);
             }
+            if (oldVersion < 3) {
+                upgradeSchemaV3(db);
+            }
             // Expand as needed.
         };
 
@@ -159,7 +181,7 @@ LocalIndexedDBStoreBackend.prototype = {
         console.log(
             `LocalIndexedDBStoreBackend.connect: awaiting connection...`,
         );
-        return promiseifyRequest(req).then((ev) => {
+        return reqAsEventPromise(req).then((ev) => {
             console.log(
                 `LocalIndexedDBStoreBackend.connect: connected`,
             );
@@ -174,6 +196,10 @@ LocalIndexedDBStoreBackend.prototype = {
             return this._init();
         });
     },
+    /** @return {bool} whether or not the database was newly created in this session. */
+    isNewlyCreated: function() {
+        return Promise.resolve(this._isNewlyCreated);
+    },
 
     /**
      * Having connected, load initial data from the database and prepare for use
@@ -254,42 +280,66 @@ LocalIndexedDBStoreBackend.prototype = {
      * marked as fetched, and getOutOfBandMembers will return an empty array instead of null
      * @param {string} roomId
      * @param {event[]} membershipEvents the membership events to store
-     * @returns {Promise} when all members have been stored
      */
-    setOutOfBandMembers: function(roomId, membershipEvents) {
+    setOutOfBandMembers: async function(roomId, membershipEvents) {
         console.log(`LL: backend about to store ${membershipEvents.length}` +
             ` members for ${roomId}`);
-        function ignoreResult() {}
-        // run everything in a promise so anything that throws will reject
-        return new Promise((resolve) =>{
-            const tx = this.db.transaction(["oob_membership_events"], "readwrite");
-            const store = tx.objectStore("oob_membership_events");
-            const eventPuts = membershipEvents.map((e) => {
-                const putPromise = promiseifyRequest(store.put(e));
-                // ignoring the result makes sure we discard the IDB success event
-                // ASAP, and not create a potentially big array containing them
-                // unneccesarily later on by calling Promise.all.
-                return putPromise.then(ignoreResult);
-            });
-            // aside from all the events, we also write a marker object to the store
-            // to mark the fact that OOB members have been written for this room.
-            // It's possible that 0 members need to be written as all where previously know
-            // but we still need to know whether to return null or [] from getOutOfBandMembers
-            // where null means out of band members haven't been stored yet for this room
-            const markerObject = {
-                room_id: roomId,
-                oob_written: true,
-                state_key: 0,
-            };
-            const markerPut = promiseifyRequest(store.put(markerObject));
-            const allPuts = eventPuts.concat(markerPut);
-            // ignore the empty array Promise.all creates
-            // as this method should just resolve
-            // to undefined on success
-            resolve(Promise.all(allPuts).then(ignoreResult));
-        }).then(() => {
-            console.log(`LL: backend done storing for ${roomId}!`);
+        const tx = this.db.transaction(["oob_membership_events"], "readwrite");
+        const store = tx.objectStore("oob_membership_events");
+        membershipEvents.forEach((e) => {
+            store.put(e);
         });
+        // aside from all the events, we also write a marker object to the store
+        // to mark the fact that OOB members have been written for this room.
+        // It's possible that 0 members need to be written as all where previously know
+        // but we still need to know whether to return null or [] from getOutOfBandMembers
+        // where null means out of band members haven't been stored yet for this room
+        const markerObject = {
+            room_id: roomId,
+            oob_written: true,
+            state_key: 0,
+        };
+        store.put(markerObject);
+        await txnAsPromise(tx);
+        console.log(`LL: backend done storing for ${roomId}!`);
+    },
+
+    clearOutOfBandMembers: async function(roomId) {
+        // the approach to delete all members for a room
+        // is to get the min and max state key from the index
+        // for that room, and then delete between those
+        // keys in the store.
+        // this should be way faster than deleting every member
+        // individually for a large room.
+        const readTx = this.db.transaction(
+            ["oob_membership_events"],
+            "readonly");
+        const store = readTx.objectStore("oob_membership_events");
+        const roomIndex = store.index("room");
+        const roomRange = IDBKeyRange.only(roomId);
+
+        const minStateKeyProm = reqAsCursorPromise(
+                roomIndex.openKeyCursor(roomRange, "next"),
+            ).then((cursor) => cursor && cursor.primaryKey[1]);
+        const maxStateKeyProm = reqAsCursorPromise(
+                roomIndex.openKeyCursor(roomRange, "prev"),
+            ).then((cursor) => cursor && cursor.primaryKey[1]);
+        const [minStateKey, maxStateKey] = await Promise.all(
+            [minStateKeyProm, maxStateKeyProm]);
+
+        const writeTx = this.db.transaction(
+            ["oob_membership_events"],
+            "readwrite");
+        const writeStore = writeTx.objectStore("oob_membership_events");
+        const membersKeyRange = IDBKeyRange.bound(
+            [roomId, minStateKey],
+            [roomId, maxStateKey],
+        );
+
+        console.log(`LL: Deleting all users + marker in storage for ` +
+            `room ${roomId}, with key range:`,
+            [roomId, minStateKey], [roomId, maxStateKey]);
+        await reqAsPromise(writeStore.delete(membersKeyRange));
     },
 
     /**
@@ -389,7 +439,7 @@ LocalIndexedDBStoreBackend.prototype = {
                 roomsData: roomsData,
                 groupsData: groupsData,
             }); // put == UPSERT
-            return promiseifyTxn(txn);
+            return txnAsPromise(txn);
         });
     },
 
@@ -406,7 +456,7 @@ LocalIndexedDBStoreBackend.prototype = {
             for (let i = 0; i < accountData.length; i++) {
                 store.put(accountData[i]); // put == UPSERT
             }
-            return promiseifyTxn(txn);
+            return txnAsPromise(txn);
         });
     },
 
@@ -428,7 +478,7 @@ LocalIndexedDBStoreBackend.prototype = {
                     event: tuple[1],
                 }); // put == UPSERT
             }
-            return promiseifyTxn(txn);
+            return txnAsPromise(txn);
         });
     },
 
@@ -494,6 +544,28 @@ LocalIndexedDBStoreBackend.prototype = {
             });
         });
     },
+
+    getClientOptions: function() {
+        return Promise.resolve().then(() => {
+            const txn = this.db.transaction(["client_options"], "readonly");
+            const store = txn.objectStore("client_options");
+            return selectQuery(store, undefined, (cursor) => {
+                if (cursor.value && cursor.value && cursor.value.options) {
+                    return cursor.value.options;
+                }
+            }).then((results) => results[0]);
+        });
+    },
+
+    storeClientOptions: async function(options) {
+        const txn = this.db.transaction(["client_options"], "readwrite");
+        const store = txn.objectStore("client_options");
+        store.put({
+            clobber: "-", // constant key so will always clobber
+            options: options,
+        }); // put == UPSERT
+        await txnAsPromise(txn);
+    },
 };
 
 export default LocalIndexedDBStoreBackend;

src/store/indexeddb-remote-backend.js

@@ -65,7 +65,10 @@ RemoteIndexedDBStoreBackend.prototype = {
     clearDatabase: function() {
         return this._ensureStarted().then(() => this._doCmd('clearDatabase'));
     },
-
+    /** @return {Promise<bool>} whether or not the database was newly created in this session. */
+    isNewlyCreated: function() {
+        return this._doCmd('isNewlyCreated');
+    },
     /**
      * @return {Promise} Resolves with a sync response to restore the
      * client state to where it was at the last save, or null if there
@@ -110,6 +113,18 @@ RemoteIndexedDBStoreBackend.prototype = {
         return this._doCmd('setOutOfBandMembers', [roomId, membershipEvents]);
     },
 
+    clearOutOfBandMembers: function(roomId) {
+        return this._doCmd('clearOutOfBandMembers', [roomId]);
+    },
+
+    getClientOptions: function() {
+        return this._doCmd('getClientOptions');
+    },
+
+    storeClientOptions: function(options) {
+        return this._doCmd('storeClientOptions', [options]);
+    },
+
     /**
      * Load all user presence events from the database. This is not cached.
      * @return {Promise<Object[]>} A list of presence events in their raw form.
@@ -169,7 +184,9 @@ RemoteIndexedDBStoreBackend.prototype = {
             if (msg.command == 'cmd_success') {
                 def.resolve(msg.result);
             } else {
-                def.reject(msg.error);
+                const error = new Error(msg.error.message);
+                error.name = msg.error.name;
+                def.reject(error);
             }
         } else {
             console.warn("Unrecognised message from worker: " + msg);

src/store/indexeddb-store-worker.js

@@ -67,6 +67,9 @@ class IndexedDBStoreWorker {
             case 'connect':
                 prom = this.backend.connect();
                 break;
+            case 'isNewlyCreated':
+                prom = this.backend.isNewlyCreated();
+                break;
             case 'clearDatabase':
                 prom = this.backend.clearDatabase().then((result) => {
                     // This returns special classes which can't be cloned
@@ -95,13 +98,22 @@ class IndexedDBStoreWorker {
             case 'getOutOfBandMembers':
                 prom = this.backend.getOutOfBandMembers(msg.args[0]);
                 break;
+            case 'clearOutOfBandMembers':
+                prom = this.backend.clearOutOfBandMembers(msg.args[0]);
+                break;
             case 'setOutOfBandMembers':
                 prom = this.backend.setOutOfBandMembers(msg.args[0], msg.args[1]);
                 break;
+            case 'getClientOptions':
+                prom = this.backend.getClientOptions();
+                break;
+            case 'storeClientOptions':
+                prom = this.backend.storeClientOptions(msg.args[0]);
+                break;
         }
 
         if (prom === undefined) {
-            postMessage({
+            this.postMessage({
                 command: 'cmd_fail',
                 seq: msg.seq,
                 // Can't be an Error because they're not structured cloneable
@@ -123,7 +135,10 @@ class IndexedDBStoreWorker {
                 command: 'cmd_fail',
                 seq: msg.seq,
                 // Just send a string because Error objects aren't cloneable
-                error: "Error running command",
+                error: {
+                    message: err.message,
+                    name: err.name,
+                },
             });
         });
     }

src/store/indexeddb.js

@@ -146,6 +146,11 @@ IndexedDBStore.prototype.getSavedSync = function() {
     return this.backend.getSavedSync();
 };
 
+/** @return {Promise<bool>} whether or not the database was newly created in this session. */
+IndexedDBStore.prototype.isNewlyCreated = function() {
+    return this.backend.isNewlyCreated();
+};
+
 /**
  * @return {Promise} If there is a saved sync, the nextBatch token
  * for this sync, otherwise null.
@@ -242,4 +247,16 @@ IndexedDBStore.prototype.setOutOfBandMembers = function(roomId, membershipEvents
     return this.backend.setOutOfBandMembers(roomId, membershipEvents);
 };
 
+IndexedDBStore.prototype.clearOutOfBandMembers = function(roomId) {
+    return this.backend.clearOutOfBandMembers(roomId);
+};
+
+IndexedDBStore.prototype.getClientOptions = function() {
+    return this.backend.getClientOptions();
+};
+
+IndexedDBStore.prototype.storeClientOptions = function(options) {
+    return this.backend.storeClientOptions(options);
+};
+
 module.exports.IndexedDBStore = IndexedDBStore;

src/store/memory.js

@@ -55,6 +55,7 @@ module.exports.MatrixInMemoryStore = function MatrixInMemoryStore(opts) {
     this._oobMembers = {
         // roomId: [member events]
     };
+    this._clientOptions = {};
 };
 
 module.exports.MatrixInMemoryStore.prototype = {
@@ -67,6 +68,10 @@ module.exports.MatrixInMemoryStore.prototype = {
         return this.syncToken;
     },
 
+    /** @return {Promise<bool>} whether or not the database was newly created in this session. */
+    isNewlyCreated: function() {
+        return Promise.resolve(true);
+    },
 
     /**
      * Set the token to stream from.
@@ -402,4 +407,13 @@ module.exports.MatrixInMemoryStore.prototype = {
         this._oobMembers[roomId] = membershipEvents;
         return Promise.resolve();
     },
+
+    getClientOptions: function() {
+        return Promise.resolve(this._clientOptions);
+    },
+
+    storeClientOptions: function(options) {
+        this._clientOptions = Object.assign({}, options);
+        return Promise.resolve();
+    },
 };

src/store/stub.js

@@ -32,6 +32,11 @@ function StubStore() {
 
 StubStore.prototype = {
 
+    /** @return {Promise<bool>} whether or not the database was newly created in this session. */
+    isNewlyCreated: function() {
+        return Promise.resolve(true);
+    },
+
     /**
      * Get the sync token.
      * @return {string}
@@ -272,6 +277,18 @@ StubStore.prototype = {
     setOutOfBandMembers: function() {
         return Promise.resolve();
     },
+
+    clearOutOfBandMembers: function() {
+        return Promise.resolve();
+    },
+
+    getClientOptions: function() {
+        return Promise.resolve();
+    },
+
+    storeClientOptions: function() {
+        return Promise.resolve();
+    },
 };
 
 /** Stub Store class. */

src/sync.js

@@ -33,6 +33,8 @@ const utils = require("./utils");
 const Filter = require("./filter");
 const EventTimeline = require("./models/event-timeline");
 
+import {InvalidStoreError} from './errors';
+
 const DEBUG = true;
 
 // /sync requests allow you to set a timeout= but the request may continue
@@ -100,6 +102,7 @@ function SyncApi(client, opts) {
     this._connectionReturnedDefer = null;
     this._notifEvents = []; // accumulator of sync events in the current sync response
     this._failedSyncCount = 0; // Number of consecutive failed /sync requests
+    this._storeIsInvalid = false; // flag set if the store needs to be cleared before we can start
 
     if (client.getNotifTimelineSet()) {
         client.reEmitter.reEmit(client.getNotifTimelineSet(),
@@ -123,6 +126,7 @@ SyncApi.prototype.createRoom = function(roomId) {
                           "Room.timelineReset",
                           "Room.localEchoUpdated",
                           "Room.accountData",
+                          "Room.myMembership",
                          ]);
     this._registerStateListeners(room);
     return room;
@@ -421,6 +425,26 @@ SyncApi.prototype.recoverFromSyncStartupError = async function(savedSyncPromise,
     await keepaliveProm;
 };
 
+/**
+ * Is the lazy loading option different than in previous session?
+ * @param {bool} lazyLoadMembers current options for lazy loading
+ * @return {bool} whether or not the option has changed compared to the previous session */
+SyncApi.prototype._wasLazyLoadingToggled = async function(lazyLoadMembers) {
+    lazyLoadMembers = !!lazyLoadMembers;
+    // assume it was turned off before
+    // if we don't know any better
+    let lazyLoadMembersBefore = false;
+    const isStoreNewlyCreated = await this.client.store.isNewlyCreated();
+    if (!isStoreNewlyCreated) {
+        const prevClientOptions = await this.client.store.getClientOptions();
+        if (prevClientOptions) {
+            lazyLoadMembersBefore = !!prevClientOptions.lazyLoadMembers;
+        }
+        return lazyLoadMembersBefore !== lazyLoadMembers;
+    }
+    return false;
+};
+
 /**
  * Main entry point
  */
@@ -443,6 +467,8 @@ SyncApi.prototype.sync = function() {
     //   1) We need to get push rules so we can check if events should bing as we get
     //      them from /sync.
     //   2) We need to get/create a filter which we can use for /sync.
+    //   3) We need to check the lazy loading option matches what was used in the
+    //       stored sync. If it doesn't, we can't use the stored sync.
 
     async function getPushRules() {
         try {
@@ -457,9 +483,47 @@ SyncApi.prototype.sync = function() {
             getPushRules();
             return;
         }
-        getFilter(); // Now get the filter and start syncing
+        checkLazyLoadStatus(); // advance to the next stage
     }
 
+    const checkLazyLoadStatus = async () => {
+        if (this.opts.lazyLoadMembers && client.isGuest()) {
+            this.opts.lazyLoadMembers = false;
+        }
+        if (this.opts.lazyLoadMembers) {
+            const supported = await client.doesServerSupportLazyLoading();
+            if (supported) {
+                this.opts.filter = await client.createFilter(
+                    Filter.LAZY_LOADING_SYNC_FILTER,
+                );
+            } else {
+                console.log("LL: lazy loading requested but not supported " +
+                    "by server, so disabling");
+                this.opts.lazyLoadMembers = false;
+            }
+        }
+        // need to vape the store when enabling LL and wasn't enabled before
+        const shouldClear = await this._wasLazyLoadingToggled(this.opts.lazyLoadMembers);
+        if (shouldClear) {
+            this._storeIsInvalid = true;
+            const reason = InvalidStoreError.TOGGLED_LAZY_LOADING;
+            const error = new InvalidStoreError(reason, !!this.opts.lazyLoadMembers);
+            this._updateSyncState("ERROR", { error });
+            // bail out of the sync loop now: the app needs to respond to this error.
+            // we leave the state as 'ERROR' which isn't great since this normally means
+            // we're retrying. The client must be stopped before clearing the stores anyway
+            // so the app should stop the client, clear the store and start it again.
+            console.warn("InvalidStoreError: store is not usable: stopping sync.");
+            return;
+        }
+        if (this.opts.lazyLoadMembers && this.opts.crypto) {
+            this.opts.crypto.enableLazyLoading();
+        }
+        await this.client._storeClientOptions();
+
+        getFilter(); // Now get the filter and start syncing
+    };
+
     async function getFilter() {
         let filter;
         if (self.opts.filter) {
@@ -587,7 +651,12 @@ SyncApi.prototype._syncFromCache = async function(savedSync) {
         console.error("Error processing cached sync", e.stack || e);
     }
 
-    this._updateSyncState("PREPARED", syncEventData);
+    // Don't emit a prepared if we've bailed because the store is invalid:
+    // in this case the client will not be usable until stopped & restarted
+    // so this would be useless and misleading.
+    if (!this._storeIsInvalid) {
+        this._updateSyncState("PREPARED", syncEventData);
+    }
 };
 
 /**
@@ -779,7 +848,18 @@ SyncApi.prototype._onSyncError = function(err, syncOptions) {
     // erroneous. We set the state to 'reconnecting'
     // instead, so that clients can observe this state
     // if they wish.
-    this._startKeepAlives().then(() => {
+    this._startKeepAlives().then((connDidFail) => {
+        // Only emit CATCHUP if we detected a connectivity error: if we didn't,
+        // it's quite likely the sync will fail again for the same reason and we
+        // want to stay in ERROR rather than keep flip-flopping between ERROR
+        // and CATCHUP.
+        if (connDidFail && this.getSyncState() === 'ERROR') {
+            this._updateSyncState("CATCHUP", {
+                oldSyncToken: null,
+                nextSyncToken: null,
+                catchingUp: true,
+            });
+        }
         this._sync(syncOptions);
     });
 
@@ -965,9 +1045,10 @@ SyncApi.prototype._processSyncResponse = async function(
     // Handle invites
     inviteRooms.forEach(function(inviteObj) {
         const room = inviteObj.room;
-        room.setSyncedMembership("invite");
         const stateEvents =
             self._mapSyncEventsFormat(inviteObj.invite_state, room);
+
+        room.updateMyMembership("invite");
         self._processRoomEvents(room, stateEvents);
         if (inviteObj.isBrandNewRoom) {
             room.recalculate();
@@ -982,7 +1063,6 @@ SyncApi.prototype._processSyncResponse = async function(
     // Handle joins
     await Promise.mapSeries(joinRooms, async function(joinObj) {
         const room = joinObj.room;
-        room.setSyncedMembership("join");
         const stateEvents = self._mapSyncEventsFormat(joinObj.state, room);
         const timelineEvents = self._mapSyncEventsFormat(joinObj.timeline, room);
         const ephemeralEvents = self._mapSyncEventsFormat(joinObj.ephemeral);
@@ -998,6 +1078,8 @@ SyncApi.prototype._processSyncResponse = async function(
             );
         }
 
+        room.updateMyMembership("join");
+
         joinObj.timeline = joinObj.timeline || {};
 
         if (joinObj.isBrandNewRoom) {
@@ -1105,8 +1187,6 @@ SyncApi.prototype._processSyncResponse = async function(
     // Handle leaves (e.g. kicked rooms)
     leaveRooms.forEach(function(leaveObj) {
         const room = leaveObj.room;
-        room.setSyncedMembership("leave");
-
         const stateEvents =
             self._mapSyncEventsFormat(leaveObj.state, room);
         const timelineEvents =
@@ -1114,6 +1194,8 @@ SyncApi.prototype._processSyncResponse = async function(
         const accountDataEvents =
             self._mapSyncEventsFormat(leaveObj.account_data);
 
+        room.updateMyMembership("leave");
+
         self._processRoomEvents(room, stateEvents, timelineEvents);
         room.addAccountData(accountDataEvents);
 
@@ -1206,13 +1288,16 @@ SyncApi.prototype._startKeepAlives = function(delay) {
  *
  * On failure, schedules a call back to itself. On success, resolves
  * this._connectionReturnedDefer.
+ *
+ * @param {bool} connDidFail True if a connectivity failure has been detected. Optional.
  */
-SyncApi.prototype._pokeKeepAlive = function() {
+SyncApi.prototype._pokeKeepAlive = function(connDidFail) {
+    if (connDidFail === undefined) connDidFail = false;
     const self = this;
     function success() {
         clearTimeout(self._keepAliveTimer);
         if (self._connectionReturnedDefer) {
-            self._connectionReturnedDefer.resolve();
+            self._connectionReturnedDefer.resolve(connDidFail);
             self._connectionReturnedDefer = null;
         }
     }
@@ -1229,7 +1314,7 @@ SyncApi.prototype._pokeKeepAlive = function() {
     ).done(function() {
         success();
     }, function(err) {
-        if (err.httpStatus == 400) {
+        if (err.httpStatus == 400 || err.httpStatus == 404) {
             // treat this as a success because the server probably just doesn't
             // support /versions: point is, we're getting a response.
             // We wait a short time though, just in case somehow the server
@@ -1237,8 +1322,9 @@ SyncApi.prototype._pokeKeepAlive = function() {
             // responses fail, this will mean we don't hammer in a loop.
             self._keepAliveTimer = setTimeout(success, 2000);
         } else {
+            connDidFail = true;
             self._keepAliveTimer = setTimeout(
-                self._pokeKeepAlive.bind(self),
+                self._pokeKeepAlive.bind(self, connDidFail),
                 5000 + Math.floor(Math.random() * 5000),
             );
             // A keepalive has failed, so we emit the
@@ -1246,7 +1332,7 @@ SyncApi.prototype._pokeKeepAlive = function() {
             // first failure).
             // Note we do this after setting the timer:
             // this lets the unit tests advance the mock
-            // clock when the get the error.
+            // clock when they get the error.
             self._updateSyncState("ERROR", { error: err });
         }
     });

travis.sh

@@ -5,7 +5,7 @@ set -ex
 npm run lint
 
 # install Olm so that we can run the crypto tests.
-npm install https://matrix.org/packages/npm/olm/olm-2.2.2.tgz
+npm install https://matrix.org/packages/npm/olm/olm-3.0.0.tgz
 
 npm run test