The tests helpers automatically convert dictionaries to
JSON payloads, no need to do it manually for each
test.

If you're trying to shut down Synapse, it's rather handy if it *actually* shuts
down before you move on.

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Refactor `backfilled` into specific behavior function arguments (`_persist_events_and_state_updates`) (#11417)

Part of https://github.com/matrix-org/synapse/issues/11300

Call stack:

 - `_persist_events_and_state_updates` (added `use_negative_stream_ordering`)
    - `_persist_events_txn`
       - `_update_room_depths_txn` (added `update_room_forward_stream_ordering`)
       - `_update_metadata_tables_txn`
          - `_store_room_members_txn` (added `inhibit_local_membership_updates`)

Using keyword-only arguments (`*`) to reduce the mistakes from `backfilled` being left as a positional argument somewhere and being interpreted wrong by our new arguments.

This also makes additional updates where the implementation
had drifted from the approved MSC.

Unstable endpoints will be removed at a later data.

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

This does not remove the unstable field and still parses both.
Handling of the unstable field will need to be removed in the
future.

If we tried to request multiple keys for the same server, we would end up
dropping some of those requests.

Signed-off-by: Tulir Asokan <tulir@beeper.com>

This would have caught the bug #11438 introduced in #11217 and fixed in #11439.

Since e81fa926, Synapse depends on
the use_float flag which has been introduced in ijson 3.1 and
is not available in 3.0. This is known to cause runtime errors
with send_join.

Signed-off-by: Daniel Molkentin <danimo@infra.run>

Co-authored-by: Daniel Molkentin <danimo@infra.run>

Update MSC2918 refresh token support to confirm with the latest revision: accept the `refresh_tokens` parameter in the request body rather than in the URL parameters. (#11430)

Also refactor the stream ID trackers/generators a bit and try to
document them better.

Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. (#11425)

The intent is to iterate through all the worker ports and only
report healthy when all are healthy, starting with the main process.

The previous fix for the ongoing event fetches counter
(8eec25a1) was both insufficient and
incorrect.

When the database is unreachable, `_do_fetch` never gets run and so
`_event_fetch_ongoing` is never decremented.

The previous fix also moved the `_event_fetch_ongoing` decrement outside
of the `_event_fetch_lock` which allowed race conditions to corrupt the
counter.

This change makes mypy complain if the constants are ever reassigned,
and, more usefully, makes mypy type them as `Literal`s instead of `str`s,
allowing code of the following form to pass mypy:
```py
def do_something(membership: Literal["join", "leave"], ...): ...

do_something(Membership.JOIN, ...)
```

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Rename unstable `access_token_lifetime` configuration option to `refreshable_access_token_lifetime` to make it clear it only concerns refreshable access tokens. (#11388)

Remove code invalidated by deprecated config flag 'trust_identity_servers_for_password_resets' (#11395)

* remove background update code related to deprecated config flag

* changelog entry

* update changelog

* Delete 11394.removal

Duplicate, wrong number

* add no-op background update and change newfragment so it will be consolidated with associated work

* remove unused code

* Remove code associated with deprecated flag from legacy docker dynamic config file

Co-authored-by: reivilibre <oliverw@matrix.org>

Synapse 1.47.1 (2021-11-23)
===========================

This release fixes a security issue in the media store, affecting all prior releases of Synapse. Server administrators are encouraged to update Synapse as soon as possible. We are not aware of these vulnerabilities being exploited in the wild.

Server administrators who are unable to update Synapse may use the workarounds described in the linked GitHub Security Advisory below.

Security advisory
-----------------

The following issue is fixed in 1.47.1.

- **[GHSA-3hfw-x7gx-437c](https://github.com/matrix-org/synapse/security/advisories/GHSA-3hfw-x7gx-437c) / [CVE-2021-41281](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41281): Path traversal when downloading remote media.**

  Synapse instances with the media repository enabled can be tricked into downloading a file from a remote server into an arbitrary directory, potentially outside the media store directory.

  The last two directories and file name of the path are chosen randomly by Synapse and cannot be controlled by an attacker, which limits the impact.

  Homeservers with the media repository disabled are unaffected. Homeservers configured with a federation whitelist are also unaffected.

  Fixed by [91f2bd09](https://github.com/matrix-org/synapse/commit/91f2bd090).