* Prettier complement logs

* Changelog

By returning all of the m.space.child state of the space, not just
the first 50. The number of rooms returned is still capped at 50.

For the federation API this implies that the requesting server will
need to individually query for any other rooms it is not joined to.

* Optionally use an on-disk sqlite db in tests

When debugging a test it is sometimes useful to inspect the state of the
DB. This is not easy when the db is in-memory: one cannot attach the
sqlite CLI to another process's DB.

With this change, if SYNAPSE_TEST_PERSIST_SQLITE_DB is set, we use
 `_trial_temp/test.db` as our sqlite database. One can then use 
`sqlite3 _trial_temp/test.db` and query to your heart's content.

The DB is destroyed and recreated between different test cases.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

This makes the serialization of events synchronous (and it no
longer access the database), but we must manually calculate and
provide the bundled aggregations.

Overall this should cause no change in behavior, but is prep work
for other improvements.

This was removed from MSC2674 before that was approved
and is not used by any known clients.

On Python 2, indexing a byte-string gives back a byte-string,
while on Python 3 it gives back the ASCII equivalent as an int.

* add some tests to verify we are stripping unauthorized fields out of unsigned

* add function to strip unauthorized fields from the unsigned object of event

* newsfragment

* update newsfragment number

* add check to on_send_membership_event

* refactor tests

* fix lint error

* slightly refactor tests and add some comments

* slight refactor

* refactor tests

* fix import error

* slight refactor

* remove unsigned filtration code from synapse/handlers/federation_event.py

* lint

* move unsigned filtering code to event base

* refactor tests

* update newsfragment

* requested changes

* remove unused retun values

Work around Mjolnir compatibility issue by adding an import for `glob_to_regex` in `synapse.util`, where it moved from. (#11696)

Make room creations denied by `user_may_create_room` cause an `M_FORBIDDEN` error to be returned, not `M_UNKNOWN` (#11672)

Co-authored-by: reivilibre <olivier@librepush.net>

as per
https://github.com/matrix-org/synapse/pull/11532#discussion_r769123269

Fixes minor discrepancies between the /hierarchy endpoint described
in MSC2946 and the implementation.

Note that the changes impact the stable and unstable /hierarchy and
unstable /spaces endpoints for both client and federation APIs.

* newsfragment

* fix newsfragment number

* update changelog

* remove extra space

If we've never made a request to a remote homeserver, we should cache the response---even if the response is "this user has no devices".

* `_auth_and_persist_outliers`: mark persisted events as outliers

Mark any events that get persisted via `_auth_and_persist_outliers` as, well,
outliers.

Currently this will be a no-op as everything will already be flagged as an
outlier, but I'm going to change that.

* `process_remote_join`: stop flagging as outlier

The events are now flagged as outliers later on, by `_auth_and_persist_outliers`.

* `send_join`: remove `outlier=True`

The events created here are returned in the result of `send_join` to
`FederationHandler.do_invite_join`. From there they are passed into
`FederationEventHandler.process_remote_join`, which passes them to
`_auth_and_persist_outliers`... which sets the `outlier` flag.

* `get_event_auth`: remove `outlier=True`

stop flagging the events returned by `get_event_auth` as outliers. This method
is only called by `_get_remote_auth_chain_for_event`, which passes the results
into `_auth_and_persist_outliers`, which will flag them as outliers.

* `_get_remote_auth_chain_for_event`: remove `outlier=True`

we pass all the events into `_auth_and_persist_outliers`, which will now flag
the events as outliers.

* `_check_sigs_and_hash_and_fetch`: remove unused `outlier` parameter

This param is now never set to True, so we can remove it.

* `_check_sigs_and_hash_and_fetch_one`: remove unused `outlier` param

This is no longer set anywhere, so we can remove it.

* `get_pdu`: remove unused `outlier` parameter

... and chase it down into `get_pdu_from_destination_raw`.

* `event_from_pdu_json`: remove redundant `outlier` param

This is never set to `True`, so can be removed.

* changelog

* update docstring

Co-authored-by: reivilibre <olivier@librepush.net>
Co-authored-by: reivilibre <oliverw@matrix.org>

Co-authored-by: reivilibre <olivier@librepush.net>

Co-authored-by: reivilibre <olivier@librepush.net>

Co-authored-by: Sean Quah <seanq@element.io>

Clarify SSO mapping provider documentation by writing `def` or `async def` before the names of methods, as appropriate. (#11681)

Co-authored-by: reivilibre <olivier@librepush.net>

this should not be a case-insensitive match.

* Fix AssertionErrors after purging events

If you purged a bunch of events from your database, and then restarted synapse
without receiving more events, then you would get a bunch of AssertionErrors on
restart.

This fixes the situation by rewinding the stream processors.

* `check-newsfragment`: ignore deleted newsfiles

Events returned by `backfill` should not be flagged as outliers.

Fixes:

```
AssertionError: null
  File "synapse/handlers/federation.py", line 313, in try_backfill
    dom, room_id, limit=100, extremities=extremities
  File "synapse/handlers/federation_event.py", line 517, in backfill
    await self._process_pulled_events(dest, events, backfilled=True)
  File "synapse/handlers/federation_event.py", line 642, in _process_pulled_events
    await self._process_pulled_event(origin, ev, backfilled=backfilled)
  File "synapse/handlers/federation_event.py", line 669, in _process_pulled_event
    assert not event.internal_metadata.is_outlier()
```

See https://sentry.matrix.org/sentry/synapse-matrixorg/issues/231992

Fixes #8894.

* Push `get_room_{min,max_stream_ordering}` into StreamStore

Both implementations of this are identical, so we may as well push it down and
get rid of the abstract base class nonsense.

* Remove redundant `StreamStore` class

This is empty now

* Remove redundant `get_current_events_token`

This was an exact duplicate of `get_room_max_stream_ordering`, so let's get rid
of it.

* newsfile

"Unknown room" can mean a multitude of things here. To help with debugging, add
some more words to the exception text.

Invites and knocks will now include the topic in the stripped state
send to clients before joining the room.

* update Trove classifiers to remove py36

* stop building bionic

* update dh-virtualenv

* newsfragment

* fix newsfragment

* update version refs

* another try at correct tag

* Update changelog