* Add some tests to characterise the problem

Some failing. Current states:

  RoomsMemberListTestCase
test_get_member_list ...
[OK]
test_get_member_list_mixed_memberships ...
[OK]
test_get_member_list_no_permission ...
[OK]
test_get_member_list_no_permission_former_member ...
[OK]
test_get_member_list_no_permission_former_member_with_at_token ...
[FAIL]
test_get_member_list_no_room ...
[OK]
test_get_member_list_no_permission_with_at_token ...
[FAIL]

* Correct the tests

* Check user is/was member before divulging room membership

* Pull out only the 1 membership event we want.

* Update tests/rest/client/v1/test_rooms.py

Co-authored-by: Erik Johnston <erik@matrix.org>

* Fixup tests (following apply review suggestion)

Co-authored-by: Erik Johnston <erik@matrix.org>

Turns out that the functionality added in #10546 to skip TLS was incompatible
with older Twisted versions, so we need to be a bit more inventive.

Also, add a test to (hopefully) not break this in future. Sadly, testing TLS is
really hard.

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

Allow using several directories in read_templates.

This adds a new API GET /_synapse/admin/v1/username_available?username=foo to check if a username is available. It is the counterpart to https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available, except that it works even if registration is disabled.

* Handle string read receipt data

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Test that we handle string read receipt data

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add changelog for #10606

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add docs

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Ignore malformed RRs

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Only surround hidden = ...

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Remove unnecessary argument

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Update changelog.d/10606.bugfix

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

As opposed to only allowing the summary of spaces which the user is
already in or has world-readable visibility.

This makes the logic consistent with whether a space/room is returned
as part of a space and whether a space summary can start at a space.

And ensure that the file is checked via mypy.

Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

This adds support for the /hierarchy endpoint, which is an
update to MSC2946. Currently this only supports rooms known
locally to the homeserver.

If a room which the requesting user was invited to was queried over
federation it will now properly appear in the spaces summary (instead
of being stripped out by the requesting server).

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

This adds support for MSC3289: room version 8. This is room version 7 + MSC3083.

* drop old-room hack

pretty sure we don't need this any more.

* Remove incorrect comment about modifying `context`

It doesn't look like the supplied context is ever modified.

* Stop `_auth_and_persist_event` modifying its parameters

This is only called in three places. Two of them don't pass `auth_events`, and
the third doesn't use the dict after passing it in, so this should be non-functional.

* Stop `_check_event_auth` modifying its parameters

`_check_event_auth` is only called in three places. `on_send_membership_event`
doesn't pass an `auth_events`, and `prep` and `_auth_and_persist_event` do not
use the map after passing it in.

* Stop `_update_auth_events_and_context_for_auth` modifying its parameters

Return the updated auth event dict, rather than modifying the parameter.

This is only called from `_check_event_auth`.

* Improve documentation on `_auth_and_persist_event`

Rename `auth_events` parameter to better reflect what it contains.

* Improve documentation on `_NewEventInfo`

* Improve documentation on `_check_event_auth`

rename `auth_events` parameter to better describe what it contains

* changelog

This is mostly useful in case the server offers TLS, but doesn't present a valid certificate.

This should have no user-visible changes, but refactors some pieces of
the SpaceSummaryHandler before adding support for the updated
MSC2946.

If there are no services providing a protocol, omit it completely
instead of returning an empty dictionary.

This fixes a long-standing spec compliance bug.

Fixes a bug introduced in rc1 that would cause Synapse to 400 on read receipts requests with empty bodies.

Broken in #10413

Ensure we only load an event from the DB once when the same event is requested multiple times at once.

Makes it easier to fetch user details in for example spam checker modules, without needing to use api._store or figure out database interactions.

Signed-off-by: Jason Robinson <jasonr@matrix.org>

Co-authored-by: Erik Johnston <erik@matrix.org>

Setting the value will help PostgreSQL free up memory by recycling
the connections in the connection pool.

Signed-off-by: Toni Spets <toni.spets@iki.fi>

Reproducible on a federated homeserver when there is a membership auth event as a floating outlier. Then when we try to backfill one of that persons messages, it has missing membership auth to fetch which caused us to mistakenly replace the `context` for the message with that of the floating membership `outlier` event. Since `outliers` have no `state` or `state_group`, the error bubbles up when we continue down the persisting route: `sqlite3.IntegrityError: NOT NULL constraint failed: event_to_state_groups.state_group`

Call stack:

```
backfill
_auth_and_persist_event
_check_event_auth
_update_auth_events_and_context_for_auth
```

Implementation of matrix-org/matrix-doc#2285

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Fix no-access-token bug in deactivation tests
* Support MSC2033: Device ID on whoami
* Test for appservices too

MSC: https://github.com/matrix-org/matrix-doc/pull/2033

The MSC has passed FCP, which means stable endpoints can be used.

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>