This is the final piece of the jigsaw for #9595. As with other changes before this one (eg #10771), we need to make sure that we auth the auth events in the right order, and actually check that their predecessors haven't been rejected.

To do this I've reused the existing code we use when persisting outliers elsewhere.

I've removed the code for attempting to fetch missing auth_events - the events should have been present in the send_join response, so the likely reason they are missing is that we couldn't verify them, so requesting them again is unlikely to help. Instead, we simply drop any state which relies on those auth events, as we do at a backwards-extremity. See also matrix-org/complement#216 for a test for this.

`synapse.config.__main__` has the possibility to read a config item. This can be used to conveniently also validate the config is valid before trying to start Synapse.

 The "read" command broke in https://github.com/matrix-org/synapse/pull/10916

 as it now requires passing in "server.server_name" for example.

 Also made the read command optional so one can just call this with just the confirm file reference and get a "Config parses OK" if things are ok.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

Adds experimental support for MSC3440's `io.element.thread` relation
type (and the aggregation for it).

* We only need to fetch users in private rooms

* Filter out `user_id` at the top

* Discard excluded users in the top loop

We weren't doing this in the "First, if they're our user" branch so this
is a bugfix.

* The caller must check that `user_id` is included

This is in the docstring. There are two call sites:
- one in `_handle_room_publicity_change`, which explicitly checks before calling;
- and another in `_handle_room_membership_event`, which returns early if
  the user is excluded.

So this change is safe.

* Test joining a private room with an excluded user

* Tweak an existing test

* Changelog

* test docstring

* lint

And require type hints for this module.

relative links don't work when it's on dockerhub.

Fix setting a user's external_id via the admin API returns 500 and deletes users existing external mappings if that external ID is already mapped (#11051)

Fixes #10846

It's been possible to configure a key inline in the homeserver.yaml since
13bc1e07. Update `sign_json` to work with this.

Signed-off-by: Robert Edström <github@legogris.se>

Remove some redundant code, and generally simplify.

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Mention callbacks introduced in v1.37.0

According to the documentation introduced in https://github.com/matrix-org/synapse/pull/10062

* Mention callbacks introduced in v1.39.0

According to https://github.com/matrix-org/synapse/pull/10386 and https://github.com/matrix-org/synapse/pull/9884

* Mention callbacks introduced in v1.42.0

According to https://github.com/matrix-org/synapse/pull/10524

* Mention callbacks introduced in v1.44.0 and v1.45.0

As per https://github.com/matrix-org/synapse/pull/10898, https://github.com/matrix-org/synapse/pull/10910 and https://github.com/matrix-org/synapse/pull/10894

* Mention callbacks introduced in v1.46.0

According to https://github.com/matrix-org/synapse/pull/10548

Temporarily revert "Add functionality to remove deactivated users from the monthly_active_users table (#10947)".

This reverts commit eda8c88b.

Updates the event rows returned from the database to be
attrs classes instead of dictionaries.

Signed-off-by: Dan Callahan <danc@element.io>

This is just a lift-and-shift, because it fits more naturally here. We do
rename it to `process_remote_join` at the same time though.

... to `_auth_and_persist_outliers`, since that reflects its purpose better.

If we find ourselves dealing with rejected events, we proably want to know
about it. Let's include it in the stringification of the event so that it gets
logged.

* Convert UserPresenceState to attrs.
* Remove args/kwargs from error classes and explicitly pass msg/errorcode.

Currently, when we receive an event whose auth_events differ from those we expect, we state-resolve between the two state sets, and check that the event passes auth based on the resolved state.

This means that it's possible for us to accept events which don't pass auth at their declared auth_events (or where the auth events themselves were rejected), leading to problems down the line like #10083.

This change means we will:

 * ignore any events where we cannot find the auth events
 * reject any events whose auth events were rejected
 * reject any events which do not pass auth at their declared auth_events.

Together with a whole raft of previous work, this is a partial fix to #9595.

Fixes #6643.

Based on #11009.

This fixes a bug where we would accept an event whose `auth_events` include
rejected events, if the rejected event was shadowed by another `auth_event`
with same `(type, state_key)`.

The approach is to pass a list of auth events into
`check_auth_rules_for_event` instead of a dict, which of course means updating
the call sites.

This is an extension of #10956.

Document Synapse's behaviour when multiple modules register the same
callback/web resource/etc.

Co-authored-by: reivilibre <oliverw@matrix.org>

Make sure that we correctly handle rooms where we do not yet have a
`min_depth`, and also add some comments and logging.

Fix broken export-data admin command and add a test for it to CI 

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>