Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).

By replacing duplicated code with parameterized tests and
avoiding unnecessary dumping of JSON data.

Merged before approval; these comments from @clokep on that PR.

* Validate device_keys for C-S /keys/query requests

Closes #10354

A small, not particularly critical fix. I'm interested in seeing if we
can find a more systematic approach though. #8445 is the place for any discussion.

Here we split on_receive_pdu into two functions (on_receive_pdu and process_pulled_event), rather than having both cases in the same method. There's a tiny bit of overlap, but not that much.

This is a follow-up to #10615: it takes the code that constructs the state at a backwards extremity, and extracts it to a separate method.

A user will still see this room if it is in a local cache, but it will
not reappear if clearing the cache and reloading.

Follow-up to #10629 which set it to true, not false.

Instead of using namedtuples. This helps with asserting type hints
and code completion.

Co-authored-by: Patrick Cloke <patrickc@matrix.org>

* drop room pdu linearizer sooner

No point holding onto it while we recheck the db

* move out `missing_prevs` calculation

we're going to need `missing_prevs` whatever we do, so we may as well calculate
it eagerly and just update it if it gets outdated.

* Add another `if missing_prevs` condition

this should be a no-op, since all the code inside the block already checks `if
missing_prevs`

* reorder if conditions

This shouldn't change the logic at all.

* Push down `min_depth` read

No point reading it from the database unless we're going to use it.

* Collect the sent_to_us_directly code together

Move the remaining `sent_to_us_directly` code inside the `if
sent_to_us_directly` block.

* Properly separate the `not sent_to_us_directly` branch

Since the only way this second block is now reachable is if we
*didn't* go into the `sent_to_us_directly` branch, we can replace it with a
simple `else`.

* changelog

By default the calls only ran on the worker configured to run background
tasks.

Fixes https://github.com/matrix-org/synapse/issues/7867

Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Several configuration sections are using separate settings for custom template directories, which can be confusing. This PR adds a new top-level configuration for a custom template directory which is then used for every module. The only exception is the consent templates, since the consent template directory require a specific hierarchy, so it's probably better that it stays separate from everything else.

If the new /hierarchy API does not exist on all destinations,
fallback to querying the /spaces API and translating the results.

This is a backwards compatibility hack since not all of the
federated homeservers will update at the same time.

Marking things as outliers to inhibit pushes is a sledgehammer to crack a
nut. Move the test further down the stack so that we just inhibit the thing we
want.

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

Allow using several directories in read_templates.

This adds a new API GET /_synapse/admin/v1/username_available?username=foo to check if a username is available. It is the counterpart to https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available, except that it works even if registration is disabled.

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>