The code to deduplicate repeated fetches of the same set of events was
N^2 (over the number of events requested), which could lead to a process
being completely wedged.

The main fix is to deduplicate the returned deferreds so we only await
on a deferred once rather than many times. Seperately, when handling the
returned events from the defrered we only add the events we care about
to the event map to be returned (so that we don't pay the price of
inserting extraneous events into the dict).

The idea here is to take anything to do with incoming events and move it out to a separate handler, as a way of making FederationHandler smaller.

Given that backfill and get_missing_events are basically the same thing, it's somewhat crazy that we have entirely separate code paths for them. This makes backfill use the existing get_missing_events code, and then clears up all the unused code.

Signed-off-by: Aaron Raimist <aaron@raim.ist>

When a user deletes an email from their account it will
now also remove all pushers for that email and that user
(even if these pushers were created by a different client)

Applies the changes from #10665 to additional modules.

I found this easy to miss (and evidently, it looks like it was missed for schema version 62).

We now always rebuild the matrixdotorg/synapse image, then
build the matrixdotorg/synapse-workers image on top of it.

Fixes #10318

Fix a bug where the prometheus metrics for SSO logins wouldn't be initialised
until the first user logged in with a given auth provider.

Signed-off-by: Hugo Delval <hugo.delval@gmail.com>

To match the maximum lengths allowed for profile data.

* Fix the titles in the OIDC documentation

Having them as links broke the table-of-contents rendering in mdbook.
Plus there's no reason for only some of the provider titles to be links.

* Changelog

* Add link to google idp docs

Setting `update_existing: true` in the `create-an-issue` GitHub Action
will avoid opening duplicate issues if an open issue already exists with
an identical title.

If no open issues match the title, then a new issue will be created.

This helps avoid spamming our issue tracker should there be a failure
when testing against Twisted's trunk.

This PR also pins the SHA of the `create-an-issue` action to mitigate
the risk of a malicious actor gaining access to JasonEtco's account.

See GitHub's page on security hardening third party actions for more:
https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions



Signed-off-by: Dan Callahan <danc@element.io>

Do not include stack traces for known exceptions when trying multiple federation destinations. (#10662)

This creates a GHA workflow which runs at 8am every day, and runs mypy, trial and sytest against Twisted's current trunk. If any of the jobs fail, it opens an issue.

Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).

By replacing duplicated code with parameterized tests and
avoiding unnecessary dumping of JSON data.

Merged before approval; these comments from @clokep on that PR.

* Validate device_keys for C-S /keys/query requests

Closes #10354

A small, not particularly critical fix. I'm interested in seeing if we
can find a more systematic approach though. #8445 is the place for any discussion.

Here we split on_receive_pdu into two functions (on_receive_pdu and process_pulled_event), rather than having both cases in the same method. There's a tiny bit of overlap, but not that much.

This is a follow-up to #10615: it takes the code that constructs the state at a backwards extremity, and extracts it to a separate method.

A user will still see this room if it is in a local cache, but it will
not reappear if clearing the cache and reloading.

Ubuntu 20.10 was not an LTS release

Signed-off-by: John-Scott Atlakson <24574+jsma@users.noreply.github.com>

Follow-up to #10629 which set it to true, not false.