The major change is moving the decision of whether to use oEmbed
further up the call-stack. This reverts the _download_url method to
being a "dumb" functionwhich takes a single URL and downloads it
(as it was before #7920).

This also makes more minor refactorings:

* Renames internal variables for clarity.
* Factors out shared code between the HTML and rich oEmbed
  previews.
* Fixes tests to preview an oEmbed image.

It's a simplification, but one that'll help make the user directory logic easier
to follow with the other changes upcoming. It's not strictly required for those
changes, but this will help simplify the resulting logic that listens for
`m.room.member` events and generally make the logic easier to follow.

This means the config option `search_all_users` ends up controlling the
search query only, and not the data we store. The cost of doing so is an
extra row in the `user_directory` and `user_directory_search` tables for
each local user which

- belongs to no public rooms
- belongs to no private rooms of size ≥ 2

I think the cost of this will be marginal (since they'll already have entries
 in `users` and `profiles` anyway).

As a small upside, a homeserver whose directory was built with this
change can toggle `search_all_users` without having to rebuild their
directory.

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

No functional changes here. This came out as I was working to tackle #5677 

* Split up the documentation in several files rather than one huge one
* Add examples for each callback category
* Other niceties like fixing https://github.com/matrix-org/synapse/issues/10632
* Add titles to callbacks so they're easier to find in the navigation panels and link to

Also refactors some of the registration of endpoints on workers.

This adds configuration options (under an `oembed` section) to
configure which URLs are matched to use oEmbed for URL
previews.

Point to the book where possible, and use hyperlinks to github to refer to files not included in the book.

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

- Removed page summaries from CONTRIBUTING and installation pages as 
this information was already in the table of contents on the right hand side
- Fixed some broken links in CONTRIBUTING
- Added margin-right tag for when table of contents is being shown
(otherwise the text in the page sometimes overlaps with it)

* Update reverse_proxy.md

* Create 10708.doc

When a user deletes an email from their account it will
now also remove all pushers for that email and that user
(even if these pushers were created by a different client)

Signed-off-by: Hugo Delval <hugo.delval@gmail.com>

* Fix the titles in the OIDC documentation

Having them as links broke the table-of-contents rendering in mdbook.
Plus there's no reason for only some of the provider titles to be links.

* Changelog

* Add link to google idp docs

Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).

Co-authored-by: Patrick Cloke <patrickc@matrix.org>

Fixes https://github.com/matrix-org/synapse/issues/7867

Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0

Several configuration sections are using separate settings for custom template directories, which can be confusing. This PR adds a new top-level configuration for a custom template directory which is then used for every module. The only exception is the consent templates, since the consent template directory require a specific hierarchy, so it's probably better that it stays separate from everything else.

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

This adds a new API GET /_synapse/admin/v1/username_available?username=foo to check if a username is available. It is the counterpart to https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available, except that it works even if registration is disabled.

Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

Signed-off-by: Drew Short <warrick@sothr.com>

This is mostly useful in case the server offers TLS, but doesn't present a valid certificate.

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>