Add a config option to change whether unread push notification counts are per-message or per-room (#8820)

This PR adds a new config option to the `push` section of the homeserver config, `group_unread_count_by_room`. By default Synapse will group push notifications by room (so if you have 1000 unread messages, if they lie in 55 rooms, you'll see an unread count on your phone of 55).

However, it is also useful to be able to send out the true count of unread messages if desired. If `group_unread_count_by_room` is set to `false`, then with the above example, one would see an unread count of 1000 (email anyone?).

These are now only available via `/_synapse/admin/v1`.

This PR updates the push config's formatting to better align with our [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).

Abstracts the SAML and OpenID Connect code which attempts to regenerate
the localpart of a matrix ID if it is already in use.

Clarify that the list media API only shows media from unencrypted events.

Some hopefully-useful notes on setting up a turnserver.

Checks that the localpart returned by mapping providers for SAML and
OIDC are valid before registering new users.

Extends the OIDC tests for existing users and invalid data.

If the SAML metadata includes multiple IdPs it is necessary to
specify which IdP to redirect users to for authentication.

Updates the sample configuration with the pysaml2 configuration for
accepting clock skew/drift between the homeserver and IdP.

Signed-off-by: Chagai Friedlander <chagai95@gmail.com>

Signed-off-by: Adrian Wannenmacher <tfld@tfld.dev>

Related to #8714. `event_reports.rst` was introduced in Synapse 1.21.0.

add some mdui:UIInfo element examples for saml2_config in homeserver.yaml

If SSO login is used (e.g. SAML) in a multi worker setup, it should be mentioned that currently all SAML logins must run on the same worker, see https://github.com/matrix-org/synapse/issues/7530

Also, if you are using different ports (for example 443 and 8448) in a reverse proxy for client and federation, the path `/_matrix/media` on the client and federation port must point to the listener of the `media_repository` worker, otherwise you'll get a 404 on the federation port for the path `/_matrix/media`, if a remote server is trying to get the media object on federation port, see https://github.com/matrix-org/synapse/issues/8695

Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users.
Related to #6094
It is the first API for statistics.
Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats

)

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`

Fixes #8550

This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.

This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.

Add an admin API `GET /_synapse/admin/v1/users/<user_id>/pushers` like https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers

Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.

Split admin API for reported events in detail und list view.
API was introduced with #8217 in synapse v.1.21.0.

It makes the list (`GET /_synapse/admin/v1/event_reports`) less complex and provides a better overview.
The details can be queried with: `GET /_synapse/admin/v1/event_reports/<report_id>`.
It is similar to room and users API.

It is a kind of regression in `GET /_synapse/admin/v1/event_reports`.  `event_json` was removed. But the api was introduced one version before and it is an admin API (not under spec).

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

Signed-off-by: Peter Krantz <peter.krantz@gmail.com>

Related to: #6459, #3479

Add `DELETE /_synapse/admin/v1/media/<server_name>/<media_id>` to delete
a single file from server.

Signed-off-by: Christopher May-Townsend <chris@maytownsend.co.uk>

* Update documentation on retention policies limits

Document the changes from https://github.com/matrix-org/synapse/pull/8104

Currently background proccesses stream the events stream use the "minimum persisted position" (i.e. `get_current_token()`) rather than the vector clock style tokens. This is broadly fine as it doesn't matter if the background processes lag a small amount. However, in extreme cases (i.e. SyTests) where we only write to one event persister the background processes will never make progress.

This PR changes it so that the `MultiWriterIDGenerator` keeps the current position of a given instance as up to date as possible (i.e using the latest token it sees if its not in the process of persisting anything), and then periodically announces that over replication. This then allows the "minimum persisted position" to advance, albeit with a small lag.