Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.

We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).

Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.

Fixes #3962

Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.

Redo of https://github.com/matrix-org/synapse/pull/4840

`__str__` depended on `self.addr`, which was absent from
ClientReplicationStreamProtocol, so attempting to call str on such an object
would raise an exception.

We can calculate the peer addr from the transport, so there is no need for addr
anyway.

Make sure that they are sent correctly over the replication stream.

Fixes: #4898

I don't have a database with the same name as my user, so leaving the database
name unset fails.

While we're at it, clear out some unused stuff in the test setup.

Fix yaml warnings by using safe_load

As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.

Deny peeking into rooms that have been blocked

Co-Authored-By: erikjohnston <erikj@jki.re>

This one should close #4841. Many thanks to @dev4223 for bringing it up and finding a solution.

Signed-off-by: Colin White

Fixup shutdown room API

Add option to disable search room lists

Rather than stubbing out the access_log, make it actually log the requests,
which makes it a lot more obvious what is going on during tests.

There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.