Also tighten validation of server names by forbidding invalid characters
in IPv6 addresses and empty domain labels.

Co-authored-by: reivilibre <oliverw@matrix.org>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

When fetching remote devices for the first time, we did not correctly include the cross signing keys in the returned results.

c.f. #11159

Co-authored-by: reivilibre <olivier@librepush.net>

`get_insertion_event_by_batch_id` -> `get_insertion_event_id_by_batch_id`

Split out from https://github.com/matrix-org/synapse/pull/11114

Fixes #11259

We might as well use a default value for `public_baseurl` based on
`server_name` - in many cases, it will be correct.

Signed-off-by: Dan Callahan <danc@element.io>

* Make lock better handle process being killed

If the process gets killed and restarted (so that it didn't have a
chance to drop its locks gracefully) then there may still be locks in
the DB that are for the same instance that haven't yet timed out but are
safe to delete.

We handle this case by a) checking if the current instance already has
taken out the lock, and b) if not then ignoring locks that are for the
same instance.

* Periodically check for old staged events

This is to protect against other instances dying and their locks timing
out.

* Remove unused Vagrant scripts

* Change package Architecture to any

* Preinstall the wheel package when building venvs.

Addresses the following warnings during Debian builds:

    Using legacy 'setup.py install' for jaeger-client, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for matrix-synapse-ldap3, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for opentracing, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for systemd-python, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for pympler, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for threadloop, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for thrift, since package 'wheel' is not installed.

* Allow /etc/default/matrix-synapse to be missing

Per the systemd.exec manpage, prefixing an EnvironmentFile with "-":

> indicates that if the file does not exist, it will not be read and no
> error or warning message is logged.

Signed-off-by: Dan Callahan <danc@element.io>

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Julian Vanden Broeck <julian.vandenbroeck@dalibo.com>

Fixes #11252

I was trying to understand how `cachedList` works, and ended up writing this
extra test. I figure we may as well keep it.

When an event fetcher aborts due to an exception, `_event_fetch_ongoing`
must be decremented, otherwise the event fetcher would never be
replaced. If enough event fetchers were to fail, no more events would be
fetched and requests would get stuck waiting for events.

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

This makes the typing stream writer config match the other stream writers
that only currently support a single worker.

We don't pin them as we execute them as commands, rather than use them
as libs.