It seems most of these blacklisted tests do actually pass most of the time.

I'm of the opinion that having them blacklisted here means there is very little incentive for us to deflake any flaky tests, and meanwhile any value in those tests is completely lost.

This commit was originally merged in #7228 but reverted in #7254 as the
associated sytest was not ready yet. Now that it is, we can merge this
again.

This reverts commit 59f0ca8b.

I'm not really sure how this was going wrong, but this seems like the
right approach anyway.

Workaround for jaraco/zipp#40

These now exist in the pipelines repo.

Remove tests that got blacklisted while torturing was enabled, and add one that
fails.

* Don't use a virtualenv

* Generate the server's signing key to allow it to start

* Add signing key paths to CI configuration files

* Use a Python script to create the postgresql database

* Improve logging

This adds:

* a test sqlite database
* a configuration file for the sqlite database
* a configuration file for a postgresql database (using the credentials in `.buildkite/docker-compose.pyXX.pgXX.yaml`)

as well as a new script named `.buildkite/scripts/test_synapse_port_db.sh` that:

1. installs Synapse
2. updates the test sqlite database to the latest schema and runs background updates on it
3. creates an empty postgresql database
4. run the `synapse_port_db` script to migrate the test sqlite database to the empty postgresql database (with coverage)

Step `2` is done via a new script located at `scripts-dev/update_database`.

The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.

* Delete format_tap.py

This python implementation of a tap formatting library for buildkite has been
replaced with a perl implementation as part of the matrix-org/sytest repo,
which is specific to sytest's language, not that of any one homeserver's.

This also adds a worker blacklist.

* Add a sytest blacklist file

* Add changelog

* Add blacklist to manifest

Including workers!

* remove 2.7 from CI and publishing

* fill out classifiers and also make it not be installed on 3.5

* some minor bumps so that the old deps work on python 3.5

Sometimes the build agents get lost or die (error codes -1 and 2). Retry automatically a maximum of 2 times if this happens.

Error code reference:

* -1: Agent was lost
* 0: Build successful
* 1: There was an error in your code
* 2: The build stopped abruptly
* 255: The build was cancelled

* Regen sample config before kicking off agents

* Add changelog