Turns out these scripts weren't getting linted.

I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+.

Apparently on tox 2.5, `usedevelop` overrides `skip_install`, so we end up
trying to install the full dependencies even for the `-old` environment.

This also pins the Twisted version in the mypy job for CI until
proper type hints are fixed throughout Synapse.

Co-authored-by: Dan Callahan <danc@element.io>

pip 21.0 stopped supporting Python 3.5.

Co-authored-by: Dan Callahan <danc@element.io>

pip 21.0 stopped supporting Python 3.5.

As we use `execute_values` with the `fetch` parameter.

The lists of source directories to lint between `tox.ini` and `lint.sh` became out of sync. This PR tightens them up and adds some comments reminding any future readers to keep the list in sync.

setuptools 51.0.0 dropped support for Python 3.5.

We were running some linters on some files and some on others. Extract a common
setting and use it everywhere.

... for running the tests with no optional deps.

... and disable coverage tracking for mypy and friends.

The test runner isn't present in the `[all]` set of extras, so the
previous instructions did not work without also installing `[test]`.

Note that this does not include the `[lint]` extras, since those do not
install on all supported Python versions (specifically, isort 5.x
requires Python 3.6, while we still support 3.5). Instructions for that
are included in our pull request template, so we should be fine there.

I've also dropped the `--no-use-pep517` arg to `pip install` since it
seems to have been added to address a temporary regression in pip 19.1
which was fixed in pip 19.1.1 the following month.

Lastly, updated the example output of the test suite to set more
realistic expectations around run time.

Signed-off-by: Dan Callahan <danc@element.io>

As expected, all tests pass locally without modification.

Signed-off-by: Dan Callahan <danc@element.io>

moves non-runtime dependencies out of synapse.python_dependencies (test and lint)

Lint dependencies can now be installed with pip install -e ".[lint]"

This should help keep the version in sync between tox and documentation.

Duplicating function signatures between server.py and server.pyi is
silly. This commit changes that by changing all `build_*` methods to
`get_*` methods and changing the `_make_dependency_method` to work work
as a descriptor that caches the produced value.

There are some changes in other files that were made to fix the typing
in server.py.

Run `isort`, `flake8` and `black` over the `contrib/` directory and `synctl` script. The latter was already being done in CI, but now the linting script does it too.

Fixes https://github.com/matrix-org/synapse/issues/7910

The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.

fix a few things to make this pass mypy.

- Remove the requirement for a specific version of Python
- Move dep comment to a separate line, Tox 3.7.0 like trailing ones

Signed-off-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>

* Expose `return_html_error`, and allow it to take a Jinja2 template instead of a raw string

* Clean up exception handling in SAML2ResponseResource

  * use the existing code in `return_html_error` instead of re-implementing it
    (giving it a jinja2 template rather than inventing a new form of template)

  * do the exception-catching in the REST layer rather than in the handler
    layer, to make sure we catch all exceptions.