We don't necessarily want to put the data in the cwd.

It's too confusing.

This will enable us to skip the unintuitive behaviour where the generated
config and default config are the same thing.

Allow configuration of the path used for ACME account keys.

This is no longer used and only serves to confuse.

Because sticking it in the same place as the config isn't necessarily the right
thing to do.

* Pull config_dir_path and data_dir_path calculation out of read_config_files

* Pass config_dir_path and data_dir_path into read_config

This makes some of the conditional-import hoop-jumping easier.

Use monotonic clock where possible for metrics

This has no useful purpose on python3, and is generally a source of confusion.

* group the arguments together into a group
* add new names "--generate-missing-config" and "--config-directory" for
  existing cmdline options "--generate-keys" and "--keys-dir", which better
  reflect their purposes.

Add some comments, and simplify `read_config_files`.

Make it a bit clearer what's going on.

Fixes https://github.com/matrix-org/synapse/issues/5431

`jinja2` was being imported even when it wasn't strictly necessary. This made it required to run Synapse, even if the functionality that required it wasn't enabled. This was causing new Synapse installations to crash on startup.

Email modules are now required.

There is a README.txt which always sets off this warning, which is a bit
alarming when you first start synapse. I don't think we need to warn about
this.

Fix /messages on worker when no token supplied

Support pagination API in client_reader worker

E_TOO_MANY_NEGATIVES

Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Add metrics for length of new extremities persisted.

Fixes intermittent errors observed on Apple hardware which were caused by
time.clock() appearing to go backwards when called from different threads.

Also fixes a bug where database activity times were logged as 1/1000 of their
correct ratio due to confusion between milliseconds and seconds.

Handle the case of `get_missing_events` failing

Add experimental option to reduce extremities.

Include xmlsec in the docker image

Fix typo in deactivation background job

Improve advice regarding poor performance

I had to add quite a lot of logging to diagnose a problem with 3pid
invites - we only logged the one failure which isn't all that
informative.

NB. I'm not convinced the logic of this loop is right: I think it
should just accept a single valid signature from a trusted source
rather than fail if *any* signature is invalid. Also it should
probably not skip the rest of middle loop if a check fails? However,
I'm deliberately not changing the logic here.