make it absolutely clear that Purge History API does not remove all traces of events and message contents

because this topic pops up too often

#890 #1621 #1730 #2260 #2315 and so on

Refactor to speed up incremental syncs

Clarify recommended network setup

Fix iteration of requests_missing_keys; list doesn't have .values()

preserve_context_over_function doesn't do what you want it to do.

Remove non-existing files, add stop, use synctl

Starting with synctl lead to "no config file found"
Stopping also leads to some (code=exited, status=1/FAILURE), but at least now we can stop the service.

Non-existing files, when running the suggested from https://github.com/matrix-org/synapse#configuring-synapse
/etc/synapse/log_config.yaml so the --log-config leads to an error
/etc/sysconfig/synapse The environment-file or even the /etc/sysconfig does not exist in arch linux

Also instead of calling python2 we use synctl, as this seems to be the proper way to start it, and it gives us a more useful error in the systemctl status. And we now allow stop (and therefore restart).

Support SRV records which point at AAAA records, as well as A records.

Fixes https://github.com/matrix-org/synapse/issues/2405

Clean up Keyring code

preserve_context_over_fn is essentially broken, because (a) it pointlessly
drops the current logcontext before calling its wrapped function, which means
we don't get any useful logcontexts for _handle_key_deferred; (b) it wraps the
resulting deferred in a _PreservingContextDeferred, which is very dangerous
because you then can't yield on it without leaking context back into the
reactor.

Instead, let's specify that the resultant deferreds call their callbacks with
no logcontext.

... which means that logcontexts can be correctly preserved for the stuff it
does.

get_server_verify_keys is now called with the logcontext, so needs to
preserve_fn when it fires off its nested inlineCallbacks function.

Also renames get_server_verify_keys to reflect the fact it's meant to be
private.

If the verify_request.deferred has already completed, then `remove_deferreds`
will be called immediately. It therefore might resolve the server_to_deferred
deferred while there are still other requests for that server in flight.

To avoid that, we should build the complete list of requests, and *then* add the
callbacks.

Define that it is run with no log context, and make sure that happens.

If we aren't careful to reset the logcontext, we can't bung the deferreds into
defer.gatherResults etc. We don't actually do that directly, but we *do*
resolve other deferreds from affected callbacks (notably the server_to_deferred
map in _start_key_lookups), and those *do* get passed into
defer.gatherResults. It turns out that this way ends up being least confusing.

... to make it easier to see what's going on.

This is a precursor to factoring some of this code out.

There's no need for this to be a nested definition; pulling it out not only
makes it more efficient, but makes it easier to check that it's not accessing
any local variables it shouldn't be.

This might make the cache slightly more efficient.

preserve_fn is a no-op unless the wrapped function returns a
Deferred. verify_json_objects_for_server returns a list, so this is doing
nothing.

Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.