Remove redundant return value from _calculate_state_delta

Factor out common code for search insert

we already have the state from _get_new_state_after_events, so returning it
from _calculate_state_delta is just confusing.

we can reuse the same code as is used for event insert, for doing the
background index population.

... as a precursor to making event storing and doing the bg update share some
code.

montoring metrics for number of cache evictions

Ensure media is in local cache before thumbnailing

Handle url_previews with no content-type

Factor out resolve_state_groups to a separate handler

factor _get_new_state_after_events out of _calculate_state_delta

Docstring fixes

Add an admin route to get all the media in a room

avoid failing with an exception if the remote server doesn't give us a
Content-Type header.

Also, clean up the exception handling a bit.

Fix a couple of errors in docstrings

The return type was a complete lie, so fix it

We extract the storage-independent bits of the state group resolution out to a
separate functiom, and stick it in a new handler, in preparation for its use
from the storage layer.

(to make way for a method that actually just does the state group resolution)

Fix sql error in quarantine_media

Signed-off-by: Travis Ralston <travpc@gmail.com>

This reduces the scope of a bunch of variables

Remove spurious log argument

Remove unused "event_type" param on state.get_current_state_ids

Improve exception handling in persist_event

Logging and metrics for the http pusher

Use a connection pool for the SimpleHttpClient

Signed-off-by: Travis Ralston <travpc@gmail.com>

Signed-off-by: Travis Ralston <travpc@gmail.com>

... which would cause scary-looking and unhelpful errors in the log on dns fail

this param doesn't seem to be used, and is a bit pointless anyway because it
can easily be replicated by the caller. It is also horrible, because it changes
the return type of the method.

1. use `deferred.errback()` instead of `deferred.errback(e)`, which means that
a Failure object will be constructed using the current exception state,
*including* its stack trace - so the stack trace is saved in the Failure,
leading to better exception reports.

2. Set `consumeErrors=True` on the ObservableDeferred, because we know that
there will always be at least one observer - which avoids a spurious "CRITICAL:
unhandled exception in Deferred" error in the logs