Add comments to the Prometheus recording rules to make it clear which set of rules you need for Grafana or Prometheus Console. (#13876)

* Add worker_main_http_uri, replace >> with >

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>

Remove unused Prometheus recording rules from `synapse-v2.rules` and add comments describing where the rest are used. (#13756)

* Add missing graph to contrib

* Update with minor but plausible changes, including positioning changes

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

Add another bash script to the contrib directory. It creates multiple stream writers and also prints out the example configuration for homeserver.yaml.

Signed-off-by: Ville Petteri Huh.

Signed-off-by: Ville Petteri Huh.

* Clarifies comments and documentation.
* Adds type-hints.
* Fixes Python 3 compatibility (and runs pyupgrade).
* Updates for changes in Synapse internals.

* Move the `snapcraft` configuration to `contrib`.

We're happy for people to package this as a snap image if it's useful,
but we don't support or maintain it. I'd like to move the config to
`contrib` to reflect this state of affairs.

* Changelog

The existing implementation of the `python_twisted_reactor_tick_time` metric is pretty useless, because it *only* 
measures the time taken to execute timed calls and callbacks from threads. That neglects everything that 
happens off the back of I/O, which is obviously quite a lot for us.

To improve this, I've hooked into a different place in the reactor - in particular, where it calls `epoll`. That call is 
the only place it should wait for something to happen - the rest of the loop *should* be quick.

I've also removed `python_twisted_reactor_pending_calls`, because I don't believe anyone ever looks at it, and
it's a nuisance to populate.

SC2089: Quotes/backslashes will be treated literally. Use an array.

https://github.com/koalaman/shellcheck/wiki/SC2089

SC2090: Quotes/backslashes in this variable will not be respected.

https://github.com/koalaman/shellcheck/wiki/SC2090



Putting literal JSON in a variable mistakenly triggers these warnings.
Instead of adding ignore directives, this can be avoided by inlining the
JSON data into the curl invocation.

Since the variable is only used in this one location, inlining is fine.

Signed-off-by: Dan Callahan <danc@element.io>

A follow-up to #10829

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>

Reformat all files with the new version.

Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>

Signed-off-by: Savyasachee Jha <savya.jha@hawkradius.com>

This reverts commit 05e8c70c.

This basically speeds up federation by "squeezing" each individual dual database call (to destinations and destination_rooms), which previously happened per every event, into one call for an entire batch (100 max).

Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>

Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl&gt;`>

Part of #9366

Adds in fixes for B006 and B008, both relating to mutable parameter lint errors.

Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>

On NixOS, `bash` isn't under `/bin/bash` but rather in some directory in `$PATH`. Locally, I've been patching those scripts to make them work.

`/usr/bin/env` seems to be the only [portable way](https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-bin-env-name-instead-of-path-to-name-as-my

) to use binaries from the PATH as interpreters.

Signed-off-by: Quentin Gliech <quentingliech@gmail.com>

 - Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version

The official dashboard uses data from these rules, but they were never added to the synapse-v2.rules. They are mentioned in this issue: https://github.com/matrix-org/synapse/issues/7917#issuecomment-661330409

, but never got added to the rules.

Adding them results in all graphs in the "Event persist rate" section to function as intended.

Signed-off-by: Johanna Dorothea Reichmann <transcaffeine@finallycoffee.eu>

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>