* add tests for checking if room search works with non-ascii char

* change encoding on parse_string to UTF-8

* lints

* properly encode search term

* lints

* add changelog file

* update changelog number

* set changelog entry filetype to .bugfix

* Revert "set changelog entry filetype to .bugfix"

This reverts commit be8e5a314251438ec4ec7dbc59ba32162c93e550.

* update changelog message and file type

* change parse_string default encoding back to ascii and update room search admin api calll to parse string

* refactor tests

* Update tests/rest/admin/test_room.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Rename `/batch_send` query parameter from `?prev_event` to more obvious usage with `?prev_event_id` (MSC2716) (#10839)

As mentioned in https://github.com/matrix-org/matrix-doc/pull/2716#discussion_r705872887
and https://github.com/matrix-org/synapse/issues/10737

It's a simplification, but one that'll help make the user directory logic easier
to follow with the other changes upcoming. It's not strictly required for those
changes, but this will help simplify the resulting logic that listens for
`m.room.member` events and generally make the logic easier to follow.

This means the config option `search_all_users` ends up controlling the
search query only, and not the data we store. The cost of doing so is an
extra row in the `user_directory` and `user_directory_search` tables for
each local user which

- belongs to no public rooms
- belongs to no private rooms of size ≥ 2

I think the cost of this will be marginal (since they'll already have entries
 in `users` and `profiles` anyway).

As a small upside, a homeserver whose directory was built with this
change can toggle `search_all_users` without having to rebuild their
directory.

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Allow membership event to unban user

Signed-off-by: Aaron Raimist <aaron@raim.ist>

This should ensure GHA runs synapse against the same-named sytest branch

Adds missing type hints to methods in the synapse.handlers
module and requires all methods to have type hints there.

This also removes the unused construct_auth_difference method
from the FederationHandler.

To avoid duplicating it between a few tests.

We added a bunch of spans in #10704, but this ended up adding a lot of
redundant spans for rooms where nothing changed, so instead we only
start the span if there might be something interesting going on.

Including admin, consent, key, synapse, and media. All REST servlets
(the synapse.rest module) now require typed method definitions.

Also enables Mypy for related tests.

In `MatrixFederationHttpClient._send_request()`, we make a HTTP request
using an `Agent`, wrap that request in a timeout and await the resulting
`Deferred`. On its own, the `Agent` performing the HTTP request
correctly stashes and restores the logging context while waiting.
The addition of the timeout introduces a path where the logging context
is not restored when execution resumes.

To address this, we wrap the timeout `Deferred` in a
`make_deferred_yieldable()` to stash the logging context and restore it
on completion of the `await`. However this is not sufficient, since by
the time we construct the timeout `Deferred`, the `Agent` has already
stashed and cleared the logging context when using
`make_deferred_yieldable()` to produce its `Deferred` for the request.

Hence, we wrap the `Agent` request in a `run_in_background()` to "fork"
and preserve the logging context so that we can stash and restore it
when `await`ing the timeout `Deferred`.

This approach is similar to the one used with `defer.gatherResults`.

Note that the code is still not fully correct. When a timeout occurs,
the request remains running in the background (existing behavior which
is nothing to do with the new call to `run_in_background`) and may
re-start the logging context after it has finished.

This is mostly an internal change, but improves type hints in the
media code.

I had one of these error messages yesterday and assumed it was an
invalid auth token (because that was an HTTP query parameter in the
test) I was working on. In fact, it was an invalid next batch token for
syncing.

Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).

This fixes mypy errors with jaeger-client >= 4.7.0 and should be a no-op
for versions before that.

By copying the `room_type` field of the create event and
migrating any non-empty `m.space.child` events to the
new room that is created.

No functional changes here. This came out as I was working to tackle #5677 

We want to blacklist only known, consistent failures. We should deflake tests rather than ignoring them.

Fixes #10732: consent flow skipped during SSO user registration if username is left at default

Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>

We've already batched up the events previously, and assume in other
places in the events.py file that we have. Removing this makes it easier
to adjust the batch sizes in one place.

Signed-off-by: Sean Quah <seanq@element.io>

Hint to clients via the room capabilities API (MSC3244) that
room version 9 should be preferred for creating a room with
restricted join rules (instead of room version 8).

This is only called in two places, and the code seems much clearer without it.

* Split up the documentation in several files rather than one huge one
* Add examples for each callback category
* Other niceties like fixing https://github.com/matrix-org/synapse/issues/10632
* Add titles to callbacks so they're easier to find in the navigation panels and link to

When releasing 1.42.0 with @Azrenbeth and talking with @clokep yesterday I realised doing the dch incantations related to releasing Synapse wasn't trivial on eg a macOS system, so this is a script to run in a Debian container to make things a bit easier.

Also refactors some of the registration of endpoints on workers.

Outlier events don't ever have push actions associated with them, so we
can skip some expensive queries during event persistence.