CHANGES.md



To find the state of this project's repository at the time of any of these versions, check out the tags.


Synapse 1.54.0rc1 (2022-03-02)
Please note that this will be the last release of Synapse that is compatible with Mjolnir 1.3.1 and earlier.
Administrators of servers which have the Mjolnir module installed are advised to upgrade Mjolnir to version 1.3.2 or later.

Features

Add support for MSC3202: sending one-time key counts and fallback key usage states to Application Services. (#11617)
Improve the preview that is produced when generating URL previews for some web pages. Contributed by @AndrewRyanChama. (#11985)
Track cache invalidations in Prometheus metrics, as already happens for cache eviction based on size or time. (#12000)
Implement experimental support for MSC3720 (account status endpoints). (#12001, #12067)
Enable modules to set a custom display name when registering a user. (#12009)
Advertise Matrix 1.1 and 1.2 support on /_matrix/client/versions. (#12020, (#12022)
Support only the stable identifier for MSC3069's is_guest on /_matrix/client/v3/account/whoami. (#12021)
Use room version 9 as the default room version (per MSC3589). (#12058)
Add module callbacks to react to user deactivation status changes (i.e. deactivations and reactivations) and profile updates. (#12062)


Bugfixes

Fix a bug introduced in Synapse 1.48.0 where an edit of the latest event in a thread would not be properly applied to the thread summary. (#11992)
Fix long-standing bug where get_rooms_for_user was not correctly invalidated for remote users when the server left a room. (#11999)
Fix a 500 error with Postgres when looking backwards with the MSC3030 /timestamp_to_event?dir=b endpoint. (#12024)
Properly fix a long-standing bug where wrong data could be inserted into the event_search table when using SQLite. This could block running synapse_port_db with an argument of type 'int' is not iterable error. This bug was partially fixed by a change in Synapse 1.44.0. (#12037)
Fix slow performance of /logout in some cases where refresh tokens are in use. The slowness existed since the initial implementation of refresh tokens in version 1.38.0. (#12056)
Fix a long-standing bug where Synapse would make additional failing requests over federation for missing data. (#12077)
Fix occasional Unhandled error in Deferred error message. (#12089)
Fix a bug introduced in Synapse 1.51.0 where incoming federation transactions containing at least one EDU would be dropped if debug logging was enabled for synapse.8631_debug. (#12098)
Fix a long-standing bug which could cause push notifications to malfunction if use_frozen_dicts was set in the configuration. (#12100)
Fix an extremely rare, long-standing bug in ReadWriteLock that would cause an error when a newly unblocked writer completes instantly. (#12105)
Make a POST to /rooms/<room_id>/receipt/m.read/<event_id> only trigger a push notification if the count of unread messages is different to the one in the last successfully sent push. This reduces server load and load on the receiving device. (#11835)


Updates to the Docker image

The Docker image no longer automatically creates a temporary volume at /data. This is not expected to affect normal usage. (#11997)
Use Python 3.9 in Docker images by default. (#12112)


Improved Documentation

Document support for the to_device, account_data, receipts, and presence stream writers for workers. (#11599)
Explain the meaning of spam checker callbacks' return values. (#12003)
Clarify information about external Identity Provider IDs. (#12004)


Deprecations and Removals

Deprecate using synctl with the config option synctl_cache_factor and print a warning if a user still uses this option. (#11865)
Remove support for the legacy structured logging configuration (please see the the upgrade notes if you are using structured: true in the Synapse configuration). (#12008)
Drop support for MSC3283 unstable flags now that the stable flags are supported. (#12018)
Remove the unstable /spaces endpoint from MSC2946. (#12073)


Internal Changes

Make the get_room_version method use get_room_version_id to benefit from caching. (#11808)
Remove unnecessary condition on knock -> leave auth rule check. (#11900)
Add tests for device list changes between local users. (#11972)
Optimise calculating device_list changes in /sync. (#11974)
Add missing type hints to storage classes. (#11984)
Refactor the search code for improved readability. (#11991)
Move common deduplication code down into _auth_and_persist_outliers. (#11994)
Limit concurrent joins from applications services. (#11996)
Preparation for faster-room-join work: when parsing the send_join response, get the m.room.create event from state, not auth_chain. (#12005, #12039)
Preparation for faster-room-join work: parse MSC3706 fields in send_join response. (#12011)
Preparation for faster-room-join work: persist information on which events and rooms have partial state to the database. (#12012)
Preparation for faster-room-join work: Support for calling /federation/v1/state on a remote server. (#12013)
Configure tox to use venv rather than virtualenv. (#12015)
Fix bug in StateFilter.return_expanded() and add some tests. (#12016)
Use Matrix v1.1 endpoints (/_matrix/client/v3/auth/...) in fallback auth HTML forms. (#12019)
Update the olddeps CI job to use an old version of markupsafe. (#12025)
Upgrade Mypy to version 0.931. (#12030)
Remove legacy HomeServer.get_datastore(). (#12031, #12070)
Minor typing fixes. (#12034, #12069)
After joining a room, create a dedicated logcontext to process the queued events. (#12041)
Tidy up GitHub Actions config which builds distributions for PyPI. (#12051)
Move configuration out of setup.cfg. (#12052, #12059)
Fix error message when a worker process fails to talk to another worker process. (#12060)
Fix using the complement.sh script without specifying a directory or a branch. Contributed by Nico on behalf of Famedly. (#12063)
Add type hints to tests/rest/client. (#12066, #12072, #12084, #12094)
Add some logging to /sync to try and track down #11916. (#12068)
Inspect application dependencies using importlib.metadata or its backport. (#12088)
Use assertEqual instead of the deprecated assertEquals in test code. (#12092)
Move experimental support for MSC3440 to /versions. (#12099)
Add stop_cancellation utility function to stop Deferreds from being cancelled. (#12106)
Improve exception handling for concurrent execution. (#12109)
Advertise support for Python 3.10 in packaging files. (#12111)
Move CI checks out of tox, to facilitate a move to using poetry. (#12119)


Synapse 1.53.0 (2022-02-22)
No significant changes since 1.53.0rc1.

Synapse 1.53.0rc1 (2022-02-15)

Features

Add experimental support for sending to-device messages to application services, as specified by MSC2409. (#11215, #11966)
Add a background database update to purge account data for deactivated users. (#11655)
Experimental support for MSC3666: including bundled aggregations in server side search results. (#11837)
Enable cache time-based expiry by default. The expiry_time config flag has been superseded by expire_caches and cache_entry_ttl. (#11849)
Add a callback to allow modules to allow or forbid a 3PID (email address, phone number) from being associated to a local account. (#11854)
Stabilize support and remove unstable endpoints for MSC3231. Clients must switch to the stable identifier and endpoint. See the upgrade notes for more information. (#11867)
Allow modules to retrieve the current instance's server name and worker name. (#11868)
Use a dedicated configurable rate limiter for 3PID invites. (#11892)
Support the stable API endpoint for MSC3283: new settings in /capabilities endpoint. (#11933, #11989)
Support the dir parameter on the /relations endpoint, per MSC3715. (#11941)
Experimental implementation of MSC3706: extensions to /send_join to support reduced response size. (#11967)


Bugfixes

Fix MSC2716 historical messages backfilling in random order on remote homeservers. (#11114)
Fix a bug introduced in Synapse 1.51.0 where incoming federation transactions containing at least one EDU would be dropped if debug logging was enabled for synapse.8631_debug. (#11890)
Fix a long-standing bug where some unknown endpoints would return HTML error pages instead of JSON M_UNRECOGNIZED errors. (#11930)
Implement an allow list of content types for which we will attempt to preview a URL. This prevents Synapse from making useless longer-lived connections to streaming media servers. (#11936)
Fix a long-standing bug where pagination tokens from /sync and /messages could not be provided to the /relations API. (#11952)
Require that modules register their callbacks using keyword arguments. (#11975)
Fix a long-standing bug where M_WRONG_ROOM_KEYS_VERSION errors would not include the specced current_version field. (#11988)


Improved Documentation

Fix typo in User Admin API: unpind -> unbind. (#11859)
Document images returned by the User List Media Admin API can include those generated by URL previews. (#11862)
Remove outdated MSC1711 FAQ document. (#11907)
Correct the structured logging configuration example. Contributed by Brad Jones. (#11946)
Add information on the Synapse release cycle. (#11954)
Fix broken link in the README to the admin API for password reset. (#11955)


Deprecations and Removals

Drop support for webclient listeners and configuring web_client_location to a non-HTTP(S) URL. Deprecated configurations are a configuration error. (#11895)
Remove deprecated user_may_create_room_with_invites spam checker callback. See the upgrade notes for more information. (#11950)
No longer build .deb packages for Ubuntu 21.04 Hirsute Hippo, which has now EOLed. (#11961)


Internal Changes

Enhance user registration test helpers to make them more useful for tests involving application services and devices. (#11615, #11616)
Improve performance when fetching bundled aggregations for multiple events. (#11660, #11752)
Fix type errors introduced by new annotations in the Prometheus Client library. (#11832)
Add missing type hints to replication code. (#11856, #11938)
Ensure that opentracing scopes are activated and closed at the right time. (#11869)
Improve opentracing for incoming federation requests. (#11870)
Improve internal docstrings in synapse.util.caches. (#11876)
Do not needlessly clear the get_users_in_room and get_users_in_room_with_profiles caches when any room state changes. (#11878)
Convert ApplicationServiceTestCase to use simple_async_mock. (#11880)
Remove experimental changes to the default push rules which were introduced in Synapse 1.19.0 but never enabled. (#11884)
Disable coverage calculation for olddeps build. (#11888)
Preparation to support sending device list updates to application services. (#11905)
Add a test that checks users receive their own device list updates down /sync. (#11909)
Run Complement tests sequentially. (#11910)
Various refactors to the application service notifier code. (#11911, #11912)
Tests: replace mocked Authenticator with the real thing. (#11913)
Various refactors to the typing notifications code. (#11914)
Use the proper type for the Content-Length header in the UploadResource. (#11927)
Remove an unnecessary ignoring of type hints due to fixes in upstream packages. (#11939)
Add missing type hints. (#11953)
Fix an import cycle in synapse.event_auth. (#11965)
Unpin frozendict but exclude the known bad version 2.1.2. (#11969)
Prepare for rename of default Complement branch. (#11971)
Fetch Synapse's version using a helper from matrix-common. (#11979)


Synapse 1.52.0 (2022-02-08)
No significant changes since 1.52.0rc1.
Note that Twisted 22.1.0
has recently been released, which fixes a security issue
within the Twisted library. We do not believe Synapse is affected by this vulnerability,
though we advise server administrators who installed Synapse via pip to upgrade Twisted
with pip install --upgrade Twisted treq as a matter of good practice. The Docker image
matrixdotorg/synapse and the Debian packages from packages.matrix.org are using the
updated library.

Synapse 1.52.0rc1 (2022-02-01)

Features

Remove account data (including client config, push rules and ignored users) upon user deactivation. (#11621, #11788, #11789)
Add an admin API to reset connection timeouts for remote server. (#11639)
Add an admin API to get a list of rooms that federate with a given remote homeserver. (#11658)
Add a config flag to inhibit M_USER_IN_USE during registration. (#11743)
Add a module callback to set username at registration. (#11790)
Allow configuring a maximum file size as well as a list of allowed content types for avatars. (#11846)


Bugfixes

Include the bundled aggregations in the /sync response, per MSC2675. (#11612)
Fix a long-standing bug when previewing Reddit URLs which do not contain an image. (#11767)
Fix a long-standing bug that media streams could cause long-lived connections when generating URL previews. (#11784)
Include a prev_content field in state events sent to Application Services. Contributed by @totallynotvaishnav. (#11798)
Fix a bug introduced in Synapse 0.33.3 causing requests to sometimes log strings such as HTTPStatus.OK instead of integer status codes. (#11827)


Improved Documentation

Update pypi installation docs to indicate that we now support Python 3.10. (#11820)
Add missing steps to the contribution submission process in the documentation.  Contributed by @sequentialread. (#11821)
Remove not needed old table of contents in documentation. (#11860)
Consolidate the access_token information at the top of each relevant page in the Admin API documentation. (#11861)


Deprecations and Removals

Drop support for Python 3.6, which is EOL. (#11683)
Remove the experimental_msc1849_support_enabled flag as the features are now stable. (#11843)


Internal Changes

Preparation for database schema simplifications: add state_key and rejection_reason columns to events table. (#11792)
Add FrozenEvent.get_state_key and use it in a couple of places. (#11793)
Preparation for database schema simplifications: stop reading from event_reference_hashes. (#11794)
Drop unused table public_room_list_stream. (#11795)
Preparation for reducing Postgres serialization errors: allow setting transaction isolation level. Contributed by Nick @ Beeper. (#11799, #11847)
Docker: skip the initial amd64-only build and go straight to multiarch. (#11810)
Run Complement on the Github Actions VM and not inside a Docker container. (#11811)
Log module names at startup. (#11813)
Improve type safety of bundled aggregations code. (#11815)
Correct a type annotation in the event validation logic. (#11817, #11830)
Minor updates and documentation for database schema delta files. (#11823)
Workaround a type annotation problem in prometheus_client 0.13.0. (#11834)
Minor performance improvement in room state lookup. (#11836)
Fix some indentation inconsistencies in the sample config. (#11838)
Add type hints to tests/rest/admin. (#11851)


Synapse 1.51.0 (2022-01-25)
No significant changes since 1.51.0rc2.
Synapse 1.51.0 deprecates webclient listeners and non-HTTP(S) web_client_locations. Support for these will be removed in Synapse 1.53.0, at which point Synapse will not be capable of directly serving a web client for Matrix. See the upgrade notes.

Synapse 1.51.0rc2 (2022-01-24)

Bugfixes

Fix a bug introduced in Synapse 1.40.0 that caused Synapse to fail to process incoming federation traffic after handling a large amount of events in a v1 room. (#11806)


Synapse 1.50.2 (2022-01-24)
This release includes the same bugfix as Synapse 1.51.0rc2.

Bugfixes

Fix a bug introduced in Synapse 1.40.0 that caused Synapse to fail to process incoming federation traffic after handling a large amount of events in a v1 room. (#11806)


Synapse 1.51.0rc1 (2022-01-21)

Features

Add track_puppeted_user_ips config flag to record client IP addresses against puppeted users, and include the puppeted users in monthly active user counts. (#11561, #11749, #11757)
Include whether the requesting user has participated in a thread when generating a summary for MSC3440. (#11577)
Return an M_FORBIDDEN error code instead of M_UNKNOWN when a spam checker module prevents a user from creating a room. (#11672)
Add a flag to the synapse_review_recent_signups script to ignore and filter appservice users. (#11675, #11770)


Bugfixes

Fix a long-standing issue which could cause Synapse to incorrectly accept data in the unsigned field of events
received over federation. (#11530)
Fix a long-standing bug where Synapse wouldn't cache a response indicating that a remote user has no devices. (#11587)
Fix an error that occurs whilst trying to get the federation status of a destination server that was working normally. This admin API was newly introduced in Synapse v1.49.0. (#11593)
Fix bundled aggregations not being included in the /sync response, per MSC2675. (#11612, #11659, #11791)
Fix the /_matrix/client/v1/room/{roomId}/hierarchy endpoint returning incorrect fields which have been present since Synapse 1.49.0. (#11667)
Fix preview of some GIF URLs (like tenor.com). Contributed by Philippe Daouadi. (#11669)
Fix a bug where only the first 50 rooms from a space were returned from the /hierarchy API. This has existed since the introduction of the API in Synapse v1.41.0. (#11695)
Fix a bug introduced in Synapse v1.18.0 where password reset and address validation emails would not be sent if their subject was configured to use the 'app' template variable. Contributed by @br4nnigan. (#11710, #11745)
Make the 'List Rooms' Admin API sort stable. Contributed by Daniël Sonck. (#11737)
Fix a long-standing bug where space hierarchy over federation would only work correctly some of the time. (#11775)
Fix a bug introduced in Synapse v1.46.0 that prevented on_logged_out module callbacks from being correctly awaited by Synapse. (#11786)


Improved Documentation

Warn against using a Let's Encrypt certificate for TLS/DTLS TURN server client connections, and suggest using ZeroSSL certificate instead. This works around client-side connectivity errors caused by WebRTC libraries that reject Let's Encrypt certificates. Contibuted by @AndrewFerr. (#11686)
Document the new SYNAPSE_TEST_PERSIST_SQLITE_DB environment variable in the contributing guide. (#11715)
Document that the minimum supported PostgreSQL version is now 10. (#11725)
Fix typo in demo docs: differnt. (#11735)
Update room spec URL in config files. (#11739)
Mention python3-venv and libpq-dev dependencies in the contribution guide. (#11740)
Update documentation for configuring login with Facebook. (#11755)
Update installation instructions to note that Python 3.6 is no longer supported. (#11781)


Deprecations and Removals

Remove the unstable /send_relation endpoint. (#11682)
Remove python_twisted_reactor_pending_calls Prometheus metric. (#11724)
Remove the password_hash field from the response dictionaries of the Users Admin API. (#11576)
Deprecate support for webclient listeners and non-HTTP(S) web_client_location configuration. (#11774, #11783)


Internal Changes

Run pyupgrade --py37-plus --keep-percent-format on Synapse. (#11685)
Use buildkit's cache feature to speed up docker builds. (#11691)
Use auto_attribs and native type hints for attrs classes. (#11692, #11768)
Remove debug logging for #4422, which has been closed since Synapse 0.99. (#11693)
Remove fallback code for Python 2. (#11699)
Add a test for an edge case in the /sync logic. (#11701)
Add the option to write SQLite test dbs to disk when running tests. (#11702)
Improve Complement test output for Gitub Actions. (#11707)
Fix docstring on add_account_data_for_user. (#11716)
Complement environment variable name change and update .gitignore. (#11718)
Simplify calculation of Prometheus metrics for garbage collection. (#11723)
Improve accuracy of python_twisted_reactor_tick_time Prometheus metric. (#11724, #11771)
Minor efficiency improvements when inserting many values into the database. (#11742)
Invite PR authors to give themselves credit in the changelog. (#11744)
Add optional debugging to investigate issue 8631. (#11760)
Remove log_function utility function and its uses. (#11761)
Add a unit test that checks both client and webclient resources will function when simultaneously enabled. (#11765)
Allow overriding complement commit using COMPLEMENT_REF. (#11766)
Add some comments and type annotations for _update_outliers_txn. (#11776)


Synapse 1.50.1 (2022-01-18)
This release fixes a bug in Synapse 1.50.0 that could prevent clients from being able to connect to Synapse if the webclient resource was enabled. Further details are available in this issue.

Bugfixes

Fix a bug introduced in Synapse 1.50.0rc1 that could cause Matrix clients to be unable to connect to Synapse instances with the webclient resource enabled. (#11764)


Synapse 1.50.0 (2022-01-18)
This release contains a critical bug that may prevent clients from being able to connect.
As such, it is not recommended to upgrade to 1.50.0. Instead, please upgrade straight to
to 1.50.1. Further details are available in this issue.
Please note that we now only support Python 3.7+ and PostgreSQL 10+ (if applicable), because Python 3.6 and PostgreSQL 9.6 have reached end-of-life.
No significant changes since 1.50.0rc2.

Synapse 1.50.0rc2 (2022-01-14)
This release candidate fixes a federation-breaking regression introduced in Synapse 1.50.0rc1.

Bugfixes

Fix a bug introduced in Synapse v1.0.0 whereby some device list updates would not be sent to remote homeservers if there were too many to send at once. (#11729)
Fix a bug introduced in Synapse v1.50.0rc1 whereby outbound federation could fail because too many EDUs were produced for device updates. (#11730)


Improved Documentation

Document that now the minimum supported PostgreSQL version is 10. (#11725)


Internal Changes

Fix a typechecker problem related to our (ab)use of nacl.signing.SigningKeys. (#11714)


Synapse 1.50.0rc1 (2022-01-05)

Features

Allow guests to send state events per MSC3419. (#11378)
Add experimental support for part of MSC3202: allowing application services to masquerade as specific devices. (#11538)
Add admin API to get users' account data. (#11664)
Include the room topic in the stripped state included with invites and knocking. (#11666)
Send and handle cross-signing messages using the stable prefix. (#10520)
Support unprefixed versions of fallback key property names. (#11541)


Bugfixes

Fix a long-standing bug where relations from other rooms could be included in the bundled aggregations of an event. (#11516)
Fix a long-standing bug which could cause AssertionErrors to be written to the log when Synapse was restarted after purging events from the database. (#11536, #11642)
Fix a bug introduced in Synapse 1.17.0 where a pusher created for an email with capital letters would fail to be created. (#11547)
Fix a long-standing bug where responses included bundled aggregations when they should not, per MSC2675. (#11592, #11623)
Fix a long-standing bug that some unknown endpoints would return HTML error pages instead of JSON M_UNRECOGNIZED errors. (#11602)
Fix a bug introduced in Synapse 1.19.3 which could sometimes cause AssertionErrors when backfilling rooms over federation. (#11632)


Improved Documentation

Update Synapse install command for FreeBSD as the package is now prefixed with py38. Contributed by @itchychips. (#11267)
Document the usage of refresh tokens. (#11427)
Add details for how to configure a TURN server when behind a NAT. Contibuted by @AndrewFerr. (#11553)
Add references for using Postgres to the Docker documentation. (#11640)
Fix the documentation link in newly-generated configuration files. (#11678)
Correct the documentation for nginx to use a case-sensitive url pattern. Fixes an error introduced in v1.21.0. (#11680)
Clarify SSO mapping provider documentation by writing def or async def before the names of methods, as appropriate. (#11681)


Deprecations and Removals

Replace mock package by its standard library version. (#11588)
Drop support for Python 3.6 and Ubuntu 18.04. (#11633)


Internal Changes

Allow specific, experimental events to be created without prev_events. Used by MSC2716. (#11243)
A test helper (wait_for_background_updates) no longer depends on classes defining a store property. (#11331)
Add type hints to synapse.appservice. (#11360)
Add missing type hints to synapse.config module. (#11480)
Add test to ensure we share the same state_group across the whole historical batch when using the MSC2716 /batch_send endpoint. (#11487)
Refactor tests.util.setup_test_homeserver and tests.server.setup_test_homeserver. (#11503)
Move glob_to_regex and re_word_boundary to matrix-python-common. (#11505, #11687)
Use HTTPStatus constants in place of literals in tests.rest.client.test_auth. (#11520)
Add a receipt types constant for m.read. (#11531)
Clean up synapse.rest.admin. (#11535)
Add missing errcode to parse_string and parse_boolean. (#11542)
Use HTTPStatus constants in place of literals in synapse.http. (#11543)
Add missing type hints to storage classes. (#11546, #11549, #11551, #11555, #11575, #11589, #11594, #11652, #11653, #11654, #11657)
Fix an inaccurate and misleading comment in the /sync code. (#11550)
Add missing type hints to synapse.logging.context. (#11556)
Stop populating unused database column state_events.prev_state. (#11558)
Minor efficiency improvements in event persistence. (#11560)
Add some safety checks that storage functions are used correctly. (#11564, #11580)
Make get_device return None if the device doesn't exist rather than raising an exception. (#11565)
Split the HTML parsing code from the URL preview resource code. (#11566)
Remove redundant COALESCE()s around COUNT()s in database queries. (#11570)
Add missing type hints to synapse.http. (#11571)
Add MSC2716 and MSC3030 to /versions -> unstable_features to detect server support. (#11582)
Add type hints to synapse/tests/rest/admin. (#11590)
Drop end-of-life Python 3.6 and Postgres 9.6 from CI. (#11595)
Update black version and run it on all the files. (#11596)
Add opentracing type stubs and fix associated mypy errors. (#11603, #11622)
Improve OpenTracing support for requests which use a ResponseCache. (#11607)
Improve OpenTracing support for incoming HTTP requests. (#11618)
A number of improvements to opentracing support. (#11619)
Refactor the way that the outlier flag is set on events received over federation. (#11634)
Improve the error messages from  get_create_event_for_room. (#11638)
Remove redundant get_current_events_token method. (#11643)
Convert namedtuples to attrs. (#11665, #11574)
Update the /capabilities response to include whether support for MSC3440 is available. (#11690)
Send the Accept header in HTTP requests made using SimpleHttpClient.get_json. (#11677)
Work around Mjolnir compatibility issue by adding an import for glob_to_regex in synapse.util, where it moved from. (#11696)


Synapse 1.49.2 (2021-12-21)
This release fixes a regression introduced in Synapse 1.49.0 which could cause /sync requests to take significantly longer. This would particularly affect "initial" syncs for users participating in a large number of rooms, and in extreme cases, could make it impossible for such users to log in on a new client.
Note: in line with our deprecation policy for platform dependencies, this will be the last release to support Python 3.6 and PostgreSQL 9.6, both of which have now reached upstream end-of-life. Synapse will require Python 3.7+ and PostgreSQL 10+.
Note: We will also stop producing packages for Ubuntu 18.04 (Bionic Beaver) after this release, as it uses Python 3.6.

Bugfixes

Fix a performance regression in /sync handling, introduced in 1.49.0. (#11583)


Internal Changes

Work around a build problem on Debian Buster. (#11625)


Synapse 1.49.1 (2021-12-21)
Not released due to problems building the debian packages.

Synapse 1.49.0 (2021-12-14)
No significant changes since version 1.49.0rc1.

Support for Ubuntu 21.04 ends next month on the 20th of January
For users of Ubuntu 21.04 (Hirsute Hippo), please be aware that upstream support for this version of Ubuntu will end next month.
We will stop producing packages for Ubuntu 21.04 after upstream support ends.

The wiki has been migrated to the documentation website
We've decided to move the existing, somewhat stagnant pages from the GitHub wiki
to the documentation website.
This was done for two reasons. The first was to ensure that changes are checked by
multiple authors before being committed (everyone makes mistakes!) and the second
was visibility of the documentation. Not everyone knows that Synapse has some very
useful information hidden away in its GitHub wiki pages. Bringing them to the
documentation website should help with visibility, as well as keep all Synapse documentation
in one, easily-searchable location.
Note that contributions to the documentation website happen through GitHub pull
requests. Please visit #synapse-dev:matrix.org
if you need help with the process!

Synapse 1.49.0rc1 (2021-12-07)

Features

Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp. (#9445)
Include bundled relation aggregations during a limited /sync request and /relations request, per MSC2675. (#11284, #11478)
Add plugin support for controlling database background updates. (#11306, #11475, #11479)
Support the stable API endpoints for MSC2946: the room /hierarchy endpoint. (#11329)
Add admin API to get some information about federation status with remote servers. (#11407)
Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. (#11425)
Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435, #11522)
Update MSC2918 refresh token support to confirm with the latest revision: accept the refresh_tokens parameter in the request body rather than in the URL parameters. (#11430)
Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. (#11445)
Expose synapse_homeserver and synapse_worker commands as entry points to run Synapse's main process and worker processes, respectively. Contributed by @Ma27. (#11449)

synctl stop will now wait for Synapse to exit before returning. (#11459, #11490)
Extend the "delete room" admin api to work correctly on rooms which have previously been partially deleted. (#11523)
Add support for the /_matrix/client/v3/login/sso/redirect/{idpId} API from Matrix v1.1. This endpoint was overlooked when support for v3 endpoints was added in Synapse 1.48.0rc1. (#11451)


Bugfixes

Fix using MSC2716 batch sending in combination with event persistence workers. Contributed by @tulir at Beeper. (#11220)
Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection, properly this time. Also fix a race condition introduced in the previous insufficient fix in Synapse 1.47.0. (#11376)
The /send_join response now includes the stable event field instead of the unstable field from MSC3083. (#11413)
Fix a bug introduced in Synapse 1.47.0 where send_join could fail due to an outdated ijson version. (#11439, #11441, #11460)
Fix a bug introduced in Synapse 1.36.0 which could cause problems fetching event-signing keys from trusted key servers. (#11440)
Fix a bug introduced in Synapse 1.47.1 where the media repository would fail to work if the media store path contained any symbolic links. (#11446)
Fix an LruCache corruption bug, introduced in Synapse 1.38.0, that would cause certain requests to fail until the next Synapse restart. (#11454)
Fix a long-standing bug where invites from ignored users were included in incremental syncs. (#11511)
Fix a regression in Synapse 1.48.0 where presence workers would not clear their presence updates over replication on shutdown. (#11518)
Fix a regression in Synapse 1.48.0 where the module API's looping_background_call method would spam errors to the logs when given a non-async function. (#11524)


Updates to the Docker image

Update Dockerfile-workers to healthcheck all workers in the container. (#11429)


Improved Documentation

Update the media repository documentation. (#11415)
Update section about backward extremities in the room DAG concepts doc to correct the misconception about backward extremities indicating whether we have fetched an events' prev_events. (#11469)


Internal Changes

Add Final annotation to string constants in synapse.api.constants so that they get typed as Literals. (#11356)
Add a check to ensure that users cannot start the Synapse master process when worker_app is set. (#11416)
Add a note about postgres memory management and hugepages to postgres doc. (#11467)
Add missing type hints to synapse.config module. (#11465)
Add missing type hints to synapse.federation. (#11483)
Add type annotations to tests.storage.test_appservice. (#11488, #11492)
Add type annotations to some of the configuration surrounding refresh tokens. (#11428)
Add type hints to synapse/tests/rest/admin. (#11501)
Add type hints to storage classes. (#11411)
Add wiki pages to documentation website. (#11402)
Clean up tests.storage.test_main to remove use of legacy code. (#11493)
Clean up tests.test_visibility to remove legacy code. (#11495)
Convert status codes to HTTPStatus in synapse.rest.admin. (#11452, #11455)
Extend the scripts-dev/sign_json script to support signing events. (#11486)
Improve internal types in push code. (#11409)
Improve type annotations in synapse.module_api. (#11029)
Improve type hints for LruCache. (#11453)
Preparation for database schema simplifications: disambiguate queries on state_key. (#11497)
Refactor backfilled into specific behavior function arguments (_persist_events_and_state_updates and downstream calls). (#11417)
Refactor get_version_string to fix-up types and duplicated code. (#11468)
Refactor various parts of the /sync handler. (#11494, #11515)
Remove unnecessary json.dumps from tests.rest.admin. (#11461)
Save the OpenID Connect session ID on login. (#11482)
Update and clean up recently ported documentation pages. (#11466)


Synapse 1.48.0 (2021-11-30)
This release removes support for the long-deprecated trust_identity_server_for_password_resets configuration flag.
This release also fixes some performance issues with some background database updates introduced in Synapse 1.47.0.
No significant changes since 1.48.0rc1.

Synapse 1.48.0rc1 (2021-11-25)

Features

Experimental support for the thread relation defined in MSC3440. (#11161)
Support filtering by relation senders & types per MSC3440. (#11236)
Add support for the /_matrix/client/v3 and /_matrix/media/v3 APIs from Matrix v1.1. (#11318, #11371)
Support the stable version of MSC2778: the m.login.application_service login type. Contributed by @tulir. (#11335)
Add a new version of delete room admin API DELETE /_synapse/admin/v2/rooms/<room_id> to run it in the background. Contributed by @dklimpel. (#11223)
Allow the admin Delete Room API to block a room without the need to join it. (#11228)
Add an admin API to un-shadow-ban a user. (#11347)
Add an admin API to run background database schema updates. (#11352)
Add an admin API for blocking a room. (#11324)
Update the JWT login type to support custom a sub claim. (#11361)
Store and allow querying of arbitrary event relations. (#11391)


Bugfixes

Fix a long-standing bug wherein display names or avatar URLs containing null bytes cause an internal server error when stored in the DB. (#11230)
Prevent MSC2716 historical state events from being pushed to an application service via /transactions. (#11265)
Fix a long-standing bug where uploading extremely thin images (e.g. 1000x1) would fail. Contributed by @Neeeflix. (#11288)
Fix a bug, introduced in Synapse 1.46.0, which caused the check_3pid_auth and on_logged_out callbacks in legacy password authentication provider modules to not be registered. Modules using the generic module interface were not affected. (#11340)
Fix a bug introduced in 1.41.0 where space hierarchy responses would be incorrectly reused if multiple users were to make the same request at the same time. (#11355)
Fix a bug introduced in 1.45.0 where the read_templates method of the module API would error. (#11377)
Fix an issue introduced in 1.47.0 which prevented servers re-joining rooms they had previously left, if their signing keys were replaced. (#11379)
Fix a bug introduced in 1.13.0 where creating and publishing a room could cause errors if room_list_publication_rules is configured. (#11392)
Improve performance of various background database updates. (#11421, #11422)


Improved Documentation

Suggest users of the Debian packages add configuration to /etc/matrix-synapse/conf.d/ to prevent, upon upgrade, being asked to choose between their configuration and the maintainer's. (#11281)
Fix typos in the documentation for the username_available admin API. Contributed by Stanislav Motylkov. (#11286)
Add Single Sign-On, SAML and CAS pages to the documentation. (#11298)
Change the word 'Home server' as one word 'homeserver' in documentation. (#11320)
Fix missing quotes for wildcard domains in federation_certificate_verification_whitelist. (#11381)


Deprecations and Removals

Remove deprecated trust_identity_server_for_password_resets configuration flag. (#11333, #11395)


Internal Changes

Add type annotations to synapse.metrics. (#10847)
Split out federated PDU retrieval function into a non-cached version. (#11242)
Clean up code relating to to-device messages and sending ephemeral events to application services. (#11247)
Fix a small typo in the error response when a relation type other than 'm.annotation' is passed to GET /rooms/{room_id}/aggregations/{event_id}. (#11278)
Drop unused database tables room_stats_historical and user_stats_historical. (#11280)
Require all files in synapse/ and tests/ to pass mypy unless specifically excluded. (#11282, #11285, #11359)
Add missing type hints to synapse.app. (#11287)
Remove unused parameters on FederationEventHandler._check_event_auth. (#11292)
Add type hints to synapse._scripts. (#11297)
Fix an issue which prevented the remove_deleted_devices_from_device_inbox background database schema update from running when updating from a recent Synapse version. (#11303)
Add type hints to storage classes. (#11307, #11310, #11311, #11312, #11313, #11314, #11316, #11322, #11332, #11339, #11342)
Add type hints to synapse.util. (#11321, #11328)
Improve type annotations in Synapse's test suite. (#11323, #11330)
Test that room alias deletion works as intended. (#11327)
Add type annotations for some methods and properties in the module API. (#11341)
Fix running scripts-dev/complement.sh, which was broken in v1.47.0rc1. (#11368)
Rename internal functions for token generation to better reflect what they do. (#11369, #11370)
Add type hints to configuration classes. (#11377)
Publish a develop image to Docker Hub. (#11380)
Keep fallback key marked as used if it's re-uploaded. (#11382)
Use auto_attribs on the attrs class RefreshTokenLookupResult. (#11386)
Rename unstable access_token_lifetime configuration option to refreshable_access_token_lifetime to make it clear it only concerns refreshable access tokens. (#11388)
Do not run the broken MSC2716 tests when running scripts-dev/complement.sh. (#11389)
Remove dead code from supporting ACME. (#11393)
Refactor including the bundled relations when serializing an event. (#11408)


Synapse 1.47.1 (2021-11-23)
This release fixes a security issue in the media store, affecting all prior releases of Synapse. Server administrators are encouraged to update Synapse as soon as possible. We are not aware of these vulnerabilities being exploited in the wild.
Server administrators who are unable to update Synapse may use the workarounds described in the linked GitHub Security Advisory below.

Security advisory
The following issue is fixed in 1.47.1.


GHSA-3hfw-x7gx-437c / CVE-2021-41281: Path traversal when downloading remote media.
Synapse instances with the media repository enabled can be tricked into downloading a file from a remote server into an arbitrary directory, potentially outside the media store directory.
The last two directories and file name of the path are chosen randomly by Synapse and cannot be controlled by an attacker, which limits the impact.
Homeservers with the media repository disabled are unaffected. Homeservers configured with a federation whitelist are also unaffected.
Fixed by 91f2bd090.


Synapse 1.47.0 (2021-11-17)
No significant changes since 1.47.0rc3.

Synapse 1.47.0rc3 (2021-11-16)

Bugfixes

Fix a bug introduced in 1.47.0rc1 which caused worker processes to not halt startup in the presence of outstanding database migrations. (#11346)
Fix a bug introduced in 1.47.0rc1 which prevented the 'remove deleted devices from device_inbox column' background process from running when updating from a recent Synapse version. (#11303, #11353)


Synapse 1.47.0rc2 (2021-11-10)
This fixes an issue with publishing the Debian packages for 1.47.0rc1.
It is otherwise identical to 1.47.0rc1.

Synapse 1.47.0rc1 (2021-11-09)

Deprecations and Removals

The user_may_create_room_with_invites module callback is now deprecated. Please refer to the upgrade notes for more information. (#11206)
Remove deprecated admin API to delete rooms (POST /_synapse/admin/v1/rooms/<room_id>/delete). (#11213)


Features

Advertise support for Client-Server API r0.6.1. (#11097)
Add search by room ID and room alias to the List Room admin API. (#11099)
Add an on_new_event third-party rules callback to allow Synapse modules to act after an event has been sent into a room. (#11126)
Add a module API method to update a user's membership in a room. (#11147)
Add metrics for thread pool usage. (#11178)
Support the stable room type field for MSC3288. (#11187)
Add a module API method to retrieve the current state of a room. (#11204)
Calculate a default value for public_baseurl based on server_name. (#11210)
Add support for serving /.well-known/matrix/server files, to redirect federation traffic to port 443. (#11211)
Add admin APIs to pause, start and check the status of background updates. (#11263)


Bugfixes

Fix a long-standing bug which allowed hidden devices to receive to-device messages, resulting in unnecessary database bloat. (#10097)
Fix a long-standing bug where messages in the device_inbox table for deleted devices would persist indefinitely. Contributed by @dklimpel and @JohannesKleine. (#10969, #11212)
Do not accept events if a third-party rule check_event_allowed callback raises an exception. (#11033)
Fix long-standing bug where verification requests could fail in certain cases if a federation whitelist was in place but did not include your own homeserver. (#11129)
Allow an empty list of state_events_at_start to be sent when using the MSC2716 /batch_send endpoint and the author of the historical messages is already part of the current room state at the given ?prev_event_id. (#11188)
Fix a bug introduced in Synapse 1.45.0 which prevented the synapse_review_recent_signups script from running. Contributed by @samuel-p. (#11191)
Delete to_device messages for hidden devices that will never be read, reducing database size. (#11199)
Fix a long-standing bug wherein a missing Content-Type header when downloading remote media would cause Synapse to throw an error. (#11200)
Fix a long-standing bug which could result in serialization errors and potentially duplicate transaction data when sending ephemeral events to application services. Contributed by @Fizzadar at Beeper. (#11207)
Fix a bug introduced in Synapse 1.35.0 which made it impossible to join rooms that return a send_join response containing floats. (#11217)
Fix long-standing bug where cross signing keys were not included in the response to /r0/keys/query the first time a remote user was queried. (#11234)
Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection. (#11240)
Fix a bug preventing Synapse from being rolled back to an earlier version when using workers. (#11255, #11276)
Fix a bug introduced in Synapse 1.37.1 which caused a remote event being processed by a worker to not get processed on restart if the worker was killed. (#11262)
Only allow old Element/Riot Android clients to send read receipts without a request body. All other clients must include a request body as required by the specification. Contributed by @rogersheu. (#11157)


Updates to the Docker image

Avoid changing user ID when started as a non-root user, and no explicit UID is set. (#11209)


Improved Documentation

Improve example HAProxy config in the docs to properly handle HTTP Host headers with port information. This is required for federation over port 443 to work correctly. (#11128)
Add documentation for using Authentik as an OpenID Connect Identity Provider. Contributed by @samip5. (#11151)
Clarify lack of support for Windows. (#11198)
Improve code formatting and fix a few typos in docs. Contributed by @sumnerevans at Beeper. (#11221)
Add documentation for using LemonLDAP as an OpenID Connect Identity Provider. Contributed by @l00ptr. (#11257)


Internal Changes

Add type annotations for the log_function decorator. (#10943)
Add type hints to synapse.events. (#11098)
Remove and document unnecessary RoomStreamToken checks in application service ephemeral event code. (#11137)
Add type hints so that synapse.http passes mypy checks. (#11164)
Update scripts to pass Shellcheck lints. (#11166)
Add knock information in admin export. Contributed by Rafael Gonçalves. (#11171)
Add tests to check that ClientIpStore.get_last_client_ip_by_device and get_user_ip_and_agents combine database and in-memory data correctly. (#11179)
Refactor Filter to check different fields depending on the data type. (#11194)
Improve type hints for the relations datastore. (#11205)
Replace outdated links in the pull request checklist with links to the rendered documentation. (#11225)
Fix a bug in unit test test_block_room_and_not_purge. (#11226)
In ObservableDeferred, run observers in the order they were registered. (#11229)
Minor speed up to start up times and getting updates for groups by adding missing index to local_group_updates.stream_id. (#11231)
Add twine and towncrier as dev dependencies, as they're used by the release script. (#11233)
Allow stream_writers.typing config to be a list of one worker. (#11237)
Remove debugging statement in tests. (#11239)
Fix MSC2716 historical messages backfilling in random order on remote homeservers. (#11244)
Add an additional test for the cachedList method decorator. (#11246)
Make minor correction to the type of auth_checkers callbacks. (#11253)
Clean up trivial aspects of the Debian package build tooling. (#11269, #11273)
Blacklist new SyTest that checks that key uploads are valid pending the validation being implemented in Synapse. (#11270)


Synapse 1.46.0 (2021-11-02)
The cause of the performance regression affecting Synapse 1.44 has been identified and fixed. (#11177)

Bugfixes

Fix a bug introduced in v1.46.0rc1 where URL previews of some XML documents would fail. (#11196)


Synapse 1.46.0rc1 (2021-10-27)

Features

Add support for Ubuntu 21.10 "Impish Indri". (#11024)
Port the Password Auth Providers module interface to the new generic interface. (#10548, #11180)
Experimental support for the thread relation defined in MSC3440. (#11088, #11181, #11192)
Users admin API can now also modify user type in addition to allowing it to be set on user creation. (#11174)


Bugfixes

Newly-created public rooms are now only assigned an alias if the room's creation has not been blocked by permission settings. Contributed by @AndrewFerr. (#10930)
Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state. (#11001, #11009, #11012)
Fix 500 error on /messages when the server accumulates more than 5 backwards extremities at a given depth for a room. (#11027)
Fix a bug where setting a user's external_id via the admin API returns 500 and deletes user's existing external mappings if that external ID is already mapped. (#11051)
Fix a long-standing bug where users excluded from the user directory were added into the directory if they belonged to a room which became public or private. (#11075)
Fix a long-standing bug when attempting to preview URLs which are in the windows-1252 character encoding. (#11077, #11089)
Fix broken export-data admin command and add test script checking the command to CI. (#11078)
Show an error when timestamp in seconds is provided to the /purge_media_cache Admin API. (#11101)
Fix local users who left all their rooms being removed from the user directory, even if the search_all_users config option was enabled. (#11103)
Fix a bug which caused the module API's get_user_ip_and_agents function to always fail on workers. get_user_ip_and_agents was introduced in 1.44.0 and did not function correctly on worker processes at the time. (#11112)
Identity server connection is no longer ignoring ip_range_whitelist. (#11120)
Fix a bug introduced in Synapse 1.45.0 breaking the configuration file parsing script. (#11145)
Fix a performance regression introduced in 1.44.0 which could cause client requests to time out when making large numbers of outbound requests. (#11177, #11190)
Resolve and share state_groups for all MSC2716 historical events in batch. (#10975)


Improved Documentation

Fix broken links relating to module API deprecation in the upgrade notes. (#11069)
Add more information about what happens when a user is deactivated. (#11083)
Clarify the the sample log config can be copied from the documentation without issue. (#11092)
Update the admin API documentation with an updated list of the characters allowed in registration tokens. (#11093)
Document Synapse's behaviour when dealing with multiple modules registering the same callbacks and/or handlers for the same HTTP endpoints. (#11096)
Fix instances of [example]{.title-ref} in the upgrade documentation as a result of prior RST to Markdown conversion. (#11118)
Document the version of Synapse each module callback was introduced in. (#11132)
Document the version of Synapse that introduced each module API method. (#11183)


Internal Changes

Fix spurious warnings about losing the logging context on the ReplicationCommandHandler when losing the replication connection. (#10984)
Include rejected status when we log events. (#11008)
Add some extra logging to the event persistence code. (#11014)
Rearrange the internal workings of the incremental user directory updates. (#11035)
Fix a long-standing bug where users excluded from the directory could still be added to the users_who_share_private_rooms table after a regular user joins a private room. (#11143)
Add and improve type hints. (#10972, #11055, #11066, #11076, #11095, #11109, #11121, #11146)
Mark the Synapse package as containing type annotations and fix export declarations so that Synapse pluggable modules may be type checked against Synapse. (#11054)
Remove dead code from MediaFilePaths. (#11056)
Be more lenient when parsing oEmbed response versions. (#11065)
Create a separate module for the retention configuration. (#11070)
Clean up some of the federation event authentication code for clarity. (#11115, #11116, #11122)
Add docstrings and comments to the application service ephemeral event sending code. (#11138)
Update the sign_json script to support inline configuration of the signing key. (#11139)
Fix broken link in the docker image README. (#11144)
Always dump logs from unit tests during CI runs. (#11068)
Add tests for MediaFilePaths class. (#11057)
Simplify the user admin API tests. (#11048)
Add a test for the workaround introduced in #11042 concerning the behaviour of third-party rule modules and SynapseErrors. (#11071)


Synapse 1.45.1 (2021-10-20)

Bugfixes

Revert change to counting of deactivated users towards the monthly active users limit, introduced in 1.45.0rc1. (#11127)


Synapse 1.45.0 (2021-10-19)
No functional changes since Synapse 1.45.0rc2.

Known Issues


A suspected performance regression which was first reported after the release of 1.44.0 remains unresolved.
We have not been able to identify a probable cause. Affected users report that setting up a federation sender worker appears to alleviate symptoms of the regression.


Improved Documentation

Reword changelog to clarify concerns about a suspected performance regression in 1.44.0. (#11117)


Synapse 1.45.0rc2 (2021-10-14)
This release candidate fixes a user directory bug present in 1.45.0rc1.

Known Issues


A suspected performance regression which was first reported after the release of 1.44.0 remains unresolved.
We have not been able to identify a probable cause. Affected users report that setting up a federation sender worker appears to alleviate symptoms of the regression.


Bugfixes

Fix a long-standing bug when using multiple event persister workers where events were not correctly sent down /sync due to a race. (#11045)
Fix a bug introduced in Synapse 1.45.0rc1 where the user directory would stop updating if it processed an event from a
user not in the users table. (#11053)
Fix a bug introduced in Synapse 1.44.0 when logging errors during oEmbed processing. (#11061)


Internal Changes

Add an 'approximate difference' method to StateFilter. (#10825)
Fix inconsistent behavior of get_last_client_by_ip when reporting data that has not been stored in the database yet. (#10970)
Fix a bug introduced in Synapse 1.21.0 that causes opentracing and Prometheus metrics for replication requests to be measured incorrectly. (#10996)
Ensure that cache config tests do not share state. (#11036)


Synapse 1.45.0rc1 (2021-10-12)