CHANGES.md



To find the state of this project's repository at the time of any of these versions, check out the tags.


Synapse 1.47.0 (2021-11-17)
No significant changes since 1.47.0rc3.

Synapse 1.47.0rc3 (2021-11-16)

Bugfixes

Fix a bug introduced in 1.47.0rc1 which caused worker processes to not halt startup in the presence of outstanding database migrations. (#11346)
Fix a bug introduced in 1.47.0rc1 which prevented the 'remove deleted devices from device_inbox column' background process from running when updating from a recent Synapse version. (#11303, #11353)


Synapse 1.47.0rc2 (2021-11-10)
This fixes an issue with publishing the Debian packages for 1.47.0rc1.
It is otherwise identical to 1.47.0rc1.

Synapse 1.47.0rc1 (2021-11-09)

Deprecations and Removals

The user_may_create_room_with_invites module callback is now deprecated. Please refer to the upgrade notes for more information. (#11206)
Remove deprecated admin API to delete rooms (POST /_synapse/admin/v1/rooms/<room_id>/delete). (#11213)


Features

Advertise support for Client-Server API r0.6.1. (#11097)
Add search by room ID and room alias to the List Room admin API. (#11099)
Add an on_new_event third-party rules callback to allow Synapse modules to act after an event has been sent into a room. (#11126)
Add a module API method to update a user's membership in a room. (#11147)
Add metrics for thread pool usage. (#11178)
Support the stable room type field for MSC3288. (#11187)
Add a module API method to retrieve the current state of a room. (#11204)
Calculate a default value for public_baseurl based on server_name. (#11210)
Add support for serving /.well-known/matrix/server files, to redirect federation traffic to port 443. (#11211)
Add admin APIs to pause, start and check the status of background updates. (#11263)


Bugfixes

Fix a long-standing bug which allowed hidden devices to receive to-device messages, resulting in unnecessary database bloat. (#10097)
Fix a long-standing bug where messages in the device_inbox table for deleted devices would persist indefinitely. Contributed by @dklimpel and @JohannesKleine. (#10969, #11212)
Do not accept events if a third-party rule check_event_allowed callback raises an exception. (#11033)
Fix long-standing bug where verification requests could fail in certain cases if a federation whitelist was in place but did not include your own homeserver. (#11129)
Allow an empty list of state_events_at_start to be sent when using the MSC2716 /batch_send endpoint and the author of the historical messages is already part of the current room state at the given ?prev_event_id. (#11188)
Fix a bug introduced in Synapse 1.45.0 which prevented the synapse_review_recent_signups script from running. Contributed by @samuel-p. (#11191)
Delete to_device messages for hidden devices that will never be read, reducing database size. (#11199)
Fix a long-standing bug wherein a missing Content-Type header when downloading remote media would cause Synapse to throw an error. (#11200)
Fix a long-standing bug which could result in serialization errors and potentially duplicate transaction data when sending ephemeral events to application services. Contributed by @Fizzadar at Beeper. (#11207)
Fix a bug introduced in Synapse 1.35.0 which made it impossible to join rooms that return a send_join response containing floats. (#11217)
Fix long-standing bug where cross signing keys were not included in the response to /r0/keys/query the first time a remote user was queried. (#11234)
Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection. (#11240)
Fix a bug preventing Synapse from being rolled back to an earlier version when using workers. (#11255, #11276)
Fix a bug introduced in Synapse 1.37.1 which caused a remote event being processed by a worker to not get processed on restart if the worker was killed. (#11262)
Only allow old Element/Riot Android clients to send read receipts without a request body. All other clients must include a request body as required by the specification. Contributed by @rogersheu. (#11157)


Updates to the Docker image

Avoid changing user ID when started as a non-root user, and no explicit UID is set. (#11209)


Improved Documentation

Improve example HAProxy config in the docs to properly handle HTTP Host headers with port information. This is required for federation over port 443 to work correctly. (#11128)
Add documentation for using Authentik as an OpenID Connect Identity Provider. Contributed by @samip5. (#11151)
Clarify lack of support for Windows. (#11198)
Improve code formatting and fix a few typos in docs. Contributed by @sumnerevans at Beeper. (#11221)
Add documentation for using LemonLDAP as an OpenID Connect Identity Provider. Contributed by @l00ptr. (#11257)


Internal Changes

Add type annotations for the log_function decorator. (#10943)
Add type hints to synapse.events. (#11098)
Remove and document unnecessary RoomStreamToken checks in application service ephemeral event code. (#11137)
Add type hints so that synapse.http passes mypy checks. (#11164)
Update scripts to pass Shellcheck lints. (#11166)
Add knock information in admin export. Contributed by Rafael Gonçalves. (#11171)
Add tests to check that ClientIpStore.get_last_client_ip_by_device and get_user_ip_and_agents combine database and in-memory data correctly. (#11179)
Refactor Filter to check different fields depending on the data type. (#11194)
Improve type hints for the relations datastore. (#11205)
Replace outdated links in the pull request checklist with links to the rendered documentation. (#11225)
Fix a bug in unit test test_block_room_and_not_purge. (#11226)
In ObservableDeferred, run observers in the order they were registered. (#11229)
Minor speed up to start up times and getting updates for groups by adding missing index to local_group_updates.stream_id. (#11231)
Add twine and towncrier as dev dependencies, as they're used by the release script. (#11233)
Allow stream_writers.typing config to be a list of one worker. (#11237)
Remove debugging statement in tests. (#11239)
Fix MSC2716 historical messages backfilling in random order on remote homeservers. (#11244)
Add an additional test for the cachedList method decorator. (#11246)
Make minor correction to the type of auth_checkers callbacks. (#11253)
Clean up trivial aspects of the Debian package build tooling. (#11269, #11273)
Blacklist new SyTest that checks that key uploads are valid pending the validation being implemented in Synapse. (#11270)


Synapse 1.46.0 (2021-11-02)
The cause of the performance regression affecting Synapse 1.44 has been identified and fixed. (#11177)

Bugfixes

Fix a bug introduced in v1.46.0rc1 where URL previews of some XML documents would fail. (#11196)


Synapse 1.46.0rc1 (2021-10-27)

Features

Add support for Ubuntu 21.10 "Impish Indri". (#11024)
Port the Password Auth Providers module interface to the new generic interface. (#10548, #11180)
Experimental support for the thread relation defined in MSC3440. (#11088, #11181, #11192)
Users admin API can now also modify user type in addition to allowing it to be set on user creation. (#11174)


Bugfixes

Newly-created public rooms are now only assigned an alias if the room's creation has not been blocked by permission settings. Contributed by @AndrewFerr. (#10930)
Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state. (#11001, #11009, #11012)
Fix 500 error on /messages when the server accumulates more than 5 backwards extremities at a given depth for a room. (#11027)
Fix a bug where setting a user's external_id via the admin API returns 500 and deletes user's existing external mappings if that external ID is already mapped. (#11051)
Fix a long-standing bug where users excluded from the user directory were added into the directory if they belonged to a room which became public or private. (#11075)
Fix a long-standing bug when attempting to preview URLs which are in the windows-1252 character encoding. (#11077, #11089)
Fix broken export-data admin command and add test script checking the command to CI. (#11078)
Show an error when timestamp in seconds is provided to the /purge_media_cache Admin API. (#11101)
Fix local users who left all their rooms being removed from the user directory, even if the search_all_users config option was enabled. (#11103)
Fix a bug which caused the module API's get_user_ip_and_agents function to always fail on workers. get_user_ip_and_agents was introduced in 1.44.0 and did not function correctly on worker processes at the time. (#11112)
Identity server connection is no longer ignoring ip_range_whitelist. (#11120)
Fix a bug introduced in Synapse 1.45.0 breaking the configuration file parsing script. (#11145)
Fix a performance regression introduced in 1.44.0 which could cause client requests to time out when making large numbers of outbound requests. (#11177, #11190)
Resolve and share state_groups for all MSC2716 historical events in batch. (#10975)


Improved Documentation

Fix broken links relating to module API deprecation in the upgrade notes. (#11069)
Add more information about what happens when a user is deactivated. (#11083)
Clarify the the sample log config can be copied from the documentation without issue. (#11092)
Update the admin API documentation with an updated list of the characters allowed in registration tokens. (#11093)
Document Synapse's behaviour when dealing with multiple modules registering the same callbacks and/or handlers for the same HTTP endpoints. (#11096)
Fix instances of [example]{.title-ref} in the upgrade documentation as a result of prior RST to Markdown conversion. (#11118)
Document the version of Synapse each module callback was introduced in. (#11132)
Document the version of Synapse that introduced each module API method. (#11183)


Internal Changes

Fix spurious warnings about losing the logging context on the ReplicationCommandHandler when losing the replication connection. (#10984)
Include rejected status when we log events. (#11008)
Add some extra logging to the event persistence code. (#11014)
Rearrange the internal workings of the incremental user directory updates. (#11035)
Fix a long-standing bug where users excluded from the directory could still be added to the users_who_share_private_rooms table after a regular user joins a private room. (#11143)
Add and improve type hints. (#10972, #11055, #11066, #11076, #11095, #11109, #11121, #11146)
Mark the Synapse package as containing type annotations and fix export declarations so that Synapse pluggable modules may be type checked against Synapse. (#11054)
Remove dead code from MediaFilePaths. (#11056)
Be more lenient when parsing oEmbed response versions. (#11065)
Create a separate module for the retention configuration. (#11070)
Clean up some of the federation event authentication code for clarity. (#11115, #11116, #11122)
Add docstrings and comments to the application service ephemeral event sending code. (#11138)
Update the sign_json script to support inline configuration of the signing key. (#11139)
Fix broken link in the docker image README. (#11144)
Always dump logs from unit tests during CI runs. (#11068)
Add tests for MediaFilePaths class. (#11057)
Simplify the user admin API tests. (#11048)
Add a test for the workaround introduced in #11042 concerning the behaviour of third-party rule modules and SynapseErrors. (#11071)


Synapse 1.45.1 (2021-10-20)

Bugfixes

Revert change to counting of deactivated users towards the monthly active users limit, introduced in 1.45.0rc1. (#11127)


Synapse 1.45.0 (2021-10-19)
No functional changes since Synapse 1.45.0rc2.

Known Issues


A suspected performance regression which was first reported after the release of 1.44.0 remains unresolved.
We have not been able to identify a probable cause. Affected users report that setting up a federation sender worker appears to alleviate symptoms of the regression.


Improved Documentation

Reword changelog to clarify concerns about a suspected performance regression in 1.44.0. (#11117)


Synapse 1.45.0rc2 (2021-10-14)
This release candidate fixes a user directory bug present in 1.45.0rc1.

Known Issues


A suspected performance regression which was first reported after the release of 1.44.0 remains unresolved.
We have not been able to identify a probable cause. Affected users report that setting up a federation sender worker appears to alleviate symptoms of the regression.


Bugfixes

Fix a long-standing bug when using multiple event persister workers where events were not correctly sent down /sync due to a race. (#11045)
Fix a bug introduced in Synapse 1.45.0rc1 where the user directory would stop updating if it processed an event from a
user not in the users table. (#11053)
Fix a bug introduced in Synapse 1.44.0 when logging errors during oEmbed processing. (#11061)


Internal Changes

Add an 'approximate difference' method to StateFilter. (#10825)
Fix inconsistent behavior of get_last_client_by_ip when reporting data that has not been stored in the database yet. (#10970)
Fix a bug introduced in Synapse 1.21.0 that causes opentracing and Prometheus metrics for replication requests to be measured incorrectly. (#10996)
Ensure that cache config tests do not share state. (#11036)


Synapse 1.45.0rc1 (2021-10-12)
Note: Media storage providers module that read from Synapse's configuration need changes as of this version, see the upgrade notes for more information.

Known Issues

We are investigating a performance issue which was reported after the release of 1.44.0.
We are aware of a bug with the user directory when using application services. A second release candidate is expected which will resolve this.


Features

Add MSC3069 support to /account/whoami. (#9655)
Support autodiscovery of oEmbed previews. (#10822)
Add a user_may_send_3pid_invite spam checker callback for modules to allow or deny 3PID invites. (#10894)
Add a spam checker callback to allow or deny room joins. (#10910)
Include an update_synapse_database script in the distribution. Contributed by @Fizzadar at Beeper. (#10954)
Include exception information in JSON logging output. Contributed by @Fizzadar at Beeper. (#11028)


Bugfixes

Fix a minor bug in the response to /_matrix/client/r0/voip/turnServer. Contributed by @lukaslihotzki. (#10922)
Fix a bug where empty yyyy-mm-dd/ directories would be left behind in the media store's url_cache_thumbnails/ directory. (#10924)
Fix a bug introduced in Synapse v1.40.0 where the signature checks for room version 8 and 9 could be applied to earlier room versions in some situations. (#10927)
Fix a long-standing bug wherein deactivated users still count towards the monthly active users limit. (#10947)
Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state. (#10956)
Fix a long-standing bug where rebuilding the user directory wouldn't exclude support and deactivated users. (#10960)
Fix MSC2716 /batch_send endpoint rejecting subsequent batches with unknown batch ID error in existing room versions from the room creator. (#10962)
Fix a bug that could leak local users' per-room nicknames and avatars when the user directory is rebuilt. (#10981)
Fix a long-standing bug where the remainder of a batch of user directory changes would be silently dropped if the server left a room early in the batch. (#10982)
Correct a bugfix introduced in Synapse v1.44.0 that would catch the wrong error if a connection is lost before a response could be written to it. (#10995)
Fix a long-standing bug where local users' per-room nicknames/avatars were visible to anyone who could see you in the user directory. (#11002)
Fix a long-standing bug where a user's per-room nickname/avatar would overwrite their profile in the user directory when a room was made public. (#11003)
Work around a regression, introduced in Synapse v1.39.0, that caused SynapseErrors raised by the experimental third-party rules module callback check_event_allowed to be ignored. (#11042)
Fix a bug in MSC2716 insertion events in rooms that could cause cross-talk/conflicts between batches. (#10877)


Improved Documentation

Change wording ("reference homeserver") in Synapse repository documentation. Contributed by @maxkratz. (#10971)
Fix a dead URL in development documentation (SAML) and change wording from "Riot" to "Element". Contributed by @maxkratz. (#10973)
Add additional content to the Welcome and Overview page of the documentation. (#10990)
Update links to MSCs in documentation. Contributed by @dklimpel. (#10991)


Internal Changes

Improve type hinting in synapse.util. (#10888)
Add further type hints to synapse.storage.util. (#10892)
Fix type hints to be compatible with an upcoming change to Twisted. (#10895)
Update utility code to handle C implementations of frozendict. (#10902)
Drop old functionality which maintained database compatibility with Synapse versions before v1.31. (#10903)
Clean-up configuration helper classes for the ServerConfig class. (#10915)
Use direct references to config flags. (#10916, #10959, #10985)
Clean up some of the federation event authentication code for clarity. (#10926, #10940, #10986, #10987, #10988, #11010, #11011)
Refactor various parts of the codebase to use RoomVersion objects instead of room version identifier strings. (#10934)
Refactor user directory tests in preparation for upcoming changes. (#10935)
Include the event id in the logcontext when handling PDUs received over federation. (#10936)
Fix logged errors in unit tests. (#10939)
Fix a broken test to ensure that consent configuration works during registration. (#10945)
Add type hints to filtering classes. (#10958)
Add type-hint to HomeserverTestcase.setup_test_homeserver. (#10961)
Fix the test utility function create_room_as so that is_public=True will explicitly set the visibility parameter of room creation requests to public. Contributed by @AndrewFerr. (#10963)
Make the release script more robust and transparent. (#10966)
Refactor MSC2716 /batch_send mega function into smaller handler functions. (#10974)
Log stack traces when a missing opentracing span is detected. (#10983)
Update GHA config to run tests against Python 3.10 and PostgreSQL 14. (#10992)
Fix a long-standing bug where ReadWriteLocks could drop logging contexts on exit. (#10993)
Add a CODEOWNERS file to automatically request reviews from the @matrix-org/synapse-core team on new pull requests. (#10994)
Add further type hints to synapse.state. (#11004)
Remove the deprecated BaseHandler object. (#11005)
Bump mypy version for CI to 0.910, and pull in new type stubs for dependencies. (#11006)
Fix CI to run the unit tests without optional deps. (#11017)
Ensure that cache config tests do not share state. (#11019)
Add additional type hints to synapse.server_notices. (#11021)
Add additional type hints for synapse.push. (#11023)
When installing the optional developer dependencies, also include the dependencies needed for type-checking and unit testing. (#11034)
Remove unnecessary list comprehension from synapse_port_db to satisfy code style requirements. (#11043)


Synapse 1.44.0 (2021-10-05)
No significant changes since 1.44.0rc3.

Synapse 1.44.0rc3 (2021-10-04)

Bugfixes

Fix a bug introduced in Synapse v1.40.0 where changing a user's display name or avatar in a restricted room would cause an authentication error. (#10933)
Fix /admin/whois/{user_id} endpoint, which was broken in v1.44.0rc1. (#10968)


Synapse 1.44.0rc2 (2021-09-30)

Bugfixes

Fix a bug introduced in v1.44.0rc1 which caused the experimental MSC2716 /batch_send endpoint to return a 500 error. (#10938)
Fix a bug introduced in v1.44.0rc1 which prevented sending presence events to application services. (#10944)


Improved Documentation

Minor updates to the installation instructions. (#10919)


Synapse 1.44.0rc1 (2021-09-29)

Features

Only allow the MSC2716 /batch_send?chunk_id=xxx endpoint to connect to an already existing insertion event. (#10776)
Improve oEmbed URL previews by processing the author name, photo, and video information. (#10814, #10819)
Speed up responding with large JSON objects to requests. (#10868, #10905)
Add a user_may_create_room_with_invites spam checker callback to allow modules to allow or deny a room creation request based on the invites and/or 3PID invites it includes. (#10898)


Bugfixes

Fix a long-standing bug that caused an AssertionError when purging history in certain rooms. Contributed by @Kokokokoka. (#10690)
Fix a long-standing bug which caused deactivated users that were later reactivated to be missing from the user directory. (#10782)
Fix a long-standing bug that caused unbanning a user by sending a membership event to fail. Contributed by @aaronraimist. (#10807)
Fix a long-standing bug where logging contexts would go missing when federation requests time out. (#10810)
Fix a long-standing bug causing an error in the deprecated /initialSync endpoint when using the undocumented from and to parameters. (#10827)
Fix a bug causing the remove_stale_pushers background job to repeatedly fail and log errors. This bug affected Synapse servers that had been upgraded from version 1.28 or older and are using SQLite. (#10843)
Fix a long-standing bug in Unicode support of the room search admin API breaking search for rooms with non-ASCII characters. (#10859)
Fix a bug introduced in Synapse 1.37.0 which caused knock membership events which we sent to remote servers to be incorrectly stored in the local database. (#10873)
Fix invalidating one-time key count cache after claiming keys. The bug was introduced in Synapse v1.41.0. Contributed by Tulir at Beeper. (#10875)
Fix a long-standing bug causing application service users to be subject to MAU blocking if the MAU limit had been reached, even if configured not to be blocked. (#10881)
Fix a long-standing bug which could cause events pulled over federation to be incorrectly rejected. (#10907)
Fix a long-standing bug causing URL cache files to be stored in storage providers. Server admins may safely delete the url_cache/ and url_cache_thumbnails/ directories from any configured storage providers to reclaim space. (#10911)
Fix a long-standing bug leading to race conditions when creating media store and config directories. (#10913)


Improved Documentation

Fix some crashes in the Module API example code, by adding JSON encoding/decoding. (#10845)
Add developer documentation about experimental configuration flags. (#10865)
Properly remove deleted files from GitHub pages when generating the documentation. (#10869)


Internal Changes

Fix GitHub Actions config so we can run sytest on synapse from parallel branches. (#10659)
Split out MSC2716 meta events to their own fields in the /batch_send response. (#10777)
Add missing type hints to REST servlets. (#10785, #10817)
Simplify the internal logic which maintains the user directory database tables. (#10796)
Use direct references to config flags. (#10812, #10885, #10893, #10897)
Specify the type of token in generic "Invalid token" error messages. (#10815)
Make StateFilter frozen so it is hashable. (#10816)
Fix a long-standing bug where an m.room.message event containing a null byte would cause an internal server error. (#10820)
Add type hints to the state database. (#10823)
Opt out of cache expiry for get_users_who_share_room_with_user, to hopefully improve /sync performance when you
haven't synced recently. (#10826)
Track cache eviction rates more finely in Prometheus's monitoring. (#10829)
Add missing type hints to synapse.handlers. (#10831, #10856)
Extend the Module API to let plug-ins check whether an ID is local and to access IP + User Agent data. (#10833)
Factor out PNG image data to a constant to be used in several tests. (#10834)
Add a test to ensure state events sent by modules get persisted correctly. (#10835)
Rename MSC2716 fields and event types from chunk to batch to match the /batch_send endpoint. (#10838)
Rename MSC2716 /batch_send query parameter from ?prev_event to more obvious usage with ?prev_event_id. (#10839)
Add type hints to synapse.http.site. (#10867)
Include outlier status when we log V2 or V3 events. (#10879)
Break down Grafana's cache expiry time series based on reason for eviction, c.f. #10829. (#10880)
Clean up some of the federation event authentication code for clarity. (#10883, #10884, #10896, #10901)
Allow the . and ~ characters when creating registration tokens as per the change to MSC3231. (#10887)
Clean up some unnecessary parentheses in places around the codebase. (#10889)
Improve type hinting in the user directory code. (#10891)
Update development testing script test_postgresql.sh to use a supported Python version and make re-runs quicker. (#10906)
Document and summarize changes in schema version 61 – 64. (#10917)
Update release script to sign the newly created git tags. (#10925)
Fix Debian builds due to dh-virtualenv no longer being able to build their docs. (#10931)


Synapse 1.43.0 (2021-09-21)
This release drops support for the deprecated, unstable API for MSC2858 (Multiple SSO Identity Providers), as well as the undocumented experimental.msc2858_enabled config option. Client authors should update their clients to use the stable API, available since Synapse 1.30.
The documentation has been updated with configuration for routing /spaces, /hierarchy and /summary to workers. See the upgrade notes for more details.
No significant changes since 1.43.0rc2.

Synapse 1.43.0rc2 (2021-09-17)

Bugfixes

Added opentracing logging to help debug #9424. (#10828)


Synapse 1.43.0rc1 (2021-09-14)

Features

Allow room creators to send historical events specified by MSC2716 in existing room versions. (#10566)
Add config option to use non-default manhole password and keys. (#10643)
Skip final GC at shutdown to improve restart performance. (#10712)
Allow configuration of the oEmbed URLs used for URL previews. (#10714, #10759)
Prefer room version 9 for restricted rooms per the room version capabilities API. (#10772)


Bugfixes

Fix a long-standing bug where room avatars were not included in email notifications. (#10658)
Fix a bug where the ordering algorithm was skipping the origin_server_ts step in the spaces summary resulting in unstable room orderings. (#10730)
Fix edge case when persisting events into a room where there are multiple events we previously hadn't calculated auth chains for (and hadn't marked as needing to be calculated). (#10743)
Fix a bug which prevented calls to /createRoom that included the room_alias_name parameter from being handled by worker processes. (#10757)
Fix a bug which prevented user registration via SSO to require consent tracking for SSO mapping providers that don't prompt for Matrix ID selection. Contributed by @AndrewFerr. (#10733)
Only return the stripped state events for the m.space.child events in a room for the spaces summary from MSC2946. (#10760)
Properly handle room upgrades of spaces. (#10774)
Fix a bug which generated invalid homeserver config when the frontend_proxy worker type was passed to the Synapse Worker-based Complement image. (#10783)


Improved Documentation

Minor fix to the media_repository developer documentation. Contributed by @cuttingedge1109. (#10556)
Update the documentation to note that the /spaces and /hierarchy endpoints can be routed to workers. (#10648)
Clarify admin API documentation on undoing room deletions. (#10735)
Split up the modules documentation and add examples for module developers. (#10758)
Correct 2 typographical errors in the Log Contexts documentation. (#10795)
Fix a wording mistake in the sample configuration. Contributed by @bramvdnheuvel:nltrix.net. (#10804)


Deprecations and Removals

Remove the unstable MSC2858 API, including the undocumented experimental.msc2858_enabled config option. The unstable API has been deprecated since Synapse 1.35. Client authors should update their clients to use the stable API introduced in Synapse 1.30 if they have not already done so. (#10693)


Internal Changes

Add OpenTracing logging to help debug stuck messages (as described by issue #9424). (#10704)
Add type annotations to the synapse.util package. (#10601)
Ensure rooms.creator field is always populated for easy lookup in MSC2716 usage later. (#10697)
Add missing type hints to REST servlets. (#10707, #10728, #10736)
Do not include rooms with unknown room versions in the spaces summary results. (#10727)
Additional error checking for the preset field when creating a room. (#10738)
Clean up some of the federation event authentication code for clarity. (#10744, #10745, #10746, #10771, #10773, #10781)
Add an index to presence_stream to hopefully speed up startups a little. (#10748)
Refactor event size checking code to simplify searching the codebase for the origins of certain error strings that are occasionally emitted. (#10750)
Move tests relating to rooms having encryption out of the user directory tests. (#10752)
Use attrs internally for the URL preview code & update documentation. (#10753)
Minor speed ups when joining large rooms over federation. (#10754, #10755, #10756, #10780, #10784)
Add a constant for m.federate. (#10775)
Add a script to update the Debian changelog in a Docker container for systems that are not Debian-based. (#10778)
Change the format of authenticated users in logs when a user is being puppeted by and admin user. (#10779)
Remove fixed and flakey tests from the Sytest blacklist. (#10788)
Improve internal details of the user directory code. (#10789)
Use direct references to config flags. (#10798)
Ensure the Rust reporter passes type checking with jaeger-client 4.7's type annotations. (#10799)


Synapse 1.42.0 (2021-09-07)
This version of Synapse removes deprecated room-management admin APIs, removes out-of-date email pushers, and improves error handling for fallback templates for user-interactive authentication. For more information on these points, server administrators are encouraged to read the upgrade notes.
No significant changes since 1.42.0rc2.

Synapse 1.42.0rc2 (2021-09-06)

Features

Support room version 9 from MSC3375. (#10747)


Internal Changes

Print a warning when using one of the deprecated template_dir settings. (#10768)


Synapse 1.42.0rc1 (2021-09-01)

Features

Add support for MSC3231: Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. (#10142)
Add support for MSC3283: Expose enable_set_displayname in capabilities. (#10452)
Port the PresenceRouter module interface to the new generic interface. (#10524)
Add pagination to the spaces summary based on updates to MSC2946. (#10613, #10725)


Bugfixes

Validate new m.room.power_levels events. Contributed by @aaronraimist. (#10232)
Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. (#10561)
Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. (#10581, #10734)
Reject Client-Server /keys/query requests which provide device_ids incorrectly. (#10593)
Rooms with unsupported room versions are no longer returned via /sync. (#10644)
Enforce the maximum length for per-room display names and avatar URLs. (#10654)
Fix a bug which caused the synapse_user_logins_total Prometheus metric not to be correctly initialised on restart. (#10677)
Improve ServerNoticeServlet to avoid duplicate requests and add unit tests. (#10679)
Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. (#10684)
Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. (#10703)
Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. (#10713)


Improved Documentation

Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. (#10192)
Advertise https://matrix-org.github.io/synapse documentation in the README and CONTRIBUTING files. (#10595)
Fix some of the titles not rendering in the OpenID Connect documentation. (#10639)
Minor clarifications to the documentation for reverse proxies. (#10708)
Remove table of contents from the top of installation and contributing documentation pages. (#10711)


Deprecations and Removals

Remove deprecated Shutdown Room and Purge Room Admin API. (#8830)


Internal Changes

Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. (#10608)
Clean up some of the federation event authentication code for clarity. (#10614, #10615, #10624, #10640)
Add a comment asking developers to leave a reason when bumping the database schema version. (#10621)
Remove not needed database updates in modify user admin API. (#10627)
Convert room member storage tuples to attrs classes. (#10629, #10642)
Use auto-attribs for the attrs classes used in sync. (#10630)
Make backfill and get_missing_events use the same codepath. (#10645)
Improve the performance of the /hierarchy API (from MSC2946) by caching responses received over federation. (#10647)
Run a nightly CI build against Twisted trunk. (#10651, #10672)
Do not print out stack traces for network errors when fetching data over federation. (#10662)
Simplify tests for device admin rest API. (#10664)
Add missing type hints to REST servlets. (#10665, #10666, #10674)
Flatten the tests.synapse.rests package by moving the contents of v1 and v2_alpha into the parent. (#10667)
Update complement.sh to rebuild the base Docker image when run with workers. (#10686)
Split the event-processing methods in FederationHandler into a separate FederationEventHandler. (#10692)
Remove unused compare_digest function. (#10706)


Synapse 1.41.1 (2021-08-31)
Due to the two security issues highlighted below, server administrators are encouraged to update Synapse. We are not aware of these vulnerabilities being exploited in the wild.

Security advisory
The following issues are fixed in v1.41.1.


GHSA-3x4c-pq33-4w3q / CVE-2021-39164: Enumerating a private room's list of members and their display names.
If an unauthorized user both knows the Room ID of a private room and that room's history visibility is set to shared, then they may be able to enumerate the room's members, including their display names.
The unauthorized user must be on the same homeserver as a user who is a member of the target room.
Fixed by 52c7a51cf.


GHSA-jj53-8fmw-f2w2 / CVE-2021-39163: Disclosing a private room's name, avatar, topic, and number of members.
If an unauthorized user knows the Room ID of a private room, then its name, avatar, topic, and number of members may be disclosed through Group / Community features.
The unauthorized user must be on the same homeserver as a user who is a member of the target room, and their homeserver must allow non-administrators to create groups (enable_group_creation in the Synapse configuration; off by default).
Fixed by cb35df940a, #10723.


Bugfixes

Fix a regression introduced in Synapse 1.41 which broke email transmission on systems using older versions of the Twisted library. (#10713)


Synapse 1.41.0 (2021-08-24)
This release adds support for Debian 12 (Bookworm), but removes support for Ubuntu 20.10 (Groovy Gorilla), which reached End of Life last month.
Note that when using workers the /_synapse/admin/v1/users/{userId}/media must now be handled by media workers. See the upgrade notes for more information.

Features

Enable room capabilities (MSC3244) by default and set room version 8 as the preferred room version when creating restricted rooms. (#10571)


Synapse 1.41.0rc1 (2021-08-18)

Features

Add get_userinfo_by_id method to ModuleApi. (#9581)
Initial local support for MSC3266, Room Summary over the unstable /rooms/{roomIdOrAlias}/summary API. (#10394)
Experimental support for MSC3288, sending room_type to the identity server for 3pid invites over the /store-invite API. (#10435)
Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. See the upgrade notes for more information. (#10596). (#10475)
Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of MSC2716). (#10498)
Add a configuration setting for the time a /sync response is cached for. (#10513)
The default logging handler for new installations is now PeriodicallyFlushingMemoryHandler, a buffered logging handler which periodically flushes itself. (#10518)
Add support for new redaction rules for historical events specified in MSC2716. (#10538)
Add a setting to disable TLS when sending email. (#10546)
Add pagination to the spaces summary based on updates to MSC2946. (#10549, #10560, #10569, #10574, #10575, #10579, #10583)
Admin API to delete several media for a specific user. Contributed by @dklimpel. (#10558, #10628)
Add support for routing /createRoom to workers. (#10564)
Update the Synapse Grafana dashboard. (#10570)
Add an admin API (GET /_synapse/admin/username_available) to check if a username is available (regardless of registration settings). (#10578)
Allow editing a user's external_ids via the "Edit User" admin API. Contributed by @dklimpel. (#10598)
The Synapse manhole no longer needs coroutines to be wrapped in defer.ensureDeferred. (#10602)
Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. (#10638)


Bugfixes

Add some clarification to the sample config file. Contributed by @Kentokamoto. (#10129)
Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via GET /_matrix/client/r0/thirdparty/protocols. (#10532)
Fix exceptions in logs when failing to get remote room list. (#10541)
Fix longstanding bug which caused the user's presence "status message" to be reset when the user went offline. Contributed by @dklimpel. (#10550)
Allow public rooms to be previewed in the spaces summary APIs from MSC2946. (#10580)
Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. (#10592)
Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for MSC2285 enabled. Contributed by @SimonBrandner. (#10606)
Additional validation for the spaces summary API to avoid errors like ValueError: Stop argument for islice() must be None or an integer. The missing validation has existed since v1.31.0. (#10611)
Revert behaviour introduced in v1.38.0 that strips org.matrix.msc2732.device_unused_fallback_key_types from /sync when its value is empty. This field should instead always be present according to MSC2732. (#10623)


Improved Documentation

Add documentation for configuring a forward proxy. (#10443)
Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. (#10551)
Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. (#10599)


Deprecations and Removals

No longer build .deb packages for Ubuntu 20.10 Groovy Gorilla, which has now EOLed. (#10588)
The template_dir configuration settings in the sso, account_validity and email sections of the configuration file are now deprecated in favour of the global templates.custom_template_directory setting. See the upgrade notes for more information. (#10596)


Internal Changes

Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. (#10119)
Reduce errors in PostgreSQL logs due to concurrent serialization errors. (#10504)
Include room ID in ignored EDU log messages. Contributed by @ilmari. (#10507)
Add pagination to the spaces summary based on updates to MSC2946. (#10527, #10530)
Fix CI to not break when run against branches rather than pull requests. (#10529)
Mark all events stemming from the MSC2716 /batch_send endpoint as historical. (#10537)
Clean up some of the federation event authentication code for clarity. (#10539, #10591)
Convert Transaction and Edu objects to attrs. (#10542)
Update /batch_send endpoint to only return state_events created by the state_events_from_before passed in. (#10552)
Update contributing.md to warn against rebasing an open PR. (#10563)
Remove the unused public rooms replication stream. (#10565)
Clarify error message when failing to join a restricted room. (#10572)
Remove references to BuildKite in favour of GitHub Actions. (#10573)
Move /batch_send endpoint defined by MSC2716 to the /v2_alpha directory. (#10576)
Allow multiple custom directories in read_templates. (#10587)
Re-organize the synapse.federation.transport.server module to create smaller files. (#10590)
Flatten the synapse.rest.client package by moving the contents of v1 and v2_alpha into the parent. (#10600)
Build Debian packages for Debian 12 (Bookworm). (#10612)
Fix up a couple of links to the database schema documentation. (#10620)
Fix a broken link to the upgrade notes. (#10631)


Synapse 1.40.0 (2021-08-10)
No significant changes.

Synapse 1.40.0rc3 (2021-08-09)

Features

Support MSC3289: room version 8. (#10449)


Bugfixes

Mark the experimental room version from MSC2716 as unstable. (#10449)


Improved Documentation

Fix broken links in upgrade.md. Contributed by @dklimpel. (#10543)


Synapse 1.40.0rc2 (2021-08-04)

Bugfixes

Fix the PeriodicallyFlushingMemoryHandler inhibiting application shutdown because of its background thread. (#10517)
Fix a bug introduced in Synapse v1.40.0rc1 that could cause Synapse to respond with an error when clients would update read receipts. (#10531)


Internal Changes

Fix release script to open the correct URL for the release. (#10516)


Synapse 1.40.0rc1 (2021-08-03)

Features

Add support for MSC2033: device_id on /account/whoami. (#9918)
Update support for MSC2716 - Incrementally importing history into existing rooms. (#10245, #10432, #10463)
Update support for MSC3083 to consider changes in the MSC around which servers can issue join events. (#10254, #10447, #10489)
Initial support for MSC3244, Room version capabilities over the /capabilities API. (#10283)
Add a buffered logging handler which periodically flushes itself. (#10407, #10515)
Add support for https connections to a proxy server. Contributed by @Bubu and @dklimpel. (#10411)
Support for MSC2285 (hidden read receipts). Contributed by @SimonBrandner. (#10413)
Email notifications now state whether an invitation is to a room or a space. (#10426)
Allow setting transaction limit for database connections. (#10440, #10511)
Add creation_ts to "list users" admin API. (#10448)


Bugfixes

Improve character set detection in URL previews by supporting underscores (in addition to hyphens). Contributed by @srividyut. (#10410)
Fix events being incorrectly rejected over federation if they reference auth events that the server needed to fetch. (#10439)
Fix synapse_federation_server_oldest_inbound_pdu_in_staging Prometheus metric to not report a max age of 51 years when the queue is empty. (#10455)
Fix a bug which caused an explicit assignment of power-level 0 to a user to be misinterpreted in rare circumstances. (#10499)


Improved Documentation

Fix hierarchy of providers on the OpenID page. (#10445)
Consolidate development documentation to docs/development/. (#10453)
Add some developer docs to explain room DAG concepts like outliers, state_groups, depth, etc. (#10464)
Document how to use Complement while developing a new Synapse feature. (#10483)


Internal Changes

Prune inbound federation queues for a room if they get too large. (#10390)
Add type hints to synapse.federation.transport.client module. (#10408)
Remove shebang line from module files. (#10415)
Drop backwards-compatibility code that was required to support Ubuntu Xenial. (#10429)
Use a docker image cache for the prerequisites for the debian package build. (#10431)
Improve servlet type hints. (#10437, #10438)
Replace usage of or_ignore in simple_insert with simple_upsert usage, to stop spamming postgres logs with spurious ERROR messages. (#10442)
Update the tests-done Github Actions status. (#10444, #10512)
Update type annotations to work with forthcoming Twisted 21.7.0 release. (#10446, #10450)
Cancel redundant GHA workflows when a new commit is pushed. (#10451)
Mitigate media repo XSS attacks on IE11 via the non-standard X-Content-Security-Policy header. (#10468)
Additional type hints in the state handler. (#10482)
Update syntax used to run complement tests. (#10488)
Fix up type annotations to work with Twisted 21.7. (#10490)
Improve type annotations for ObservableDeferred. (#10491)
Extend release script to also tag and create GitHub releases. (#10496)
Fix a bug which caused production debian packages to be incorrectly marked as 'prerelease'. (#10500)


Synapse 1.39.0 (2021-07-29)
No significant changes.

Synapse 1.39.0rc3 (2021-07-28)

Bugfixes

Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. (#10477)
Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. (#10485)
Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. (#10486)


Internal Changes

Fix an error which prevented the Github Actions workflow to build the docker images from running. (#10461)
Fix release script to correctly version debian changelog when doing RCs. (#10465)


Synapse 1.39.0rc2 (2021-07-22)
This release also includes the changes in v1.38.1.

Internal Changes

Move docker image build to Github Actions. (#10416)


Synapse 1.38.1 (2021-07-22)

Bugfixes

Always include device_one_time_keys_count key in /sync response to work around a bug in Element Android that broke encryption for new devices. (#10457)


Synapse 1.39.0rc1 (2021-07-20)
The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the upgrade notes for more information.

Features

Add the ability to override the account validity feature with a module. (#9884)
The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. (#10298, #10305)
Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. (#10332, #10427)
Allow providing credentials to http_proxy. (#10360)


Bugfixes

Fix error while dropping locks on shutdown. Introduced in v1.38.0. (#10433)
Add base starting insertion event when no chunk ID is specified in the historical batch send API. (#10250)
Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. (#10276)
Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. (#10317)
Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). (#10343)
Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. (#10344)
Fix newly added synapse_federation_server_oldest_inbound_pdu_in_staging prometheus metric to measure age rather than timestamp. (#10355)
Fix PostgreSQL sometimes using table scans for queries against state_groups_state table, taking a long time and a large amount of IO. (#10359)
Fix make_room_admin failing for users that have left a private room. (#10367)
Fix a number of logged errors caused by remote servers being down. (#10400, #10414)
Responses from /make_{join,leave,knock} no longer include signatures, which will turn out to be invalid after events are returned to /send_{join,leave,knock}. (#10404)


Improved Documentation

Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. (#9971)
Simplify structure of room admin API. (#10313)
Refresh the logcontext dev documentation. (#10353), (#10337)
Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. (#10368)
Fix and clarify some links in docs and contrib. (#10370), (#10322), (#10399)
Make deprecation notice of the spam checker doc more obvious. (#10395)
Add instructions on installing Debian packages for release candidates. (#10396)


Deprecations and Removals

Remove functionality associated with the unused room_stats_historical and user_stats_historical tables. Contributed by @xmunoz. (#9721)
The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the upgrade notes for more information. (#10386)


Internal Changes

Convert room_depth.min_depth column to a BIGINT. (#10289)
Add tests to characterise the current behaviour of R30 phone-home metrics. (#10315)
Rebuild event context and auth when processing specific results from ThirdPartyEventRules modules. (#10316)
Minor change to the code that populates user_daily_visits. (#10324)
Re-enable Sytests that were disabled for the 1.37.1 release. (#10345, #10357)
Run pyupgrade on the codebase. (#10347, #10348)
Switch application_services_txns.txn_id database column to BIGINT. (#10349)
Convert internal type variable syntax to reflect wider ecosystem use. (#10350, #10380, #10381, #10382, #10418)
Make the Github Actions workflow configuration more efficient. (#10383)
Add type hints to get_{domain,localpart}_from_id. (#10385)
When building Debian packages for prerelease versions, set the Section accordingly. (#10391)
Add type hints and comments to event auth code. (#10393)
Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. (#10398)
Remove unused events_by_room code (tech debt). (#10421)
Add a github actions job which records success of other jobs. (#10430)


Synapse 1.38.0 (2021-07-13)
This release includes a database schema update which could result in elevated disk usage. See the upgrade notes for more information.
No significant changes since 1.38.0rc3.

Synapse 1.38.0rc3 (2021-07-13)

Internal Changes

Build the Debian packages in CI. (#10247, #10379)


Synapse 1.38.0rc2 (2021-07-09)

Bugfixes

Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. (#10336)


Improved Documentation