CHANGES.md



To find the state of this project's repository at the time of any of these versions, check out the tags.


Synapse 1.10.0rc2 (2020-02-06)

Bugfixes

Fix an issue with cross-signing where device signatures were not sent to remote servers. (#6844)
Fix to the unknown remote device detection which was introduced in 1.10.rc1. (#6848)


Internal Changes

Detect unexpected sender keys on remote encrypted events and resync device lists. (#6850)


Synapse 1.10.0rc1 (2020-01-31)
WARNING to client developers: As of this release Synapse validates client_secret parameters in the Client-Server API as per the spec. See #6766 for details.

Features

Add experimental support for updated authorization rules for aliases events, from MSC2260. (#6787, #6790, #6794)


Bugfixes

Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). (#6734)
Minor fixes to PUT /_synapse/admin/v2/users admin api. (#6761)
Validate client_secret parameter using the regex provided by the Client-Server API, temporarily allowing : characters for older clients. The : character will be removed in a future release. (#6767)
Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). (#6771)
Fix outbound federation request metrics. (#6795)
Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device. (#6796)
Fix race in federation sender worker that delayed sending of device updates. (#6799, #6800)
Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room. (#6801)
Fix waking up other workers when remote server is detected to have come back online. (#6811)


Improved Documentation

Clarify documentation related to user_dir and federation_reader workers. (#6775)


Internal Changes

Record room versions in the rooms table. (#6729, #6788, #6810)
Propagate cache invalidates from workers to other workers. (#6748)
Remove some unnecessary admin handler abstraction methods. (#6751)
Add some debugging for media storage providers. (#6757)
Detect unknown remote devices and mark cache as stale. (#6776, #6819)
Attempt to resync remote users' devices when detected as stale. (#6786)
Delete current state from the database when server leaves a room. (#6792)
When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale. (#6797)
Add background update to clean out left rooms from current state. (#6802, #6816)
Refactoring work in preparation for changing the event redaction algorithm. (#6803, #6805, #6806, #6807, #6820)


Synapse 1.9.1 (2020-01-28)

Bugfixes

Fix bug where setting mau_limit_reserved_threepids config would cause Synapse to refuse to start. (#6793)


Synapse 1.9.0 (2020-01-23)
WARNING: As of this release, Synapse no longer supports versions of SQLite before 3.11, and will refuse to start when configured to use an older version. Administrators are recommended to migrate their database to Postgres (see instructions here).
If your Synapse deployment uses workers, note that the reverse-proxy configurations for the synapse.app.media_repository, synapse.app.federation_reader and synapse.app.event_creator workers have changed, with the addition of a few paths (see the updated configurations here). Existing configurations will continue to work.

Improved Documentation

Fix endpoint documentation for the List Rooms admin API. (#6770)


Synapse 1.9.0rc1 (2020-01-22)

Features

Allow admin to create or modify a user. Contributed by Awesome Technologies Innovationslabor GmbH. (#5742)
Add new quarantine media admin APIs to quarantine by media ID or by user who uploaded the media. (#6681, #6756)
Add org.matrix.e2e_cross_signing to unstable_features in /versions as per MSC1756. (#6712)
Add a new admin API to list and filter rooms on the server. (#6720)


Bugfixes

Correctly proxy HTTP errors due to API calls to remote group servers. (#6654)
Fix media repo admin APIs when using a media worker. (#6664)
Fix "CRITICAL" errors being logged when a request is received for a uri containing non-ascii characters. (#6682)
Fix a bug where we would assign a numeric user ID if somebody tried registering with an empty username. (#6690)
Fix purge_room admin API. (#6711)
Fix a bug causing Synapse to not always purge quiet rooms with a low max_lifetime in their message retention policies when running the automated purge jobs. (#6714)
Fix the synapse_port_db not correctly running background updates. Thanks @tadzik for reporting. (#6718)
Fix changing password via user admin API. (#6730)
Fix /events/:event_id deprecated API. (#6731)
Fix monthly active user limiting support for worker mode, fixes #4639. (#6742)
Fix bug when setting account_validity to an empty block in the config. Thanks to @Sorunome for reporting. (#6747)
Fix AttributeError: 'NoneType' object has no attribute 'get' in hash_password when configuration has an empty password_config. Contributed by @ivilata. (#6753)
Fix the docker-compose.yaml overriding the entire /etc folder of the container. Contributed by Fabian Meyer. (#6656)


Improved Documentation

Fix a typo in the configuration example for purge jobs in the sample configuration file. (#6621)
Add complete documentation of the message retention policies support. (#6624, #6665)
Add some helpful tips about changelog entries to the GitHub pull request template. (#6663)
Clarify the account_validity and email sections of the sample configuration. (#6685)
Add more endpoints to the documentation for Synapse workers. (#6698)


Deprecations and Removals

Synapse no longer supports versions of SQLite before 3.11, and will refuse to start when configured to use an older version. Administrators are recommended to migrate their database to Postgres (see instructions here). (#6675)


Internal Changes

Add local_current_membership table for tracking local user membership state in rooms. (#6655, #6728)
Port synapse.replication.tcp to async/await. (#6666)
Fixup synapse.replication to pass mypy checks. (#6667)
Allow additional_resources to implement IResource directly. (#6686)
Allow REST endpoint implementations to raise a RedirectException, which will redirect the user's browser to a given location. (#6687)
Updates and extensions to the module API. (#6688)
Updates to the SAML mapping provider API. (#6689, #6723)
Remove redundant RegistrationError class. (#6691)
Don't block processing of incoming EDUs behind processing PDUs in the same transaction. (#6697)
Remove duplicate check for the session query parameter on the /auth/xxx/fallback/web Client-Server endpoint. (#6702)
Attempt to retry sending a transaction when we detect a remote server has come back online, rather than waiting for a transaction to be triggered by new data. (#6706)
Add StateMap type alias to simplify types. (#6715)
Add a DeltaState to track changes to be made to current state during event persistence. (#6716)
Add more logging around message retention policies support. (#6717)
When processing a SAML response, log the assertions for easier configuration. (#6724)
Fixup synapse.rest to pass mypy. (#6732, #6764)
Fixup synapse.api to pass mypy. (#6733)
Allow streaming cache 'invalidate all' to workers. (#6749)
Remove unused CI docker compose files. (#6754)


Synapse 1.8.0 (2020-01-09)
WARNING: As of this release Synapse will refuse to start if the log_file config option is specified. Support for the option was removed in v1.3.0.

Bugfixes

Fix GET request on /_synapse/admin/v2/users endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. (#6563)
Fix incorrect signing of responses from the key server implementation. (#6657)


Synapse 1.8.0rc1 (2020-01-07)

Features

Add v2 APIs for the send_join and send_leave federation endpoints (as described in MSC1802). (#6349)
Add a develop script to generate full SQL schemas. (#6394)
Add custom SAML username mapping functionality through an external provider plugin. (#6411)
Automatically delete empty groups/communities. (#6453)
Add option limit_profile_requests_to_users_who_share_rooms to prevent requirement of a local user sharing a room with another user to query their profile information. (#6523)
Add an export_signing_key script to extract the public part of signing keys when rotating them. (#6546)
Add experimental config option to specify multiple databases. (#6580)
Raise an error if someone tries to use the log_file config option. (#6626)


Bugfixes

Prevent redacted events from being returned during message search. (#6377, #6522)
Prevent error on trying to search a upgraded room when the server is not in the predecessor room. (#6385)
Improve performance of looking up cross-signing keys. (#6486)
Fix race which occasionally caused deleted devices to reappear. (#6514)
Fix missing row in device_max_stream_id that could cause unable to decrypt errors after server restart. (#6555)
Fix a bug which meant that we did not send systemd notifications on startup if acme was enabled. (#6571)
Fix exception when fetching the matrix.org:ed25519:auto key. (#6625)
Fix bug where a moderator upgraded a room and became an admin in the new room. (#6633)
Fix an error which was thrown by the PresenceHandler _on_shutdown handler. (#6640)
Fix exceptions in the synchrotron worker log when events are rejected. (#6645)
Ensure that upgraded rooms are removed from the directory. (#6648)
Fix a bug causing Synapse not to fetch missing events when it believes it has every event in the room. (#6652)


Improved Documentation

Document the Room Shutdown Admin API. (#6541)
Reword sections of docs/federate.md that explained delegation at time of Synapse 1.0 transition. (#6601)
Added the section 'Configuration' in docs/turn-howto.md. (#6614)


Deprecations and Removals

Remove redundant code from event authorisation implementation. (#6502)
Remove unused, undocumented /_matrix/content API. (#6628)


Internal Changes

Add experimental support for multiple physical databases and split out state storage to separate data store. (#6245, #6510, #6511, #6513, #6564, #6565)
Port sections of code base to async/await. (#6496, #6504, #6505, #6517, #6559, #6647, #6653)
Remove SnapshotCache in favour of ResponseCache. (#6506)
Silence mypy errors for files outside those specified. (#6512)
Clean up some logging when handling incoming events over federation. (#6515)
Test more folders against mypy. (#6534)
Update mypy to new version. (#6537)
Adjust the sytest blacklist for worker mode. (#6538)
Remove unused get_pagination_rows methods from EventSource classes. (#6557)
Clean up logs from the push notifier at startup. (#6558)
Improve diagnostics on database upgrade failure. (#6570)
Reduce the reconnect time when worker replication fails, to make it easier to catch up. (#6617)
Simplify http handling by removing redundant SynapseRequestFactory. (#6619)
Add a workaround for synapse raising exceptions when fetching the notary's own key from the notary. (#6620)
Automate generation of the sample log config. (#6627)
Simplify event creation code by removing redundant queries on the event_reference_hashes table. (#6629)
Fix errors when frozen_dicts are enabled. (#6642)


Synapse 1.7.3 (2019-12-31)
This release fixes a long-standing bug in the state resolution algorithm.

Bugfixes

Fix exceptions caused by state resolution choking on malformed events. (#6608)


Synapse 1.7.2 (2019-12-20)
This release fixes some regressions introduced in Synapse 1.7.0 and 1.7.1.

Bugfixes

Fix a regression introduced in Synapse 1.7.1 which caused errors when attempting to backfill rooms over federation. (#6576)
Fix a bug introduced in Synapse 1.7.0 which caused an error on startup when upgrading from versions before 1.3.0. (#6578)


Synapse 1.7.1 (2019-12-18)
This release includes several security fixes as well as a fix to a bug exposed by the security fixes. Administrators are encouraged to upgrade as soon as possible.

Security updates

Fix a bug which could cause room events to be incorrectly authorized using events from a different room. (#6501, #6503, #6521, #6524, #6530, #6531)
Fix a bug causing responses to the /context client endpoint to not use the pruned version of the event. (#6553)
Fix a cause of state resets in room versions 2 onwards. (#6556, #6560)


Bugfixes

Fix a bug which could cause the federation server to incorrectly return errors when handling certain obscure event graphs. (#6526, #6527)


Synapse 1.7.0 (2019-12-13)
This release changes the default settings so that only local authenticated users can query the server's room directory. See the upgrade notes for details.
Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11.
Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available here. A future release of synapse will, by default, disable federation for servers using SQLite.
No significant changes since 1.7.0rc2.

Synapse 1.7.0rc2 (2019-12-11)

Bugfixes

Fix incorrect error message for invalid requests when setting user's avatar URL. (#6497)
Fix support for SQLite 3.7. (#6499)
Fix regression where sending email push would not work when using a pusher worker. (#6507, #6509)


Synapse 1.7.0rc1 (2019-12-09)

Features

Implement per-room message retention policies. (#5815, #6436)
Add etag and count fields to key backup endpoints to help clients guess if there are new keys. (#5858)
Add /admin/v2/users endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. (#5925)
Require User-Interactive Authentication for /account/3pid/add, meaning the user's password will be required to add a third-party ID to their account. (#6119)
Implement the /_matrix/federation/unstable/net.atleastfornow/state/<context> API as drafted in MSC2314. (#6176)
Configure privacy-preserving settings by default for the room directory. (#6355)
Add ephemeral messages support by partially implementing MSC2228. (#6409)
Add support for MSC 2367, which allows specifying a reason on all membership events. (#6434)


Bugfixes

Transfer non-standard power levels on room upgrade. (#6237)
Fix error from the Pillow library when uploading RGBA images. (#6241)
Correctly apply the event filter to the state, events_before and events_after fields in the response to /context requests. (#6329)
Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. (#6332)
Prevent account data syncs getting lost across TCP replication. (#6333)
Fix bug: TypeError in register_user() while using LDAP auth module. (#6406)
Fix an intermittent exception when handling read-receipts. (#6408)
Fix broken guest registration when there are existing blocks of numeric user IDs. (#6420)
Fix startup error when http proxy is defined. (#6421)
Fix error when using synapse_port_db on a vanilla synapse db. (#6449)
Fix uploading multiple cross signing signatures for the same user. (#6451)
Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. (#6462)
Fix synapse_port_db not exiting with a 0 code if something went wrong during the port process. (#6470)
Improve sanity-checking when receiving events over federation. (#6472)
Fix inaccurate per-block Prometheus metrics. (#6491)
Fix small performance regression for sending invites. (#6493)
Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. (#6494)


Improved Documentation

Update documentation and variables in user contributed systemd reference file. (#6369, #6490)
Fix link in the user directory documentation. (#6388)
Add build instructions to the docker readme. (#6390)
Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. (#6443)
Write some docs for the quarantine_media api. (#6458)
Convert CONTRIBUTING.rst to markdown (among other small fixes). (#6461)


Deprecations and Removals

Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. (#5925)
Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. (#6488)


Internal Changes

Add benchmarks for structured logging and improve output performance. (#6266)
Improve the performance of outputting structured logging. (#6322)
Refactor some code in the event authentication path for clarity. (#6343, #6468, #6480)
Clean up some unnecessary quotation marks around the codebase. (#6362)
Complain on startup instead of 500'ing during runtime when public_baseurl isn't set when necessary. (#6379)
Add a test scenario to make sure room history purges don't break /messages in the future. (#6392)
Clarifications for the email configuration settings. (#6423)
Add more tests to the blacklist when running in worker mode. (#6429)
Refactor data store layer to support multiple databases in the future. (#6454, #6464, #6469, #6487)
Port synapse.rest.client.v1 to async/await. (#6482)
Port synapse.rest.client.v2_alpha to async/await. (#6483)
Port SyncHandler to async/await. (#6484)


Synapse 1.6.1 (2019-11-28)

Security updates
This release includes a security fix (#6426, below). Administrators are encouraged to upgrade as soon as possible.

Bugfixes

Clean up local threepids from user on account deactivation. (#6426)
Fix startup error when http proxy is defined. (#6421)


Synapse 1.6.0 (2019-11-26)

Bugfixes

Fix phone home stats reporting. (#6418)


Synapse 1.6.0rc2 (2019-11-25)

Bugfixes

Fix a bug which could cause the background database update hander for event labels to get stuck in a loop raising exceptions. (#6407)


Synapse 1.6.0rc1 (2019-11-20)

Features

Add federation support for cross-signing. (#5727)
Increase default room version from 4 to 5, thereby enforcing server key validity period checks. (#6220)
Add support for outbound http proxying via http_proxy/HTTPS_PROXY env vars. (#6238)
Implement label-based filtering on /sync and /messages (MSC2326). (#6301, #6310, #6340)


Bugfixes

Fix LruCache callback deduplication for Python 3.8. Contributed by @V02460. (#6213)
Remove a room from a server's public rooms list on room upgrade. (#6232, #6235)
Delete keys from key backup when deleting backup versions. (#6253)
Make notification of cross-signing signatures work with workers. (#6254)
Fix exception when remote servers attempt to join a room that they're not allowed to join. (#6278)
Prevent errors from appearing on Synapse startup if git is not installed. (#6284)
Appservice requests will no longer contain a double slash prefix when the appservice url provided ends in a slash. (#6306)
Fix /purge_room admin API. (#6307)
Fix the hidden field in the devices table for SQLite versions prior to 3.23.0. (#6313)
Fix bug which casued rejected events to be persisted with the wrong room state. (#6320)
Fix bug where rc_login ratelimiting would prematurely kick in. (#6335)
Prevent the server taking a long time to start up when guest registration is enabled. (#6338)
Fix bug where upgrading a guest account to a full user would fail when account validity is enabled. (#6359)
Fix to_device stream ID getting reset every time Synapse restarts, which had the potential to cause unable to decrypt errors. (#6363)
Fix permission denied error when trying to generate a config file with the docker image. (#6389)


Improved Documentation

Contributor documentation now mentions script to run linters. (#6164)
Modify CAPTCHA_SETUP.md to update the terms private key and public key to secret key and site key respectively. Contributed by Yash Jipkate. (#6257)
Update INSTALL.md Email section to talk about account_threepid_delegates. (#6272)
Fix a small typo in account_threepid_delegates configuration option. (#6273)


Internal Changes

Add a CI job to test the synapse_port_db script. (#6140, #6276)
Convert EventContext to an attrs. (#6218)
Move persist_events out from main data store. (#6240, #6300)
Reduce verbosity of user/room stats. (#6250)
Reduce impact of debug logging. (#6251)
Expose some homeserver functionality to spam checkers. (#6259)
Change cache descriptors to always return deferreds. (#6263, #6291)
Fix incorrect comment regarding the functionality of an if statement. (#6269)
Update CI to run isort over the scripts and scripts-dev directories. (#6270)
Replace every instance of logger.warn method with logger.warning as the former is deprecated. (#6271, #6314)
Port replication http server endpoints to async/await. (#6274)
Port room rest handlers to async/await. (#6275)
Remove redundant CLI parameters on CI's flake8 step. (#6277)
Port federation_server.py to async/await. (#6279)
Port receipt and read markers to async/wait. (#6280)
Split out state storage into separate data store. (#6294, #6295)
Refactor EventContext for clarity. (#6298)
Update the version of black used to 19.10b0. (#6304)
Add some documentation about worker replication. (#6305)
Move admin endpoints into separate files. Contributed by Awesome Technologies Innovationslabor GmbH. (#6308)
Document the use of lint.sh for code style enforcement & extend it to run on specified paths only. (#6312)
Add optional python dependencies and dependant binary libraries to snapcraft packaging. (#6317)
Remove the dependency on psutil and replace functionality with the stdlib resource module. (#6318, #6336)
Improve documentation for EventContext fields. (#6319)
Add some checks that we aren't using state from rejected events. (#6330)
Add continuous integration for python 3.8. (#6341)
Correct spacing/case of various instances of the word "homeserver". (#6357)
Temporarily blacklist the failing unit test PurgeRoomTestCase.test_purge_room. (#6361)


Synapse 1.5.1 (2019-11-06)

Features

Limit the length of data returned by url previews, to prevent DoS attacks. (#6331, #6334)


Synapse 1.5.0 (2019-10-29)

Security updates
This release includes a security fix (#6262, below). Administrators are encouraged to upgrade as soon as possible.

Bugfixes

Fix bug where room directory search was case sensitive. (#6268)


Synapse 1.5.0rc2 (2019-10-28)

Bugfixes

Update list of boolean columns in synapse_port_db. (#6247)
Fix /keys/query API on workers. (#6256)
Improve signature checking on some federation APIs. (#6262)


Internal Changes

Move schema delta files to the correct data store. (#6248)
Small performance improvement by removing repeated config lookups in room stats calculation. (#6255)


Synapse 1.5.0rc1 (2019-10-24)

Features

Improve quality of thumbnails for 1-bit/8-bit color palette images. (#2142)
Add ability to upload cross-signing signatures. (#5726)
Allow uploading of cross-signing keys. (#5769)
CAS login now provides a default display name for users if a displayname_attribute is set in the configuration file. (#6114)
Reject all pending invites for a user during deactivation. (#6125)
Add config option to suppress client side resource limit alerting. (#6173)


Bugfixes

Return an HTTP 404 instead of 400 when requesting a filter by ID that is unknown to the server. Thanks to @krombel for contributing this! (#2380)
Fix a bug where users could be invited twice to the same group. (#3436)
Fix /createRoom failing with badly-formatted MXIDs in the invitee list. Thanks to @wener291! (#4088)
Make the synapse_port_db script create the right indexes on a new PostgreSQL database. (#6102, #6178, #6243)
Fix bug when uploading a large file: Synapse responds with M_UNKNOWN while it should be M_TOO_LARGE according to spec. Contributed by Anshul Angaria. (#6109)
Fix user push rules being deleted from a room when it is upgraded. (#6144)
Don't 500 when trying to exchange a revoked 3PID invite. (#6147)
Fix transferring notifications and tags when joining an upgraded room that is new to your server. (#6155)
Fix bug where guest account registration can wedge after restart. (#6161)
Fix monthly active user reaping when reserved users are specified. (#6168)
Fix /federation/v1/state endpoint not supporting newer room versions. (#6170)
Fix bug where we were updating censored events as bytes rather than text, occaisonally causing invalid JSON being inserted breaking APIs that attempted to fetch such events. (#6186)
Fix occasional missed updates in the room and user directories. (#6187)
Fix tracing of non-JSON APIs, /media, /key etc. (#6195)
Fix bug where presence would not get timed out correctly if a synchrotron worker is used and restarted. (#6212)
synapse_port_db: Add 2 additional BOOLEAN_COLUMNS to be able to convert from database schema v56. (#6216)
Fix a bug where the Synapse demo script blacklisted ::1 (ipv6 localhost) from receiving federation traffic. (#6229)


Updates to the Docker image

Fix logging getting lost for the docker image. (#6197)


Internal Changes

Update user_filters table to have a unique index, and non-null columns. Thanks to @pik for contributing this. (#1172, #6175, #6184)
Allow devices to be marked as hidden, for use by features such as cross-signing.
This adds a new field with a default value to the devices field in the database,
and so the database upgrade may take a long time depending on how many devices
are in the database. (#5759)
Move lookup-related functions from RoomMemberHandler to IdentityHandler. (#5978)
Improve performance of the public room list directory. (#6019, #6152, #6153, #6154)
Edit header dicts docstrings in SimpleHttpClient to note that str or bytes can be passed as header keys. (#6077)
Add snapcraft packaging information. Contributed by @devec0. (#6084, #6191)
Kill off half-implemented password-reset via sms. (#6101)
Remove get_user_by_req opentracing span and add some tags. (#6108)
Drop some unused database tables. (#6115)
Add env var to turn on tracking of log context changes. (#6127)
Refactor configuration loading to allow better typechecking. (#6137)
Log responder when responding to media request. (#6139)
Improve performance of find_next_generated_user_id DB query. (#6148)
Expand type-checking on modules imported by synapse.config. (#6150)
Use Postgres ANY for selecting many values. (#6156)
Add more caching to _get_joined_users_from_context DB query. (#6159)
Add some metrics on the federation sender. (#6160)
Add some logging to the rooms stats updates, to try to track down a flaky test. (#6167)
Remove unused timeout parameter from _get_public_room_list. (#6179)
Reject (accidental) attempts to insert bytes into postgres tables. (#6186)
Make version optional in body of PUT /room_keys/version/{version}, since it's redundant. (#6189)
Make storage layer responsible for adding device names to key, rather than the handler. (#6193)
Port synapse.rest.admin module to use async/await. (#6196)
Enforce that all boolean configuration values are lowercase in CI. (#6203)
Remove some unused event-auth code. (#6214)
Remove Auth.check method. (#6217)
Remove format_tap.py script in favour of a perl reimplementation in Sytest's repo. (#6219)
Refactor storage layer in preparation to support having multiple databases. (#6231)
Remove some extra quotation marks across the codebase. (#6236)


Synapse 1.4.1 (2019-10-18)
No changes since 1.4.1rc1.

Synapse 1.4.1rc1 (2019-10-17)

Bugfixes

Fix bug where redacted events were sometimes incorrectly censored in the database, breaking APIs that attempted to fetch such events. (#6185, 5b0e9948)


Synapse 1.4.0 (2019-10-03)

Bugfixes

Redact client_secret in server logs. (#6158)


Synapse 1.4.0rc2 (2019-10-02)

Bugfixes

Fix bug in background update that adds last seen information to the devices table, and improve its performance on Postgres. (#6135)
Fix bad performance of censoring redactions background task. (#6141)
Fix fetching censored redactions from DB, which caused APIs like initial sync to fail if it tried to include the censored redaction. (#6145)
Fix exceptions when storing large retry intervals for down remote servers. (#6146)


Internal Changes

Fix up sample config entry for redaction_retention_period option. (#6117)


Synapse 1.4.0rc1 (2019-09-26)
Note that this release includes significant changes around 3pid
verification. Administrators are reminded to review the upgrade notes.

Features

Changes to 3pid verification:

Add the ability to send registration emails from the homeserver rather than delegating to an identity server. (#5835, #5940, #5993, #5994, #5868)
Replace trust_identity_server_for_password_resets config option with account_threepid_delegates, and make the id_server parameteter optional on */requestToken endpoints, as per MSC2263. (#5876, #5969, #6028)
Switch to using the v2 Identity Service /lookup API where available, with fallback to v1. (Implements MSC2134 plus id_access_token authentication for v2 Identity Service APIs from MSC2140). (#5897)
Remove bind_email and bind_msisdn parameters from /register ala MSC2140. (#5964)
Add m.id_access_token to unstable_features in /versions as per MSC2264. (#5974)
Use the v2 Identity Service API for 3PID invites. (#5979)
Add POST /_matrix/client/unstable/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account. (#5980, #6062)
Use account_threepid_delegate.email and account_threepid_delegate.msisdn for validating threepid sessions. (#6011)
Allow homeserver to handle or delegate email validation when adding an email to a user's account. (#6042)
Implement new Client Server API endpoints /account/3pid/add and /account/3pid/bind as per MSC2290. (#6043)
Add an unstable feature flag for separate add/bind 3pid APIs. (#6044)
Remove bind parameter from Client Server POST /account endpoint as per MSC2290. (#6067)
Add POST /add_threepid/msisdn/submit_token endpoint for proxying submitToken on an account_threepid_handler. (#6078)
Add submit_url response parameter to */msisdn/requestToken endpoints. (#6079)
Add m.require_identity_server flag to /version's unstable_features. (#5972)


Enhancements to OpenTracing support:

Make OpenTracing work in worker mode. (#5771)
Pass OpenTracing contexts between servers when transmitting EDUs. (#5852)
OpenTracing for device list updates. (#5853)
Add a tag recording a request's authenticated entity and corresponding servlet in OpenTracing. (#5856)
Add minimum OpenTracing for client servlets. (#5983)
Check at setup that OpenTracing is installed if it's enabled in the config. (#5985)
Trace replication send times. (#5986)
Include missing OpenTracing contexts in outbout replication requests. (#5982)
Fix sending of EDUs when OpenTracing is enabled with an empty whitelist. (#5984)
Fix invalid references to None while OpenTracing if the log context slips. (#5988, #5991)
OpenTracing for room and e2e keys. (#5855)
Add OpenTracing span over HTTP push processing. (#6003)


Add an admin API to purge old rooms from the database. (#5845)
Retry well-known lookups if we have recently seen a valid well-known record for the server. (#5850)
Add support for filtered room-directory search requests over federation (MSC2197, in order to allow upcoming room directory query performance improvements. (#5859)
Correctly retry all hosts returned from SRV when we fail to connect. (#5864)
Add admin API endpoint for setting whether or not a user is a server administrator. (#5878)
Enable cleaning up extremities with dummy events by default to prevent undue build up of forward extremities. (#5884)
Add config option to sign remote key query responses with a separate key. (#5895)
Add support for config templating. (#5900)
Users with the type of "support" or "bot" are no longer required to consent. (#5902)
Let synctl accept a directory of config files. (#5904)
Increase max display name size to 256. (#5906)
Add admin API endpoint for getting whether or not a user is a server administrator. (#5914)
Redact events in the database that have been redacted for a week. (#5934)
New prometheus metrics:


synapse_federation_known_servers: represents the total number of servers your server knows about (i.e. is in rooms with), including itself. Enable by setting metrics_flags.known_servers to True in the configuration.(#5981)

synapse_build_info: exposes the Python version, OS version, and Synapse version of the running server. (#6005)


Give appropriate exit codes when synctl fails. (#5992)
Apply the federation blacklist to requests to identity servers. (#6000)
Add report_stats_endpoint option to configure where stats are reported to, if enabled. Contributed by @Sorunome. (#6012)
Add config option to increase ratelimits for room admins redacting messages. (#6015)
Stop sending federation transactions to servers which have been down for a long time. (#6026)
Make the process for mapping SAML2 users to matrix IDs more flexible. (#6037)
Return a clearer error message when a timeout occurs when attempting to contact an identity server. (#6073)
Prevent password reset's submit_token endpoint from accepting trailing slashes. (#6074)
Return 403 on /register/available if registration has been disabled. (#6082)
Explicitly log when a homeserver does not have the trusted_key_servers config field configured. (#6090)
Add support for pruning old rows in user_ips table. (#6098)


Bugfixes

Don't create broken room when power_level_content_override.users does not contain creator_id. (#5633)
Fix database index so that different backup versions can have the same sessions. (#5857)
Fix Synapse looking for config options password_reset_failure_template and password_reset_success_template, when they are actually password_reset_template_failure_html, password_reset_template_success_html. (#5863)
Fix stack overflow when recovering an appservice which had an outage. (#5885)
Fix error message which referred to public_base_url instead of public_baseurl. Thanks to @aaronraimist for the fix! (#5909)
Fix 404 for thumbnail download when dynamic_thumbnails is false and the thumbnail was dynamically generated. Fix reported by rkfg. (#5915)
Fix a cache-invalidation bug for worker-based deployments. (#5920)
Fix admin API for listing media in a room not being available with an external media repo. (#5966)
Fix list media admin API always returning an error. (#5967)
Fix room and user stats tracking. (#5971, #5998, #6029)
Return a M_MISSING_PARAM if sid is not provided to /account/3pid. (#5995)

federation_certificate_verification_whitelist now will not cause TypeErrors to be raised (a regression in 1.3). Additionally, it now supports internationalised domain names in their non-canonical representation. (#5996)
Only count real users when checking for auto-creation of auto-join room. (#6004)
Ensure support users can be registered even if MAU limit is reached. (#6020)
Fix bug where login error was shown incorrectly on SSO fallback login. (#6024)
Fix bug in calculating the federation retry backoff period. (#6025)
Prevent exceptions being logged when extremity-cleanup events fail due to lack of user consent to the terms of service. (#6053)
Remove POST method from password-reset submit_token endpoint until we implement submit_url functionality. (#6056)
Fix logcontext spam on non-Linux platforms. (#6059)
Ensure query parameters in email validation links are URL-encoded. (#6063)
Fix a bug which caused SAML attribute maps to be overridden by defaults. (#6069)
Fix the logged number of updated items for the users_set_deactivated_flag background update. (#6092)
Add sid to next_link for email validation. (#6097)
Threepid validity checks on msisdns should not be dependent on threepid_behaviour_email. (#6104)
Ensure that servers which are not configured to support email address verification do not offer it in the registration flows. (#6107)


Updates to the Docker image

Avoid changing UID/GID if they are already correct. (#5970)
Provide SYNAPSE_WORKER envvar to specify python module. (#6058)


Improved Documentation

Convert documentation to markdown (from rst) (#5849)
Update INSTALL.md to say that Python 2 is no longer supported. (#5953)
Add developer documentation for using SAML2. (#6032)
Add some notes on rolling back to v1.3.1. (#6049)
Update the upgrade notes. (#6050)


Deprecations and Removals

Remove shared-secret registration from /_matrix/client/r0/register endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. (#5877)
Deprecate the trusted_third_party_id_servers option. (#5875)


Internal Changes

Lay the groundwork for structured logging output. (#5680)
Retry well-known lookup before the cache expires, giving a grace period where the remote well-known can be down but we still use the old result. (#5844)
Remove log line for debugging issue #5407. (#5860)
Refactor the Appservice scheduler code. (#5886)
Compatibility with v2 Identity Service APIs other than /lookup. (#5892, #6013)
Stop populating some unused tables. (#5893, #6047)
Add missing index on users_in_public_rooms to improve the performance of directory queries. (#5894)
Improve the logging when we have an error when fetching signing keys. (#5896)
Add support for database engine-specific schema deltas, based on file extension. (#5911)
Update Buildkite pipeline to use plugins instead of buildkite-agent commands. (#5922)
Add link in sample config to the logging config schema. (#5926)
Remove unnecessary parentheses in return statements. (#5931)
Remove unused jenkins/prepare_sytest.sh file. (#5938)
Move Buildkite pipeline config to the pipelines repo. (#5943)
Remove unnecessary return statements in the codebase which were the result of a regex run. (#5962)
Remove left-over methods from v1 registration API. (#5963)
Cleanup event auth type initialisation. (#5975)
Clean up dependency checking at setup. (#5989)
Update OpenTracing docs to use the unified trace method. (#5776)
Small refactor of function arguments and docstrings in RoomMemberHandler. (#6009)
Remove unused origin argument on FederationHandler.add_display_name_to_third_party_invite. (#6010)
Add a failure_ts column to the destinations database table. (#6016, #6072)
Clean up some code in the retry logic. (#6017)
Fix the structured logging tests stomping on the global log configuration for subsequent tests. (#6023)
Clean up the sample config for SAML authentication. (#6064)
Change mailer logging to reflect Synapse doesn't just do chat notifications by email now. (#6075)
Move last-seen info into devices table. (#6089)
Remove unused parameter to get_user_id_by_threepid. (#6099)
Refactor the user-interactive auth handling. (#6105)
Refactor code for calculating registration flows. (#6106)


Synapse 1.3.1 (2019-08-17)

Features

Drop hard dependency on sdnotify python package. (#5871)


Bugfixes

Fix startup issue (hang on ACME provisioning) due to ordering of Twisted reactor startup. Thanks to @chrismoos for supplying the fix. (#5867)


Synapse 1.3.0 (2019-08-15)

Bugfixes

Fix 500 Internal Server Error on publicRooms when the public room list was
cached. (#5851)


Synapse 1.3.0rc1 (2019-08-13)

Features

Use M_USER_DEACTIVATED instead of M_UNKNOWN for errcode when a deactivated user attempts to login. (#5686)
Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify. (#5732)
Synapse will no longer serve any media repo admin endpoints when enable_media_repo is set to False in the configuration. If a media repo worker is used, the admin APIs relating to the media repo will be served from it instead. (#5754, #5848)
Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers. (#5783)
Allow defining HTML templates to serve the user on account renewal attempt when using the account validity feature. (#5807)


Bugfixes

Fix UISIs during homeserver outage. (#5693, #5789)
Fix stack overflow in server key lookup code. (#5724)
start.sh no longer uses deprecated cli option. (#5725)
Log when we receive an event receipt from an unexpected origin. (#5743)
Fix debian packaging scripts to correctly build sid packages. (#5775)
Correctly handle redactions of redactions. (#5788)
Return 404 instead of 403 when accessing /rooms/{roomId}/event/{eventId} for an event without the appropriate permissions. (#5798)
Fix check that tombstone is a state event in push rules. (#5804)
Fix error when trying to login as a deactivated user when using a worker to handle login. (#5806)
Fix bug where user /sync stream could get wedged in rare circumstances. (#5825)
The purge_remote_media.sh script was fixed. (#5839)


Deprecations and Removals

Synapse now no longer accepts the -v/--verbose, -f/--log-file, or --log-config command line flags, and removes the deprecated verbose and log_file configuration file options. Users of these options should migrate their options into the dedicated log configuration. (#5678, #5729)
Remove non-functional 'expire_access_token' setting. (#5782)


Internal Changes

Make Jaeger fully configurable. (#5694)
Add precautionary measures to prevent future abuse of window.opener in default welcome page. (#5695)
Reduce database IO usage by optimising queries for current membership. (#5706, #5738, #5746, #5752, #5770, #5774, #5792, #5793)
Improve caching when fetching get_filtered_current_state_ids. (#5713)
Don't accept opentracing data from clients. (#5715)
Speed up PostgreSQL unit tests in CI. (#5717)
Update the coding style document. (#5719)
Improve database query performance when recording retry intervals for remote hosts. (#5720)
Add a set of opentracing utils. (#5722)
Cache result of get_version_string to reduce overhead of /version federation requests. (#5730)
Return 'user_type' in admin API user endpoints results. (#5731)
Don't package the sytest test blacklist file. (#5733)
Replace uses of returnValue with plain return, as returnValue is not needed on Python 3. (#5736)
Blacklist some flakey tests in worker mode. (#5740)
Fix some error cases in the caching layer. (#5749)
Add a prometheus metric for pending cache lookups. (#5750)
Stop trying to fetch events with event_id=None. (#5753)
Convert RedactionTestCase to modern test style. (#5768)
Allow looping calls to be given arguments. (#5780)
Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO. (#5785)
Remove DelayedCall debugging from the test suite, as it is no longer required in the vast majority of Synapse's tests. (#5787)
Remove some spurious exceptions from the logs where we failed to talk to a remote server. (#5790)
Improve performance when making .well-known requests by sharing the SSL options between requests. (#5794)
Disable codecov GitHub comments on PRs. (#5796)
Don't allow clients to send tombstone events that reference the room it's sent in. (#5801)
Deny redactions of events sent in a different room. (#5802)
Deny sending well known state types as non-state events. (#5805)
Handle incorrectly encoded query params correctly by returning a 400. (#5808)
Handle pusher being deleted during processing rather than logging an exception. (#5809)
Return 502 not 500 when failing to reach any remote server. (#5810)
Reduce global pauses in the events stream caused by expensive state resolution during persistence. (#5826)
Add a lower bound to well-known lookup cache time to avoid repeated lookups. (#5836)
Whitelist history visbility sytests in worker mode tests. (#5843)


Synapse 1.2.1 (2019-07-26)

Security update
This release includes four security fixes:

Prevent an attack where a federated server could send redactions for arbitrary events in v1 and v2 rooms. (#5767)
Prevent a denial-of-service attack where cycles of redaction events would make Synapse spin infinitely. Thanks to @lrizika:matrix.org for identifying and responsibly disclosing this issue. (0f2ecb961)
Prevent an attack where users could be joined or parted from public rooms without their consent. Thanks to @dylangerdaly for identifying and responsibly disclosing this issue. (#5744)
Fix a vulnerability where a federated server could spoof read-receipts from
users on other servers. Thanks to @dylangerdaly for identifying this issue too. (#5743)

Additionally, the following fix was in Synapse 1.2.0, but was not correctly
identified during the original release:

It was possible for a room moderator to send a redaction for an m.room.create event, which would downgrade the room to version 1. Thanks to /dev/ponies for identifying and responsibly disclosing this issue! (#5701)


Synapse 1.2.0 (2019-07-25)
No significant changes.

Synapse 1.2.0rc2 (2019-07-24)

Bugfixes

Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. (#5734)


Synapse 1.2.0rc1 (2019-07-22)

Security fixes
This update included a security fix which was initially incorrectly flagged as
a regular bug fix.

It was possible for a room moderator to send a redaction for an m.room.create event, which would downgrade the room to version 1. Thanks to /dev/ponies for identifying and responsibly disclosing this issue! (#5701)


Features

Add support for opentracing. (#5544, #5712)
Add ability to pull all locally stored events out of synapse that a particular user can see. (#5589)
Add a basic admin command app to allow server operators to run Synapse admin commands separately from the main production instance. (#5597)
Add sender and origin_server_ts fields to m.replace. (#5613)
Add default push rule to ignore reactions. (#5623)
Include the original event when asking for its relations. (#5626)
Implement session_lifetime configuration option, after which access tokens will expire. (#5660)
Return "This account has been deactivated" when a deactivated user tries to login. (#5674)
Enable aggregations support by default (#5714)


Bugfixes

Fix 'utime went backwards' errors on daemonization. (#5609)
Various minor fixes to the federation request rate limiter. (#5621)
Forbid viewing relations on an event once it has been redacted. (#5629)
Fix requests to the /store_invite endpoint of identity servers being sent in the wrong format. (#5638)
Fix newly-registered users not being able to lookup their own profile without joining a room. (#5644)
Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to /relations. (#5654)
Fix 3PID bind requests being sent to identity servers as application/x-form-www-urlencoded data, which is deprecated. (#5658)
Fix some problems with authenticating redactions in recent room versions. (#5699, #5700, #5707)


Updates to the Docker image

Base Docker image on a newer Alpine Linux version (3.8 -> 3.10). (#5619)
Add missing space in default logging file format generated by the Docker image. (#5620)


Improved Documentation

Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks! (#5397)
--no-pep517 should be --no-use-pep517 in the documentation to setup the development environment. (#5651)
Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks! (#5661)
Minor tweaks to postgres documentation. (#5675)


Deprecations and Removals

Remove support for the invite_3pid_guest configuration setting. (#5625)


Internal Changes

Move logging code out of synapse.util and into synapse.logging. (#5606, #5617)
Add a blacklist file to the repo to blacklist certain sytests from failing CI. (#5611)
Make runtime errors surrounding password reset emails much clearer. (#5616)
Remove dead code for persiting outgoing federation transactions. (#5622)
Add lint.sh to the scripts-dev folder which will run all linting steps required by CI. (#5627)
Move RegistrationHandler.get_or_create_user to test code. (#5628)
Add some more common python virtual-environment paths to the black exclusion list. (#5630)
Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See docs/metrics-howto.rst for details. (#5636)
Unblacklist some user_directory sytests. (#5637)
Factor out some redundant code in the login implementation. (#5639)
Update ModuleApi to avoid register(generate_token=True). (#5640)
Remove access-token support from RegistrationHandler.register, and rename it. (#5641)
Remove access-token support from RegistrationStore.register, and rename it. (#5642)
Improve logging for auto-join when a new user is created. (#5643)
Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure. (#5645)
Fix a small typo in a code comment. (#5655)
Clean up exception handling around client access tokens. (#5656)
Add a mechanism for per-test homeserver configuration in the unit tests. (#5657)
Inline issue_access_token. (#5659)
Update the sytest BuildKite configuration to checkout Synapse in /src. (#5664)
Add a docker type to the towncrier configuration. (#5673)
Convert synapse.federation.transport.server to async. Might improve some stack traces. (#5689)
Documentation for opentracing. (#5703)


Synapse 1.1.0 (2019-07-04)
As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4.
See the upgrade notes for more details.
This release also deprecates the use of environment variables to configure the
docker image. See the docker README
for more details.
No changes since 1.1.0rc2.

Synapse 1.1.0rc2 (2019-07-03)

Bugfixes

Fix regression in 1.1rc1 where OPTIONS requests to the media repo would fail. (#5593)
Removed the SYNAPSE_SMTP_* docker container environment variables. Using these environment variables prevented the docker container from starting in Synapse v1.0, even though they didn't actually allow any functionality anyway. (#5596)
Fix a number of "Starting txn from sentinel context" warnings. (#5605)


Internal Changes

Update github templates. (#5552)


Synapse 1.1.0rc1 (2019-07-02)
As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4.
See the upgrade notes for more details.

Features

Added possibilty to disable local password authentication. Contributed by Daniel Hoffend. (#5092)
Add monthly active users to phonehome stats. (#5252)
Allow expired user to trigger renewal email sending manually. (#5363)
Statistics on forward extremities per room are now exposed via Prometheus. (#5384, #5458, #5461)
Add --no-daemonize option to run synapse in the foreground, per issue #4130. Contributed by Soham Gumaste. (#5412, #5587)
Fully support SAML2 authentication. Contributed by Alexander Trost - thank you! (#5422)
Allow server admins to define implementations of extra rules for allowing or denying incoming events. (#5440, #5474, #5477)
Add support for handling pagination APIs on client reader worker. (#5505, #5513, #5531)
Improve help and cmdline option names for --generate-config options. (#5512)
Allow configuration of the path used for ACME account keys. (#5516, #5521, #5522)
Add --data-dir and --open-private-ports options. (#5524)
Split public rooms directory auth config in two settings, in order to manage client auth independently from the federation part of it. Obsoletes the "restrict_public_rooms_to_local_users" configuration setting. If "restrict_public_rooms_to_local_users" is set in the config, Synapse will act as if both new options are enabled, i.e. require authentication through the client API and deny federation requests. (#5534)
The minimum TLS version used for outgoing federation requests can now be set with federation_client_minimum_tls_version. (#5550)
Optimise devices changed query to not pull unnecessary rows from the database, reducing database load. (#5559)
Add new metrics for number of forward extremities being persisted and number of state groups involved in resolution. (#5476)


Bugfixes

Fix bug processing incoming events over federation if call to /get_missing_events fails. (#5042)
Prevent more than one room upgrade happening simultaneously on the same room. (#5051)
Fix a bug where running synapse_port_db would cause the account validity feature to fail because it didn't set the type of the email_sent column to boolean. (#5325)
Warn about disabling email-based password resets when a reset occurs, and remove warning when someone attempts a phone-based reset. (#5387)
Fix email notifications for unnamed rooms with multiple people. (#5388)
Fix exceptions in federation reader worker caused by attempting to renew attestations, which should only happen on master worker. (#5389)
Fix handling of failures fetching remote content to not log failures as exceptions. (#5390)
Fix a bug where deactivated users could receive renewal emails if the account validity feature is on. (#5394)
Fix missing invite state after exchanging 3PID invites over federaton. (#5464)
Fix intermittent exceptions on Apple hardware. Also fix bug that caused database activity times to be under-reported in log lines. (#5498)
Fix logging error when a tampered event is detected. (#5500)
Fix bug where clients could tight loop calling /sync for a period. (#5507)
Fix bug with jinja2 preventing Synapse from starting. Users who had this problem should now simply need to run pip install matrix-synapse. (#5514)
Fix a regression where homeservers on private IP addresses were incorrectly blacklisted. (#5523)
Fixed m.login.jwt using unregistred user_id and added pyjwt>=1.6.4 as jwt conditional dependencies. Contributed by Pau Rodriguez-Estivill. (#5555, #5586)
Fix a bug that would cause invited users to receive several emails for a single 3PID invite in case the inviter is rate limited. (#5576)


Updates to the Docker image

Add ability to change Docker containers timezone with the TZ variable. (#5383)
Update docker image to use Python 3.7. (#5546)
Deprecate the use of environment variables for configuration, and make the use of a static configuration the default. (#5561, #5562, #5566, #5567)
Increase default log level for docker image to INFO. It can still be changed by editing the generated log.config file. (#5547)
Send synapse logs to the docker logging system, by default. (#5565)
Open the non-TLS port by default. (#5568)
Fix failure to start under docker with SAML support enabled. (#5490)
Use a sensible location for data files when generating a config file. (#5563)


Deprecations and Removals

Python 2.7 is no longer a supported platform. Synapse now requires Python 3.5+ to run. (#5425)
PostgreSQL 9.4 is no longer supported. Synapse requires Postgres 9.5+ or above for Postgres support. (#5448)
Remove support for cpu_affinity setting. (#5525)


Improved Documentation

Improve README section on performance troubleshooting. (#4276)
Add information about how to install and run black on the codebase to code_style.rst. (#5537)
Improve install docs on choosing server_name. (#5558)


Internal Changes

Add logging to 3pid invite signature verification. (#5015)
Update example haproxy config to a more compatible setup. (#5313)
Track deactivated accounts in the database. (#5378, #5465, #5493)
Clean up code for sending federation EDUs. (#5381)
Add a sponsor button to the repo. (#5382, #5386)
Don't log non-200 responses from federation queries as exceptions. (#5383)
Update Python syntax in contrib/ to Python 3. (#5446)
Update federation_client dev script to support .well-known and work with python3. (#5447)
SyTest has been moved to Buildkite. (#5459)
Demo script now uses python3. (#5460)
Synapse can now handle RestServlets that return coroutines. (#5475, #5585)
The demo servers talk to each other again. (#5478)
Add an EXPERIMENTAL config option to try and periodically clean up extremities by sending dummy events. (#5480)
Synapse's codebase is now formatted by black. (#5482)
Some cleanups and sanity-checking in the CPU and database metrics. (#5499)
Improve email notification logging. (#5502)
Fix "Unexpected entry in 'full_schemas'" log warning. (#5509)
Improve logging when generating config files. (#5510)
Refactor and clean up Config parser for maintainability. (#5511)
Make the config clearer in that email.template_dir is relative to the Synapse's root directory, not the synapse/ folder within it. (#5543)
Update v1.0.0 release changelog to include more information about changes to password resets. (#5545)
Remove non-functioning check_event_hash.py dev script. (#5548)
Synapse will now only allow TLS v1.2 connections when serving federation, if it terminates TLS. As Synapse's allowed ciphers were only able to be used in TLSv1.2 before, this does not change behaviour. (#5550)
Logging when running GC collection on generation 0 is now at the DEBUG level, not INFO. (#5557)
Reduce the amount of stuff we send in the docker context. (#5564)
Point the reverse links in the Purge History contrib scripts at the intended location. (#5570)


Synapse 1.0.0 (2019-06-11)

Bugfixes

Fix bug where attempting to send transactions with large number of EDUs can fail. (#5418)


Improved Documentation

Expand the federation guide to include relevant content from the MSC1711 FAQ (#5419)


Internal Changes

Move password reset links to /_matrix/client/unstable namespace. (#5424)


Synapse 1.0.0rc3 (2019-06-10)
Security: Fix authentication bug introduced in 1.0.0rc1. Please upgrade to rc3 immediately

Synapse 1.0.0rc2 (2019-06-10)

Bugfixes

Remove redundant warning about key server response validation. (#5392)
Fix bug where old keys stored in the database with a null valid until timestamp caused all verification requests for that key to fail. (#5415)
Fix excessive memory using with default federation_verify_certificates: true configuration. (#5417)


Synapse 1.0.0rc1 (2019-06-07)

Features


Synapse now more efficiently collates room statistics. (#4338, #5260, #5324)


Add experimental support for relations (aka reactions and edits). (#5220)


Ability to configure default room version. (#5223, #5249)


Allow configuring a range for the account validity startup job. (#5276)


CAS login will now hit the r0 API, not the deprecated v1 one. (#5286)


Validate federation server TLS certificates by default (implements MSC1711). (#5359)


Update /_matrix/client/versions to reference support for r0.5.0. (#5360)


Add a script to generate new signing-key files. (#5361)


Update upgrade and installation guides ahead of 1.0. (#5371)


Replace the perspectives configuration section with trusted_key_servers, and make validating the signatures on responses optional (since TLS will do this job for us). (#5374)


Add ability to perform password reset via email without trusting the identity server. As a result of this PR, password resets will now be disabled on the default configuration.
Password reset emails are now sent from the homeserver by default, instead of the identity server. To enable this functionality, ensure email and public_baseurl config options are filled out.
If you would like to re-enable password resets being sent from the identity server (warning: this is dangerous! See #5345), set email.trust_identity_server_for_password_resets to true. (#5377)


Set default room version to v4. (#5379)


Bugfixes

Fixes client-server API not sending "m.heroes" to lazy-load /sync requests when a rooms name or its canonical alias are empty. Thanks to @dnaf for this work! (#5089)
Prevent federation device list updates breaking when processing multiple updates at once. (#5156)
Fix worker registration bug caused by ClientReaderSlavedStore being unable to see get_profileinfo. (#5200)
Fix race when backfilling in rooms with worker mode. (#5221)
Fix appservice timestamp massaging. (#5233)
Ensure that server_keys fetched via a notary server are correctly signed. (#5251)
Show the correct error when logging out and access token is missing. (#5256)
Fix error code when there is an invalid parameter on /_matrix/client/r0/publicRooms (#5257)
Fix error when downloading thumbnail with missing width/height parameter. (#5258)
Fix schema update for account validity. (#5268)
Fix bug where we leaked extremities when we soft failed events, leading to performance degradation. (#5274, #5278, #5291)
Fix "db txn 'update_presence' from sentinel context" log messages. (#5275)
Fix dropped logcontexts during high outbound traffic. (#5277)
Fix a bug where it is not possible to get events in the federation format with the request GET /_matrix/client/r0/rooms/{roomId}/messages. (#5293)
Fix performance problems with the rooms stats background update. (#5294)
Fix noisy 'no key for server' logs. (#5300)
Fix bug where a notary server would sometimes forget old keys. (#5307)
Prevent users from setting huge displaynames and avatar URLs. (#5309)
Fix handling of failures when processing incoming events where calling /event_auth on remote server fails. (#5317)
Ensure that we have an up-to-date copy of the signing key when validating incoming federation requests. (#5321)
Fix various problems which made the signing-key notary server time out for some requests. (#5333)
Fix bug which would make certain operations (such as room joins) block for 20 minutes while attemoting to fetch verification keys. (#5334)
Fix a bug where we could rapidly mark a server as unreachable even though it was only down for a few minutes. (#5335, #5340)
Fix a bug where account validity renewal emails could only be sent when email notifs were enabled. (#5341)
Fix failure when fetching batches of events during backfill, etc. (#5342)
Add a new room version where the timestamps on events are checked against the validity periods on signing keys. (#5348, #5354)
Fix room stats and presence background updates to correctly handle missing events. (#5352)
Include left members in room summaries' heroes. (#5355)
Fix federation_custom_ca_list configuration option. (#5362)
Fix missing logcontext warnings on shutdown. (#5369)


Improved Documentation

Fix docs on resetting the user directory. (#5282)
Fix notes about ACME in the MSC1711 faq. (#5357)


Internal Changes

Synapse will now serve the experimental "room complexity" API endpoint. (#5216)
The base classes for the v1 and v2_alpha REST APIs have been unified. (#5226, #5328)
Simplifications and comments in do_auth. (#5227)
Remove urllib3 pin as requests 2.22.0 has been released supporting urllib3 1.25.2. (#5230)
Preparatory work for key-validity features. (#5232, #5234, #5235, #5236, #5237, #5244, #5250, #5296, #5299, #5343, #5347, #5356)
Specify the type of reCAPTCHA key to use. (#5283)
Improve sample config for monthly active user blocking. (#5284)
Remove spurious debug from MatrixFederationHttpClient.get_json. (#5287)
Improve logging for logcontext leaks. (#5288)
Clarify that the admin change password API logs the user out. (#5303)
New installs will now use the v54 full schema, rather than the full schema v14 and applying incremental updates to v54. (#5320)
Improve docstrings on MatrixFederationClient. (#5332)
Clean up FederationClient.get_events for clarity. (#5344)
Various improvements to debug logging. (#5353)
Don't run CI build checks until sample config check has passed. (#5370)
Automatically retry buildkite builds (max twice) when an agent is lost. (#5380)


Synapse 0.99.5.2 (2019-05-30)

Bugfixes

Fix bug where we leaked extremities when we soft failed events, leading to performance degradation. (#5274, #5278, #5291)


Synapse 0.99.5.1 (2019-05-22)
0.99.5.1 supersedes 0.99.5 due to malformed debian changelog - no functional changes.

Synapse 0.99.5 (2019-05-22)
No significant changes.

Synapse 0.99.5rc1 (2019-05-21)

Features

Add ability to blacklist IP ranges for the federation client. (#5043)
Ratelimiting configuration for clients sending messages and the federation server has been altered to match login ratelimiting. The old configuration names will continue working. Check the sample config for details of the new names. (#5181)
Drop support for the undocumented /_matrix/client/v2_alpha API prefix. (#5190)
Add an option to disable per-room profiles. (#5196)
Stick an expiration date to any registered user missing one at startup if account validity is enabled. (#5204)
Add experimental support for relations (aka reactions and edits). (#5209, #5211, #5203, #5212)
Add a room version 4 which uses a new event ID format, as per MSC2002. (#5210, #5217)


Bugfixes

Fix image orientation when generating thumbnails (needs pillow>=4.3.0). Contributed by Pau Rodriguez-Estivill. (#5039)
Exclude soft-failed events from forward-extremity candidates: fixes "No forward extremities left!" error. (#5146)
Re-order stages in registration flows such that msisdn and email verification are done last. (#5174)
Fix 3pid guest invites. (#5177)
Fix a bug where the register endpoint would fail with M_THREEPID_IN_USE instead of returning an account previously registered in the same session. (#5187)
Prevent registration for user ids that are too long to fit into a state key. Contributed by Reid Anderson. (#5198)
Fix incompatibility between ACME support and Python 3.5.2. (#5218)
Fix error handling for rooms whose versions are unknown. (#5219)


Internal Changes

Make /sync attempt to return device updates for both joined and invited users. Note that this doesn't currently work correctly due to other bugs. (#3484)
Update tests to consistently be configured via the same code that is used when loading from configuration files. (#5171, #5185)
Allow client event serialization to be async. (#5183)
Expose DataStore._get_events as get_events_as_list. (#5184)
Make generating SQL bounds for pagination generic. (#5191)
Stop telling people to install the optional dependencies by default. (#5197)


Synapse 0.99.4 (2019-05-15)
No significant changes.

Synapse 0.99.4rc1 (2019-05-13)

Features

Add systemd-python to the optional dependencies to enable logging to the systemd journal. Install with pip install matrix-synapse[systemd]. (#4339)
Add a default .m.rule.tombstone push rule. (#4867)
Add ability for password provider modules to bind email addresses to users upon registration. (#4947)
Implementation of MSC1711 including config options for requiring valid TLS certificates for federation traffic, the ability to disable TLS validation for specific domains, and the ability to specify your own list of CA certificates. (#4967)
Remove presence list support as per MSC 1819. (#4989)
Reduce CPU usage starting pushers during start up. (#4991)
Add a delete group admin API. (#5002)
Add config option to block users from looking up 3PIDs. (#5010)
Add context to phonehome stats. (#5020)
Configure the example systemd units to have a log identifier of matrix-synapse
instead of the executable name, python.
Contributed by Christoph Müller. (#5023)
Add time-based account expiration. (#5027, #5047, #5073, #5116)
Add support for handling /versions, /voip and /push_rules client endpoints to client_reader worker. (#5063, #5065, #5070)
Add a configuration option to require authentication on /publicRooms and /profile endpoints. (#5083)
Move admin APIs to /_synapse/admin/v1. (The old paths are retained for backwards-compatibility, for now). (#5119)
Implement an admin API for sending server notices. Many thanks to @krombel who provided a foundation for this work. (#5121, #5142)


Bugfixes

Avoid redundant URL encoding of redirect URL for SSO login in the fallback login page. Fixes a regression introduced in #4220. Contributed by Marcel Fabian Krüger ("zaugin"). (#4555)
Fix bug where presence updates were sent to all servers in a room when a new server joined, rather than to just the new server. (#4942, #5103)
Fix sync bug which made accepting invites unreliable in worker-mode synapses. (#4955, #4956)
start.sh: Fix the --no-rate-limit option for messages and make it bypass rate limit on registration and login too. (#4981)
Transfer related groups on room upgrade. (#4990)
Prevent the ability to kick users from a room they aren't in. (#4999)
Fix issue #4596 so synapse_port_db script works with --curses option on Python 3. Contributed by Anders Jensen-Waud anders@jensenwaud.com. (#5003)
Clients timing out/disappearing while downloading from the media repository will now no longer log a spurious "Producer was not unregistered" message. (#5009)
Fix "cannot import name execute_batch" error with postgres. (#5032)
Fix disappearing exceptions in manhole. (#5035)
Workaround bug in twisted where attempting too many concurrent DNS requests could cause it to hang due to running out of file descriptors. (#5037)
Make sure we're not registering the same 3pid twice on registration. (#5071)
Don't crash on lack of expiry templates. (#5077)
Fix the ratelimiting on third party invites. (#5104)
Add some missing limitations to room alias creation. (#5124, #5128)
Limit the number of EDUs in transactions to 100 as expected by synapse. Thanks to @superboum for this work! (#5138)


Internal Changes

Add test to verify threepid auth check added in #4435. (#4474)
Fix/improve some docstrings in the replication code. (#4949)
Split synapse.replication.tcp.streams into smaller files. (#4953)
Refactor replication row generation/parsing. (#4954)
Run black to clean up formatting on synapse/storage/roommember.py and synapse/storage/events.py. (#4959)
Remove log line for password via the admin API. (#4965)
Fix typo in TLS filenames in docker/README.md. Also add the '-p' commandline option to the 'docker run' example. Contributed by Jurrie Overgoor. (#4968)
Refactor room version definitions. (#4969)
Reduce log level of .well-known/matrix/client responses. (#4972)
Add config.signing_key_path that can be read by synapse.config utility. (#4974)
Track which identity server is used when binding a threepid and use that for unbinding, as per MSC1915. (#4982)
Rewrite KeyringTestCase as a HomeserverTestCase. (#4985)
README updates: Corrected the default POSTGRES_USER. Added port forwarding hint in TLS section. (#4987)
Remove a number of unused tables from the database schema. (#4992, #5028, #5033)
Run black on the remainder of synapse/storage/. (#4996)
Fix grammar in get_current_users_in_room and give it a docstring. (#4998)
Clean up some code in the server-key Keyring. (#5001)
Convert SYNAPSE_NO_TLS Docker variable to boolean for user friendliness. Contributed by Gabriel Eckerson. (#5005)
Refactor synapse.storage._base._simple_select_list_paginate. (#5007)
Store the notary server name correctly in server_keys_json. (#5024)
Rewrite Datastore.get_server_verify_keys to reduce the number of database transactions. (#5030)
Remove extraneous period from copyright headers. (#5046)
Update documentation for where to get Synapse packages. (#5067)
Add workarounds for pep-517 install errors. (#5098)
Improve logging when event-signature checks fail. (#5100)
Factor out an "assert_requester_is_admin" function. (#5120)
Remove the requirement to authenticate for /admin/server_version. (#5122)
Prevent an exception from being raised in a IResolutionReceiver and use a more generic error message for blacklisted URL previews. (#5155)
Run black on the tests directory. (#5170)
Fix CI after new release of isort. (#5179)
Fix bogus imports in unit tests. (#5154)


Synapse 0.99.3.2 (2019-05-03)

Internal Changes

Ensure that we have urllib3 <1.25, to resolve incompatibility with requests. (#5135)


Synapse 0.99.3.1 (2019-05-03)

Security update
This release includes two security fixes:

Switch to using a cryptographically-secure random number generator for token strings, ensuring they cannot be predicted by an attacker. Thanks to @opnsec for identifying and responsibly disclosing this issue! (#5133)
Blacklist 0.0.0.0 and :: by default for URL previews. Thanks to @opnsec for identifying and responsibly disclosing this issue too! (#5134)


Synapse 0.99.3 (2019-04-01)