Upgrading Synapse
Before upgrading check if any special steps are required to upgrade from the version you currently have installed to the current version of Synapse. The extra instructions that may be required are listed later in this document.
-
Check that your versions of Python and PostgreSQL are still supported.
Synapse follows upstream lifecycles for Python and PostgreSQL, and removes support for versions which are no longer maintained.
The website https://endoflife.date also offers convenient summaries.
-
If Synapse was installed using prebuilt packages, you will need to follow the normal process for upgrading those packages.
-
If Synapse was installed from source, then:
-
Activate the virtualenv before upgrading. For example, if Synapse is installed in a virtualenv in
~/synapse/env
then run:source ~/synapse/env/bin/activate
-
If Synapse was installed using pip then upgrade to the latest version by running:
pip install --upgrade matrix-synapse
If Synapse was installed using git then upgrade to the latest version by running:
git pull pip install --upgrade .
-
Restart Synapse:
./synctl restart
-
To check whether your update was successful, you can check the running server version with:
# you may need to replace 'localhost:8008' if synapse is not configured
# to listen on port 8008.
curl http://localhost:8008/_synapse/admin/v1/server_version
Rolling back to older versions
Rolling back to previous releases can be difficult, due to database schema changes between releases. Where we have been able to test the rollback process, this will be noted below.
In general, you will need to undo any changes made during the upgrade process, for example:
-
pip:
source env/bin/activate # replace `1.3.0` accordingly: pip install matrix-synapse==1.3.0
-
Debian:
# replace `1.3.0` and `stretch` accordingly: wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to v1.44.0
The URL preview cache is no longer mirrored to storage providers
The url_cache/
and url_cache_thumbnails/
directories in the media store are
no longer mirrored to storage providers. These two directories can be safely
deleted from any configured storage providers to reclaim space.
Upgrading to v1.43.0
The spaces summary APIs can now be handled by workers
The available worker applications documentation
has been updated to reflect that calls to the /spaces
, /hierarchy
, and
/summary
endpoints can now be routed to workers for both client API and
federation requests.
Upgrading to v1.42.0
Removal of old Room Admin API
The following admin APIs were deprecated in Synapse 1.25 (released on 2021-01-13) and have now been removed:
POST /_synapse/admin/v1/purge_room
POST /_synapse/admin/v1/shutdown_room/<room_id>
Any scripts still using the above APIs should be converted to use the Delete Room API.
User-interactive authentication fallback templates can now display errors
This may affect you if you make use of custom HTML templates for the reCAPTCHA or terms fallback pages.
The template is now provided an error
variable if the authentication
process failed. See the default templates linked above for an example.
Removal of out-of-date email pushers
Users will stop receiving message updates via email for addresses that were once, but not still, linked to their account.
Upgrading to v1.41.0
Add support for routing outbound HTTP requests via a proxy for federation
Since Synapse 1.6.0 (2019-11-26) you can set a proxy for outbound HTTP requests via http_proxy/https_proxy environment variables. This proxy was set for:
- push
- url previews
- phone-home stats
- recaptcha validation
- CAS auth validation
- OpenID Connect
- Federation (checking public key revocation)
In this version we have added support for outbound requests for:
- Outbound federation
- Downloading remote media
- Fetching public keys of other servers
These requests use the same proxy configuration. If you have a proxy configuration we
recommend to verify the configuration. It may be necessary to adjust the no_proxy
environment variable.
See using a forward proxy with Synapse documentation for details.
template_dir
Deprecation of The template_dir
settings in the sso
, account_validity
and email
sections of the
configuration file are now deprecated. Server admins should use the new
templates.custom_template_directory
setting in the configuration file and use one single
custom template directory for all aforementioned features. Template file names remain
unchanged. See the related documentation
for more information and examples.
We plan to remove support for these settings in October 2021.
/_synapse/admin/v1/users/{userId}/media
must be handled by media workers
The media repository worker documentation
has been updated to reflect that calls to /_synapse/admin/v1/users/{userId}/media
must now be handled by media repository workers. This is due to the new DELETE
method
of this endpoint modifying the media store.
Upgrading to v1.39.0
Deprecation of the current third-party rules module interface
The current third-party rules module interface is deprecated in favour of the new generic modules system introduced in Synapse v1.37.0. Authors of third-party rules modules can refer to this documentation to update their modules. Synapse administrators can refer to this documentation to update their configuration once the modules they are using have been updated.
We plan to remove support for the current third-party rules interface in September 2021.
Upgrading to v1.38.0
events
table on Postgres databases
Re-indexing of This release includes a database schema update which requires re-indexing one of
the larger tables in the database, events
. This could result in increased
disk I/O for several hours or days after upgrading while the migration
completes. Furthermore, because we have to keep the old indexes until the new
indexes are ready, it could result in a significant, temporary, increase in
disk space.
To get a rough idea of the disk space required, check the current size of one
of the indexes. For example, from a psql
shell, run the following sql:
SELECT pg_size_pretty(pg_relation_size('events_order_room'));
We need to rebuild four indexes, so you will need to multiply this result by four to give an estimate of the disk space required. For example, on one particular server:
synapse=# select pg_size_pretty(pg_relation_size('events_order_room'));
pg_size_pretty
----------------
288 MB
(1 row)
On this server, it would be wise to ensure that at least 1152MB are free.
The additional disk space will be freed once the migration completes.
SQLite databases are unaffected by this change.
Upgrading to v1.37.0
Deprecation of the current spam checker interface
The current spam checker interface is deprecated in favour of a new generic modules system. Authors of spam checker modules can refer to this documentation to update their modules. Synapse administrators can refer to this documentation to update their configuration once the modules they are using have been updated.
We plan to remove support for the current spam checker interface in August 2021.
More module interfaces will be ported over to this new generic system in future versions of Synapse.
Upgrading to v1.34.0
room_invite_state_types
configuration setting
The room_invite_state_types
configuration setting has been deprecated
and replaced with room_prejoin_state
. See the sample configuration
file.
If you have set room_invite_state_types
to the default value you
should simply remove it from your configuration file. The default value
used to be:
room_invite_state_types:
- "m.room.join_rules"
- "m.room.canonical_alias"
- "m.room.avatar"
- "m.room.encryption"
- "m.room.name"
If you have customised this value, you should remove
room_invite_state_types
and configure room_prejoin_state
instead.
Upgrading to v1.33.0
Account Validity HTML templates can now display a user's expiration date
This may affect you if you have enabled the account validity feature,
and have made use of a custom HTML template specified by the
account_validity.template_dir
or
account_validity.account_renewed_html_path
Synapse config options.
The template can now accept an expiration_ts
variable, which
represents the unix timestamp in milliseconds for the future date of
which their account has been renewed until. See the default
template
for an example of usage.
ALso note that a new HTML template, account_previously_renewed.html
,
has been added. This is is shown to users when they attempt to renew
their account with a valid renewal token that has already been used
before. The default template contents can been found
here,
and can also accept an expiration_ts
variable. This template replaces
the error message users would previously see upon attempting to use a
valid renewal token more than once.
Upgrading to v1.32.0
Regression causing connected Prometheus instances to become overwhelmed
This release introduces a regression that can overwhelm connected Prometheus instances. This issue is not present in Synapse v1.32.0rc1.
If you have been affected, please downgrade to 1.31.0. You then may need to remove excess writeahead logs in order for Prometheus to recover. Instructions for doing so are provided here.
Dropping support for old Python, Postgres and SQLite versions
In line with our deprecation policy, we've dropped support for Python 3.5 and PostgreSQL 9.5, as they are no longer supported upstream.
This release of Synapse requires Python 3.6+ and PostgresSQL 9.6+ or SQLite 3.22+.
Removal of old List Accounts Admin API
The deprecated v1 "list accounts" admin API
(GET /_synapse/admin/v1/users/<user_id>
) has been removed in this
version.
The v2 list accounts API
has been available since Synapse 1.7.0 (2019-12-13), and is accessible
under GET /_synapse/admin/v2/users
.
The deprecation of the old endpoint was announced with Synapse 1.28.0 (released on 2021-02-25).
m.login.application_service
when registering users
Application Services must use type In compliance with the Application Service
spec,
Application Services are now required to use the
m.login.application_service
type when registering users via the
/_matrix/client/r0/register
endpoint. This behaviour was deprecated in
Synapse v1.30.0.
Please ensure your Application Services are up to date.
Upgrading to v1.29.0
Requirement for X-Forwarded-Proto header
When using Synapse with a reverse proxy (in particular, when using the [x_forwarded]{.title-ref} option on an HTTP listener), Synapse now expects to receive an [X-Forwarded-Proto]{.title-ref} header on incoming HTTP requests. If it is not set, Synapse will log a warning on each received request.
To avoid the warning, administrators using a reverse proxy should ensure that the reverse proxy sets [X-Forwarded-Proto]{.title-ref} header to [https]{.title-ref} or [http]{.title-ref} to indicate the protocol used by the client.
Synapse also requires the [Host]{.title-ref} header to be preserved.
See the reverse proxy documentation, where the example configurations have been updated to show how to set these headers.
(Users of Caddy are unaffected, since we believe it sets [X-Forwarded-Proto]{.title-ref} by default.)
Upgrading to v1.27.0
Changes to callback URI for OAuth2 / OpenID Connect and SAML2
This version changes the URI used for callbacks from OAuth2 and SAML2 identity providers:
-
If your server is configured for single sign-on via an OpenID Connect or OAuth2 identity provider, you will need to add
[synapse public baseurl]/_synapse/client/oidc/callback
to the list of permitted "redirect URIs" at the identity provider.See the OpenID docs for more information on setting up OpenID Connect.
-
If your server is configured for single sign-on via a SAML2 identity provider, you will need to add
[synapse public baseurl]/_synapse/client/saml2/authn_response
as a permitted "ACS location" (also known as "allowed callback URLs") at the identity provider.The "Issuer" in the "AuthnRequest" to the SAML2 identity provider is also updated to
[synapse public baseurl]/_synapse/client/saml2/metadata.xml
. If your SAML2 identity provider uses this property to validate or otherwise identify Synapse, its configuration will need to be updated to use the new URL. Alternatively you could create a new, separate "EntityDescriptor" in your SAML2 identity provider with the new URLs and leave the URLs in the existing "EntityDescriptor" as they were.
Changes to HTML templates
The HTML templates for SSO and email notifications now have Jinja2's
autoescape
enabled for files ending in .html
, .htm
, and .xml
. If you have
customised these templates and see issues when viewing them you might
need to update them. It is expected that most configurations will need
no changes.
If you have customised the templates names for these templates, it is
recommended to verify they end in .html
to ensure autoescape is
enabled.
The above applies to the following templates:
add_threepid.html
add_threepid_failure.html
add_threepid_success.html
notice_expiry.html
notice_expiry.html
-
notif_mail.html
(which, by default, includesroom.html
andnotif.html
) password_reset.html
password_reset_confirmation.html
password_reset_failure.html
password_reset_success.html
registration.html
registration_failure.html
registration_success.html
sso_account_deactivated.html
sso_auth_bad_user.html
sso_auth_confirm.html
sso_auth_success.html
sso_error.html
sso_login_idp_picker.html
sso_redirect_confirm.html
Upgrading to v1.26.0
Rolling back to v1.25.0 after a failed upgrade
v1.26.0 includes a lot of large changes. If something problematic occurs, you may want to roll-back to a previous version of Synapse. Because v1.26.0 also includes a new database schema version, reverting that version is also required alongside the generic rollback instructions mentioned above. In short, to roll back to v1.25.0 you need to:
-
Stop the server
-
Decrease the schema version in the database:
UPDATE schema_version SET version = 58;
-
Delete the ignored users & chain cover data:
DROP TABLE IF EXISTS ignored_users; UPDATE rooms SET has_auth_chain_index = false;
For PostgreSQL run:
TRUNCATE event_auth_chain_links; TRUNCATE event_auth_chains;
For SQLite run:
DELETE FROM event_auth_chain_links; DELETE FROM event_auth_chains;
-
Mark the deltas as not run (so they will re-run on upgrade).
DELETE FROM applied_schema_deltas WHERE version = 59 AND file = "59/01ignored_user.py"; DELETE FROM applied_schema_deltas WHERE version = 59 AND file = "59/06chain_cover_index.sql";
-
Downgrade Synapse by following the instructions for your installation method in the "Rolling back to older versions" section above.
Upgrading to v1.25.0
Last release supporting Python 3.5
This is the last release of Synapse which guarantees support with Python 3.5, which passed its upstream End of Life date several months ago.
We will attempt to maintain support through March 2021, but without guarantees.
In the future, Synapse will follow upstream schedules for ending support of older versions of Python and PostgreSQL. Please upgrade to at least Python 3.6 and PostgreSQL 9.6 as soon as possible.
Blacklisting IP ranges
Synapse v1.25.0 includes new settings, ip_range_blacklist
and
ip_range_whitelist
, for controlling outgoing requests from Synapse for
federation, identity servers, push, and for checking key validity for
third-party invite events. The previous setting,
federation_ip_range_blacklist
, is deprecated. The new
ip_range_blacklist
defaults to private IP ranges if it is not defined.
If you have never customised federation_ip_range_blacklist
it is
recommended that you remove that setting.
If you have customised federation_ip_range_blacklist
you should update
the setting name to ip_range_blacklist
.
If you have a custom push server that is reached via private IP space
you may need to customise ip_range_blacklist
or ip_range_whitelist
.
Upgrading to v1.24.0
Custom OpenID Connect mapping provider breaking change
This release allows the OpenID Connect mapping provider to perform normalisation of the localpart of the Matrix ID. This allows for the mapping provider to specify different algorithms, instead of the default way.
If your Synapse configuration uses a custom mapping provider ([oidc_config.user_mapping_provider.module]{.title-ref} is specified and not equal to [synapse.handlers.oidc_handler.JinjaOidcMappingProvider]{.title-ref}) then you must ensure that [map_user_attributes]{.title-ref} of the mapping provider performs some normalisation of the [localpart]{.title-ref} returned. To match previous behaviour you can use the [map_username_to_mxid_localpart]{.title-ref} function provided by Synapse. An example is shown below:
from synapse.types import map_username_to_mxid_localpart
class MyMappingProvider:
def map_user_attributes(self, userinfo, token):
# ... your custom logic ...
sso_user_id = ...
localpart = map_username_to_mxid_localpart(sso_user_id)
return {"localpart": localpart}
Removal historical Synapse Admin API
Historically, the Synapse Admin API has been accessible under:
/_matrix/client/api/v1/admin
/_matrix/client/unstable/admin
/_matrix/client/r0/admin
/_synapse/admin/v1
The endpoints with /_matrix/client/*
prefixes have been removed as of
v1.24.0. The Admin API is now only accessible under:
/_synapse/admin/v1
The only exception is the [/admin/whois]{.title-ref} endpoint, which is also available via the client-server API.
The deprecation of the old endpoints was announced with Synapse 1.20.0 (released on 2020-09-22) and makes it easier for homeserver admins to lock down external access to the Admin API endpoints.
Upgrading to v1.23.0
Structured logging configuration breaking changes
This release deprecates use of the structured: true
logging
configuration for structured logging. If your logging configuration
contains structured: true
then it should be modified based on the
structured logging documentation.
The structured
and drains
logging options are now deprecated and
should be replaced by standard logging configuration of handlers
and
formatters
.
A future will release of Synapse will make using structured: true
an
error.
Upgrading to v1.22.0
ThirdPartyEventRules breaking changes
This release introduces a backwards-incompatible change to modules
making use of ThirdPartyEventRules
in Synapse. If you make use of a
module defined under the third_party_event_rules
config option, please
make sure it is updated to handle the below change:
The http_client
argument is no longer passed to modules as they are
initialised. Instead, modules are expected to make use of the
http_client
property on the ModuleApi
class. Modules are now passed
a module_api
argument during initialisation, which is an instance of
ModuleApi
. ModuleApi
instances have a http_client
property which
acts the same as the http_client
argument previously passed to
ThirdPartyEventRules
modules.
Upgrading to v1.21.0
/_synapse/client
through your reverse proxy
Forwarding The reverse proxy documentation
has been updated to include reverse proxy directives for
/_synapse/client/*
endpoints. As the user password reset flow now uses
endpoints under this prefix, you must update your reverse proxy
configurations for user password reset to work.
- Additionally, note that the Synapse worker documentation has been updated to
-
state that the
/_synapse/client/password_reset/email/submit_token
endpoint can be handled
by all workers. If you make use of Synapse's worker feature, please update your reverse proxy configuration to reflect this change.
New HTML templates
A new HTML template,
password_reset_confirmation.html,
has been added to the synapse/res/templates
directory. If you are
using a custom template directory, you may want to copy the template
over and modify it.
Note that as of v1.20.0, templates do not need to be included in custom template directories for Synapse to start. The default templates will be used if a custom template cannot be found.
This page will appear to the user after clicking a password reset link that has been emailed to them.
To complete password reset, the page must include a way to make a
[POST]{.title-ref} request to
/_synapse/client/password_reset/{medium}/submit_token
with the query
parameters from the original link, presented as a URL-encoded form. See
the file itself for more details.
Updated Single Sign-on HTML Templates
The saml_error.html
template was removed from Synapse and replaced
with the sso_error.html
template. If your Synapse is configured to use
SAML and a custom sso_redirect_confirm_template_dir
configuration then
any customisations of the saml_error.html
template will need to be
merged into the sso_error.html
template. These templates are similar,
but the parameters are slightly different:
- The
msg
parameter should be renamed toerror_description
. - There is no longer a
code
parameter for the response code. - A string
error
parameter is available that includes a short hint of why a user is seeing the error page.
Upgrading to v1.18.0
Docker [-py3]{.title-ref} suffix will be removed in future versions
From 10th August 2020, we will no longer publish Docker images with the [-py3]{.title-ref} tag suffix. The images tagged with the [-py3]{.title-ref} suffix have been identical to the non-suffixed tags since release 0.99.0, and the suffix is obsolete.
On 10th August, we will remove the [latest-py3]{.title-ref} tag. Existing per-release tags (such as [v1.18.0-py3]{.title-ref}) will not be removed, but no new [-py3]{.title-ref} tags will be added.
Scripts relying on the [-py3]{.title-ref} suffix will need to be updated.
Redis replication is now recommended in lieu of TCP replication
When setting up worker processes, we now recommend the use of a Redis server for replication. The old direct TCP connection method is deprecated and will be removed in a future release. See workers for more details.
Upgrading to v1.14.0
This version includes a database update which is run as part of the upgrade, and which may take a couple of minutes in the case of a large server. Synapse will not respond to HTTP requests while this update is taking place.
Upgrading to v1.13.0
Incorrect database migration in old synapse versions
A bug was introduced in Synapse 1.4.0 which could cause the room directory to be incomplete or empty if Synapse was upgraded directly from v1.2.1 or earlier, to versions between v1.4.0 and v1.12.x.
- This will not be a problem for Synapse installations which were:
-
- created at v1.4.0 or later,
- upgraded via v1.3.x, or
- upgraded straight from v1.2.1 or earlier to v1.13.0 or later.
If completeness of the room directory is a concern, installations which are affected can be repaired as follows:
-
Run the following sql from a [psql]{.title-ref} or [sqlite3]{.title-ref} console:
INSERT INTO background_updates (update_name, progress_json, depends_on) VALUES ('populate_stats_process_rooms', '{}', 'current_state_events_membership'); INSERT INTO background_updates (update_name, progress_json, depends_on) VALUES ('populate_stats_process_users', '{}', 'populate_stats_process_rooms');
-
Restart synapse.
New Single Sign-on HTML Templates
New templates (sso_auth_confirm.html
, sso_auth_success.html
, and
sso_account_deactivated.html
) were added to Synapse. If your Synapse
is configured to use SSO and a custom
sso_redirect_confirm_template_dir
configuration then these templates
will need to be copied from
synapse/res/templates into that directory.
Synapse SSO Plugins Method Deprecation
Plugins using the complete_sso_login
method of
synapse.module_api.ModuleApi
should update to using the async/await
version complete_sso_login_async
which includes additional checks. The
non-async version is considered deprecated.
Rolling back to v1.12.4 after a failed upgrade
v1.13.0 includes a lot of large changes. If something problematic occurs, you may want to roll-back to a previous version of Synapse. Because v1.13.0 also includes a new database schema version, reverting that version is also required alongside the generic rollback instructions mentioned above. In short, to roll back to v1.12.4 you need to:
-
Stop the server
-
Decrease the schema version in the database:
UPDATE schema_version SET version = 57;
-
Downgrade Synapse by following the instructions for your installation method in the "Rolling back to older versions" section above.
Upgrading to v1.12.0
This version includes a database update which is run as part of the upgrade, and which may take some time (several hours in the case of a large server). Synapse will not respond to HTTP requests while this update is taking place.