-
Patrick Cloke authored
MSC1772 specifies the m.room.create event should be sent as part of the invite_state. This was done optionally behind an experimental flag, but is now done by default due to MSC1772 being approved.
Patrick Cloke authoredMSC1772 specifies the m.room.create event should be sent as part of the invite_state. This was done optionally behind an experimental flag, but is now done by default due to MSC1772 being approved.
Upgrading Synapse
Before upgrading check if any special steps are required to upgrade from the version you currently have installed to the current version of Synapse. The extra instructions that may be required are listed later in this document.
-
Check that your versions of Python and PostgreSQL are still supported.
Synapse follows upstream lifecycles for Python and PostgreSQL, and removes support for versions which are no longer maintained.
The website https://endoflife.date also offers convenient summaries.
-
If Synapse was installed using prebuilt packages, you will need to follow the normal process for upgrading those packages.
-
If Synapse was installed from source, then:
-
Activate the virtualenv before upgrading. For example, if Synapse is installed in a virtualenv in
~/synapse/env
then run:source ~/synapse/env/bin/activate
-
If Synapse was installed using pip then upgrade to the latest version by running:
pip install --upgrade matrix-synapse
If Synapse was installed using git then upgrade to the latest version by running:
git pull pip install --upgrade .
-
Restart Synapse:
./synctl restart
-
To check whether your update was successful, you can check the running server version with:
# you may need to replace 'localhost:8008' if synapse is not configured
# to listen on port 8008.
curl http://localhost:8008/_synapse/admin/v1/server_version
Rolling back to older versions
Rolling back to previous releases can be difficult, due to database schema changes between releases. Where we have been able to test the rollback process, this will be noted below.
In general, you will need to undo any changes made during the upgrade process, for example:
-
pip:
source env/bin/activate # replace `1.3.0` accordingly: pip install matrix-synapse==1.3.0
-
Debian:
# replace `1.3.0` and `stretch` accordingly: wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to v1.34.0
room_invite_state_types configuration setting
The room_invite_state_types
configuration setting has been deprecated and
replaced with room_prejoin_state
. See the sample configuration file.
If you have set room_invite_state_types
to the default value you should simply
remove it from your configuration file. The default value used to be:
room_invite_state_types:
- "m.room.join_rules"
- "m.room.canonical_alias"
- "m.room.avatar"
- "m.room.encryption"
- "m.room.name"
If you have customised this value by adding addition state types, you should
remove room_invite_state_types
and configure additional_event_types
with
your customisations.
If you have customised this value by removing state types, you should rename
room_invite_state_types
to additional_event_types
, and set
disable_default_event_types
to true
.
Upgrading to v1.33.0
Account Validity HTML templates can now display a user's expiration date
This may affect you if you have enabled the account validity feature, and have made use of a
custom HTML template specified by the account_validity.template_dir
or account_validity.account_renewed_html_path
Synapse config options.
The template can now accept an expiration_ts
variable, which represents the unix timestamp in milliseconds for the
future date of which their account has been renewed until. See the
default template
for an example of usage.
ALso note that a new HTML template, account_previously_renewed.html
, has been added. This is is shown to users
when they attempt to renew their account with a valid renewal token that has already been used before. The default
template contents can been found
here,
and can also accept an expiration_ts
variable. This template replaces the error message users would previously see
upon attempting to use a valid renewal token more than once.
Upgrading to v1.32.0
Regression causing connected Prometheus instances to become overwhelmed
This release introduces a regression that can overwhelm connected Prometheus instances. This issue is not present in Synapse v1.32.0rc1.
If you have been affected, please downgrade to 1.31.0. You then may need to remove excess writeahead logs in order for Prometheus to recover. Instructions for doing so are provided here.
Dropping support for old Python, Postgres and SQLite versions
In line with our deprecation policy, we've dropped support for Python 3.5 and PostgreSQL 9.5, as they are no longer supported upstream.
This release of Synapse requires Python 3.6+ and PostgresSQL 9.6+ or SQLite 3.22+.
Removal of old List Accounts Admin API
The deprecated v1 "list accounts" admin API (GET /_synapse/admin/v1/users/<user_id>
) has been removed in this version.
The v2 list accounts API
has been available since Synapse 1.7.0 (2019-12-13), and is accessible under GET /_synapse/admin/v2/users
.
The deprecation of the old endpoint was announced with Synapse 1.28.0 (released on 2021-02-25).
Application Services must use type m.login.application_service
when registering users
In compliance with the
Application Service spec,
Application Services are now required to use the m.login.application_service
type when registering users via the
/_matrix/client/r0/register
endpoint. This behaviour was deprecated in Synapse v1.30.0.
Please ensure your Application Services are up to date.
Upgrading to v1.29.0
Requirement for X-Forwarded-Proto header
When using Synapse with a reverse proxy (in particular, when using the x_forwarded option on an HTTP listener), Synapse now expects to receive an X-Forwarded-Proto header on incoming HTTP requests. If it is not set, Synapse will log a warning on each received request.
To avoid the warning, administrators using a reverse proxy should ensure that the reverse proxy sets X-Forwarded-Proto header to https or http to indicate the protocol used by the client.
Synapse also requires the Host header to be preserved.
See the reverse proxy documentation, where the example configurations have been updated to show how to set these headers.
(Users of Caddy are unaffected, since we believe it sets X-Forwarded-Proto by default.)
Upgrading to v1.27.0
Changes to callback URI for OAuth2 / OpenID Connect and SAML2
This version changes the URI used for callbacks from OAuth2 and SAML2 identity providers:
-
If your server is configured for single sign-on via an OpenID Connect or OAuth2 identity provider, you will need to add
[synapse public baseurl]/_synapse/client/oidc/callback
to the list of permitted "redirect URIs" at the identity provider.See docs/openid.md for more information on setting up OpenID Connect.
-
If your server is configured for single sign-on via a SAML2 identity provider, you will need to add
[synapse public baseurl]/_synapse/client/saml2/authn_response
as a permitted "ACS location" (also known as "allowed callback URLs") at the identity provider.The "Issuer" in the "AuthnRequest" to the SAML2 identity provider is also updated to
[synapse public baseurl]/_synapse/client/saml2/metadata.xml
. If your SAML2 identity provider uses this property to validate or otherwise identify Synapse, its configuration will need to be updated to use the new URL. Alternatively you could create a new, separate "EntityDescriptor" in your SAML2 identity provider with the new URLs and leave the URLs in the existing "EntityDescriptor" as they were.
Changes to HTML templates
The HTML templates for SSO and email notifications now have Jinja2's autoescape
enabled for files ending in .html
, .htm
, and .xml
. If you have customised
these templates and see issues when viewing them you might need to update them.
It is expected that most configurations will need no changes.
If you have customised the templates names for these templates, it is recommended
to verify they end in .html
to ensure autoescape is enabled.
The above applies to the following templates:
add_threepid.html
add_threepid_failure.html
add_threepid_success.html
notice_expiry.html
notice_expiry.html
-
notif_mail.html
(which, by default, includesroom.html
andnotif.html
) password_reset.html
password_reset_confirmation.html
password_reset_failure.html
password_reset_success.html
registration.html
registration_failure.html
registration_success.html
sso_account_deactivated.html
sso_auth_bad_user.html
sso_auth_confirm.html
sso_auth_success.html
sso_error.html
sso_login_idp_picker.html
sso_redirect_confirm.html
Upgrading to v1.26.0
Rolling back to v1.25.0 after a failed upgrade
v1.26.0 includes a lot of large changes. If something problematic occurs, you may want to roll-back to a previous version of Synapse. Because v1.26.0 also includes a new database schema version, reverting that version is also required alongside the generic rollback instructions mentioned above. In short, to roll back to v1.25.0 you need to:
-
Stop the server
-
Decrease the schema version in the database:
UPDATE schema_version SET version = 58;
-
Delete the ignored users & chain cover data:
DROP TABLE IF EXISTS ignored_users; UPDATE rooms SET has_auth_chain_index = false;
For PostgreSQL run:
TRUNCATE event_auth_chain_links; TRUNCATE event_auth_chains;
For SQLite run:
DELETE FROM event_auth_chain_links; DELETE FROM event_auth_chains;
-
Mark the deltas as not run (so they will re-run on upgrade).
DELETE FROM applied_schema_deltas WHERE version = 59 AND file = "59/01ignored_user.py"; DELETE FROM applied_schema_deltas WHERE version = 59 AND file = "59/06chain_cover_index.sql";
-
Downgrade Synapse by following the instructions for your installation method in the "Rolling back to older versions" section above.
Upgrading to v1.25.0
Last release supporting Python 3.5
This is the last release of Synapse which guarantees support with Python 3.5, which passed its upstream End of Life date several months ago.
We will attempt to maintain support through March 2021, but without guarantees.
In the future, Synapse will follow upstream schedules for ending support of older versions of Python and PostgreSQL. Please upgrade to at least Python 3.6 and PostgreSQL 9.6 as soon as possible.
Blacklisting IP ranges
Synapse v1.25.0 includes new settings, ip_range_blacklist
and
ip_range_whitelist
, for controlling outgoing requests from Synapse for federation,
identity servers, push, and for checking key validity for third-party invite events.
The previous setting, federation_ip_range_blacklist
, is deprecated. The new
ip_range_blacklist
defaults to private IP ranges if it is not defined.
If you have never customised federation_ip_range_blacklist
it is recommended
that you remove that setting.
If you have customised federation_ip_range_blacklist
you should update the
setting name to ip_range_blacklist
.
If you have a custom push server that is reached via private IP space you may
need to customise ip_range_blacklist
or ip_range_whitelist
.
Upgrading to v1.24.0
Custom OpenID Connect mapping provider breaking change
This release allows the OpenID Connect mapping provider to perform normalisation of the localpart of the Matrix ID. This allows for the mapping provider to specify different algorithms, instead of the [default way](https://matrix.org/docs/spec/appendices#mapping-from-other-character-sets).
If your Synapse configuration uses a custom mapping provider (oidc_config.user_mapping_provider.module is specified and not equal to synapse.handlers.oidc_handler.JinjaOidcMappingProvider) then you must ensure that map_user_attributes of the mapping provider performs some normalisation of the localpart returned. To match previous behaviour you can use the map_username_to_mxid_localpart function provided by Synapse. An example is shown below:
from synapse.types import map_username_to_mxid_localpart
class MyMappingProvider:
def map_user_attributes(self, userinfo, token):
# ... your custom logic ...
sso_user_id = ...
localpart = map_username_to_mxid_localpart(sso_user_id)
return {"localpart": localpart}
Removal historical Synapse Admin API
Historically, the Synapse Admin API has been accessible under:
/_matrix/client/api/v1/admin
/_matrix/client/unstable/admin
/_matrix/client/r0/admin
/_synapse/admin/v1
The endpoints with /_matrix/client/*
prefixes have been removed as of v1.24.0.
The Admin API is now only accessible under:
/_synapse/admin/v1
The only exception is the /admin/whois endpoint, which is also available via the client-server API.
The deprecation of the old endpoints was announced with Synapse 1.20.0 (released on 2020-09-22) and makes it easier for homeserver admins to lock down external access to the Admin API endpoints.
Upgrading to v1.23.0
Structured logging configuration breaking changes
This release deprecates use of the structured: true
logging configuration for
structured logging. If your logging configuration contains structured: true
then it should be modified based on the structured logging documentation.
The structured
and drains
logging options are now deprecated and should
be replaced by standard logging configuration of handlers
and formatters
.
A future will release of Synapse will make using structured: true
an error.
Upgrading to v1.22.0
ThirdPartyEventRules breaking changes
This release introduces a backwards-incompatible change to modules making use of
ThirdPartyEventRules
in Synapse. If you make use of a module defined under the
third_party_event_rules
config option, please make sure it is updated to handle
the below change:
The http_client
argument is no longer passed to modules as they are initialised. Instead,
modules are expected to make use of the http_client
property on the ModuleApi
class.
Modules are now passed a module_api
argument during initialisation, which is an instance of
ModuleApi
. ModuleApi
instances have a http_client
property which acts the same as
the http_client
argument previously passed to ThirdPartyEventRules
modules.
Upgrading to v1.21.0
Forwarding /_synapse/client
through your reverse proxy
The reverse proxy documentation has been updated
to include reverse proxy directives for /_synapse/client/*
endpoints. As the user password
reset flow now uses endpoints under this prefix, you must update your reverse proxy
configurations for user password reset to work.
Additionally, note that the Synapse worker documentation has been updated to
state that the /_synapse/client/password_reset/email/submit_token
endpoint can be handled
by all workers. If you make use of Synapse's worker feature, please update your reverse proxy configuration to reflect this change.
New HTML templates
A new HTML template,
password_reset_confirmation.html,
has been added to the synapse/res/templates
directory. If you are using a
custom template directory, you may want to copy the template over and modify it.
Note that as of v1.20.0, templates do not need to be included in custom template directories for Synapse to start. The default templates will be used if a custom template cannot be found.
This page will appear to the user after clicking a password reset link that has been emailed to them.
To complete password reset, the page must include a way to make a POST
request to
/_synapse/client/password_reset/{medium}/submit_token
with the query parameters from the original link, presented as a URL-encoded form. See the file
itself for more details.
Updated Single Sign-on HTML Templates
The saml_error.html
template was removed from Synapse and replaced with the
sso_error.html
template. If your Synapse is configured to use SAML and a
custom sso_redirect_confirm_template_dir
configuration then any customisations
of the saml_error.html
template will need to be merged into the sso_error.html
template. These templates are similar, but the parameters are slightly different:
- The
msg
parameter should be renamed toerror_description
. - There is no longer a
code
parameter for the response code. - A string
error
parameter is available that includes a short hint of why a user is seeing the error page.
Upgrading to v1.18.0
Docker -py3 suffix will be removed in future versions
From 10th August 2020, we will no longer publish Docker images with the -py3 tag suffix. The images tagged with the -py3 suffix have been identical to the non-suffixed tags since release 0.99.0, and the suffix is obsolete.
On 10th August, we will remove the latest-py3 tag. Existing per-release tags (such as v1.18.0-py3) will not be removed, but no new -py3 tags will be added.
Scripts relying on the -py3 suffix will need to be updated.
Redis replication is now recommended in lieu of TCP replication
When setting up worker processes, we now recommend the use of a Redis server for replication. The old direct TCP connection method is deprecated and will be removed in a future release. See docs/workers.md for more details.
Upgrading to v1.14.0
This version includes a database update which is run as part of the upgrade, and which may take a couple of minutes in the case of a large server. Synapse will not respond to HTTP requests while this update is taking place.
Upgrading to v1.13.0
Incorrect database migration in old synapse versions
A bug was introduced in Synapse 1.4.0 which could cause the room directory to be incomplete or empty if Synapse was upgraded directly from v1.2.1 or earlier, to versions between v1.4.0 and v1.12.x.
- This will not be a problem for Synapse installations which were:
-
- created at v1.4.0 or later,
- upgraded via v1.3.x, or
- upgraded straight from v1.2.1 or earlier to v1.13.0 or later.
If completeness of the room directory is a concern, installations which are affected can be repaired as follows:
-
Run the following sql from a psql or sqlite3 console:
INSERT INTO background_updates (update_name, progress_json, depends_on) VALUES ('populate_stats_process_rooms', '{}', 'current_state_events_membership'); INSERT INTO background_updates (update_name, progress_json, depends_on) VALUES ('populate_stats_process_users', '{}', 'populate_stats_process_rooms');
-
Restart synapse.
New Single Sign-on HTML Templates
New templates (sso_auth_confirm.html
, sso_auth_success.html
, and
sso_account_deactivated.html
) were added to Synapse. If your Synapse is
configured to use SSO and a custom sso_redirect_confirm_template_dir
configuration then these templates will need to be copied from
synapse/res/templates into that directory.
Synapse SSO Plugins Method Deprecation
Plugins using the complete_sso_login
method of
synapse.module_api.ModuleApi
should update to using the async/await
version complete_sso_login_async
which includes additional checks. The
non-async version is considered deprecated.
Rolling back to v1.12.4 after a failed upgrade
v1.13.0 includes a lot of large changes. If something problematic occurs, you may want to roll-back to a previous version of Synapse. Because v1.13.0 also includes a new database schema version, reverting that version is also required alongside the generic rollback instructions mentioned above. In short, to roll back to v1.12.4 you need to:
-
Stop the server
-
Decrease the schema version in the database:
UPDATE schema_version SET version = 57;
-
Downgrade Synapse by following the instructions for your installation method in the "Rolling back to older versions" section above.
Upgrading to v1.12.0
This version includes a database update which is run as part of the upgrade, and which may take some time (several hours in the case of a large server). Synapse will not respond to HTTP requests while this update is taking place.