Change device list streams to have one row per ID (#7010)
* Add 'device_lists_outbound_pokes' as extra table. This makes sure we check all the relevant tables to get the current max stream ID. Currently not doing so isn't problematic as the max stream ID in `device_lists_outbound_pokes` is the same as in `device_lists_stream`, however that will change. * Change device lists stream to have one row per id. This will make it possible to process the streams more incrementally, avoiding having to process large chunks at once. * Change device list replication to match new semantics. Instead of sending down batches of user ID/host tuples, send down a row per entity (user ID or host). * Newsfile * Remove handling of multiple rows per ID * Fix worker handling * Comments from review
No related branches found
No related tags found
Showing
- changelog.d/7010.misc 1 addition, 0 deletionschangelog.d/7010.misc
- synapse/app/generic_worker.py 7 additions, 3 deletionssynapse/app/generic_worker.py
- synapse/replication/slave/storage/devices.py 23 additions, 13 deletionssynapse/replication/slave/storage/devices.py
- synapse/replication/tcp/streams/_base.py 9 additions, 4 deletionssynapse/replication/tcp/streams/_base.py
- synapse/storage/data_stores/main/__init__.py 4 additions, 1 deletionsynapse/storage/data_stores/main/__init__.py
- synapse/storage/data_stores/main/devices.py 68 additions, 64 deletionssynapse/storage/data_stores/main/devices.py
- tests/storage/test_devices.py 0 additions, 45 deletionstests/storage/test_devices.py
Loading
Please register or sign in to comment