synapse/tests
Erik Johnston c46d452c7c
Fix bug where purging history could lead to increase in disk space usage (#18131)
When purging history, we try and delete any state groups that become
unreferenced (i.e. there are no longer any events that directly
reference them). When we delete a state group that is referenced by
another state group, we "de-delta" that state group so that it no longer
refers to the state group that is deleted.

There are two bugs with this approach that we fix here:
1. There is a common pattern where we end up storing two state groups
when persisting a state event: the state before and after the new state
event, where the latter is stored as a delta to the former. When
deleting state groups we only deleted the "new" state and left (and
potentially de-deltaed) the old state. This was due to a bug/typo when
trying to find referenced state groups.
2. There are times where we store unreferenced state groups in the DB,
during the purging of history these would not get rechecked and instead
always de-deltaed. Instead, we should check for this case and delete any
unreferenced state groups rather than de-deltaing them.

The effect of the above bugs is that when purging history we'd end up
with lots of unreferenced state groups that had been de-deltaed (i.e.
stored as the full state). This can lead to dramatic increases in
storage space used.
2025-02-03 19:04:19 +00:00
..
api Consolidate SSO redirects through /_matrix/client/v3/login/sso/redirect(/{idpId}) (#17972) 2024-11-29 11:26:37 -06:00
app Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
appservice Format files with Ruff (#17643) 2024-09-02 12:39:04 +01:00
config Add macaroon_secret_key_path config option (#17983) 2024-12-16 18:01:33 -06:00
crypto Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
events Never autojoin deactivated & suspended users. (#18073) 2025-01-28 00:37:24 +00:00
federation Fix join being denied after being invited over federation (#18075) 2025-01-27 11:21:10 -06:00
handlers Add locking to more safely delete state groups: Part 1 (#18107) 2025-02-03 17:29:15 +00:00
http Fix mypy errors on Twisted 24.11.0 (#17998) 2024-12-18 11:49:38 +00:00
logging Removal: Remove support for experimental msc3886 (#17638) 2024-11-13 14:10:20 +00:00
media Enable authenticated media by default (#17889) 2024-11-20 14:48:22 +00:00
metrics Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
module_api Format files with Ruff (#17643) 2024-09-02 12:39:04 +01:00
push MSC4076: Add disable_badge_count to pusher configuration (#17975) 2024-12-03 22:58:43 +00:00
replication Fix join being denied after being invited over federation (#18075) 2025-01-27 11:21:10 -06:00
rest Add locking to more safely delete state groups: Part 2 (#18130) 2025-02-03 17:58:55 +00:00
scripts Update license headers 2023-11-21 15:29:58 -05:00
server_notices Sliding Sync: Add cache to get_tags_for_room(...) (#17730) 2024-09-19 12:43:26 +01:00
state Update license headers 2023-11-21 15:29:58 -05:00
storage Fix bug where purging history could lead to increase in disk space usage (#18131) 2025-02-03 19:04:19 +00:00
test_utils Add media tests for a CMYK JPEG image (#17786) 2024-10-23 18:26:01 +01:00
types Use immutabledict instead of frozendict (#15113) 2023-03-22 17:15:34 +00:00
util Implement MSC4133 to support custom profile fields. (#17488) 2025-01-21 11:11:04 +00:00
__init__.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
server.py Removal: Remove support for experimental msc3886 (#17638) 2024-11-13 14:10:20 +00:00
test_distributor.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_event_auth.py Format files with Ruff (#17643) 2024-09-02 12:39:04 +01:00
test_mau.py Update license headers 2023-11-21 15:29:58 -05:00
test_phone_home.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_rust.py Add missing type hints to tests. (#15027) 2023-02-08 19:52:37 +00:00
test_server.py Removal: Remove support for experimental msc3886 (#17638) 2024-11-13 14:10:20 +00:00
test_state.py Add locking to more safely delete state groups: Part 1 (#18107) 2025-02-03 17:29:15 +00:00
test_terms_auth.py Update license headers 2023-11-21 15:29:58 -05:00
test_test_utils.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_types.py Format files with Ruff (#17643) 2024-09-02 12:39:04 +01:00
test_visibility.py Include user membership on events (#17282) 2024-06-13 21:45:54 +00:00
unittest.py Support for MSC4190: device management for application services (#17705) 2024-12-04 12:04:49 +01:00
utils.py Fix join being denied after being invited over federation (#18075) 2025-01-27 11:21:10 -06:00