Commit Graph

24918 Commits

Author SHA1 Message Date
dependabot[bot]
7de9ac01a0
Bump once_cell from 1.19.0 to 1.21.3 (#18710) 2025-07-22 11:55:28 +01:00
Devon Hudson
4e118aecd0
Reduce log spam when client stops downloading media while it is being streamed to them (ConsumerStopProducingError) (#18699)
The case where a consumer stops downloading media that is currently
being streamed is now able to be handled explicitly.
That scenario isn't really an error, it is expected behaviour.

This PR adds a custom exception which allows us to drop the log level
for this specific case from `WARNING` to `INFO`.


### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [X] Pull request is based on the develop branch
* [X] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [X] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))

---------

Co-authored-by: Eric Eastwood <erice@element.io>
2025-07-21 20:11:46 +00:00
Shay
11a11414c5
Add an option to issue redactions as admin user on admin redaction endpoint (#18671)
Currently the [admin redaction
endpoint](https://element-hq.github.io/synapse/latest/admin_api/user_admin_api.html#redact-all-the-events-of-a-user)
defaults to puppeting the user being redacted. This PR adds an optional
param `use_admin`, which when provided issues the redactions as the
admin user instead.
2025-07-21 16:40:45 +00:00
Quentin Gliech
8a4e2e826d
Dedicated MAS API (#18520)
This introduces a dedicated API for MAS to consume. Companion PR on the
MAS side: element-hq/matrix-authentication-service#4801

This has a few advantages over the previous admin API:

- it works on workers (this will be documented once we stabilise MSC3861
as a whole)
 - it is more efficient because more focused
 - it propagates trace contexts from MAS
- it is only accessible to MAS (through the shared secret) and will let
us remove the weird hack that made this token 'admin' with a ghost
'@__oidc_admin:' user

The next MAS version should support it, but will be opt-in. The version
after that should use this new API by default

---------

Co-authored-by: Eric Eastwood <erice@element.io>
2025-07-21 16:17:43 +00:00
reivilibre
875269eb53
Add experimental and incomplete support for MSC4306: Thread Subscriptions. (#18674)
Implements:
[MSC4306](https://github.com/matrix-org/matrix-spec-proposals/blob/rei/msc_thread_subscriptions/proposals/4306-thread-subscriptions.md)
(partially)

What's missing:
- Changes to push rules

Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2025-07-21 15:54:28 +01:00
Eric Eastwood
56f5097d1c
Prevent dirty Cargo.lock changes from install (#18693)
Spawning from https://github.com/element-hq/synapse/pull/18689

Example CI failure that will stop people from leaving stray `Cargo.lock`
changes behind,

```
Error: Cargo.lock has uncommitted changes after install. Please run 'poetry install --extras all' and commit the Cargo.lock changes.
```
2025-07-18 10:28:10 -05:00
Strac Consulting Engineers Pty Ltd
797fa5728d
18697.misc (#18698) 2025-07-18 15:37:34 +01:00
Strac Consulting Engineers Pty Ltd
c58d7ade38
Update msc3861_delegated.py spelling correction (#18697) 2025-07-18 15:37:17 +01:00
Alex Durham
6127aa0d50
Don't allow tagnames longer than 255 bytes (#18660) 2025-07-18 14:23:28 +01:00
Quentin Gliech
5ea2cf2484
Move device changes off the main process (#18581)
The main goal of this PR is to handle device list changes onto multiple
writers, off the main process, so that we can have logins happening
whilst Synapse is rolling-restarting.

This is quite an intrusive change, so I would advise to review this
commit by commit; I tried to keep the history as clean as possible.

There are a few things to consider:

- the `device_list_key` in stream tokens becomes a
`MultiWriterStreamToken`, which has a few implications in sync and on
the storage layer
- we had a split between `DeviceHandler` and `DeviceWorkerHandler` for
master vs. worker process. I've kept this split, but making it rather
writer vs. non-writer worker, using method overrides for doing
replication calls when needed
- there are a few operations that need to happen on a single worker at a
time. Instead of using cross-worker locks, for now I made them run on
the first writer on the list

---------

Co-authored-by: Eric Eastwood <erice@element.io>
2025-07-18 09:06:14 +02:00
Eric Eastwood
66504d1144
Fix dirty Cargo.lock changes appearing after install (base64) (#18689)
Normal install results in `Cargo.lock` changes constantly popping up for me as I navigate my
branches. This was probably caused by some Depdendabot PR updating the
`Cargo.toml` without `Cargo.lock` or something.

```
poetry install --extras all
```

In another PR, I've also added CI to ensure we don't leave `Cargo.lock`
changes behind to avoid this annoyance in the future ->
https://github.com/element-hq/synapse/pull/18693
2025-07-17 15:20:42 -05:00
Eric Eastwood
cda922830e
Clean up MetricsResource and Prometheus hacks (#18687)
Clean up `MetricsResource`, Prometheus hacks
(`_set_prometheus_client_use_created_metrics`), and better document why
we care about having a separate `metrics` listener type.

These clean-up changes have been split out from
https://github.com/element-hq/synapse/pull/18584 since that PR was
closed.
2025-07-17 11:57:19 -05:00
dependabot[bot]
f0f9a82ca4
Bump lxml from 5.4.0 to 6.0.0 (#18631)
Bumps [lxml](https://github.com/lxml/lxml) from 5.4.0 to 6.0.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/lxml/lxml/blob/master/CHANGES.txt">lxml's
changelog</a>.</em></p>
<blockquote>
<h1>6.0.0 (2025-06-26)</h1>
<h2>Features added</h2>
<ul>
<li>
<p>GH#463: <code>lxml.html.diff</code> is faster and provides
structurally better diffs.
Original patch by Steven Fernandez.</p>
</li>
<li>
<p>GH#405: The factories <code>Element</code> and
<code>ElementTree</code> can now be used in type hints.</p>
</li>
<li>
<p>GH#448: Parsing from <code>memoryview</code> and other buffers is
supported to allow zero-copy parsing.</p>
</li>
<li>
<p>GH#437: <code>lxml.html.builder</code> was missing several HTML5 tag
names.
Patch by Nick Tarleton.</p>
</li>
<li>
<p>GH#458: <code>CDATA</code> can now be written into the incremental
<code>xmlfile()</code> writer.
Original patch by Lane Shaw.</p>
</li>
<li>
<p>A new parser option <code>decompress=False</code> was added that
controls the automatic
input decompression when using libxml2 2.15.0 or later. Disabling this
option
by default will effectively prevent decompression bombs when handling
untrusted
input. Code that depends on automatic decompression must enable this
option.
Note that libxml2 2.15.0 was not released yet, so this option currently
has no
effect but can already be used.</p>
</li>
<li>
<p>The set of compile time / runtime supported libxml2 feature names is
available as
<code>etree.LIBXML_COMPILED_FEATURES</code> and
<code>etree.LIBXML_FEATURES</code>.
This currently includes
<code>catalog</code>, <code>ftp</code>, <code>html</code>,
<code>http</code>, <code>iconv</code>, <code>icu</code>,
<code>lzma</code>, <code>regexp</code>, <code>schematron</code>,
<code>xmlschema</code>, <code>xpath</code>, <code>zlib</code>.</p>
</li>
</ul>
<h2>Bugs fixed</h2>
<ul>
<li>
<p>GH#353: Predicates in <code>.find*()</code> could mishandle tag
indices if a default namespace is provided.
Original patch by Luise K.</p>
</li>
<li>
<p>GH#272: The <code>head</code> and <code>body</code> properties of
<code>lxml.html</code> elements failed if no such element
was found.  They now return <code>None</code> instead.
Original patch by FVolral.</p>
</li>
<li>
<p>Tag names provided by code (API, not data) that are longer than
<code>INT_MAX</code>
could be truncated or mishandled in other ways.</p>
</li>
<li>
<p><code>.text_content()</code> on <code>lxml.html</code> elements
accidentally returned a &quot;smart string&quot;
without additional information.  It now returns a plain string.</p>
</li>
<li>
<p>LP#2109931: When building lxml with coverage reporting, it now
disables the <code>sys.monitoring</code>
support due to the lack of support in <a
href="https://redirect.github.com/nedbat/coveragepy/issues/1790">nedbat/coveragepy#1790</a></p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="2a67034bc2"><code>2a67034</code></a>
Prepare release of 6.0.0.</li>
<li><a
href="e0b4e02182"><code>e0b4e02</code></a>
Update changelog.</li>
<li><a
href="d3f4dcf689"><code>d3f4dcf</code></a>
Build: Upgrade libxml2 to latest 2.14.4.</li>
<li><a
href="014e51cce3"><code>014e51c</code></a>
Build: Add Windows arm64 wheel builds (<a
href="https://redirect.github.com/lxml/lxml/issues/465">GH-465</a>)</li>
<li><a
href="d3914dcb6a"><code>d3914dc</code></a>
Only use &quot;xmlCtxtIsStopped()&quot; from libxml2 2.15.0 on since it
fails to cover ...</li>
<li><a
href="6e41390275"><code>6e41390</code></a>
Avoid reading the deprecated &quot;disableSAX&quot; attribute of
&quot;xmlParserCtxt&quot;.</li>
<li><a
href="f85da81b1d"><code>f85da81</code></a>
Use newer &quot;language_level=3&quot; in ElementPath module.</li>
<li><a
href="787315eb54"><code>787315e</code></a>
Build: bump pypa/cibuildwheel in the github-actions group (<a
href="https://redirect.github.com/lxml/lxml/issues/464">#464</a>)</li>
<li><a
href="fb3adb1dce"><code>fb3adb1</code></a>
Readme: Add project income report for 2024.</li>
<li><a
href="8e61a757c8"><code>8e61a75</code></a>
Fit cached tuple more nicely into cachelines.</li>
<li>Additional commits viewable in <a
href="https://github.com/lxml/lxml/compare/lxml-5.4.0...lxml-6.0.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=lxml&package-manager=pip&previous-version=5.4.0&new-version=6.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-17 17:24:57 +02:00
Quentin Gliech
f031105eee
Start and stop the Tokio runtime with the Twisted reactor (#18691)
Fixes https://github.com/element-hq/synapse/issues/18659

This changes the Tokio runtime to be attached to the Twisted reactor.
This way, the Tokio runtime starts when the Twisted reactor starts, and
*not* when the module gets loaded.

This is important as starting the runtime on module load meant that it
broke when Synapse was started with `daemonize`/`synctl`, as forks only
retain the calling threads, breaking the Tokio runtime.

This also changes so that the HttpClient gets the Twisted reactor
explicitly as parameter instead of loading it from
`twisted.internet.reactor`
2025-07-17 15:15:11 +00:00
Kim Brose
a0d6469069
fix schema and docs of rc_delayed_event_mgmt (#18692)
Signed-off-by: Kim Brose <kim.brose@nordeck.net>
2025-07-17 15:54:33 +01:00
Patrick Cloke
84991317d0
Stabilize support for custom profile fields. (#18635) 2025-07-17 14:15:07 +01:00
Tulir Asokan
56c166cbf0
Include event_id when getting state with ?format=event (#18675) 2025-07-17 11:28:17 +01:00
Märt
a07e26a936
Fix sequence migration for autoincrement tables in synapse_port_db (#18677)
Closes https://github.com/element-hq/synapse/issues/18053 - the
sliding sync tables will now migrate properly.
2025-07-17 11:13:36 +01:00
dependabot[bot]
b07dc6a27d
Bump types-jsonschema from 4.24.0.20250528 to 4.24.0.20250708 (#18682) 2025-07-17 11:07:15 +01:00
dependabot[bot]
42297bfceb
Bump ruff from 0.12.2 to 0.12.3 (#18683) 2025-07-17 11:06:54 +01:00
Eric Eastwood
88785dbaeb
Refactor cache metrics to be homeserver-scoped (#18604)
(add `server_name` label to cache metrics).

Part of https://github.com/element-hq/synapse/issues/18592
2025-07-16 16:04:57 -05:00
Eric Eastwood
fc10a5ee29
Refactor Measure block metrics to be homeserver-scoped (v2) (#18601)
Refactor `Measure` block metrics to be homeserver-scoped (add
`server_name` label to block metrics).

Part of https://github.com/element-hq/synapse/issues/18592


### Testing strategy

#### See behavior of previous `metrics` listener

 1. Add the `metrics` listener in your `homeserver.yaml`
    ```yaml
    listeners:
      - port: 9323
        type: metrics
        bind_addresses: ['127.0.0.1']
    ```
1. Start the homeserver: `poetry run synapse_homeserver --config-path
homeserver.yaml`
 1. Fetch `http://localhost:9323/metrics`
1. Observe response includes the block metrics
(`synapse_util_metrics_block_count`,
`synapse_util_metrics_block_in_flight`, etc)


#### See behavior of the `http` `metrics` resource

1. Add the `metrics` resource to a new or existing `http` listeners in
your `homeserver.yaml`
    ```yaml
    listeners:
      - port: 9322
        type: http
        bind_addresses: ['127.0.0.1']
        resources:
          - names: [metrics]
            compress: false
    ```
1. Start the homeserver: `poetry run synapse_homeserver --config-path
homeserver.yaml`
1. Fetch `http://localhost:9322/_synapse/metrics` (it's just a `GET`
request so you can even do in the browser)
1. Observe response includes the block metrics
(`synapse_util_metrics_block_count`,
`synapse_util_metrics_block_in_flight`, etc)
2025-07-15 15:55:23 -05:00
Eric Eastwood
d72c278a07
Remove allow_no_prev_events option (MSC2716 cleanup) (#18676)
This option is no longer used
since we backed out the MSC2716 changes in
https://github.com/matrix-org/synapse/pull/15748 and is even mentioned
as a follow-up task in the PR description there.

The `allow_no_prev_events` option was first introduced in
https://github.com/matrix-org/synapse/pull/11243 to support MSC2716 back
in the day.
2025-07-15 15:53:56 -05:00
Johannes Marbach
b274d6561c
Document that some config options for the user directory are in violation of the Matrix spec (#18548)
Fix #17534

Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
2025-07-15 13:25:25 -05:00
Andrew Morgan
49cb78376e
Advertise support for Matrix v1.12 (#18647) 2025-07-15 15:07:20 +01:00
Eric Eastwood
88f38ea149
Correct version that recaptcha_{private,public}_key_path config options were introduced (#18684)
Introduced in https://github.com/element-hq/synapse/pull/17984

I already see a
[`v1.134.0rc1`](https://github.com/element-hq/synapse/releases/tag/v1.134.0rc1)
tag from 5 days ago so I assume
https://github.com/element-hq/synapse/pull/17984 will actually ship in
the next release (which will be `v1.135.0`)
2025-07-15 09:05:45 -05:00
Andrew Morgan
5f027adb33
Update URL Preview code to work with lxml 6.0.0 (#18622) 2025-07-15 15:04:29 +01:00
Erik Johnston
e6dbbbb315 Merge remote-tracking branch 'origin/master' into develop 2025-07-15 14:55:25 +01:00
dependabot[bot]
78ce4dc26f
Bump mypy from 1.13.0 to 1.16.1 (#18653) 2025-07-15 14:42:54 +01:00
Erik Johnston
60be549c0c 1.134.0 2025-07-15 14:22:54 +01:00
reivilibre
97d2738eef
Fix CPU and database spinning when retrying sending events to servers whilst at the same time purging those events. (#18499)
Fixes: #18491

Fix hotlooping due to skipped PDUs if there is still no progress to be
made.
This could bite if the event was purged since being skipped during
catch-up.

Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2025-07-15 12:01:41 +01:00
dependabot[bot]
945e22303c
Bump phonenumbers from 9.0.8 to 9.0.9 (#18681) 2025-07-15 11:47:59 +01:00
V02460
481c4e2b55
Add recaptcha_{private,public}_key_path config option (#17984)
Another config option on my quest to a `*_path` variant for every
secret. Adds the config options `recaptcha_private_key_path` and
`recaptcha_public_key_path`. Tests and docs are included.

A public key is of course no secret, but it is closely related to the
private key, so it’s still useful to have a `*_path` variant for it.
2025-07-14 11:37:36 -05:00
Travis Ralston
5129668449
Allow admins to see soft failed events (if they want to) (#18238) 2025-07-14 16:55:19 +01:00
dependabot[bot]
3c13c3bebf
Bump base64 from 0.21.7 to 0.22.1 (#18666)
Bumps [base64](https://github.com/marshallpierce/rust-base64) from
0.21.7 to 0.22.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md">base64's
changelog</a>.</em></p>
<blockquote>
<h1>0.22.1</h1>
<ul>
<li>Correct the symbols used for the predefined
<code>alphabet::BIN_HEX</code>.</li>
</ul>
<h1>0.22.0</h1>
<ul>
<li><code>DecodeSliceError::OutputSliceTooSmall</code> is now
conservative rather than precise. That is, the error will only occur if
the decoded output <em>cannot</em> fit, meaning that
<code>Engine::decode_slice</code> can now be used with exactly-sized
output slices. As part of this, <code>Engine::internal_decode</code> now
returns <code>DecodeSliceError</code> instead of
<code>DecodeError</code>, but that is not expected to affect any
external callers.</li>
<li><code>DecodeError::InvalidLength</code> now refers specifically to
the <em>number of valid symbols</em> being invalid (i.e. <code>len % 4
== 1</code>), rather than just the number of input bytes. This avoids
confusing scenarios when based on interpretation you could make a case
for either <code>InvalidLength</code> or <code>InvalidByte</code> being
appropriate.</li>
<li>Decoding is somewhat faster (5-10%)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="e144006974"><code>e144006</code></a>
v0.22.1</li>
<li><a
href="64cca59ddb"><code>64cca59</code></a>
Merge pull request <a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/271">#271</a>
from JobanSD/patch-1</li>
<li><a
href="838355e0ac"><code>838355e</code></a>
Correct BinHex 4.0 alphabet according to specifications</li>
<li><a
href="bf15ccf30a"><code>bf15ccf</code></a>
Merge pull request <a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/270">#270</a>
from marshallpierce/mp/clippy</li>
<li><a
href="fc6aabee8a"><code>fc6aabe</code></a>
Appease clippy</li>
<li><a
href="9a518a2d5d"><code>9a518a2</code></a>
Merge pull request <a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/267">#267</a>
from bdura/patch-1</li>
<li><a
href="d96c80f242"><code>d96c80f</code></a>
Merge branch 'marshallpierce:master' into patch-1</li>
<li><a
href="5d70ba7576"><code>5d70ba7</code></a>
Merge pull request <a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/269">#269</a>
from marshallpierce/mp/decode-precisely</li>
<li><a
href="efb6c006c7"><code>efb6c00</code></a>
Release notes</li>
<li><a
href="2b91084a31"><code>2b91084</code></a>
Add some tests to boost coverage</li>
<li>Additional commits viewable in <a
href="https://github.com/marshallpierce/rust-base64/compare/v0.21.7...v0.22.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=base64&package-manager=cargo&previous-version=0.21.7&new-version=0.22.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-11 13:26:59 +00:00
Quentin Gliech
1e5e6a48be
Use markdown-it-py instead of commonmark in the release script. (#18637)
`commonmark` has been deprecated in favor of `markdown-it-py`, and its
type hints have been [removed from
typeshed](https://github.com/python/typeshed/issues/13725).

This switches the release script to use `markdown-it-py` instead of
`commonmark` for parsing the `CHANGES.md`
2025-07-11 12:54:46 +00:00
Andrew Morgan
947216abc0
Update latest_deps workflow to migrate poetry --no-dev -> --without dev (#18617) 2025-07-11 12:34:37 +01:00
V02460
c5999cf452
Document config file merge behavior (#18664)
Explains in the doc comment of `synapse.config._base.read_config_file`
how config files are merged.
2025-07-11 11:15:12 +01:00
Quentin Gliech
28c9ed3ccb
Remove unnecessary replication calls (#18564)
This should be reviewed commit by commit.

Nowadays it's trivial to propagate cache invalidations, which means we
can move some things off the main process, and not go through HTTP
replication.

`ReplicationGetQueryRestServlet` appeared to be unused, and was very
weird, as it was being called if the current instance is the main one…
to RPC to the main one (if no instance is set on a replication client,
it makes it to the main process)

The other two handlers could be relatively trivially moved to any
workers, moving some methods to the worker store.

**I've intentionally not removed the replication servlets yet** so that
it's safe to rollout, and will do another PR that clean those up to
remove on the N+1 version
2025-07-11 08:47:54 +00:00
Quentin Gliech
1dc29563c1
Move registrations off the main worker (#18552)
This is mainly moving a few store methods around. Note that this doesn't
yet remove the replication servlet to avoid breaking during rollout.
2025-07-10 13:13:27 +00:00
Erik Johnston
66daf0bfae
Add ability to limit amount uploaded by a user (#18527)
You can now configure how much media can be uploaded by a user in a
given time period.

Note the first commit here is a refactor of create/upload content
function
2025-07-10 13:39:09 +01:00
Johannes Marbach
b9b8775db7
Add plain-text handling for rich-text topics as per MSC3765 (#18195)
This implements
https://github.com/matrix-org/matrix-spec-proposals/pull/3765 which is
already merged and, therefore, can use stable identifiers.

For `/publicRooms` and `/hierarchy`, the topic is read from the
eponymous field of the `current_state_events` table. Rather than
introduce further columns in this table, I changed the insertion /
update logic to write the plain-text topic from the rich topic into the
existing field. This will not take effect for existing rooms unless
their topic is changed. However, existing rooms shouldn't have rich
topics to begin with.

Similarly, for server-side search, I changed the insertion logic of the
`event_search` table to prefer the value from the rich topic. Again,
existing events shouldn't have rich topics and, therefore, don't need to
be migrated in the table.

Spec doc: https://spec.matrix.org/v1.15/client-server-api/#mroomtopic

Part of supporting Matrix v1.15:
https://spec.matrix.org/v1.15/client-server-api/#mroomtopic

Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
Co-authored-by: Eric Eastwood <erice@element.io>
2025-07-09 14:13:54 -05:00
Johannes Marbach
e1b429d88e
Add experimental support for MSC4277: Harmonizing the reporting endpoints (#18263)
[MSC4277](https://github.com/matrix-org/matrix-spec-proposals/pull/4277):
Harmonizing the reporting endpoints
2025-07-09 14:08:21 -05:00
Andrew Morgan
8c1e60045c Merge branch 'release-v1.134' into develop 2025-07-09 14:38:52 +01:00
Erik Johnston
bf0370162f
Speed up inserting into stream_positions (#18672)
By ensuring we don't do a no-op `UPDATE`, as this causes new tuples to
be written in postgres.
2025-07-09 11:48:06 +01:00
Andrew Morgan
5c2765bd7a 1.134.0rc1 2025-07-09 11:31:01 +01:00
Quentin Gliech
f8a7872ddb
Use native arm runners for building wheels (#18618)
This takes down the CI time to build wheels from 50 minutes to <10
minutes.

**It also fixes macOS ARM builds, and includes more ARM builds in
general** (we were ignoring pypy and musl before). This doesn't cost
much for us to do this, reasons for not doing this is 1. space on PyPI
and 2. keeping them 'officially' supported?

This is the list of wheels this built (`+` are the ones added):

```diff
  matrix_synapse-1.133.0-cp39-abi3-macosx_10_9_x86_64.whl
+ matrix_synapse-1.133.0-cp39-abi3-macosx_11_0_arm64.whl
  matrix_synapse-1.133.0-cp39-abi3-manylinux_2_28_aarch64.whl
  matrix_synapse-1.133.0-cp39-abi3-manylinux_2_28_x86_64.whl
+ matrix_synapse-1.133.0-cp39-abi3-musllinux_1_2_aarch64.whl
  matrix_synapse-1.133.0-cp39-abi3-musllinux_1_2_x86_64.whl
  matrix_synapse-1.133.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl
+ matrix_synapse-1.133.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl
+ matrix_synapse-1.133.0-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl
  matrix_synapse-1.133.0-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl
  matrix_synapse-1.133.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl
+ matrix_synapse-1.133.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl
+ matrix_synapse-1.133.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl
  matrix_synapse-1.133.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl
```

And the numbers aaaaare 🥁 

-
[before](https://github.com/element-hq/synapse/actions/runs/16072488018):
54 minutes
-
[after](https://github.com/element-hq/synapse/actions/runs/16004034949?pr=18618):
10 minutes

**Revert
[e43b0f9](e43b0f9bd1)
before merging**
2025-07-08 11:30:36 +00:00
Andrew Morgan
b8ad9bf1b7
Add .zed/ to .gitignore (#18623) 2025-07-08 11:57:23 +01:00
Quentin Gliech
9b86458900
Speed up the building of Docker images using ARM CI workers (#18620)
This splits the building of docker images in 2 jobs, one for each
platform, using the native ARM runners for arm64.

The tricky part here is to get back a nice multi-arch manifest.
Previously, you'd do that by pushing each platform image in two distinct
tags, then referencing them in a multi-arch manifest. Nowadays, it's
possible to push images by their digest only, then creating the manifest
for those pushed digests separately

This is inspired by the Docker docs on how to distribute multi-platform
image builds:
https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners

`ghcr.io/element-hq/synapse:sha-c733dd6` is an example image that got
built by this workflow (there is a temporary sha-* tag on
workflow_dispatch runs to help trying out the workflow)

I also had to make sure we sign the manifests correctly:


```
$ cosign verify --certificate-oidc-issuer https://token.actions.githubusercontent.com --certificate-identity-regexp 'https://github.com/element-hq/synapse/.github/workflows/docker.yml@.*' ghcr.io/element-hq/synapse:sha-c733dd6

Verification for ghcr.io/element-hq/synapse:sha-c733dd6 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The code-signing certificate was verified using trusted certificate authority certificates
```

And the numbers aaaaare 🥁 

-
[before](https://github.com/element-hq/synapse/actions/runs/16118229296/job/45477093703):
30 minutes
-
[after](https://github.com/element-hq/synapse/actions/runs/16021743575):
4 minutes

---------

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2025-07-08 10:53:12 +00:00
dependabot[bot]
57220706b2
Bump typing-extensions from 4.14.0 to 4.14.1 (#18668)
Bumps [typing-extensions](https://github.com/python/typing_extensions)
from 4.14.0 to 4.14.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/python/typing_extensions/releases">typing-extensions's
releases</a>.</em></p>
<blockquote>
<h2>4.14.1</h2>
<h1>Release 4.14.1 (July 4, 2025)</h1>
<ul>
<li>Fix usage of <code>typing_extensions.TypedDict</code> nested inside
other types
(e.g., <code>typing.Type[typing_extensions.TypedDict]</code>). This is
not allowed by the
type system but worked on older versions, so we maintain support.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/python/typing_extensions/blob/main/CHANGELOG.md">typing-extensions's
changelog</a>.</em></p>
<blockquote>
<h1>Release 4.14.1 (July 4, 2025)</h1>
<ul>
<li>Fix usage of <code>typing_extensions.TypedDict</code> nested inside
other types
(e.g., <code>typing.Type[typing_extensions.TypedDict]</code>). This is
not allowed by the
type system but worked on older versions, so we maintain support.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="42027aba35"><code>42027ab</code></a>
Prepare release 4.14.1 (<a
href="https://redirect.github.com/python/typing_extensions/issues/620">#620</a>)</li>
<li><a
href="59d2c20858"><code>59d2c20</code></a>
Fix off by one in pickle protocol tests (<a
href="https://redirect.github.com/python/typing_extensions/issues/618">#618</a>)</li>
<li><a
href="40e22ebb2c"><code>40e22eb</code></a>
Do not use slots for <code>_TypedDictSpecialForm</code> (<a
href="https://redirect.github.com/python/typing_extensions/issues/616">#616</a>)</li>
<li><a
href="d17c456d36"><code>d17c456</code></a>
allow TypedDict as a type argument (<a
href="https://redirect.github.com/python/typing_extensions/issues/614">#614</a>)</li>
<li>See full diff in <a
href="https://github.com/python/typing_extensions/compare/4.14.0...4.14.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=typing-extensions&package-manager=pip&previous-version=4.14.0&new-version=4.14.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-07 13:24:30 +00:00