Replace post meta sync storage with dedicated wp_collaboration table

[josephfusco]

The real-time collaboration sync layer currently stores messages as post meta, which works but creates side effects at scale. This moves it to a dedicated wp_collaboration table purpose-built for the workload.

Additionally, the subsystem has been renamed from 'sync' to 'collaboration' throughout. A deprecated wp-sync/v1 REST route alias is preserved for backward compatibility with the Gutenberg plugin during its transition.

The beta 1–3 implementation stores sync messages as post meta on a private wp_sync_storage post type. Post meta is designed for static key-value data, not high-frequency transient message passing. This mismatch causes:

1. Data loss during compaction

When the sync log gets long, a client cleans it up by deleting all messages then re-inserting the ones that still matter. Between delete and re-insert, the history is empty. Any editor polling during that window sees nothing; any edit sent during it is silently dropped. Editors poll multiple times per second, so the gap gets hit regularly with two or more people editing.

Reproduce with the included script:

npm run env:cli -- eval-file tools/local-env/scripts/collaboration-perf/DO_NOT_RELEASE_prove-beta3-post_meta_data_loss.php

2. Silent cursor collisions

Each sync message gets a millisecond timestamp as its ID. When two editors send updates within the same millisecond, both messages get the same ID. The second one is skipped. The edit vanishes with no error.

3. Cache thrashing

Every sync write touches post meta. Every post meta write invalidates the site-wide post query cache (wp_cache_set_posts_last_changed()). One open editor tab continuously invalidates caches for every visitor on the entire site. Five active editors can cause 20–30 cache invalidations per second, degrading object caching for front-end traffic.

Comparison

Problem	Beta 1–3 (post meta)	This PR (wp_collaboration table)
Compaction	Multi-step delete + re-insert; race window	Single atomic DELETE … WHERE cursor <
Message IDs	Millisecond timestamps; collisions possible	Auto-increment; guaranteed unique ordering
Cache impact	Every write invalidates posts_last_changed	No post meta hooks; no cache churn

A purpose-built table with auto-increment IDs eliminates all three at the root: no post meta hooks fire, compaction is a single atomic DELETE, and auto-increment IDs guarantee unique ordering. The WP_Collaboration_Storage interface and WP_HTTP_Polling_Collaboration_Server are unchanged.

Testing

# Unit tests
npm run test:php -- --filter WP_Test_REST_Collaboration_Server

# E2E tests — 3 concurrent users (presence, sync, undo/redo)
npm run test:e2e -- tests/e2e/specs/collaboration/

# Performance benchmark
npm run test:performance:collaboration

Note for beta 1–3 testers: If upgrading from a previous beta, the old wp_sync_updates table is no longer used and can be dropped: npm run env:cli -- db query "DROP TABLE IF EXISTS wp_sync_updates"

Credits

• @westonruter — cache thrashing analysis (wordpress-develop #11002)
• @maxschmeling — compaction race condition (Gutenberg #75835)
• @mindctrl — meta_id cursor approach, awareness→transients, race condition testing (wordpress-develop #11067)
• @chriszarate — meta_value indexing limitations

Trac ticket: https://core.trac.wordpress.org/ticket/64696

Use of AI Tools

Co-authored with Claude Code (Opus 4.6), used to synthesize discussion across related tickets and PRs into a single implementation. All code was reviewed and tested before submission.

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props joefusco, peterwilsoncc, mindctrl, westonruter, paulkevan, dd32, czarate.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[peterwilsoncc]

First pass note inline.

This approach had occurred to me and I think it's probably quite a good idea.

But, as always, there is a but.

I know @m has been reluctant in the past to add new tables to the database schema. I guess (and Matt will be able to correct me) that's to avoid ending up with a database scheme of dozens and dozens of tables.

As not all sites run the database upgrade routine on a regular basis (I think wordpress.org is often behind) as it can be quite a burden on a site, if this approach is to be taken it would need to include a check for the table's existence in a couple of locations:

• The option_{$option} hook
• Before displaying the options field on the writing page

Some form of filtering would need to be available for sites that are using custom data stores.

Sorry to be that person. I'm not trying to be negative, just share knowledge I've learnt over the last few years.

[josephfusco]

@peterwilsoncc Thanks for the thorough review — please don't apologize, this is exactly the kind of institutional knowledge that's invaluable.

As not all sites run the database upgrade routine on a regular basis (I think wordpress.org is often behind) as it can be quite a burden on a site, if this approach is to be taken it would need to include a check for the table's existence in a couple of locations:

Completely agree. I believe I've audited every code path that reads wp_enable_real_time_collaboration to map out what actually needs protection.

• The option_{$option} hook

When the setting is saved, the flow through options.php → update_option() writes to the wp_options table. No code is hooked onto that option change that would interact with the sync_updates table, so toggling the setting is safe regardless of table state.

• Before displaying the options field on the writing page

The checkbox in options-writing.php calls get_option() to check/uncheck itself — it doesn't query the sync_updates table, so it renders safely whether or not the upgrade has run.

The code path that touches the table is the REST route registration in rest-api.php:433, which instantiates WP_Sync_Table_Storage. That's already gated with a db_version >= 61697 check (c13f1cd), so the table is never accessed on sites that haven't run the upgrade.

Some form of filtering would need to be available for sites that are using custom data stores.

The wp_sync_storage filter (line 443) should cover this — it lets sites swap out WP_Sync_Table_Storage for any custom WP_Sync_Storage implementation (Redis, etc.).

Let me know if I've missed anything or if you think additional safeguards would be worthwhile!

[westonruter]

@peterwilsoncc:

As not all sites run the database upgrade routine on a regular basis (I think wordpress.org is often behind) as it can be quite a burden on a site, if this approach is to be taken it would need to include a check for the table's existence in a couple of locations:

• The option_{$option} hook
• Before displaying the options field on the writing page

Some form of filtering would need to be available for sites that are using custom data stores.

Yeah, this has been an issue, but I think it's primarily/only really an issue for database tables that are used for frontend features. Given that the proposed wp_sync_updates table here is exclusively used in the post editor which is in the admin, and access to the admin requires first doing a DB upgrade screen, then I think this wouldn't be a concern.

I know Matt has been reluctant in the past to add new tables to the database schema. I guess (and Matt will be able to correct me) that's to avoid ending up with a database scheme of dozens and dozens of tables.

I haven't touched custom tables for a long time, so I'm not aware of the state of the art here. But I do know that WooCommerce made the switch to using custom tables instead of postmeta for the sake of scalabilitiy, simplicity, and reliability. Doing a quick search in WPDirectory and I see that Jetpack, Yoast, Elementor, WPForms, WordFence, and many others use custom tables as well.

[peterwilsoncc]

Yeah, this has been an issue, but I think it's primarily/only really an issue for database tables that are used for frontend features. Given that the proposed wp_sync_updates table here is exclusively used in the post editor which is in the admin, and access to the admin requires first doing a DB upgrade screen, then I think this wouldn't be a concern.

This isn't quite correct, on multi site installs site (ie, non super admins) don't get prompted to upgrade the database tables and globally upgrading tables by super admins can be prevented using a filter:

add_filter( 'map_meta_cap', function( $caps, $cap ) {
	if ( 'upgrade_network' === $cap ) {
		$caps[] = 'do_not_allow';
	}
	return $caps;
}, 10, 4 );

To reproduce:

1. Install MS
2. Create a sub site
3. Add a user to the network
4. Add the new user as an admin on the sub site
5. Login as sub site admin
6. Bump DB version in version.php
7. Visit dashboard as sub site admin
8. Observe lack of db upgrade prompt
9. Login as super admin
10. Observe but don't click db upgrade prompt
11. You can ignore the prompt as long as you wish as a super admin
12. Add the code above as a mu-plugin
13. Prompt disappears
14. Visit /wp-admin/network/upgrade.php
15. Observe permission denied message
16. Remove mu-plugin added above to avoid confusion later.

Keep in mind that the sub site admin can enable site RTC on the writing page, even though they can't update the tables.

I haven't touched custom tables for a long time, so I'm not aware of the state of the art here. But I do know that WooCommerce made the switch to using custom tables instead of postmeta for the sake of scalabilitiy, simplicity, and reliability. Doing a quick search in WPDirectory and I see that Jetpack, Yoast, Elementor, WPForms, WordFence, and many others use custom tables as well.

Not really relevant for the core schema.

[westonruter]

Not really relevant for the core schema.

Seems relevant to me, as plugins may opt to introduce new tables if the core schema doesn't provide an efficient way to store the desired data. In the same way, if a new feature for core can't be represented efficiently in the core schema, then so too should a new table be considered.

[peterwilsoncc]

@westonruter Well, no, because my point wasn't about whether the tables would be more performant (I said it's probably a good idea in my comment), my point was that Matt has a reluctance to add more tables to the database schema. Whether that reluctance still remains is worth asking but if it does then the plugin actions aren't relevant to core.

[westonruter]

@josephfusco It looks like the types can be made more specific. For example:

diff --git a/src/wp-includes/collaboration/class-wp-collaboration-table-storage.php b/src/wp-includes/collaboration/class-wp-collaboration-table-storage.php
index 894777e608..8c62cc1ab0 100644
--- a/src/wp-includes/collaboration/class-wp-collaboration-table-storage.php
+++ b/src/wp-includes/collaboration/class-wp-collaboration-table-storage.php
@@ -15,6 +15,8 @@
  * @since 7.0.0
  *
  * @access private
+ *
+ * @phpstan-import-type AwarenessState from WP_Collaboration_Storage
  */
 class WP_Collaboration_Table_Storage implements WP_Collaboration_Storage {
 	/**
@@ -73,7 +75,8 @@ class WP_Collaboration_Table_Storage implements WP_Collaboration_Storage {
 	 *
 	 * @param string $room    Room identifier.
 	 * @param int    $timeout Seconds before an awareness entry is considered expired.
-	 * @return array<int, array{client_id: int, state: mixed, wp_user_id: int}> Awareness entries.
+	 * @return array<int, array> Awareness entries.
+	 * @phpstan-return array<int, AwarenessState>
 	 */
 	public function get_awareness_state( string $room, int $timeout = 30 ): array {
 		global $wpdb;
@@ -241,10 +244,10 @@ class WP_Collaboration_Table_Storage implements WP_Collaboration_Storage {
 	 *
 	 * @global wpdb $wpdb WordPress database abstraction object.
 	 *
-	 * @param string $room       Room identifier.
-	 * @param int    $client_id  Client identifier.
-	 * @param array  $state      Serializable awareness state for this client.
-	 * @param int    $wp_user_id WordPress user ID that owns this client.
+	 * @param string               $room       Room identifier.
+	 * @param int                  $client_id  Client identifier.
+	 * @param array<string, mixed> $state      Serializable awareness state for this client.
+	 * @param int                  $wp_user_id WordPress user ID that owns this client.
 	 * @return bool True on success, false on failure.
 	 */
 	public function set_awareness_state( string $room, int $client_id, array $state, int $wp_user_id ): bool {
diff --git a/src/wp-includes/collaboration/interface-wp-collaboration-storage.php b/src/wp-includes/collaboration/interface-wp-collaboration-storage.php
index 9550384da5..13dec0d8da 100644
--- a/src/wp-includes/collaboration/interface-wp-collaboration-storage.php
+++ b/src/wp-includes/collaboration/interface-wp-collaboration-storage.php
@@ -10,6 +10,8 @@
  * Interface for collaboration storage backends used by the collaborative editing server.
  *
  * @since 7.0.0
+ *
+ * @phpstan-type AwarenessState array{client_id: int, state: array<string, mixed>, wp_user_id: int}
  */
 interface WP_Collaboration_Storage {
 	/**
@@ -32,7 +34,8 @@ interface WP_Collaboration_Storage {
 	 *
 	 * @param string $room    Room identifier.
 	 * @param int    $timeout Seconds before an awareness entry is considered expired.
-	 * @return array<int, array{client_id: int, state: mixed, wp_user_id: int}> Awareness entries.
+	 * @return array<int, array> Awareness entries.
+	 * @phpstan-return array<int, AwarenessState>
 	 */
 	public function get_awareness_state( string $room, int $timeout = 30 ): array;
 
@@ -86,10 +89,10 @@ interface WP_Collaboration_Storage {
 	 *
 	 * @since 7.0.0
 	 *
-	 * @param string $room       Room identifier.
-	 * @param int    $client_id  Client identifier.
-	 * @param array  $state      Serializable awareness state for this client.
-	 * @param int    $wp_user_id WordPress user ID that owns this client.
+	 * @param string               $room       Room identifier.
+	 * @param int                  $client_id  Client identifier.
+	 * @param array<string, mixed> $state      Serializable awareness state for this client.
+	 * @param int                  $wp_user_id WordPress user ID that owns this client.
 	 * @return bool True on success, false on failure.
 	 */
 	public function set_awareness_state( string $room, int $client_id, array $state, int $wp_user_id ): bool;