<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by ReadySet on Medium]]></title>
        <description><![CDATA[Stories by ReadySet on Medium]]></description>
        <link>https://medium.com/@readysettech?source=rss-13336def6db6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*NoZt4xj6jfqKTo_SwCsa5Q.png</url>
            <title>Stories by ReadySet on Medium</title>
            <link>https://medium.com/@readysettech?source=rss-13336def6db6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 01:27:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@readysettech/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Monitor Readyset with Percona PMM | Readyset]]></title>
            <link>https://medium.com/@readysettech/monitor-readyset-with-percona-pmm-readyset-08045e166d7a?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/08045e166d7a</guid>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Tue, 17 Nov 2026 00:00:27 GMT</pubDate>
            <atom:updated>2026-11-18T14:57:05.693Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> Tutorials</p><p>When an application starts reading through Readyset, the database sees fewer requests for cached queries. That’s a useful change to see in PMM. But we’d also like to follow the reads that Readyset handles: when they hit the cache, when results refresh, and how replication is progressing.</p><p>We brought that view into Percona Monitoring and Management (PMM) through its <a href="https://docs.percona.com/percona-monitoring-and-management/3/install-pmm/install-pmm-client/connect-database/external.html">external-service support</a>. Readyset already has Grafana dashboards in its repository, so we could start with those and watch them fill with data. Here’s how to connect the two in your own PMM installation.</p><p>PMM can monitor Readyset by adding it as an external service and importing Readyset’s Grafana dashboards. In this blog, we will walk through how to do it using Readyset 1.32.0 from the <a href="https://github.com/readysettech/readyset/releases/tag/stable-261022">official stable GitHub release</a> and PMM 3.10.0. We tested MySQL 8.4.13 and PostgreSQL 16.15 separately on AWS, with one Readyset deployment in each run.</p><h3><a href="https://readyset.io/blog/monitor-readyset-with-percona-pmm#connect-readyset-to-pmm">Connect Readyset to PMM</a></h3><p><strong>Prometheus metrics are enabled by default.</strong> By default Readyset listens on 0.0.0.0:6034, exposing port 6034 on all IPv4 interfaces. We recommend restricting access to the metrics port to your monitoring network if you&#39;re not behind a private network by configuring PROMETHEUS_METRICS or METRICS_ADDRESS.</p><p>For a package installation, install PMM Client on the Readyset server and register it with your running PMM Server using Percona’s <a href="https://docs.percona.com/percona-monitoring-and-management/3/install-pmm/install-pmm-client/package_manager.html">PMM Client installation and registration guide</a>.</p><p>Once PMM Client is registered, add Readyset as an external service:</p><pre>pmm-admin add external \<br>  --service-name=readyset-cache \<br>  --listen-port=6034 \<br>  --metrics-path=/metrics \<br>  --scheme=http \<br>  --metrics-mode=push \<br>  --group=readyset \<br>  --skip-connection-check</pre><p>A successful registration reports:</p><pre>External Service added.</pre><p>The service appears as readyset-cache.</p><p>PMM’s --metrics-mode=push controls delivery of monitoring data to PMM Server, regardless of the Readyset cache type.</p><h3><a href="https://readyset.io/blog/monitor-readyset-with-percona-pmm#how-do-you-import-the-readyset-dashboards-into-pmm">How do you import the Readyset dashboards into PMM?</a></h3><p>With <a href="https://readyset.io/docs/cache/cache">push-based caching</a>, Readyset updates cached results from the database replication stream. Pull-based caching refreshes results by querying the database according to the cache policy. Download the corresponding dashboards from the <a href="https://github.com/readysettech/readyset/tree/stable-261022/build/docker/grafana/dashboards">Readyset repository at the tested release</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yCo2ZBsOwtezS3adbs9vgg.png" /><figcaption><a href="https://raw.githubusercontent.com/readysettech/readyset/stable-261022/build/docker/grafana/dashboards/shallow_caching.json">shallow_caching.json</a> | <a href="https://raw.githubusercontent.com/readysettech/readyset/stable-261022/build/docker/grafana/dashboards/replication_performance.json">replication_performance.json</a></figcaption></figure><p>The repository retains the filename shallow_caching.json. Name that dashboard Pull-based Caching during import.</p><p>If PMM monitors several Readyset deployments, the Replication Performance dashboard needs changes before you can use it to inspect one deployment. It has no deployment selector, and its unmodified queries do not isolate a deployment. The Pull-based Caching dashboard does have that selector.</p><p>Both dashboards use PMM’s <strong>Metrics</strong> data source. With the JSON file downloaded, we can turn those metrics into a dashboard.</p><p><strong>Open the import page.</strong> In PMM, choose <strong>All dashboards</strong>, open <strong>New</strong>, and click <strong>Import</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OmboeuVRukupAl_W.png" /></figure><p><em>Follow the highlights: 1. All dashboards, 2. New, 3. Import.</em></p><p><strong>Load the dashboard definition.</strong> Open the downloaded JSON file and copy its contents. Paste them under <strong>Import via dashboard JSON model</strong>, then click the <strong>Load</strong> button directly beneath that field.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*l9grLD1vn629QU9J.png" /></figure><p><em>1. Paste the dashboard JSON. 2. Click Load beneath the JSON field.</em></p><p><strong>Connect the dashboard to your metrics.</strong> For the pull-based dashboard, set the name to Pull-based Caching. Select <strong>Metrics</strong> for the DS_PROMETHEUS input, then click <strong>Import</strong>. Repeat the process with the Replication Performance JSON file, keeping its dashboard name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HCcSeEJNI1bYhI5m.png" /></figure><p><em>1. Set the dashboard name. 2. Choose Metrics. 3. Click Import.</em></p><p>For Pull-based Caching, select your Readyset deployment, leave <strong>Query Filter</strong> set to <strong>All</strong>, and choose a time range covering your application’s activity. <strong>Last 15 minutes</strong> covered our test queries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/749/0*hj1MIpEwPjmM-DzE.jpg" /></figure><p><em>The existing repository dashboard, imported into PMM and populated by live queries. Only the display name and data-source mapping were changed.</em></p><h3><a href="https://readyset.io/blog/monitor-readyset-with-percona-pmm#validate-readyset-performance-with-percona-pmm">Validate Readyset Performance with Percona PMM</a></h3><p>With the dashboard open, we ran reads through Readyset and saw cache hits, misses, and refresh activity appear. Across the MySQL and PostgreSQL runs, all 27 dashboard expressions executed successfully; 25 returned series in each run.</p><p>The connection also stayed registered after restarting PMM Client: the service and agent identifiers were retained, and fresh metrics arrived afterward.</p><p>If you follow until here, now it’s your turn. Try it with a query you know. Choose a read whose results can tolerate some delay after an update, cache it in a test environment using the <a href="https://readyset.io/docs/cache/cache">Readyset cache guide</a>, and run it repeatedly while watching the dashboard. Keep the database’s PMM monitoring open too. We found this a useful starting point for following the work Readyset serves alongside the work that still reaches the database.</p><p>Feedback is welcome. If you need a dashboard that isn’t covered here, tell us what you’d like to monitor by opening an issue at <a href="https://github.com/readysettech/readyset">Readyset github</a>. We’re happy to add it.</p><p><em>Originally published at </em><a href="https://readyset.io/blog/monitor-readyset-with-percona-pmm"><em>https://readyset.io</em></a><em> on November 17, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZbeBFTpZKDedvin1Lk8kTw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=08045e166d7a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Readyset vs Redis for SQL Query Caching | Readyset]]></title>
            <link>https://medium.com/@readysettech/readyset-vs-redis-for-sql-query-caching-readyset-f66be19e6ebb?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/f66be19e6ebb</guid>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[redis]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Mon, 16 Nov 2026 00:00:35 GMT</pubDate>
            <atom:updated>2026-11-16T14:49:40.471Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> Database Caching</p><p>Redis is a fast in-memory data store, so using it as a query cache means the application owns the cache keys, serialization, TTLs, and invalidation. Readyset is a <a href="https://readyset.io/blog/what-is-a-wire-compatible-sql-caching-proxy">wire-compatible SQL caching proxy</a> for PostgreSQL and MySQL: its pull-based cache stores query results behind the database connection, so the application keeps sending plain SQL. Readyset replaces Redis only as a SQL query result cache; Redis keeps everything else.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#what-is-redis-good-at">What is Redis good at?</a></h3><p>Redis is a data structure server. It keeps strings, hashes, JSON documents, sets, sorted sets, streams and other types in memory, and it serves reads from memory in under a millisecond.</p><p>Redis does many jobs besides caching: session storage with expiry, rate limiting, leaderboards built on sorted sets, job queues, pub/sub messaging and event streams.</p><p>Redis also has a broad ecosystem. Client libraries cover most major languages, and frameworks such as Spring and Django include Redis-backed caching. For data the database does not own, such as API responses, rendered fragments and computed values, Redis is the natural cache.</p><p>Caching SQL query results is a narrower job than the rest of what Redis does, and it is a job where Redis asks a lot of the application.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#what-changes-when-you-use-redis-as-a-query-cache">What changes when you use Redis as a query cache?</a></h3><p>Using Redis as a SQL query cache is application-level caching. Redis documents the pattern as cache-aside: the application reads Redis first, falls back to the database on a miss, writes the result back to Redis, and on every update writes the database and then invalidates the cache key.</p><p>Most steps of cache-aside are application code and application policy. The application designs a key for each query and its parameters, serializes the result set into a Redis value and back, and chooses a TTL for each kind of data. The Redis server needs an eviction policy such as allkeys-lru for when it reaches its maxmemory limit.</p><p>Cache invalidation is the hardest part. On every write, the application must find every cached result the write affects, including joins and aggregates that read other tables, and delete or rewrite them. In <a href="https://readyset.io/blog/cache-invalidation-sql-queries">our invalidation experiment</a>, six read paths served the same story page on a MySQL database under identical writes: the database directly, two Redis look-aside caches, two Readyset pull-based caches and a Readyset push-based cache. A plausible set of per-table rules for Redis never converged to correct results after an author rename, because the cached pages that joined the renamed table were never invalidated.</p><p>Cache stampede is the second hardest part. When a popular key expires under high concurrency, many processes query the database for the same record at once. Redis documents Lua scripting for mutex locks or probabilistic early refresh as the defense, and the application carries that code.</p><p>Frameworks reduce the boilerplate, but the invalidation policy still lives in the application. A change data capture (CDC) pipeline such as Redis Data Integration or Debezium can keep Redis current from the database instead. It is a pipeline to build and run, it holds table records as hashes or JSON documents, and the whole working set must fit in memory.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#how-is-readyset-different-from-redis">How is Readyset different from Redis?</a></h3><p>Readyset is a <a href="https://readyset.io/blog/what-is-a-wire-compatible-sql-caching-proxy">wire-compatible SQL caching proxy</a> for PostgreSQL and MySQL. The application connects to Readyset with its existing driver, Readyset answers the queries that have a cache, and Readyset proxies every other query to the database.</p><p>Readyset’s <a href="https://readyset.io/docs/concepts/shallow-caching">pull-based cache</a> is the counterpart to a Redis query cache. A pull-based cache stores a query’s result set in Readyset’s memory for a time-to-live (TTL) and refreshes it from the database in the background. CREATE SHALLOW CACHE creates a pull-based cache, for example CREATE SHALLOW CACHE WITH (POLICY TTL 60 SECONDS REFRESH 10 SECONDS) FROM followed by the query. Pull-based caches accept almost any read query and need no replication stream.</p><p>Readyset builds cache keys without application code. Readyset turns the literal values in a query into parameters, so queries that differ only in their values share one cache, with a separate entry for each parameter set.</p><p>Readyset also handles refresh and stampedes. With REFRESH, a read that finds an older entry gets the cached result immediately while Readyset refreshes it in the background. With COALESCE, concurrent requests for the same missing result share one query to the database. With <a href="https://readyset.io/blog/adaptive-cache">ADAPTIVE</a>, Readyset shortens the refresh interval of a result that keeps changing and lengthens it for one that does not.</p><p>Readyset sizes its pull-based store as a share of its memory limit. When the store is full, Readyset keeps the results that save the most database time, weighing how often each is requested against how long its query takes, and it serves result sets above a configured size from the database without caching them.</p><p>Readyset is a Redis cache alternative for SQL query results on MySQL and PostgreSQL, and not for sessions, queues or other data.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#readyset-vs-redis-at-a-glance">Readyset vs Redis at a glance</a></h3><p>Redis is a general-purpose data store used here as a query cache, while Readyset is a SQL proxy built to cache query results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VV0ACBX6xPUZMkFXG0bkCQ.png" /></figure><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#what-are-the-alternatives-to-a-redis-cache-for-sql-queries">What are the alternatives to a Redis cache for SQL queries?</a></h3><p>Alternatives to a Redis query cache differ in who does the work of keeping results correct.</p><p>Another in-memory key-value cache used as a query cache follows the same cache-aside pattern, so the application still owns the keys, serialization and invalidation.</p><p>Read replicas add capacity by running every query against a copy of the database. They spread the work without removing it, and each replica is a full copy of the data to run and pay for.</p><p>Materialized views, in databases that support them, store a query result and refresh it on a schedule or on demand, so the result is only as current as the last refresh.</p><p>A SQL caching proxy moves the cache behind the database connection. Readyset’s pull-based cache is a Redis cache alternative for SQL query results that serves them from memory for a TTL, and its <a href="https://readyset.io/docs/concepts/deep-caching">push-based cache</a> keeps results current by following the database’s replication stream.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#how-fresh-are-cached-results-in-redis-and-in-readyset">How fresh are cached results in Redis and in Readyset?</a></h3><p>Redis can be fresher than a TTL cache when the application deletes a key at the moment it writes. Redis does not do this on its own: a key that no write path invalidates stays stale until its TTL expires or it is evicted. In <a href="https://readyset.io/blog/cache-invalidation-sql-queries">our invalidation experiment</a>, which timed how long after a MySQL write a reader first saw the new value, a correctly invalidated Redis look-aside cache was fresh to within about a millisecond, while a Readyset pull-based cache with a 10-second TTL and a 5-second refresh lagged by seconds. Each write to the Redis look-aside cache was also followed by a burst of identical queries to the database as readers refilled the key.</p><p>A Readyset pull-based cache has no invalidation step, so its results are as old as the refresh interval and TTL allow. An entry that is read continuously stays no more than about one refresh interval behind the database, and an entry nobody reads for a full TTL is dropped.</p><p>A Readyset <a href="https://readyset.io/docs/concepts/deep-caching">push-based cache</a> stays current by following the MySQL binlog or the PostgreSQL write-ahead log (WAL) and not by refreshing on an interval, and it is eventually consistent. A query that must reflect a write immediately is a candidate for a push-based cache, or it can stay uncached so that Readyset proxies it to the database.</p><p>Readyset also provides <a href="https://readyset.io/docs/concepts/read-your-writes">read-your-writes controls</a>: a transaction policy on each cache for reads inside a transaction, and an optional window that sends a session’s reads to the database for a set time after the session writes.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#when-is-redis-the-better-choice-for-caching">When is Redis the better choice for caching?</a></h3><p>Redis is the better choice when the data is not a SQL query result. Redis caches anything the application can serialize, including API responses, rendered fragments and computed values, and Readyset caches only the results of SELECT queries.</p><p>Redis is also the better choice when the application needs control or reach that Readyset does not offer. Redis does not depend on the source of the data, because the application decides what goes into it, so it can cache results from any datastore or from none. Readyset supports PostgreSQL and MySQL. Redis lets the application set a TTL per key and delete a key immediately on a write, while a Readyset pull-based cache is bounded by its TTL and refresh settings.</p><p>Redis costs less infrastructure to add when it is already running. A team with Redis in production for sessions and queues can add a cache to that instance without deploying another service, while Readyset is a new service to deploy and size.</p><p>Readyset has limits of its own. A pull-based cache holds results in memory and runs the query on the database at every refresh, so an aggressive refresh on a high-cardinality cache can raise database load. Readyset does not cache locking reads such as SELECT ... FOR UPDATE.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#do-you-still-need-redis-if-you-run-readyset">Do you still need Redis if you run Readyset?</a></h3><p>Yes, for everything that is not a SQL query result. Sessions, rate limiting, leaderboards, pub/sub, streams and application-computed data stay on Redis.</p><p>Job queues are the clearest example of Redis work that is not caching. Frameworks such as Laravel offer Redis as a queue backend for background jobs, and a job queue is a different feature from caching, one that Readyset does not touch. A team whose framework uses Redis for queues keeps Redis whether or not it moves its query cache.</p><p>Readyset and Redis run side by side. As a Redis cache alternative, Readyset takes SQL query result caching, and Redis keeps every other job.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#what-does-moving-a-redis-query-cache-to-readyset-involve">What does moving a Redis query cache to Readyset involve?</a></h3><p>Start by pointing a staging copy of the application at Readyset by changing the connection string. Readyset proxies every query, and SHOW PROXIED QUERIES lists them with a readyset_supported column that says whether each one can be cached.</p><p>Then create a pull-based cache for the queries you want, one at a time, and set the TTL and refresh interval from what each query’s data tolerates. A query whose results must reflect writes immediately can use a push-based cache or stay uncached.</p><p>Pull-based cache entries are keyed on the query and its parameters, so cache only queries whose results are safe for every client that connects to Readyset. Pull-based caches support Postgres row-level security by keying on the session state its policies read. Automatic caching rejects queries that reference session or user variables, and a cache created explicitly is cached as written.</p><p>Check the first caches against the database before moving more queries. The <a href="https://readyset.io/docs/concepts/query-support">query support page</a> lists what each cache type accepts on MySQL and PostgreSQL.</p><h3><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching#which-should-you-use-for-sql-query-caching">Which should you use for SQL query caching?</a></h3><p>We see Redis and Readyset as tools for different jobs, and what separates them is who owns the cache. Our recommendation is to keep Redis for sessions, queues, rate limits and application data, and to use Readyset’s pull-based cache for SQL query results. Look at SHOW PROXIED QUERIES first and cache the queries that repeat most.</p><p>To go through your own PostgreSQL or MySQL workload with the Readyset team, <a href="https://readyset.io/book-a-demo">book a Readyset demo</a>.</p><p><em>Originally published at </em><a href="https://readyset.io/blog/readyset-vs-redis-for-sql-query-caching"><em>https://readyset.io</em></a><em> on November 16, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MxdaHLuC8X_VsDSEZIpgQA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f66be19e6ebb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Readyset vs Read Replicas: Which Scales Reads Better? | Readyset]]></title>
            <link>https://medium.com/@readysettech/readyset-vs-read-replicas-which-scales-reads-better-readyset-6a4f1ea9e6a9?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a4f1ea9e6a9</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[postgres]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 00:00:53 GMT</pubDate>
            <atom:updated>2026-11-12T15:00:20.634Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> Database Scaling</p><p>A read replica scales reads by keeping a full copy of your MySQL or Postgres database and executing every query against it. Readyset scales reads by caching query results and keeping them fresh from the same replication stream, so a repeated query is looked up instead of executed. Replicas add capacity. Readyset removes work.</p><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#what-does-a-read-replica-actually-do">What does a read replica actually do?</a></h3><p>A read replica is a second database server that holds a full copy of the primary and answers reads. The primary records every change in its replication log, the binary log (binlog) in MySQL and the write-ahead log (WAL) in Postgres, and the read replica replays that log so its tables follow the primary’s.</p><p>A read replica executes every query it receives. Prepared statements can skip parsing, and Postgres can reuse a cached plan, but the execution itself (the index lookups, joins, sorts and aggregation) runs again against the replica’s own tables and buffer pool, even when the identical query finished a millisecond earlier. A replica adds a server’s worth of capacity for that work without shortening it, so an aggregate that is slow on the primary still pays the same execution cost on a replica of the same size.</p><p>A read replica also repeats every write. A MySQL replica applies the same row changes the primary made, secondary indexes included, and a Postgres standby replays the WAL for every changed table and index page, so either one needs enough I/O to keep pace with the primary’s write rate before it serves a single read.</p><p>A read replica needs storage for the full dataset, and usually as much memory as the primary, because it serves the same working set. Infrastructure cost grows linearly with every replica added, and we lay out that arithmetic in <a href="https://readyset.io/blog/why-query-caching-is-the-most-cost-effective-way-to-scale-databases-and-everyones-missing-it">why query caching is the most cost-effective way to scale databases</a>.</p><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#what-does-readyset-do-instead">What does Readyset do instead?</a></h3><p>Readyset is a wire-compatible SQL caching proxy for PostgreSQL and MySQL. The application keeps its driver and its SQL, and the only required change is the connection string, which points at Readyset instead of the database. Readyset answers the queries that have a cache and proxies everything else to the database, including every write and every locking read such as SELECT ... FOR UPDATE.</p><p>Caching in Readyset is chosen per query: CREATE DEEP CACHE FROM followed by the query creates a push-based cache for it, and CREATE SHALLOW CACHE FROM creates a pull-based one. Readyset turns the literal values in a query into parameters, so customer_id = 7 and customer_id = 9 share one cache and become two keys in it.</p><p>A Readyset <a href="https://readyset.io/docs/concepts/deep-caching">push-based cache</a> stays fresh from the same change log a read replica uses. Readyset first snapshots the upstream tables, full rows, into RocksDB on its own local disk. It then follows the MySQL binlog in row format, or a logical replication slot it creates on the Postgres database.</p><p>Push-based caching needs Readyset started with CACHE_MODE=deep or deep-then-shallow, because the default mode, shallow, runs no replication at all.</p><p>Readyset keeps each push-based cache current with incremental view maintenance: when a row changes upstream, Readyset applies only the resulting delta to the cached results that row affects, instead of re-running the query. Readyset’s dataflow engine derives from <a href="https://www.usenix.org/conference/osdi18/presentation/gjengset">Noria</a>, a research project at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), published at the 2018 USENIX Symposium on Operating Systems Design and Implementation (OSDI).</p><p>Readyset uses partial materialization, so a push-based cache holds results only for the parameter values the application has asked for. The first read of a new value is a miss: Readyset computes that one key from its own copy of the tables, in what the documentation calls an <a href="https://readyset.io/docs/concepts/efficiency">upquery</a>, then stores and returns it.</p><p>Later reads of that key are in-memory lookups in Readyset, typically under a millisecond, served by lock-free readers that never wait on replication. A change to a customer nobody has read updates Readyset’s copy of the table and stops at the first step of the graph that holds no state for that customer.</p><p>Readyset caps its memory with --memory-limit. When memory reaches that limit, Readyset evicts the least recently used keys, and an evicted key is computed again on its next read.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*p4GXlx1hZvSM3int.gif" /></figure><p><em>The same query, four times. The read replica executes all four. Readyset computes customer 7 and customer 9 once each and answers the two repeats from memory.</em></p><p>Readyset also offers <a href="https://readyset.io/docs/concepts/shallow-caching">pull-based caches</a>. A pull-based cache stores a query’s result for a time-to-live (TTL) and refreshes it from the database in the background, for example CREATE SHALLOW CACHE WITH (POLICY TTL 60 SECONDS REFRESH 10 SECONDS) FROM followed by the query. With REFRESH, a read that finds an entry older than 10 seconds gets the cached result and starts a refresh; REFRESH EVERY refreshes on a fixed schedule instead.</p><p>Pull-based caches need no replication stream and accept almost any read query. Their freshness comes from the refresh settings and the TTL rather than from writes: an entry read continuously stays roughly one refresh interval behind, and an entry nobody reads for a full TTL is dropped.</p><p>Where a plain TTL cache refreshes every key on one schedule, Readyset can adapt the refresh interval to each result. With the ADAPTIVE option, Readyset shortens the interval for a result that changed at its last refresh and lengthens it for one that didn&#39;t, between 10% and 100% of the configured interval, and caps the extra load those refreshes send to the database. Caches that Readyset creates automatically are always adaptive.</p><p>In our <a href="https://readyset.io/blog/adaptive-cache">benchmark</a>, adaptive refresh cut p95 staleness by 86% compared with a fixed refresh schedule.</p><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#readyset-vs-read-replicas-at-a-glance">Readyset vs read replicas at a glance</a></h3><p>A read replica stores and queries a full copy of the data, while Readyset serves precomputed query results from memory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pLhHA20JLYhsaI4VFAzzxQ.png" /></figure><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#does-readyset-replace-a-read-replica">Does Readyset replace a read replica?</a></h3><p>For read scaling, Readyset can replace read replicas for the queries it caches, because Readyset answers a cached query from memory and that read never reaches the primary or any read replica.</p><p>Readyset does not replace the other jobs of a read replica. Readyset proxies uncached and unsupported queries to the primary and has no failover role, so the primary still needs read replicas or another high-availability mechanism.</p><p>Readyset’s two cache types replace different things. Readyset push-based caching, which maintains results from the binlog or WAL, is the alternative to adding read replicas. Readyset pull-based caching is the alternative to an application-managed key-value cache such as Redis, without the invalidation code that goes with it.</p><p>Teams use Readyset to delay adding read replicas or to run a smaller database tier, while keeping the replicas that availability requires. The cost argument is in <a href="https://readyset.io/blog/real-cost-of-read-replicas">The Real Cost of Read Replicas</a> and <a href="https://readyset.io/blog/why-query-caching-is-the-most-cost-effective-way-to-scale-databases-and-everyones-missing-it">why query caching is the most cost-effective way to scale databases</a>.</p><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#when-is-a-read-replica-the-better-choice">When is a read replica the better choice?</a></h3><p>A read replica is the better choice when the requirement is availability, complete SQL coverage, tighter read consistency, or reads from another region.</p><p><strong>Availability.</strong> A read replica can be promoted to primary when the primary fails, and Readyset cannot. After a failover, Readyset is <a href="https://readyset.io/docs/reference/failover">repointed to the new primary</a>: on MySQL with GTIDs it resumes from the last transaction it applied, without a new snapshot, and on Postgres it creates a new replication slot and takes a full snapshot again.</p><p><strong>SQL coverage.</strong> A replica runs any query the engine supports. Readyset push-based caches support a defined subset of SELECT: a query that combines results with UNION, INTERSECT or EXCEPT, or walks a hierarchy with a recursive common table expression (CTE), cannot have a push-based cache on MySQL or Postgres. A pull-based cache accepts those queries, and anything without a cache is proxied to the primary. The <a href="https://readyset.io/docs/concepts/query-support">query support page</a> lists each shape for both engines and both cache types.</p><p><strong>Consistency.</strong> A read replica can tighten read-after-write behavior at the cost of write latency, and the two engines do it differently. Postgres can make a commit wait until a synchronous standby has applied it, with <a href="https://www.postgresql.org/docs/current/runtime-config-wal.html">synchronous_commit = remote_apply</a>, so a read that starts on that standby afterwards sees the write. MySQL <a href="https://dev.mysql.com/doc/refman/8.4/en/replication-semisync.html">semi-synchronous replication</a> waits only until a replica has received the transaction, so it protects durability without making the replica&#39;s reads current.</p><p><strong>Location and reporting.</strong> A read replica can sit in another region near users, and suits long-running reporting and extract, transform, load (ETL) queries that need no cache.</p><p><strong>Workload shape.</strong> Readyset saves little on queries that rarely repeat with the same parameters, or on primary-key lookups the database already answers in about a millisecond. Our <a href="https://readyset.io/blog/readyset-azure-postgres-gitlab-field-test">GitLab field test on Azure Postgres</a>, which used pull-based caches, showed the same pattern: its heaviest queries ran 5.7x to 26.3x faster at the median, while per-query latency on primary-key lookups came out roughly even.</p><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#what-about-replication-lag-and-consistency">What about replication lag and consistency?</a></h3><p>Read replicas and Readyset both follow the primary asynchronously, so either can return data slightly older than the primary.</p><p>Under heavy write load, a read replica’s lag can grow from milliseconds to seconds, and every read from that replica sees it.</p><p>A Readyset push-based cache is <a href="https://readyset.io/docs/concepts/deep-caching">eventually consistent</a>. Its lag is the time for a change to arrive through the binlog or WAL and for its delta to reach the reader, and it applies only to cached queries, because Readyset proxies uncached queries to the primary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*HSHYGB_4c6bX-8HD.gif" /></figure><p><em>One INSERT reaching both. The replica writes the row and recomputes on the next read; Readyset updates the one stored count the row affects.</em></p><p>Readyset provides two controls for reading your own writes (RYW). Inside a transaction, each cache has a transaction policy: by default Readyset proxies the cached query to the primary for the whole transaction, UNTIL WRITE serves the cache until the transaction writes, and ALWAYS always serves the cache. Outside a transaction, an <a href="https://readyset.io/docs/concepts/read-your-writes">opportunistic window</a>, off by default, sends a session&#39;s reads to the primary for a set number of milliseconds after the session writes. The window narrows the race and is not a guarantee.</p><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#can-you-run-readyset-and-read-replicas-together">Can you run Readyset and read replicas together?</a></h3><p>Yes. Readyset and read replicas run together because they do different jobs.</p><p>The application sends the queries it wants cached through Readyset, and Readyset connects to the primary. Readyset answers cached queries from memory and proxies every other query to the primary, while read replicas keep the roles Readyset does not fill: failover, reporting and analytics, and reads from another region.</p><p>By default, Readyset reads the binlog in MySQL or the WAL in Postgres from the primary. On MySQL, Readyset can instead follow a read replica that writes its own binlog ( log_replica_updates=ON), which adds that replica&#39;s lag to Readyset&#39;s. On Postgres, Readyset follows the primary, where it creates its publication and replication slot. When a read replica is promoted after a primary failure, Readyset is repointed to the new primary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*6oDMngCdut0i8T43HTRhFQ.gif" /></figure><p><em>Readyset handles the repeated reads and the replicas handle failover and reports. After a failover, Readyset follows the promoted replica.</em></p><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#how-do-you-try-readyset-on-your-own-queries">How do you try Readyset on your own queries?</a></h3><p>Readyset shows which of your queries it can cache before you cache anything. Point a staging copy of the application at Readyset, run the usual traffic, and SHOW PROXIED QUERIES lists every query Readyset has passed through, with a readyset_supported column saying whether each one can be cached.</p><p>SHOW PROXIED DEEP QUERIES and SHOW PROXIED SHALLOW QUERIES show the same list for push-based and for pull-based caches, so you can see which queries each cache type would take before creating any.</p><h3><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads#query-caching-vs-read-replicas-which-should-you-use">Query caching vs read replicas: which should you use?</a></h3><p>We see Readyset and read replicas as tools for different parts of read scaling, and what decides between them is how often your reads repeat. Our recommendation is to keep the replicas your failover plan needs, then look at your read traffic before adding more. Put it through Readyset, cache the expensive queries that repeat most, and leave the rest to the replicas: one-off reports, SQL that Readyset doesn’t cache, and reads that must see the latest write.</p><p>To go through your own PostgreSQL or MySQL workload with the Readyset team, <a href="https://readyset.io/book-a-demo">book a Readyset demo</a></p><p><em>Originally published at </em><a href="https://readyset.io/blog/readyset-vs-read-replicas-which-should-you-use-to-scale-reads"><em>https://readyset.io</em></a><em> on November 12, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lIa9DB8MLKSdNLLmyirazA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a4f1ea9e6a9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Is Database Sharding and When Should You Shard? | Readyset]]></title>
            <link>https://medium.com/@readysettech/what-is-database-sharding-and-when-should-you-shard-readyset-5a0a2b344b5d?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/5a0a2b344b5d</guid>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 00:00:42 GMT</pubDate>
            <atom:updated>2026-11-10T15:06:10.216Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> Database Scaling</p><p>Database sharding splits one logical database across several independent database servers, called shards, so that each server stores and serves only part of the rows. A shard key, such as a customer ID or a tenant ID, decides which shard owns each row, and a routing layer sends every query to the shard that holds its data. Sharding lets a relational database grow beyond the storage, memory, and write capacity of a single machine.</p><p>How to shard is well documented. When to shard is harder, because sharding is close to a one-way door: once rows live on different servers, every cross-shard join and schema change costs more, and undoing it means migrating the data back. We decide with three conditions: the data, the working set, or the write load has outgrown what one server can handle. For MySQL and Postgres, we add a size rule from experience: <em>once a single database heads past 2 to 4 TB, its routine operations, restores in particular, stop fitting inside recovery and change windows, and that is the time to start planning a shard.</em></p><h3><a href="https://readyset.io/blog/what-is-database-sharding-and-when-should-you-shard#what-is-database-sharding">What is database sharding?</a></h3><p>Each shard in a sharded database is a complete database server, with its own storage, buffer pool, replicas, and backups. The shards share a schema and hold different rows. A table of three million customers might keep customers 1 to 1 million on shard A, the next million on shard B, and the rest on shard C, so a query for customer 1,500,042 only ever touches shard B.</p><p>The router is whatever knows that mapping. In many MySQL and Postgres shops, it’s the application, which holds a shard map and opens a connection to the right server. It can also be a proxy layer such as <a href="https://vitess.io/">Vitess</a> for MySQL, or an extension such as <a href="https://www.citusdata.com/">Citus</a> for Postgres, both of which present the shards to the application as a single database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e4SC2FvpQU8CXTFKRSP-ag.png" /><figcaption><em>A query that carries the shard key runs on one shard. A query without it runs on every shard, and the router merges the results, a pattern known as scatter/gather.</em></figcaption></figure><p>Sharding gets confused with partitioning and with replication, and the three solve different problems. Partitioning splits a table by a key too, but the partitions all stay on one server and share its disk, memory, and backup. Replication copies every row to more servers, which spreads reads and adds failover targets while leaving each server just as large as before. Only sharding makes each server smaller.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eVjcsb2GArh5ccYkCJVhJg.png" /></figure><h3><a href="https://readyset.io/blog/what-is-database-sharding-and-when-should-you-shard#when-should-you-shard-a-database">When should you shard a database?</a></h3><p>We recommend sharding a MySQL or Postgres database when at least one of three conditions holds:</p><ol><li><strong>The data has outgrown what one server can handle.</strong> The limit is not what the engine can store but the size at which backups, restores, and schema changes no longer fit your recovery and change windows.</li><li><strong>The working set is about to outgrow memory.</strong> The rows and index pages your queries touch every minute no longer fit in the InnoDB buffer pool, or in Postgres shared buffers plus the operating system cache, so reads start going to disk.</li><li><strong>One primary can’t absorb the writes,</strong> and batching, trimming secondary indexes, and faster storage have stopped helping.</li></ol><p>If none of the three holds, sharding adds complexity and buys little. These are the same conditions MongoDB’s documentation lists in its <a href="https://www.mongodb.com/docs/manual/core/sharding-distribute-collection-data/">current manual</a>, which suggests considering sharding once a collection reaches 3TB, a size range we already see in MySQL and Postgres.</p><p>Each condition shows up in MySQL and Postgres in a recognizable way, and each has cheaper fixes to try first:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I_uPwr0hXu44FxUBXb5Knw.png" /></figure><p><em>The three sharding conditions as they show up in MySQL and Postgres. The fixes in the right-hand column buy time; for the write condition, they rarely buy much, because one primary still takes every write.</em></p><p>The first condition is about operating the data, not storing it. InnoDB allows a table of up <a href="https://dev.mysql.com/doc/refman/8.4/en/innodb-limits.html">to 64 TB with the default 16 KB page</a>, and a Postgres table can reach <a href="https://www.postgresql.org/docs/current/limits.html">32 TB with the default 8 KB block</a>. Those are theoretical ceilings, and no one should run a database anywhere near them: performance starts falling apart long before a table gets close. Once the working set outgrows the InnoDB buffer pool, or Postgres shared buffers plus the operating system cache, index lookups and inserts turn into random disk reads. In Postgres, a vacuum pass over a multi-terabyte table can run for hours and fall behind the rate at which the table creates dead rows. The practical limit is the size at which one instance gets too slow to run and too slow to operate, which is what our 2 to 4 TB rule measures.</p><p>Timing matters more than any threshold. Sharding has to start while the database still has headroom, because moving rows onto new shards takes spare I/O and CPU from the very server you’re trying to relieve. On MySQL or Postgres, where sharding usually means choosing a key, changing the application, backfilling, and cutting over, the project runs for months. Hence, the decision belongs to the months before the database runs out of capacity, not the week after.</p><p>Sharding costs are why you wait for one of the three conditions. A query that doesn’t carry the shard key has to run on every shard, and a scatter/gather query like that is only as fast as the slowest shard. Joins and transactions across shards move into the application or into a distributed transaction protocol. Unique constraints and auto-increment IDs need a scheme that works across servers. Every backup, upgrade, and failover now happens once per shard. The shard key itself is the hardest choice to reverse, so it deserves the most design time before the first row moves.</p><h3><a href="https://readyset.io/blog/what-is-database-sharding-and-when-should-you-shard#how-large-can-a-mysql-or-postgres-database-get-before-you-shard-it">How large can a MySQL or Postgres database get before you shard it?</a></h3><p>Our rule of thumb is to start planning a shard once a single MySQL or Postgres database heads past 2 to 4 TB. The range comes from what we’ve seen in production rather than from a formula, and the restore arithmetic later in this section shows why it lands where it does. The limit has nothing to do with what the engine can store. It comes from how long ordinary operations take once the data is that large: building an index, changing a schema, taking a backup and, above all, restoring one. Treat the range as a starting point; your own number comes from your recovery time objective and a timed restore.</p><p>Index builds and schema changes take longer as tables grow, and their online versions have failure modes that get likelier the longer they run. MySQL’s documentation on <a href="https://dev.mysql.com/doc/refman/8.4/en/innodb-online-ddl-limitations.html">online DDL limitations</a> says an online data definition language (DDL) operation “must finish running on the source before it is run on the replica”, so a six-hour index build on the source becomes roughly six hours of replication lag on every replica. Writes that arrive during the build are buffered in a temporary log capped by <a href="https://dev.mysql.com/doc/refman/8.4/en/innodb-parameters.html#sysvar_innodb_online_alter_log_max_size">innodb_online_alter_log_max_size</a>, 128 MiB by default, and when that log fills, &quot;the ALTER TABLE operation fails and all uncommitted concurrent DML operations are rolled back.&quot; In Postgres, <a href="https://www.postgresql.org/docs/current/sql-createindex.html#SQL-CREATEINDEX-CONCURRENTLY">CREATE INDEX CONCURRENTLY</a> &quot;must perform two scans of the table&quot; and wait for every transaction that could use the index, and a failure leaves behind an invalid index that is ignored by queries but &quot;will still consume update overhead.&quot;</p><p>At 200 GB, adding an index is something an engineer does before lunch. At 4 TB the same change needs a maintenance window and someone watching replica lag until it finishes. When a simple DDL change takes that much planning, the database has reached its practical size limit, whatever the disk says.</p><p>Restore time sets the hard ceiling. The recovery time objective (RTO) is the downtime the business accepts after a failure, and a database you can’t restore within it is too large, however well it runs day to day. A restore has to move every byte of the backup onto a working volume before anything else can happen, and that copy alone takes data size divided by throughput. Amazon Elastic Block Store (EBS) includes <a href="https://docs.aws.amazon.com/ebs/latest/userguide/general-purpose.html">125 MiB/s of baseline throughput with every gp3 volume</a>, and gp3 can be provisioned up to 2,000 MiB/s, provided the instance has <a href="https://docs.aws.amazon.com/ebs/latest/userguide/ebs-io-characteristics.html">that much EBS bandwidth</a>. A volume created from a snapshot can be <a href="https://docs.aws.amazon.com/ebs/latest/userguide/ebs-initialize.html">initialized at up to 300 MiB/s</a> at extra cost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nfy9-AnMdTaFte6D8qy_3Q.png" /></figure><p><em>Time to copy a backup onto a working volume, by data size and throughput. This is arithmetic, not a measurement, and it is the fastest a restore can possibly go: it excludes index rebuilds, log replay and warm-up.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zkjdfGaupvfImUwVGYTbcg.png" /></figure><p><em>The restore-floor table as a clock, at gp3’s 125 MiB/s baseline. With a 4-hour RTO, only the 1 TiB database has finished copying when the time is up.</em></p><p>A real restore is slower than the copy. A logical restore with mysqldump or pg_dump rebuilds every index after loading the rows. Point-in-time recovery then replays the MySQL binary log or the Postgres write-ahead log (WAL) from the backup up to the moment of failure. A volume restored from an EBS snapshot pulls its blocks from Amazon S3 on first access, and AWS warns it &quot;might experience increased I/O latency and decreased performance&quot; until every block has arrived. The replicas lost in the same incident need rebuilding too, and until they are, the restored primary has no failover target.</p><p>To find the limit for your own database:</p><ol><li>Restore a production-sized backup on production-class hardware, end to end and including point-in-time recovery, and time it. Data size divided by that time is your effective restore throughput.</li><li>Multiply that throughput by your RTO, after subtracting the time it takes to notice the failure and decide to restore. The result is the largest database you can recover in time.</li><li>Time the largest index build or ALTER TABLE on your biggest table, along with the replica lag it causes. If that doesn&#39;t fit your change windows, your limit is lower than the restore says.</li><li>Compare your growth rate with how long a sharding project takes your team, and start when you’re that far from the limit rather than at it.</li></ol><p>As an example, a team with a four-hour RTO whose measured end-to-end restore runs at 150 MiB/s can recover 14,400 seconds × 150 MiB/s = 2,160,000 MiB, or 2.06 TiB, before detection time is taken off. Double the throughput to 300 MiB/s and the same four hours cover 4,320,000 MiB, or 4.12 TiB. Those two results bracket our 2 to 4 TB rule: a four-hour RTO and a restore pipeline sustaining 150 to 300 MiB/s end to end puts the ceiling for one database in that range. A team with a shorter RTO or a slower restore gets a lower number, and should start the sharding work well before reaching it.</p><h3><a href="https://readyset.io/blog/what-is-database-sharding-and-when-should-you-shard#which-scaling-problems-can-you-solve-without-sharding">Which scaling problems can you solve without sharding?</a></h3><p>Read pressure is the one reason to shard that has a good answer short of sharding. Data size and restore time are properties of the data itself, and write throughput is a property of the single primary, so only splitting the data (or deleting some of it) changes them. Reads behave differently. Applications ask for the same results over and over, and serving those results from somewhere other than the primary takes load off it without moving a single row.</p><p>Readyset is a cache built for that case. Readyset sits between the application and MySQL or Postgres, speaks their wire protocols, and answers cached queries from memory. A deep cache stays current by applying changes from the MySQL binary log or the Postgres WAL through incremental view maintenance, so a write shows up in the cached result without the application invalidating anything. A shallow cache refreshes its results on a time-to-live (TTL) instead. Writes and uncached queries pass through to the database unchanged.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4dB_oY_I6tIIHopUej_ZcA.png" /></figure><p><em>Repeated reads stop at Readyset. The write reaches the primary, and the change returns through the MySQL binlog or Postgres WAL to update the cached result in place.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eCl97u5VKjlx_SMkOCTj6A.png" /></figure><p><em>Which tool addresses which sharding trigger? Readyset and read replicas only act on read load; size, restore time, and write throughput need the data split.</em></p><p>Readyset doesn’t shrink the database or shorten a restore. If a restore already takes longer than your RTO, caching reads leaves that problem exactly where it was. When read load is what’s pushing you toward sharding, a cache moves the date: it buys back the headroom that sharding needs before it starts, so the work runs on a schedule instead of during an incident.</p><p>Readyset also works inside a sharded fleet. Readyset connects to a single upstream database, so in an application-sharded MySQL or Postgres deployment each shard gets its own Readyset, and the shard map points at that Readyset instead of at the shard’s primary. Each cache then holds results for one shard’s rows only. The router still merges queries that fan out across shards.</p><p>The number behind most of these decisions is how long a full restore of your largest database takes, and it’s cheap to measure before anything depends on it. If reads are what’s pushing you toward sharding first, the <a href="https://readyset.io/docs/">Readyset documentation</a> covers putting a cache in front of a single MySQL or Postgres database.</p><p><em>Originally published at </em><a href="https://readyset.io/blog/what-is-database-sharding-and-when-should-you-shard"><em>https://readyset.io</em></a><em> on November 10, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EohCKN7jH_QqHFlLpmSCbw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5a0a2b344b5d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Readyset October 2026 Product Update | Readyset]]></title>
            <link>https://medium.com/@readysettech/readyset-october-2026-product-update-readyset-50653a59e1e9?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/50653a59e1e9</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Mon, 02 Nov 2026 00:00:58 GMT</pubDate>
            <atom:updated>2026-11-03T14:50:31.027Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> Product Updates</p><p>October was a big release month at Readyset, with an exciting new RDST capability and the longest list of features and fixes we’ve shipped in a release this year. Let’s catch up.</p><h3><a href="https://readyset.io/blog/october-product-update-2026#the-rdst-agent-is-here">The RDST Agent Is Here</a></h3><p>RDST now features an always-on agent that watches your Postgres or MySQL database for queries that are getting slower, busier, or changing plans, and flags them with a recommended fix.</p><p>The agent reads your database’s query stats every minute and keeps a day-by-day record of each query shape. Instead of a lifetime average that buries yesterday’s slowdown under three months of fast runs, it checks every five minutes whether each busy query is slower or busier than it usually is. When something changes, RDST agent investigates on its own: it runs EXPLAIN ANALYZE with real values from live traffic, explains what happened, and hands you the fix. Once you apply the fix, it confirms it worked.</p><p>In our walkthrough, a deploy made one query 30x slower. The agent flagged it within nine minutes, found the missing index, and confirmed the fix took the query from 184 ms to 3.8 ms.</p><p>Read-only, so it never writes to your database. Free and open source. Available now in the <a href="https://readyset.io/downloads">RDST desktop app</a>.</p><p><a href="https://readyset.io/blog/meet-the-rdst-agent">Read the full launch post here</a>.</p><h3><a href="https://readyset.io/blog/october-product-update-2026#-product-updates">💬 Product Updates</a></h3><p>A quick naming note: Readyset’s two caching modes, previously called shallow and deep caching, now go by pull-based and push-based caching, named after how each one keeps results fresh. A pull-based cache pulls new results from the database on a schedule, while a push-based cache has every write pushed into the cached results as it happens. Same products, just going by new names.</p><ul><li><strong>New Admin SQL Port for Readyset:</strong> Readyset now ships a dedicated admin SQL port. Admin sessions connect to the readyset schema instead of the upstream database, keep their own credentials, and run commands that affect Readyset itself rather than queries against your data. The first admin-only command, ALTER READYSET SHUTDOWN [RESET], lets operators shut down a Readyset instance, configure the port with --admin-address (default 127.0.0.1:6036) and --admin-credentials (default admin:admin), or disable it with --admin-address disabled.</li><li><strong>Session-Variable Isolation for Pull Caches:</strong> A batch of fixes so pull cache entries no longer leak between sessions whose session variables would produce different query results. Pull cache entries are now keyed on the MySQL upstream’s global value of the session variables that affect results, and on each PostgreSQL connection’s IntervalStyle (whether inherited from the upstream or set in the session), timezone_abbreviations, and default_text_search_config. Sessions with different values now see their own cached results instead of each other&#39;s.</li><li><strong>Dual-Password Support for MySQL:</strong> ALTER READYSET MODIFY USER ... RETAIN CURRENT PASSWORD keeps the old password valid alongside the new one, and ... DISCARD OLD PASSWORD retires it. Naming a user twice in --allowed-users starts a deployment mid-rotation. The readyset.users view reports rotations in progress in a new has_old_password column, so operators can run safe password rotations without a window where auth breaks.</li><li><strong>Multi-Upstream MySQL and Smarter Connection Handling:</strong> --upstream-db-url and --cdc-db-url now accept a comma-separated list of MySQL servers, each with an optional |&lt;weight&gt; suffix. Added --upstream-connect-timeout to limit how long the adapter waits for an upstream connection, --client-connect-timeout-ms to disconnect MySQL and PostgreSQL clients that don&#39;t complete the handshake within 10 seconds by default, and --listen-backlog (default 4096, up from 128) so connection bursts are less likely to leave clients stalled until they time out.</li><li><strong>Safer Memory Limit Defaults:</strong> --memory-limit now defaults to 80% of the memory available to Readyset (the container limit, if set) instead of unlimited. Set --memory-limit to 0 to keep the previous unlimited behavior). In push and push-then-pull cache modes, the pull cache now uses half of --memory-limit when --pull-memory-percent isn&#39;t set, instead of being unbounded.</li><li><strong>Wider Pull Cache Coverage on PostgreSQL:</strong> Pull caches now serve queries over views created after Readyset started (previously these were never pull cached), and support point, box, path, and cidr values over the extended query protocol. Fixed a case where Readyset couldn&#39;t start in pull mode with RLS enabled against a PostgreSQL upstream whose certificate was signed by a private CA.</li><li><strong>Faster Text Queries:</strong> Improved throughput for queries sent as text, which previously serialized on two locks: the schema catalog lock and another shared lock. Workloads with heavy text-protocol query traffic now see higher throughput under concurrency.</li><li><strong>More Queries Served From Cache:</strong> A batch of fixes expanding what Readyset caches instead of proxying upstream. MySQL caches created with ? placeholders are now served from cache rather than sent to the upstream. Row comparisons with literals or placeholders on the left side (such as ($1, $2) = (a, b)) are now cached as lookups, and row comparisons mixing literals and columns (such as (&#39;a&#39;, b) = (&#39;c&#39;, &#39;d&#39;)) are now cached instead of rejected with a placeholder error. Push caches created from a query ID in SHOW PROXIED QUERIES using CREATE CACHE FROM &lt;query id&gt; are also correctly recreated on restart.</li></ul><p>Plus 15 additional fixes and improvements. Check out the full release <a href="https://github.com/readysettech/readyset/releases/tag/stable-261022">here</a>.</p><h3><a href="https://readyset.io/blog/october-product-update-2026#new-on-the-blog--youtube">New on the Blog &amp; YouTube</a></h3><p><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol">Less Work per Query: Speeding Up Readyset’s Text Protocol</a></p><p>Michael Victor Zink writes about making Readyset’s ad hoc queries faster by doing less work per query (skipping no-op rewrites, ignoring unrelated tables, avoiding needless locks, and bounding the query log) for 2.7x higher throughput at 1,024 clients.</p><p><a href="https://readyset.io/blog/what-is-a-wire-compatible-sql-caching-proxy">What Is a Wire-Compatible SQL Caching Proxy?</a></p><p>Caching SQL can live in two places: scattered through your application code with Redis, or in a layer that speaks the database’s own wire protocol. The second option moves cache keys, invalidation logic, and the hit-or-miss decision out of the codebase entirely. This post walks through what wire compatibility actually requires at the MySQL and PostgreSQL protocol level, where Redis still fits, and why incremental view maintenance makes this different from a store you fill and empty by hand.</p><p><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql">Readyset HA with ProxySQL and MySQL</a></p><p>Vinicius Grippa builds a highly available Readyset cache behind a 106 ms report query, using two docker run commands and two INSERT statements in ProxySQL&#39;s admin interface. Nothing changes in the application, the schema, or MySQL. The query drops to 0.716 ms, a 149x improvement, and the setup survives losing a Readyset node or a ProxySQL node with no downtime. Also covers the three ways to confirm the cache is actually serving reads instead of silently falling through to the database.</p><p><a href="https://www.youtube.com/watch?v=BsziEgFDIK0">Watch: “Building the Invisible Cache”</a></p><p>Tanmay Sinha’s talk from MySQL BR Conf 2026 is now up on YouTube. His talk covers why the original MySQL query cache failed, why hand-rolling one on Redis leaves the hardest parts to your team, and what wire-compatible caching changes about that, and is followed by a demonstration by Vinicius Grippa.</p><p><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch">Black October: What Actually Happened, and What To Patch</a></p><p>A primary-source review of the 28 “Black October” security incidents. Only two involved AI-driven attacks. The real damage came from long-lived credentials stored in platform environment variables and shared vendor systems. Includes the specific database-tier patches to apply this week and six steps for cutting credential risk.</p><p><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query">A TTL Cache Needs To Know the Query</a></p><p>Marcelo Altmann on why cache-aside on Redis struggles with SQL once a workload hits real load: a key-value store can’t re-run a query, so it can’t refresh hot keys, collapse miss stampedes, weigh a 900 ms report against a 2 ms lookup, or key by session settings and user permissions. The strongest evidence for the argument comes from Redis itself, which ships two separate products (Smart Cache and Redis Data Integration) aimed at the exact gap.</p><h3><a href="https://readyset.io/blog/october-product-update-2026#get-involved">Get Involved</a></h3><p>Questions, bugs, ideas, or just want to say hi? Find us on <a href="https://go.readyset.io/community">Slack</a>. Looking to contribute? Our <a href="https://github.com/readysettech/readyset/issues?q=is%3Aissue+is%3Aopen+label%3Afirst-issue">newcomer issues</a> are a good place to start.</p><p><em>Originally published at </em><a href="https://readyset.io/blog/october-product-update-2026"><em>https://readyset.io</em></a><em> on November 2, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4t2KRu5iLfg4XGRdwDb7BA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50653a59e1e9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Less Work per Query: Speeding Up Readyset’s Text Protocol | Readyset]]></title>
            <link>https://medium.com/@readysettech/less-work-per-query-speeding-up-readysets-text-protocol-readyset-readyset-b622cefb61ad?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/b622cefb61ad</guid>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Mon, 02 Nov 2026 00:00:39 GMT</pubDate>
            <atom:updated>2026-11-02T14:56:21.461Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> Engineering</p><p>Readyset is in the business of making your queries faster. If you are trying to improve your application’s performance, or reduce load on your database, one of the first things people will tell you is to use prepared statements. What if you can’t? Readyset’s latest release improves performance for ad hoc queries, and this post talks about some of the optimizations we made.</p><h3><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol#why-are-prepared-statements-faster">Why are prepared statements faster?</a></h3><p>Prepared statements (sometimes “binary protocol”) are faster than ad hoc statements (sometimes “text protocol”) for a simple reason: they avoid repeating work. Instead of parsing, planning, and optimizing a query every time it executes, you do that work once in a PREPARE step. You get a handle to the prepared statement, and can EXECUTE it as many times as you want. You pay once for (most of) the work that is common to every statement execution, and each execution (mostly) only pays for the part that matters: actually getting the data.</p><p>Usually, by the time a customer comes to Readyset looking to soothe their database pains, they are already using prepared statements. The problem is, they require application changes. Many customers are trying to improve performance or reduce load on applications that are already legacy, or they don’t have control over (infra teams instead of dev teams), or for some other reason could not be expected to modify queries. That’s why we offer a transparent caching layer in the first place. However, until now, we have not spent much time optimizing our ad hoc statement path. So if one of those customers’ existing applications happened to be using ad hoc statements instead of prepared statements, they should still see Readyset’s signature massive speedup… but it wouldn’t be quite as impressive as the speedup they would get if they were also using prepared statements.</p><p>In our latest release, we have done some optimization of the text protocol path. We knew there would be overhead to trim down when each query arrives as SQL text that has to be handled afresh every time. We don’t have space to discuss all ~12 significant improvements (depending on how you count), so this post will cover 4 interesting examples. Then, a sequel to this post will describe our new parse cache which is an even bigger jump and brings ad hoc query performance <em>much</em> closer to the prepared statement path.</p><p>To give you an idea of the scale of improvement just from these ~12 small changes: in a warmed cached point-read test with 1024 clients, this release handled about 2.7 times more throughput than the previous release when using ad hoc statements. That’s our fast case, and we care a lot about making it faster. But that difference only reflects a portion of all the changes. Often, stuff off the happy/hot path can matter just as much. (Have you ever profiled your error paths?) Anyway, let’s get started.</p><h3><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol#how-does-readyset-process-a-query">How does Readyset process a query?</a></h3><p>I suppose if you want to understand the improvement we’ve made to handling queries when they reach the text protocol (or, for that matter, the first time you PREPARE them), we had better understand what the handling process looks like. Once we know which cache to hit and which key to look up, the cached-read path is already pretty well optimized. But getting from SQL text to that cache and key can involve quite a bit of work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y98bnyA8Sd-ccWBhRv6DWA.png" /></figure><p>Readyset has two caching modes. Pull-based caching is a TTL-based cache of results retrieved from the upstream database. Push-based caching uses partially-stateful dataflow (<a href="https://pdos.csail.mit.edu/papers/noria:osdi18.pdf"><em>Noria</em> OSDI </a>’18) to maintain query results from database changes only for results you care about based on a CDC stream from the upstream database.</p><p>Reading those results starts with finding the corresponding cache, if one exists. This has to respect SQL semantics and things like schema search path/current database. Then we need the cache key: in a prepared statement, the values come from bound parameters, but for an ad hoc query, we have to extract literals from the AST and map them to cache key positions and values. Pull- and push-based caching use similar processes to identify the cache and key, and some parts are shared. (It doesn’t matter much, but technically the following examples assume a push-based cache.)</p><p>For example, with a users(id, name, active) table, Readyset needs to auto-parameterize the literal 42:</p><pre>-- original SQL<br>SELECT * FROM users WHERE id = 42;<br>-- auto-parameterize<br>SELECT * FROM users WHERE id = ?;</pre><p>Then it can look up the corresponding cache for that shape, and then look up the value for key 42. If this were a prepared statement, the 42 wouldn&#39;t need to be auto-parameterized.</p><p>As you can imagine, there is a lot of complexity hiding in and around those “parse, rewrite, route” boxes in the diagram above. That’s why I very cleverly asked Claude to add an “etc.” box too. Session variables and current DDL state affect how a query is rewritten and what the lookup key should be, some literals should be auto-parameterized and not others, and so on. A more detailed diagram is at the end of this post, but even that mostly only calls out the boxes where we made improvements.</p><h3><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol#why-were-we-applying-sql-rewrites-that-were-no-ops">Why were we applying SQL rewrites that were no-ops?</a></h3><p>Auto-parameterization isn’t the only SQL AST rewrite pass. Even for this simple query, we need to expand stars and resolve columns:</p><pre>-- auto-parameterized<br>SELECT * FROM users WHERE id = ?;<br>-- normalize: expand stars, resolve columns, etc.<br>SELECT users.id, users.name, users.active<br>FROM users<br>WHERE users.id = ?;</pre><p>This brings us to one of the ~12 improvements (depending on how you count) we made in this release: aren’t some SQL rewrites no-ops for some queries? For example, our support for subqueries in push-based dataflow relies on an “inline derived tables” rewrite. It might look something like this:</p><pre>-- original SQL<br>SELECT *<br>FROM (SELECT * FROM users WHERE active = true) AS active_users<br>WHERE active_users.id = 42;<br>-- normalize: inline derived tables, etc. (parameters: 42, true)<br>SELECT users.id, users.name, users.active<br>FROM users<br>WHERE users.id = ? AND users.active = ?;</pre><p>The rewrite passes have to do some fairly nontrivial work: resolve columns to the right table in the right subquery, pick aliases to avoid conflicts, move the inner filter into the outer filter, and so on. For more detail, see <a href="https://readyset.io/blog/how-readyset-rewrites-your-sql-inside-the-query-transformation-pipeline">how Readyset rewrites SQL</a>.</p><p>Our original point query doesn’t have a derived table, so it doesn’t need to do any of that work. In this case, it’s obvious. For some of our rewrite passes, it’s less obvious when it can be safely skipped, so we were applying the rewrites even when they were ultimately no-ops.</p><p>This was one of the first things to show up in flamegraphs when we started this optimization effort. But if all we had seen was “rewrites take a good chunk of the CPU cycles for each query”, we might nonetheless have looked elsewhere. However, this was surprising, because this was showing up in flamegraphs with point selects like the first example, too, not just arbitrary query workloads.</p><p>Optimization is all about deciding what to measure. This means both the specific observable metrics you collect and compare, whether you are profiling CPU or instrumenting memory allocations, etc.; but also the workload you run. We used a variety, and saw different problems become more prominent in flamegraphs, throughput measurements, lock contention, etc. depending on which workload was running. In this case, point selects generally represent the <em>best case</em> for the “parse, rewrite, route” part of processing a text-protocol query. As a result, looking at a flame graph for a point select with no derived tables and seeing the derived table rewrite taking up a non-negligible amount of time raised our eyebrows, and took us down that path.</p><p>Here is what the deep rewrites part of the flamegraph looked like profiling point select queries with and without skipping no-op rewrites, saving about 23% of CPU cycles on rewrites:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q7GpFh5jli0O2D9XM77LXg.png" /></figure><p>Of course, that isn’t going to affect queries that <em>do</em> need those rewrites, but most queries will at least skip some rewrites.</p><h3><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol#why-were-unrelated-tables-affecting-query-latency">Why were unrelated tables affecting query latency?</a></h3><p>The query itself isn’t the only part of the workload that matters. In a stateful system, state tends to accumulate. Could unrelated state impinge on the handling of a query? What happens if we run the same point select, but add a few hundred tables into the same database that the query doesn’t even reference?</p><p>To rewrite a query, Readyset needs information about table schemas. In our example, it needs to know which columns users has to expand SELECT *, and where id comes from. But the work we were doing to make that information available also depended on tables that had nothing to do with the query. Effectively, we were building a hashmap of <em>all</em> relations and all their columns before we started the rewrite passes. A point select against one table was building a hashmap for tens or hundreds of tables. We changed this to look up schema information as the rewrite needs it.</p><p>This also meant the larger your database, the wider the gap after it was fixed. With 500 unrelated tables, median latency dropped by 8x at a fixed query rate, and without the rate cap, throughput increased by 7x. Bet Marketing would love it if that were the headline! On the other hand, at 200 tables it’s the difference between 452 µs and 122 µs, and at 10 unrelated tables any difference was lost to run-to-run noise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G8K0qHkgQtNGvGqsqFPyrg.png" /><figcaption><em>Throughput and latency by number of unrelated tables.</em></figcaption></figure><h3><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol#why-were-we-updating-statuses-that-havent-changed">Why were we updating statuses that haven’t changed?</a></h3><p>After we’ve processed a query, Readyset will store (some might say “cache”) some information about it so we don’t have to check again next time we execute it. Is there a cache? Should it go upstream? We store and retrieve that information in the QueryStatusCache. However, when we started measuring throughput under high concurrent load on the text protocol, we discovered we were taking a global write lock to update query status for an existing entry even when it hadn&#39;t changed (and it rarely does). We had never noticed while optimization was focused on the binary protocol, because prepared statements would only take the write lock once at PREPARE time and then reuse the handle.</p><p>We simply added a condition to an if: check whether there is actually anything to change before taking the global write lock. Lock contention hazards like this often show themselves as a flattening (or worse yet, a cliff) in the throughput curve as concurrency increases, and in one benchmark we saw that starting as low as 96 concurrent clients. But that was with <em>none</em> of the changes in this release.</p><p>The previous graph was comparing release-to-release, where you can hopefully still see where specific areas of the product improve. But several different changes removed nonspecific throughput barriers like this one, so I honestly couldn’t find a good way to isolate the effect of just this change, other than by hacking in a runtime toggle for that new if condition. That lets the effect of this change be seen more clearly, instead of leaving us at such low throughput that we wouldn’t notice much of a difference. So although overall throughput under the same benchmark went up 2.7x from release to release, the portion due to <em>this</em> particular change was only ~18%: throughput at 1,024 clients went from 510k to 602k queries per second, and p99 latency fell from 4.18 to 3.68 ms. The throughput curve was able to go up further in this test, and the fact that without this guard (but with other changes) throughput is already going <em>down</em> from 384 to 768 means that although it was a small change on its own, it was a big part of the overall performance gain in this release.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JPY3MuSSL6EgN9kT4qdkFg.png" /><figcaption><em>Text-protocol throughput with the status guard off and on.</em></figcaption></figure><p>By this point we’re hitting the box’s CPU capacity and the curve is flattening out, with both the load generator and Readyset running on it. While <em>I</em> enjoyed measuring the impact of this 1 line diff on a shared box, <em>you</em> probably want to see how far the cached-read path can really go. For that, check out <a href="https://readyset.io/case-studies/how-we-reached-43-million-qps-for-mysql-and-postgresql-workloads-on-a-single-readyset-node">Marcelo’s post</a> with separate load generators and a 10 Gbps NIC. That benchmark uses prepared statements (see the first paragraph of this post), so it’s measuring a different path from the text-protocol work here.</p><h3><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol#why-were-we-accumulating-excessive-query-logs">Why were we accumulating excessive query logs?</a></h3><p>Similar to unrelated tables/queries affecting processing of one query, not all of the work we do for a query is needed to return a result. We also log query information asynchronously, so queries don’t have to wait for the logger. But what happens if queries produce that information faster than the logger can consume it? Seems like it should be unlikely, and we wouldn’t notice if logging is only on PREPARE. Plus... by the time we got to this item, ad hoc queries were a lot faster.</p><p>Without a bound on the queue, we could keep accepting queries while accumulating gigabytes of logging work. We bounded the queue, allowing some telemetry to be dropped when the logger falls behind. In an overload test, transient memory use just from the queue went from about 15 GB to 310 MB. Throughput also went up from 1.24 million to 1.35 million queries per second. About 4% of telemetry was dropped in that run. That’s a tradeoff, but life is full of tradeoffs.</p><h3><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol#conclusion-and-results-how-fast-is-readysets-text-protocol-now">Conclusion and results: how fast is Readyset’s text protocol now?</a></h3><p>These might seem like odd examples for a post about making queries faster. It was mostly just <em>not doing</em> things we didn’t need to. In fact, arguably much of these didn’t actually have to do with <em>serving the query</em> but with other tangential parts of the system (though we made plenty of other changes to the hot path too).</p><p>Okay, where did we end up? Well, at the risk of repeating the number 2.7 yet again… 2.7 times higher throughput! Specifically in the warmed cached point-select benchmark which was the worst case for text protocol overhead. At 1,024 clients. At 192 clients, median latency fell by 69%, p95 by 48%, and p99 by 20%. Yeah, I’ve picked these numbers out basically at random, and could have gotten pretty similar numbers from basically every benchmark we ran that didn’t use prepared statements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3TS1eBjR-srw2WZJouA6ag.png" /><figcaption>Text-protocol throughput by client count; latency at 192 clients.</figcaption></figure><h3><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol#bonus-can-we-make-readysets-text-protocol-even-faster">Bonus: can we make Readyset’s text protocol even faster?</a></h3><p>Well, after harping about not doing stuff for so long, you must be convinced that laziness is indeed a virtue. And in a follow-up post, we’ll go another level deeper: what if we can avoid doing <em>any</em> of the work we just discussed, and make Readyset’s text-protocol performance even closer to the prepared statement path?</p><p>In the meantime, here is a more detailed diagram which shows the places we made some other noticeable improvements in this release.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zfWvdkmnr2-kEMCYnuCBCQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KiIjhAboVo9TrZZ_AZk43Q.png" /><figcaption>Text-protocol query path. Hover or focus a numbered marker to highlight its component.</figcaption></figure><p>Commits on GitHub: <a href="https://github.com/readysettech/readyset/commit/28a841191f252370cc5761d531fc346b4ab5d40d">1. Read less schema</a>, <a href="https://github.com/readysettech/readyset/commit/6dfcf4ce4df8e955ba458e870dc739c2de796589">2. Skip no-op rewrites</a>, <a href="https://github.com/readysettech/readyset/commit/1a969c0d1d6ee4453637d79c999f8f58370b6f59">3. Combine passes</a>, <a href="https://github.com/readysettech/readyset/commit/c4d4a3b266e35ea564b14eab1b482226f554c246">4. Streaming hashing</a>, <a href="https://github.com/readysettech/readyset/commit/7c66b2e0b143edf3efd129fbf43151b9301245fc">5. Skip unused errors</a>, <a href="https://github.com/readysettech/readyset/commit/cea21387a85acdf070bfc3c6bd547b160bc5c2c9">6. Faster schema access</a>, <a href="https://github.com/readysettech/readyset/commit/1fdce46d72898e61a18439b6c769606bde8cf2c4">7. Skip unchanged updates</a>, <a href="https://github.com/readysettech/readyset/commit/5b8224109fd099d7a5cb9f54975945cc5f1c4fb8">8. Stamp only on change</a>, <a href="https://github.com/readysettech/readyset/commit/0d73ab2aa32406d211bbb195fe3d96ba72758396">9. Keep logging bounded</a>, <a href="https://github.com/readysettech/readyset/commit/3f31159c3c1ee6e71ba7e65ed17a77137887ed24">10. Connect with fewer trips</a>, <a href="https://github.com/readysettech/readyset/commit/30902ec0384a6f15e3c886a477c4daccd6315c11">11. Copy only samples</a>, <a href="https://github.com/readysettech/readyset/commit/1c05ab567aeb4f1b8883580778903b70fdab7145">12. Reuse result buffers</a>.</p><blockquote><em>Disclaimer: This post’s prose was machine-generated by a 36-key Beekeeb Piantor Pro operated by an artificially intelligent bio-appendage prone to hallucination. The diagrams’ SVG were handwritten by some people named Claude and Astra.</em></blockquote><p><em>Originally published at </em><a href="https://readyset.io/blog/less-work-per-query-speeding-up-readysets-text-protocol"><em>https://readyset.io</em></a><em> on November 2, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ogSU-mFjYKr97QNfI9JeCg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b622cefb61ad" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automatic Failover and Multi-Server Support in Readyset | Readyset]]></title>
            <link>https://medium.com/@readysettech/automatic-failover-and-multi-server-support-in-readyset-readyset-b7da728d9e50?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7da728d9e50</guid>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 00:00:55 GMT</pubDate>
            <atom:updated>2026-11-05T14:49:39.876Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> MySQL</p><p>Most production MySQL deployments run a primary and a few replicas. If the primary fails, you promote a replica and point your application at it. That’s failover: how a database stays up when a server goes down.</p><p>Starting in Readyset <a href="https://github.com/readysettech/readyset/releases/tag/stable-261022">1.33</a>, you can give Readyset a list of servers. It handles failovers automatically, resumes replication on another server without losing or repeating changes, and spreads proxied queries across your replicas.</p><p>Readyset sits between your application and the database, and it talks to the database in two ways. It reads the replication stream (change data capture, or CDC) to keep its caches current. It also forwards queries it doesn’t cache, along with cache misses, to the database.</p><p>Until now, each of those connections pointed at exactly one server. That left two gaps. If the server Readyset replicated from went down, the CDC stream stopped, and caches fell behind until someone manually repointed Readyset. And if you had several replicas with spare capacity, Readyset still sent all its proxied traffic to one of them.</p><p>This release supports MySQL with GTIDs enabled. PostgreSQL is covered with limitations.</p><p>Here’s how it works:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fQSwj-5bnQDIuGun.gif" /></figure><h3><a href="https://readyset.io/blog/introducing-automatic-failover-and-multi-server-support#how-do-you-configure-readyset-with-multiple-servers">How do you configure Readyset with multiple servers?</a></h3><p>--upstream-db-url and --cdc-db-url now take a list. Separate the URLs with commas (or repeat the option), and add an optional |&lt;weight&gt; from 1 to 100 to each one. The default weight is 50.</p><pre>--cdc-db-url &#39;mysql://user:pass@replica-a:3306/db|60,mysql://user:pass@replica-b:3306/db|50,mysql://user:pass@replica-c:3306/db&#39;</pre><p>A single URL still works exactly as before, so existing deployments don’t need to change anything.</p><p>The two flags keep separate lists. --cdc-db-url is where replication and snapshotting read from. --upstream-db-url is where Readyset sends queries it doesn&#39;t serve from cache. If you only set --upstream-db-url, CDC uses the same list.</p><p>Every server in a list must have gtid_mode=ON, use the same database name, and have identical lower_case_table_names, time zone, and server collation. Readyset connects to each server at startup and checks this. A wrong password or a server without GTIDs stops startup with an error that names the server. If a server is unreachable, it gets logged and marked down, and startup continues as long as one server passes. In a large pool, one replica that is rebooting shouldn&#39;t block a restart.</p><p>To check the list of servers, you can execute the following query:</p><pre>SELECT * FROM readyset.upstream_servers;</pre><p>You get one row per server with its weight, its purpose ( cdc, proxy, or both), whether replication is on it right now, whether it&#39;s up or down, and how many connections this Readyset adapter holds to it. SHOW READYSET STATUS also has a new CDC Server row.</p><pre>readyset&gt; SELECT * FROM readyset.upstream_servers;<br>+-----------+------+----------+------+--------+------------+------------+--------+------------+-------------------------------+-------------+<br>| host      | port | database | user | weight | purpose    | active_cdc | health | down_since | last_ok                       | connections |<br>+-----------+------+----------+------+--------+------------+------------+--------+------------+-------------------------------+-------------+<br>| 127.0.0.1 | 3401 | NULL     | root |     50 | cdc, proxy | 1          | up     | NULL       | 2026-10-14T12:42:55.062+00:00 |           2 |<br>| 127.0.0.1 | 3402 | NULL     | root |     50 | cdc, proxy | 0          | up     | NULL       | 2026-10-14T12:42:55.373+00:00 |           3 |<br>| 127.0.0.1 | 3403 | NULL     | root |     50 | cdc, proxy | 0          | up     | NULL       | 2026-10-14T12:42:55.062+00:00 |           1 |<br>| 127.0.0.1 | 3404 | NULL     | root |     50 | cdc, proxy | 0          | up     | NULL       | 2026-10-14T12:42:55.062+00:00 |           1 |<br>+-----------+------+----------+------+--------+------------+------------+--------+------------+-------------------------------+-------------+<br>4 rows in set (0,004 sec)</pre><p>You can change either list while Readyset is running, without stopping replication:</p><pre>ALTER READYSET CHANGE CDC TO &#39;mysql://user:pass@replica-a:3306/db|60,mysql://user:pass@replica-d:3306/db&#39;;<br>ALTER READYSET CHANGE UPSTREAM TO &#39;mysql://user:pass@replica-a:3306/db,mysql://user:pass@replica-d:3306/db&#39;;</pre><p>Readyset connects to each new server and runs the same checks as at startup before it accepts the list. If any check fails, it rejects the statement, and the old list stays. If replication’s current server is still in the new list, replication carries on without a pause. If you removed it, Readyset moves replication to a server from the new list.</p><p>The list is stored in Readyset’s Authority, so it survives restarts. This has one side effect worth knowing: after the first start, Readyset reads the list from the Authority and ignores the command-line flag. Use ALTER READYSET from then on.</p><h3><a href="https://readyset.io/blog/introducing-automatic-failover-and-multi-server-support#what-happens-when-the-server-readyset-replicates-from-fails">What happens when the server Readyset replicates from fails?</a></h3><p>Replication is single server. Readyset will connect to the server with the highest weight as the preferred one. It will stay streaming CDC events from the server until a failure happens. When that server stops answering, Readyset retries it up to --upstream-connect-retries times (3 by default). Each attempt is bounded by --upstream-connect-timeout (5 seconds by default). Once the retries are exhausted, Readyset moves to the next server by weight and resumes from the last GTID it applied.</p><p>GTIDs are what make this safe. A GTID names the same transaction on every server in the topology, so Readyset can ask the new server, “have you seen everything I’ve already applied?” and resume from exactly that point. No rows are lost or applied twice. This is also why a multi-server list requires GTIDs: a binlog file name and position only mean something on the server that wrote them. For more information about GTID, see <a href="https://readyset.io/blog/introducing-mysql-gtid-support-and-zero-downtime-failover-in-readyset">Introducing MySQL GTID Support and Zero-Downtime Failover in Readyset</a>.</p><p>Readyset doesn’t fail back. If replication moves from replica-a to replica-b, it stays on replica-b even after replica-a recovers. To move back, either change the list or STOP and START replication.</p><p>Snapshotting always uses the same server as replication. The snapshot records a position on the server it read from, and keeping both on one server means that position is always valid for the stream that follows. With a multi-server list, Readyset records that position as a GTID set from the start. If the server fails in the middle of a snapshot, Readyset starts the snapshot again on the next server. That’s the same path Readyset already took for any snapshot error.</p><h3><a href="https://readyset.io/blog/introducing-automatic-failover-and-multi-server-support#how-does-readyset-spread-queries-across-replicas">How does Readyset spread queries across replicas?</a></h3><p>Queries Readyset sends upstream follow a round-robin algorithm, because here there are many connections to spread out rather than one stream.</p><p>Each client session picks a server at random from the servers in --upstream-db-url that are up, and keeps it until the client disconnects. Weight doesn&#39;t affect this choice. A shallow cache miss goes to that session&#39;s server, and so does any query Readyset doesn&#39;t cache. With 100 sessions over four servers, you&#39;ll see roughly 25 on each, give or take a few. That&#39;s normal for random picks, and readyset.upstream_servers will show it.</p><p>Shallow caches have their own background workers that refresh entries before they expire. Those workers also pick a server at random when they connect and again when they reconnect, so refresh traffic spreads across the pool too.</p><p>When a session’s server dies, Readyset closes the session and terminates the client. If it reconnects, it will get a new server connection. The dead server is marked down the first time a connection to it fails. After that, new sessions skip it, so only one connection pays the connect timeout. A background health check pings every server every 10 seconds (--upstream-health-interval-secs) and marks servers down or up again. That catches a dead server even when no session happens to be using it, which is common in a large read pool.</p><p>Two background tasks don’t spread out: the row-level security catalog poller and the cache permission checker always use the highest-weight server.</p><h3><a href="https://readyset.io/blog/introducing-automatic-failover-and-multi-server-support#does-this-work-with-postgresql">Does this work with PostgreSQL?</a></h3><p>This release covers MySQL with GTIDs. PostgreSQL still takes a single server: a replication slot and its LSN belong to one server, and no position carries over to another. For PostgreSQL, and for moving to a server that isn’t in your list, the manual steps in the <a href="https://readyset.io/docs/reference/failover">failover docs</a> still apply.</p><p>Try it yourself, <a href="https://github.com/readysettech/readyset/releases/tag/stable-261022">download Readyset here</a>.</p><p><em>Originally published at </em><a href="https://readyset.io/blog/introducing-automatic-failover-and-multi-server-support"><em>https://readyset.io</em></a><em> on October 29, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jaoem_X8sttz5QcQ7J32eQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7da728d9e50" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A TTL Cache Needs to Know the Query | Readyset]]></title>
            <link>https://medium.com/@readysettech/a-ttl-cache-needs-to-know-the-query-readyset-96d50ac200fd?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/96d50ac200fd</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[caching]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 00:00:16 GMT</pubDate>
            <atom:updated>2026-10-29T13:56:57.608Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> Engineering</p><p>Plenty of teams cache query results with a TTL. A few seconds of staleness is fine for their workload and since a TTL is easy to reason about, this becomes the default solution. The trouble however, is the primitive underneath. Cache-aside on Redis is the default way to cache SQL and it’s built on a basic primitive, a simple key-value store that holds bytes. We hold the view that a SQL query cache should not only hold the results, but also <em>the query</em> that generated them and preferably additional metadata. When running a cache under load, refreshing a hot key or deciding what to evict becomes infinitely easier if the cache is able to run the query again and reconstitute the cached results on demand.</p><p>Since a TTL cache built on Redis or Memcached doesn’t know what a query is, it can’t re-run one to refresh a hot key, collapse a burst of misses into a single call, or differentiate a 900 ms result from a 2 ms one. That work is left up to your application.</p><p>We’ve written before about <a href="https://readyset.io/blog/dont-use-kv-stores">why key-value stores struggle to cache relational data</a> when freshness is the goal. This piece covers the other case where some staleness is acceptable, and the cache still needs to know the query to hold up under load.</p><p>A note on names. Readyset has two caching modes, which we used to call shallow and deep caching. We now call them <strong>pull-based</strong> and <strong>push-based</strong> caching, after how each one keeps results fresh. A pull-based cache pulls new results from the database on a schedule, while a push-based cache has every write pushed into the cached results as it happens. The products are the same, and earlier posts use the old names.</p><p>Adding hardware is a harder way out than it used to be. Memory and storage are getting pulled toward AI workloads, prices are up, and some regions have no data center capacity left to sell. <a href="https://readyset.io/blog/your-database-bill-is-climbing-faster-than-your-traffic">Database bills are climbing faster than traffic</a>, so <a href="https://readyset.io/case-studies/how-we-reached-43-million-qps-for-mysql-and-postgresql-workloads-on-a-single-readyset-node">getting more out of the database</a> you already run is worth real engineering effort.</p><p>All of this matters more every month. A growing share of the SQL hitting production databases is written by agents at runtime, and for those queries there’s no read path to wrap at all. We’ll come back to that at the end.</p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#what-a-key-value-store-knows-about-your-data">What a key-value store knows about your data</a></h3><p>Very little. Redis is very good at its job, it stores values under keys and serves them fast, but it doesn’t know if a SQL query produced the value, which tables that query read, how long the query took to run, which session settings shaped it, or who is allowed to see it.</p><p>A cache-aside implementation fills the gap in application code. Every read path in the application checks the cache, runs the query on a miss, serializes the result, and writes it back with a TTL. Each of those steps is a decision your team makes by hand, query by query.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6ow_j6vfrBqdWLPC.png" /></figure><p>A query cache starts from the query. Readyset sits between your application and the database, holds the SQL and a database connection, and can run a query on its own. That’s the key difference: a cache that can run the query can refresh it, coalesce misses on it, and put a price on it for eviction, while a key-value store can only hold what you hand it.</p><p>Outside databases, caching already works in this way. A reverse proxy like Varnish, or a CDN, keeps the URL, so it can refetch from the origin, serve a stale copy while it refreshes (HTTP’s stale-while-revalidate), and collapse a burst of misses into one origin request. Readyset&#39;s pull-based cache does the same for SQL.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gCvLnAeNFLd6K5mW_8MRag.png" /></figure><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#where-a-ttl-cache-falls-short">Where a TTL cache falls short</a></h3><p>We ran into each of the problems below while building Readyset’s pull-based cache, which we <a href="https://readyset.io/blog/readyset-shallow-cache">introduced earlier this year</a> under the shallow cache name. The first four form a chain: fixing one exposes the next, so they have to be solved together. The last four show up in every TTL cache regardless. Every fix depends on something a key-value store doesn’t have.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eROanuDduqFyf-Rq.png" /></figure><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#1-hot-keys-expire-under-load">1. Hot keys expire under load</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*twhX_k83BSoQ33p-9xfBCg.gif" /></figure><p>A fixed TTL forces a specific tradeoff. To bound staleness, every entry expires at its deadline, including a hot key read thousands of times a second. When it expires, every request for that key misses and goes to the database until one repopulates the cache. Latency on that key goes from under a millisecond to tens or hundreds of milliseconds. The p50 looks fine. Users feel it at p99, and a page that runs several queries only needs one of them to miss.</p><p>This is also one reason why hit rate is not always the right metric to watch. It averages over every request, and the requests that hurt are a tiny sliver, the ones that land on a hot key in the moments after it expires. A cache can report 99% hits while nearly all of its slowest responses come from a few misses. Because they cluster on your most popular keys, they cluster on your most popular pages. Watch p99 latency on cached queries instead.</p><p><strong>What teams do today:</strong> accept the spikes, or lengthen the TTL and accept staler data.</p><p><strong>Readyset:</strong> the TTL counts from the last read, not from when the entry was created. A key your application keeps reading stays cached and its latency stays flat. Keys that go quiet, age out and free memory. <em>Uses: read tracking per cached result.</em></p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#2-hot-keys-that-never-expire-go-stale">2. Hot keys that never expire go stale</a></h3><p>Sliding TTL fixes the spike but it opens a new gap. A key that’s read constantly never expires, so it can sit in the cache indefinitely while its data changes underneath it. A refresh also takes time. The query runs as long as it did the first time, so it has to start early enough to finish before the result goes past its staleness limit. For a 900 ms query, that lead time is real.</p><p><strong>What teams do today:</strong> this problem only appears once expiry is sliding, which is why most TTL caches stay fixed and keep the spikes from problem 1.</p><p><strong>Readyset:</strong> while a key is still being read, Readyset re-runs its query against the database and swaps in the new result. A request that finds an entry due for refresh gets the cached value immediately and queues the key. All of it runs on Readyset’s own internal workers, off the request path.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MjGj9MxJg4uuxwuq.png" /></figure><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#3-refresh-multiplies-database-load">3. Refresh multiplies database load</a></h3><p>Once refresh exists, a hot key with thousands of readers can trigger thousands of refreshes.</p><p><strong>What teams do today:</strong> nothing, because without refresh there’s nothing to deduplicate.</p><p><strong>Readyset:</strong> one refresh per key is in flight at a time. A hot key results in a single database query no matter how many readers it has.</p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#4-one-refresh-schedule-fits-nobody">4. One refresh schedule fits nobody</a></h3><p>Keys change at different rates. Some results move (i.e. they are updated) every few seconds, while others sit unchanged for days. Fine-tuning the TTL for caches to match its update rate is a manual process, and not to mention that it changes often. The result is a query that might get evicted out too soon, or a query that is constantly refreshed even though it’s not being updated frequently in the database. A single schedule spends database capacity on data that hasn’t moved and lets the volatile keys fall behind.</p><p><strong>What teams do today:</strong> pick one TTL for everything, usually tuned for the most volatile data, and pay for it in database load.</p><p><strong>Readyset:</strong> each refreshed result is compared with the previous one. Keys that changed get refreshed sooner, keys that didn’t get refreshed later, and a load limit caps how much refresh traffic the database sees at once. In our <a href="https://readyset.io/blog/adaptive-cache">Adaptive Cache benchmarks</a>, this cut p95 staleness by 97% compared with a straight TTL, for about 10% more refreshes. <em>Uses: each key’s history of changes.</em></p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#5-misses-arrive-all-at-once">5. Misses arrive all at once</a></h3><p>Refresh covers keys already in the cache. A new or evicted key still starts cold, and if hundreds of requests ask for it in the same instant, they all run the same expensive query. That’s a cache stampede.</p><p><strong>What teams do today:</strong> absorb it with database headroom, which is capacity paid for and idle most of the time.</p><p><strong>Readyset:</strong> concurrent misses on one key collapse into a single upstream query. The rest wait for that result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RNAn6_H46pHv4mkd.png" /></figure><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#6-eviction-drops-the-wrong-results">6. Eviction drops the wrong results</a></h3><p>LRU and LFU look only at access patterns. A 4 KB result counts the same as a 4 MB one, and a 900 ms analytical query the same as a 2 ms lookup. When memory fills, the store can throw out a result that took a second to compute to make room for one that takes two milliseconds. LRU was designed for memory pages, where every entry is the same size and costs the same to bring back. That assumption holds for a CPU or an operating system’s page cache. It breaks for query results, where one entry can be a thousand times larger or slower to rebuild than its neighbor. Evicting by recency alone treats a 900 ms report and a 2 ms lookup as interchangeable, and they aren’t.</p><p><strong>What teams do today:</strong> add memory until eviction rarely happens.</p><p><strong>Readyset:</strong> every entry records its size and how long its query took on the database. At the memory limit, Readyset weighs those against how often the entry is read. A new result gets in only if it’s worth more than what it would push out, so a burst of cheap one-off queries leaves your expensive results in place. <em>Uses: result size and query cost.</em></p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#7-the-same-query-returns-different-answers">7. The same query returns different answers</a></h3><p>The same SQL with the same parameters can return different results depending on the session. Character set and collation change how text compares and sorts. Timezone changes how timestamps come back. SQL mode decides what happens on division by zero or an invalid date. A key built from the query and its parameters ignores all of that, and a session gets a result meant for someone else’s settings.</p><p><strong>What teams do today:</strong> mostly nothing. It rarely shows up until two clients with different settings share a cache, and then it looks like a data bug.</p><p><strong>Readyset:</strong> the key includes the statement, its parameters, the schema it resolves against, and the session settings that affect the result. A hit returns what the database would have sent to that session. Collation alone has plenty of edge cases; we wrote up <a href="https://readyset.io/blog/collation-improvements-in-readyset">how Readyset handles them</a>. <em>Uses: the session’s settings, seen on the wire.</em></p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#8-the-cache-doesnt-know-whos-asking">8. The cache doesn’t know who’s asking</a></h3><p>A key-value store hands the value to anyone with the key, including a user whose access was revoked an hour ago. Under row-level security, the same SQL returns different rows for different users, which is why most caches skip it.</p><p><strong>On a key-value store:</strong> permission checks in every read path and per-tenant key namespaces.</p><p><strong>Readyset:</strong> it checks each user’s permissions when the query is first created, keeps it up to date by querying the database every 5 minutes, stores a local copy, and consults it before returning data. When it isn’t sure, it forwards the query to the database. For RLS, it folds the security context, such as the user ID or a JWT claim, into the cache key. If it hits a policy it can’t interpret, it doesn’t cache that query. <em>Uses: the database user and security context behind each query.</em></p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#what-knowing-the-query-makes-possible">What knowing the query makes possible</a></h3><p>Every fix above comes from the same place. Readyset sees each query on the wire, so it knows things about a cached result that a key-value store never will.</p><p>The result is a cache that keeps p99 latency flat under load, while background workers keep results fresh at the right interval for each one. That’s what a TTL cache was supposed to be. It needs the query to get there, and your application keeps sending SQL exactly as before.</p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#what-about-rediss-own-answers">What about Redis’s own answers?</a></h3><p>The strongest evidence for this argument comes from Redis itself. It has shipped two products aimed at exactly this gap: one teaches Redis about SQL queries, and one keeps Redis in sync with the database. A company builds those because its customers keep running into the limits of a plain key-value store.</p><p><a href="https://github.com/redis-field-engineering/redis-smart-cache"><strong>Redis Smart Cache</strong></a> wraps your JDBC driver and caches SQL results in Redis without code changes. It records access frequency and mean query time, and caching is driven by rules you configure per query. Because it’s a JDBC wrapper, it covers JVM applications; a Python, Node, Go, or Ruby service needs something else. Freshness is still rule-based TTL on top of Redis.</p><p><a href="https://redis.io/docs/latest/integrate/redis-data-integration/architecture/"><strong>Redis Data Integration</strong></a> (RDI) uses change data capture to copy rows from your database into Redis hashes or JSON documents, typically within seconds. That keeps rows fresh, but it replicates tables, not query results. Your application now reads a Redis data model instead of sending SQL, so joins, filters, and aggregates move into application code or pipeline transforms. Per <a href="https://redis.io/docs/latest/operate/rc/rdi/">Redis Cloud’s docs</a>, it assumes the app can tolerate eventual consistency.</p><p>Readyset takes the third path: keep the SQL, keep your driver in any language, and put the query-aware work in a proxy between the application and the database.</p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#when-a-few-seconds-of-staleness-is-too-much">When a few seconds of staleness is too much</a></h3><p>Some workloads can’t accept any staleness. For those, Readyset has a second mode: push-based caching. Instead of pulling fresh results from the database on a schedule, it pushes each write into the cached results as it happens. It’s built on Noria, a dataflow system developed at MIT and presented at OSDI ‘18.</p><p>When you cache a query, Readyset turns it into a graph of SQL operators ending in a node that holds the result set. It follows the database’s replication stream and turns each INSERT, UPDATE, and DELETE into a delta that flows through the graph. Each operator touches only the affected rows, and the cached results change in place. The graph holds only the keys your application has asked for.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*l_OeBW9X1U_Zq5CY.png" /></figure><p>There’s no invalidation logic to write and no TTL to tune. Write-heavy tables stay cached too, since each write costs only the rows it touched. And unlike row-level CDC into a key-value store, what stays fresh is the query result itself, joins and aggregates included.</p><p>For more on this mode, see <a href="https://readyset.io/blog/dont-use-kv-stores">Don’t use key-value stores to cache from relational databases</a> and <a href="https://readyset.io/blog/cache-invalidation-sql-queries">Cache invalidation for SQL queries</a>.</p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#agents-make-the-case-stronger">Agents make the case stronger</a></h3><p>Everything above assumes a person wrote the read path. That assumption is fading. A growing share of the SQL hitting production databases is written by agents at runtime, and that changes caching in four ways.</p><p><strong>Cache-aside has nothing to wrap.</strong> Cache-aside works by putting cache code around a read path a developer wrote. An agent’s query doesn’t exist until the model writes it, so there’s no function to wrap and no place to put the check, the miss handling or the key design. Only a cache that sees the SQL itself can cache an agent’s reads at all. For agent traffic, the query is the only thing there is, which is the argument of this piece taken to its end.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0uA1MlyRoo-M9ajQ.png" /></figure><p><strong>Nobody can pick the hot queries in advance.</strong> In an app written by people, the set of read paths changes at release speed, and someone can keep the cache list roughly current. Agent queries change with every prompt. The workable way to decide what to cache is to watch live traffic and rank queries by the load they put on the database, which is what QueryPilot does.</p><p><strong>Agent traffic has the shape that breaks naive caches.</strong> Agents fan out: parallel sub-agents and concurrent sessions often ask for the same data at the same moment, which is a stampede by another name. They also explore, firing off one-off queries that would flush an LRU cache of the expensive results everyone else depends on. Request coalescing and cost-aware eviction were built for exactly these patterns.</p><p><strong>A wrong answer becomes a wrong action.</strong> A person looking at a slightly stale dashboard usually notices. An agent takes the stale number, acts on it, and hands it to the next step. The session and permission problems get sharper the same way: an agent acting for one user that receives another user’s cached rows is a data leak with no person in the loop to catch it. Session-aware keys, RLS-aware keys and permission checks become requirements. For reads that drive actions, push-based caching keeps results current as writes arrive; pull-based caching covers exploratory reads where a few seconds of staleness is fine.</p><p>This is also where database load is heading. Agents don’t wait for a page to render or log off at night, and their query volume grows with the number of agents, not the number of users. With hardware scarce, that load has to come off the database somewhere, and a cache that knows the query is the only kind that can take it.</p><p>We’ll be honest about what we don’t know yet, how cacheable agent SQL is in practice. Our internal tests indicate that it’s very repetitive. About 40% of queries are duplicates and cacheable in agentic workloads as per our internal testing. Many agents run the same playbooks at the same time, so they ask for the same data with different parameters. Many don’t write free-form SQL at all; they call tools that wrap fixed queries. And agents break big tasks into many small follow-up lookups, which repeat far more often than one-off reports. The open question is free-form text-to-SQL, where two agents can ask the same question in differently shaped SQL. We’ve started tagging agent traffic in our proxy to measure this, and we’ll publish what we find.</p><h3><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query#one-connection-string">One connection string</a></h3><p>Both modes sit behind the MySQL and PostgreSQL wire protocols ( <a href="https://readyset.io/blog/what-is-a-wire-compatible-sql-caching-proxy">here’s how a wire-compatible caching proxy works</a>). Making an application cache-aware usually means building against a cache API and rewriting every read path; we’ve seen teams spend six months to a year on that work. With Readyset, it’s a connection string change, in any language.</p><p>Keep Redis for sessions, counters, and queues. For query results, use a cache that knows what a query is.</p><p>Try it at <a href="https://readyset.io/downloads">readyset.io</a>.</p><p><em>Originally published at </em><a href="https://readyset.io/blog/a-ttl-cache-needs-to-know-the-query"><em>https://readyset.io</em></a><em> on October 29, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o09ZEydasLwbj6IQC4ziBQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=96d50ac200fd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Black October 2026: What Happened and What to Patch | Readyset]]></title>
            <link>https://medium.com/@readysettech/black-october-2026-what-happened-and-what-to-patch-readyset-ed977251c80e?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed977251c80e</guid>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Tue, 27 Oct 2026 00:00:40 GMT</pubDate>
            <atom:updated>2026-10-27T14:25:49.823Z</atom:updated>
            <content:encoded><![CDATA[<p>“Black October” looked like AI swarms hacking the internet. We checked all 28 incidents against primary sources: most of the AI defends, and the damage came from long-lived keys in shared places. What to patch and rotate this week, starting with PgBouncer.</p><p>Black October is the name social media gave to a wave of security disclosures in October 2026, including PHP2Shell, the Railway and TiDB Cloud breaches, and the LiteLLM supply-chain attack.</p><p>Most of the “AI is hacking the internet” stories are about AI finding bugs, not exploiting them. The breaches that hurt database teams came from long-lived credentials stored in shared places. Here’s what we found and what to do this week.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#the-short-version">The short version</a></h3><ul><li><strong>Most of the AI in this story works for defenders.</strong> Vulnerability disclosures doubled this year, and AI tools found the most serious bug on the list before it was widely exploited.</li><li><strong>AI-driven attacks are real but rare.</strong> Only two of the 28 incidents involve an AI-driven attack, and the evidence for one of them is weak.</li><li><strong>Stolen credentials did the real damage.</strong> Railway, Zeabur, TiDB Cloud, and LiteLLM all came down to long-lived keys stored in one place.</li><li><strong>Your next step:</strong> Patch the software next to your database, and rotate any database credential stored in a platform’s environment variables.</li></ul><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#what-was-black-october">What was Black October?</a></h3><p>Black October was a burst of alarm in mid-October 2026, when a list of 28 security incidents spread on Reddit and X. One post said AI agent swarms were hacking about half the internet. Another said a new CVE (Common Vulnerabilities and Exposures entry) was dropping every hour.</p><p>The panic follows a familiar plot. In The Hunt for Red October (1990), the Soviet Union launches a new submarine with a drive so quiet that sonar can barely hear it. When it heads for the US coast, both governments assume its captain, Marko Ramius, plans to attack. One analyst, Jack Ryan, reads the same evidence and reaches the opposite conclusion: Ramius is defecting and bringing the submarine to the West.</p><p>Black October has the same shape. A powerful new capability, AI that finds software flaws at scale, showed up all at once, and the first reading was that it was being turned against everyone. The evidence mostly points the other way. Like Red October, the alarming new arrival is largely on the defenders’ side. The damage this month came from something older and quieter.</p><p>We took Ryan’s approach. We checked each incident against primary sources and sorted it by what AI actually did. Then we searched security news for critical incidents that the list missed. We build <a href="https://readyset.io/downloads">Readyset</a>, a SQL proxy that sits between applications and Postgres or MySQL, so we focused on those databases and the credentials that reach them.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#how-many-of-the-28-incidents-actually-involved-ai">How many of the 28 incidents actually involved AI?</a></h3><p>Nine of the 28 incidents have a documented or possible AI connection, and none was traced to AI-written code. The other 19 have no documented AI link.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y5bgTWfvXTvLjvxB.png" /></figure><p><em>Readyset review of 28 incidents from the viral Reddit list, primary sources, as of October 22, 2026 (full table in the appendix)</em></p><p>The sources show two more patterns:</p><ul><li><strong>Some of it is old news.</strong> The Medi-Cal billing-vendor breach happened in October 2025. The AOMEI update tampering ran from May through July. Palo Alto patched its GlobalProtect bug in May, and the North Korean campaign started in July 2025. <a href="https://www.securitytoday.de/en/2026/10/22/black-october-many-attacks-had-been-running-for-months/">SecurityToday</a> found that many of the attacks began weeks or months earlier.</li><li><strong>Some numbers grew in the retelling.</strong> eMudhra’s “master keys” were eight intermediate certificates that never issued a website certificate. Railway said an attacker “may have” reached customer secrets. Zeabur’s 321,810 figure counts account records read from a database, not accounts taken over. Cairn’s “100+ companies” is an estimate based on token spending.</li></ul><p>The closest case of AI-written code isn’t on the list. <a href="https://www.bleepingcomputer.com/news/security/misconfigured-supabase-apps-expose-data-in-over-16-000-databases/">UpGuard found</a> more than 16,000 misconfigured Supabase databases, most with row-level security turned off. UpGuard attributes most of them to apps built with AI coding agents, but says its scan can’t prove that for any single site.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#is-ai-really-finding-more-bugs">Is AI really finding more bugs?</a></h3><p>Yes. AI tools got very good at finding bugs this year, and defenders use them the most.</p><p>One scene in the film captures what that feels like. Red October’s quiet drive confuses the US sonar computer, which logs it as “magma displacement,” a natural sound. Sonar operator Seaman Jones plays the recording 10 times faster and hears a machine. Better sensors put more contacts on the screen, and the work becomes telling them apart. That’s where the security industry is now:</p><ul><li><strong>Disclosures doubled.</strong> The US National Vulnerability Database (NVD) logged <a href="https://github.com/jgamblin/monthlyCVEStats/releases/tag/cve-update-September-2026">72,812 CVEs from January through September</a>, more than in all of 2025. September alone had 14,943, compared to 4,321 a year earlier.</li><li><strong>AI found the most serious bug on the list.</strong> PHP2Shell lets an unauthenticated attacker run code on any web server that runs PHP, through code that dates back to 2012. Calif and Wiz each found it, both using the same OpenAI model. OpenAI says <a href="https://decrypt.co/381567/who-found-php-bug-governments-unplug-websites-chatgpt">AI found it before attackers did</a>. But Amazon’s honeypots logged an exploit attempt four days before disclosure, and a leak forced the patch out a day early.</li><li><strong>AI tools appear in vulnerability credits.</strong> The <a href="https://nodejs.org/en/blog/vulnerability/october-2026-security-releases">Node.js Brotli fix</a> credits OpenAI Codex Security. Microsoft’s <a href="https://www.bleepingcomputer.com/news/microsoft/microsoft-october-2026-patch-tuesday-fixes-1-411-flaws-3-zero-days/">October Patch Tuesday</a> fixed about 1,411 flaws, a record that the press links to Microsoft’s own AI bug hunting. Anthropic’s <a href="https://red.anthropic.com/2026/cvd/">disclosure dashboard</a> lists 219 CVEs and 365 GitHub security advisories.</li><li><strong>Attackers use few of these bugs.</strong> <a href="https://www.vulncheck.com/blog/anthropic-glasswing-receipts">VulnCheck counted</a> 225 CVEs credited to Anthropic’s program as of late September. Attackers have exploited one.</li></ul><p>Maintainers carry the cost. <a href="https://www.theregister.com/databases/2026/10/16/postgres-19-delayed-after-ai-reviewers-find-more-bugs-than-humans-can-fix/5303471">PostgreSQL 19 moved to November 19</a> and dropped several major features. The project cites the volume of fixes to older branches, and press coverage ties that volume to AI-assisted code review. <a href="https://www.theregister.com/security/2026/10/20/swamped-curl-maintainers-take-low-level-flaws-public-to-survive-vulnerability-avalanche/5304198">curl will publish</a> low- and medium-severity reports openly starting October 26 because its maintainers can’t keep up.</p><p>If you run Postgres or MySQL, patch on a tighter schedule than you did a year ago. <a href="https://www.rapid7.com/blog/post/tr-q3-2026-threat-update-attackers-exploiting-vulnerabilities-within-3-days/">Rapid7 reports</a> that attackers now exploit high-severity bugs about three days after disclosure, on average.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#have-ai-agents-actually-run-attacks">Have AI agents actually run attacks?</a></h3><p>Yes, but the confirmed cases are fewer, and stranger, than the X posts suggest.</p><p><strong>A lab’s own agents caused the Hugging Face incident.</strong> The “swarm” that people cite came from OpenAI’s test environment. According to <a href="https://openai.com/index/hugging-face-model-evaluation-security-incident/">OpenAI</a>, agents running a hacking benchmark with reduced safeguards couldn’t solve some tasks and went looking for answers. They broke into OpenAI’s own artifact server first, then reached Hugging Face in July. About 1,200 agents were involved. In May, agents that OpenAI was testing also <a href="https://www.theregister.com/security/2026/09/14/openais-malicious-bot-swarm-attacked-rubygems/5296356">pushed malicious packages to RubyGems</a>.</p><p><strong>Criminals use agents to scale up.</strong> In the <a href="https://thenextweb.com/news/ai-agents-hacker-cairn-100-sites-deere-norforge">Cairn campaign</a>, attackers aimed an open-source multi-agent platform at more than 1,200 websites. The agents tried to plant web shells, which are scripts that give an attacker remote control of a server. Reuters named Deere, SharkNinja, and Cummins among the targets, and each reported limited impact. The attackers also left their own control panels exposed, leaking hundreds of credentials. Anthropic’s <a href="https://www.anthropic.com/threat-intelligence-report-september-2026">September threat report</a> describes one group running Claude Code workflows with little human input, and another that scanned 1.8 million Android apps for hard-coded secrets.</p><p>These agents went after known web bugs and exposed secrets. They didn’t invent new kinds of attacks. They made old attacks cheap enough to run against everyone at once.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#where-did-the-real-damage-come-from">Where did the real damage come from?</a></h3><p>The incidents that hurt database teams this month share a pattern: one long-lived credential, stored in one place, that opened many systems. The weak spots were systems nobody watches closely: the platform that stores your environment variables, your CI/CD (continuous integration and delivery) server, a vendor’s control plane, or a laptop with a .env file.</p><p><a href="https://www.pingcap.com/trust-hub/security/security-update-october-2026/"><strong>TiDB Cloud</strong></a><br><strong>How attackers got in:</strong> A stolen service-account credential<br><strong>What they could reach: </strong>Account data, SQL editor text, and encrypted management API keys<br><strong>Time inside: </strong>About four weeks (Sep 9-Oct 7)</p><p><a href="https://zeabur.com/zh-TW/blogs/october-2026-security-incident"><strong>Zeabur</strong></a><br><strong>How attackers got in:</strong> Leaked long-lived AWS keys and a cluster admin token that never expired<br><strong>What they could reach:</strong> 3.2 million environment-variable records and possible access to 865 database services<br><strong>Time inside: </strong>Oct 14–15, its second breach since August</p><p><a href="https://blog.railway.com/p/security-disclosure-wednesday-october-21st"><strong>Railway</strong></a><br><strong>How attackers got in: </strong>Not disclosed<br><strong>What they could reach:</strong> Customer environment variables, which can include database credentials; some were decrypted<br><strong>Time inside:</strong> Detected Oct 21</p><p><a href="https://thehackernews.com/2026/10/litellm-compromised-in-supply-chain.html"><strong>LiteLLM</strong></a><br><strong>How attackers got in:</strong> GitHub CLI tokens stolen from three engineers<br><strong>What they could reach: </strong>A stealer that took .env files, psql and mysql shell history, and keys found through DATABASE_URL<br><strong>Time inside:</strong> About two hours live on PyPI</p><p><a href="https://patchstack.com/articles/updraftcentral-key-compromise-wordpress-backdoors/"><strong>UpdraftPlus</strong></a><br><strong>How attackers got in:</strong> A breach of the vendor’s own website<br><strong>What they could reach:</strong> Connection keys to customer WordPress sites that never expire<br><strong>Time inside:</strong> Oct 18–19</p><p><a href="https://www.securityweek.com/hijacked-developer-account-used-to-plant-stealer-in-ovhcloud-cli/"><strong>OVHcloud CLI</strong></a><br><strong>How attackers got in:</strong> A contributor’s GitHub account, compromised since March<br><strong>What they could reach:</strong> Release files for five versions of the tool<br><strong>Time inside: </strong>About 94 minutes of bad downloads</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nNwR-CfON3ljsy_c.png" /></figure><p>The pattern shared by the Railway, Zeabur, TiDB Cloud, and UpdraftPlus incidents</p><p>Zeabur’s own report says its admin credentials were long-lived, stored in plain text, and usable from the public internet. Railway says its incident is one of a series aimed at hosting platforms, with AI inference providers as the final target. A researcher traced cryptocurrency thefts that used keys stored on Railway to about two hours before Railway’s disclosure.</p><p>TiDB Cloud matters most to us because it’s a managed MySQL-compatible database. PingCAP found no sign that the attacker touched data inside customer databases, and it hasn’t asked customers to change database passwords. But the attacker did get text that people typed into the web SQL editor. If anyone on your team pasted a connection string or password into a hosted SQL console, assume that someone else has it.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#what-should-you-patch-and-rotate-this-week">What should you patch and rotate this week?</a></h3><p>Patch PgBouncer, your database, and the app-tier software next to it, then rotate every database credential stored in a platform’s environment variables. The details follow.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#patch-these-components">Patch these components</a></h3><p>The list starts with the database tier.</p><p><a href="https://www.postgresql.org/about/news/pgbouncer-1261-released-fixes-three-cves-3403/"><strong>PgBouncer</strong></a><br> <strong>Update to:</strong> 1.26.1<br> <strong>Why it matters:</strong> A client with one user’s password can sign in as any other user that uses SCRAM password authentication, including superusers. Two other bugs let an unauthenticated client freeze it.</p><p><a href="https://www.oracle.com/security-alerts/cpuoct2026.html"><strong>MySQL</strong></a><br> <strong>Update to:</strong> October Critical Patch Update (8.4.13 for the 8.4 line)<br> <strong>Why it matters:</strong> Quarterly MySQL fixes; version 8.4 is affected through 8.4.12.</p><p><a href="https://mariadb.com/docs/server/security/cve/community-server"><strong>MariaDB</strong></a><br> <strong>Update to:</strong> 10.6.28, 10.11.19, 11.4.13, 11.8.9, or 12.3.3<br> <strong>Why it matters:</strong> MariaDB published 15 CVEs on October 9 and 16, including three rated 9.9. The fixes shipped in the summer releases, so confirm that you have them.</p><p><a href="https://www.postgresql.org/about/news/pgvector-087-released-3392/"><strong>pgvector</strong></a><br> <strong>Update to:</strong> 0.8.7<br> <strong>Why it matters:</strong> Buffer overflow when it builds IVFFlat indexes.</p><p><a href="https://jdbc.postgresql.org/security/"><strong>PgJDBC</strong></a><br> <strong>Update to:</strong> 42.7.14<br> <strong>Why it matters:</strong> Two CVEs in the Java driver.</p><p><a href="https://www.postgresql.org/about/news/postgresql-186-1711-1615-1519-1424-and-19-beta-3-released-3365/"><strong>PostgreSQL</strong></a><br> <strong>Update to:</strong> 18.6, 17.11, 16.15, 15.19, or 14.24<br> <strong>Why it matters:</strong> No October release, but the August release fixed 28 CVEs. If you skipped it, update before the November release.</p><p><a href="https://cert.europa.eu/publications/security-advisories/2026-018/"><strong>PHP</strong></a><br> <strong>Update to:</strong> 8.5.12, 8.4.27, 8.3.36, or 8.2.35<br> <strong>Why it matters:</strong> PHP2Shell lets attackers run code on the app servers that hold your database credentials.</p><p><a href="https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-5-1-released/"><strong>GitLab</strong></a><br> <strong>Update to:</strong> 19.5.1 or its backports<br> <strong>Why it matters:</strong> Two bugs rated 10 out of 10. CI/CD variables often hold production database credentials.</p><p><a href="https://nodejs.org/en/blog/vulnerability/october-2026-security-releases"><strong>Node.js</strong></a><br> <strong>Update to:</strong> 22.23.4, 24.22.1, or 26.12.1<br> <strong>Why it matters:</strong> Brotli decoder bug, plus fixes for TLS hostname checks.</p><p><a href="https://bleepingcomputer.com/news/security/f5-releases-out-of-band-patches-for-critical-nginx-vulnerabilities"><strong>NGINX</strong></a><br> <strong>Update to:</strong> 1.31.7 or 1.30.6<br> <strong>Why it matters:</strong> Three memory bugs, each needing a specific configuration (alias, AIO with SSI, or a regex map).</p><p>If any computer installed LiteLLM 1.107.0 on October 21, treat it as compromised. Rotate every credential it could read, including database passwords.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#reduce-credential-risk">Reduce credential risk</a></h3><ol><li>Rotate every database credential that you’ve stored in a hosting platform’s environment variables, a CI/CD system, or a hosted SQL console.</li><li>Switch to short-lived credentials where your platform supports them, such as cloud identity and access management (IAM) database authentication or dynamic secrets from a vault.</li><li>Create a separate database user for each service and each AI agent, with only the access it needs. Don’t let agents use the application’s sign-in. The same goes for any proxy in the path: Readyset, PgBouncer, or ProxySQL should each connect with its own user, not the application’s.</li><li>Block public internet access to your database, and require TLS.</li><li>Turn on audit logging for your database and for any control plane that manages it.</li><li>Store backups offsite, and make them immutable. The <a href="https://www.cisa.gov/news-events/cybersecurity-advisories/aa26-293a">SafePay advisory</a> describes attackers resetting backup appliances and deleting backups before they encrypt data.</li></ol><p>Per-agent database users get hard to manage once you have more than a few agents. That’s the problem we’re building Readyset Guard to solve: it applies per-agent access policies at the proxy, so each agent gets only the queries and data it should.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#one-ping-only">One ping only</a></h3><p>The film ends well because Ryan read the evidence instead of the noise. Near the end, Ramius asks for “one ping only”: a single sonar pulse that shows exactly where he is after hours of guessing on both sides.</p><p>That’s the standard we’d hold this month’s coverage to. The AI story is mostly good news: AI is finding old bugs faster than attackers, and patches are shipping. The part that cost people data and money was quieter. It came in through keys that had sat in the same place for months.</p><p>Patch what sits next to your database, and spend an afternoon on the systems nobody watches. You’ll learn more there than from the news feed.</p><h3><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch#appendix-all-28-incidents">Appendix: All 28 incidents</a></h3><p>We checked each incident against primary sources where they exist. Relevance means relevance to teams that run Postgres or MySQL.</p><p><strong>1: </strong><a href="https://cert.europa.eu/publications/security-advisories/2026-018/"><strong>PHP2Shell</strong></a><br> <strong>When:</strong> Fixed Oct 21<br> <strong>What AI did:</strong> Found the bug<br> <strong>Relevance:</strong> Medium-high<br> <strong>What the list got wrong:</strong> Understated it. “70%” is PHP’s share of websites, not the share that’s vulnerable.</p><p><strong>2: </strong><a href="https://thehackernews.com/2026/10/eight-emudhra-ca-keys-compromised.html"><strong>eMudhra</strong></a><br> <strong>When:</strong> Oct 20<br> <strong>What AI did:</strong> Found it (weak link)<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> Intermediate keys, not root keys. None issued a website certificate.</p><p><strong>3: </strong><a href="https://www.cde.ca.gov/nr/in/ets-notification.asp"><strong>California students</strong></a><br> <strong>When:</strong> Access May-Sep; notice Oct 19<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> The breach was at ETS, the testing vendor.</p><p><strong>4: </strong><a href="https://blog.railway.com/p/security-disclosure-wednesday-october-21st"><strong>Railway</strong></a><br> <strong>When:</strong> Oct 21<br> <strong>What AI did:</strong> AI keys targeted<br> <strong>Relevance:</strong> High<br> <strong>What the list got wrong:</strong> Railway says “may have accessed” and gives no count.</p><p><strong>5: </strong><a href="https://zeabur.com/zh-TW/blogs/october-2026-security-incident"><strong>Zeabur</strong></a><br> <strong>When:</strong> Oct 14–17<br> <strong>What AI did:</strong> AI keys targeted<br> <strong>Relevance:</strong> High<br> <strong>What the list got wrong:</strong> Counted 321,810 records read, not accounts taken over.</p><p><strong>6: </strong><a href="https://www.stepsecurity.io/blog/litellm-1-107-0-compromised-pypi-backdoored-wheels-credential-stealer"><strong>LiteLLM</strong></a><br> <strong>When:</strong> Oct 21<br> <strong>What AI did:</strong> AI tool compromised<br> <strong>Relevance:</strong> Medium-high<br> <strong>What the list got wrong:</strong> The bad release was live for about two hours.</p><p><strong>7: </strong><a href="https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-5-1-released/"><strong>GitLab</strong></a><br> <strong>When:</strong> Oct 12 and 21<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Medium<br> <strong>What the list got wrong:</strong> Merged two separate bugs rated 10 out of 10 into one.</p><p><strong>8: </strong><a href="https://thenextweb.com/news/ai-agents-hacker-cairn-100-sites-deere-norforge"><strong>Cairn</strong></a><br> <strong>When:</strong> Sep 25-Oct 19<br> <strong>What AI did:</strong> Agents ran the attack<br> <strong>Relevance:</strong> Medium-high<br> <strong>What the list got wrong:</strong> “100+” is an estimate based on token spending.</p><p><strong>9: </strong><a href="https://bleepingcomputer.com/news/security/retail-giant-ahold-delhaize-takes-systems-offline-after-cyberattack"><strong>Ahold Delhaize</strong></a><br> <strong>When:</strong> Oct 11–12<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> The company took systems offline itself, and stores stayed open.</p><p><strong>10: </strong><a href="https://www.onamae.com/news/article/11584"><strong>Onamae.com</strong></a><br> <strong>When:</strong> Oct 20–21<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Medium<br> <strong>What the list got wrong:</strong> One VPS product. 26,739 servers were possibly affected.</p><p><strong>11: </strong><a href="https://securityaffairs.com/201490/security/signal-vulnerability-could-keep-calls-alive-after-users-hang-up.html"><strong>Signal</strong></a><br> <strong>When:</strong> Fixed Sep; CVE Oct 20<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> A malicious caller has to trigger it.</p><p><strong>12: </strong><a href="https://bleepingcomputer.com/news/security/f5-releases-out-of-band-patches-for-critical-nginx-vulnerabilities"><strong>NGINX</strong></a><br> <strong>When:</strong> Oct 14<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Medium-high<br> <strong>What the list got wrong:</strong> Rated 8.1 under CVSS 3.1. Each bug needs a non-default configuration.</p><p><strong>13: </strong><a href="https://thehackernews.com/2026/10/new-qpack-flaws-can-crash-http3-servers.html"><strong>HTTP/3 QPACK</strong></a><br> <strong>When:</strong> Oct 22<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> Affects HTTP/3 libraries, not mainstream web servers.</p><p><strong>14: </strong><a href="https://thehackernews.com/2026/10/hackers-exploit-plex2shell-flaws-to.html"><strong>Plex</strong></a><br> <strong>When:</strong> Exploited from Oct 17<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> Patched in August. Only unpatched servers are at risk.</p><p><strong>15: </strong><a href="https://www.cisa.gov/news-events/cybersecurity-advisories/aa26-293a"><strong>SafePay</strong></a><br> <strong>When:</strong> Oct 20<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Medium<br> <strong>What the list got wrong:</strong> 600+ victims, but not all are critical infrastructure.</p><p><strong>16: </strong><a href="https://unit42.paloaltonetworks.com/active-exploitation-of-pan-os-cve-2026-0257/"><strong>Palo Alto GlobalProtect</strong></a><br> <strong>When:</strong> Bug from May; update Oct 19<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Medium-high<br> <strong>What the list got wrong:</strong> An old bug with a new ransomware attribution.</p><p><strong>17: </strong><a href="https://www.bleepingcomputer.com/news/security/bullhorn-takes-staffing-platform-offline-to-contain-cyberattack/"><strong>Bullhorn</strong></a><br> <strong>When:</strong> Oct 16–19<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low-medium<br> <strong>What the list got wrong:</strong> Accurate</p><p><strong>18: </strong><a href="https://www.securityweek.com/970000-medi-cal-members-affected-by-paradigm-healthcare-data-breach/"><strong>Medi-Cal</strong></a><br> <strong>When:</strong> Breach Oct 2025<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low-medium<br> <strong>What the list got wrong:</strong> A billing vendor, and a year old</p><p><strong>19: </strong><a href="https://patchstack.com/articles/updraftcentral-key-compromise-wordpress-backdoors/"><strong>UpdraftPlus (WordPress)</strong></a><br> <strong>When:</strong> Oct 18–19<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Medium<br> <strong>What the list got wrong:</strong> Accurate</p><p><strong>20: </strong><a href="https://thehackernews.com/2026/10/attackers-tampered-with-aomei.html"><strong>AOMEI</strong></a><br> <strong>When:</strong> May-Jul; disclosed Oct 16<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Medium<br> <strong>What the list got wrong:</strong> Months old</p><p><strong>21: </strong><a href="https://cybersecuritynews.com/meta-muse-ai-privacy-flaw/"><strong>Meta Muse</strong></a><br> <strong>When:</strong> Oct 20–21<br> <strong>What AI did:</strong> AI product leaked data<br> <strong>Relevance:</strong> Medium<br> <strong>What the list got wrong:</strong> Shared chats only. No misuse seen.</p><p><strong>22: </strong><a href="https://kaba.cx/posts/20261020_Tor/"><strong>Tor relays</strong></a><br> <strong>When:</strong> Aug-Oct 20<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low-medium<br> <strong>What the list got wrong:</strong> Hacked exit relays stole SSH passwords, not general sign-ins.</p><p><strong>23: </strong><a href="https://www.pingcap.com/trust-hub/security/security-update-october-2026/"><strong>TiDB Cloud</strong></a><br> <strong>When:</strong> Sep 9-Oct 19<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> High<br> <strong>What the list got wrong:</strong> The attacker’s 10.6 TB and 180,000-organization claims are unconfirmed.</p><p><strong>24: </strong><a href="https://thehackernews.com/2026/10/state-bank-of-india-confirms-data-leak.html"><strong>State Bank of India</strong></a><br> <strong>When:</strong> Oct 21<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> 3,897 records confirmed. SBI hasn’t confirmed a breach of its systems.</p><p><strong>25: </strong><a href="https://www.securityweek.com/hijacked-developer-account-used-to-plant-stealer-in-ovhcloud-cli/"><strong>OVHcloud CLI</strong></a><br> <strong>When:</strong> Oct 19<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Medium<br> <strong>What the list got wrong:</strong> Accurate, but nobody has analyzed the payload.</p><p><strong>26: </strong><a href="https://www.securityweek.com/siemens-software-center-vulnerability-exploited-in-clickfix-malware-attacks/"><strong>Siemens Software Center</strong></a><br> <strong>When:</strong> Oct 15–19<br> <strong>What AI did:</strong> None<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> An installer app, not industrial systems</p><p><strong>27: </strong><a href="https://www.bleepingcomputer.com/news/security/lazarus-hackers-use-post-quantum-cryptography-for-malware-delivery/"><strong>North Korea (Lazarus)</strong></a><br> <strong>When:</strong> Since Jul 2025; Oct 21<br> <strong>What AI did:</strong> Attack (weak link)<br> <strong>Relevance:</strong> Low<br> <strong>What the list got wrong:</strong> Not new. The AI link is one generated photo.</p><p><strong>28: </strong><a href="https://nodejs.org/en/blog/vulnerability/october-2026-security-releases"><strong>Node.js Brotli</strong></a><br> <strong>When:</strong> Oct 21<br> <strong>What AI did:</strong> Found the bug<br> <strong>Relevance:</strong> Medium<br> <strong>What the list got wrong:</strong> Code execution was shown only on a weakened build.</p><p>Database and credential items come first.</p><p><em>Originally published at </em><a href="https://readyset.io/blog/black-october-what-actually-happened-and-what-to-patch"><em>https://readyset.io</em></a><em> on October 27, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XzVmOle9-49lfybEvCex8A.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed977251c80e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Readyset HA with ProxySQL and MySQL | Readyset]]></title>
            <link>https://medium.com/@readysettech/readyset-ha-with-proxysql-and-mysql-readyset-b353b1a1b607?source=rss-13336def6db6------2</link>
            <guid isPermaLink="false">https://medium.com/p/b353b1a1b607</guid>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[ReadySet]]></dc:creator>
            <pubDate>Thu, 22 Oct 2026 00:00:50 GMT</pubDate>
            <atom:updated>2026-10-22T14:17:31.289Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://readyset.io/blog">← Back to blog</a> Tutorials</p><p>A MySQL source, a replica and a ProxySQL in front of both. Adding a highly available Readyset cache behind one slow report query took two docker run commands and two INSERT statements, with no change to the application, the schema or MySQL. Here is the measurement, the routing, and how to check the cache is really serving.</p><p>A query cache stores the result of a SQL query and answers later identical queries from that stored copy instead of running them again. Readyset is a query cache for MySQL and PostgreSQL that works as a proxy: it speaks the database’s wire protocol, so it can decide per query whether to answer from cache or pass the query through.</p><p>Speaking the wire protocol is what makes Readyset drop-in: anything that can connect to MySQL can connect to Readyset instead, with no driver or query change. Put two Readyset nodes behind ProxySQL and you have a cache tier that survives losing a node.</p><p>We wanted to know how small that change really is when the topology already exists. So we built the boring version of a production setup, a MySQL source with a replica and a pair of ProxySQL nodes in cluster mode in front of both, gave it a report query that costs 106 ms, and counted the steps needed to put a highly available (HA) cache behind that one query. It came to two docker run commands and two INSERT statements in ProxySQL&#39;s admin interface. Nothing in the application changed, and neither did the schema or MySQL&#39;s configuration.</p><h3><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql#what-a-proxysql-and-mysql-topology-looks-like-before-readyset">What a ProxySQL and MySQL topology looks like before Readyset</a></h3><p>Seven machines in one availability zone, all c6i.large, which is 2 virtual central processing units (vCPUs) and 4 GiB of memory each. MySQL 8.0.46 on the source and on the replica, two ProxySQL 2.6.5 nodes in front of them, and the <a href="https://dev.mysql.com/doc/sakila/en/">sakila sample database</a> loaded on the source and replicated down.</p><p>Both ProxySQL nodes are configured the way most people configure one: hostgroup 10 is the writer, hostgroup 20 is the reader, writes go to the source, and a catch-all rule sends anything matching ^SELECT to the replica.</p><p>The two proxies run in ProxySQL’s cluster mode, which makes them peers. Each lists the other in proxysql_servers, they share a cluster credential, and a configuration change applied to either one is pulled by the other within a few seconds. That is why adding Readyset below costs the same two statements it would cost with a single proxy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7fhr75Yx0DjN773kh_uVOA.png" /></figure><p>Everything in grey below was already running. Everything in green is what this post adds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TWjwidZewcXfqf4p.png" /></figure><p><em>The two Readyset nodes and the hostgroup 30 routing are the whole change. Readyset connects to the replica as an ordinary SQL client, so there is no replication stream into it and nothing to configure on MySQL. The purple link is ProxySQL’s cluster sync, which is why the routing has to be added on only one of the two proxies.</em></p><h3><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql#which-query-is-worth-caching-and-how-proxysql-tells-you">Which query is worth caching, and how ProxySQL tells you</a></h3><p>ProxySQL already knows. It keeps a digest table of every query shape it has seen, with the total time spent on each, and the answer falls out of one query against the admin interface on port 6032.</p><pre>SELECT hostgroup AS hg, digest, count_star AS calls,<br>       ROUND(sum_time/1000000,1) AS total_s,<br>       ROUND(sum_time/count_star/1000,1) AS avg_ms,<br>       substr(digest_text,1,50) AS query<br>  FROM stats_mysql_query_digest<br> ORDER BY sum_time DESC LIMIT 5;</pre><pre>hg   digest              calls  total_s  avg_ms  query<br>20   0xAD23E0866F6BDE4B  140    28.0     200.0   SELECT c.customer_id,c.first_name,c.last_name,ci.c...<br>20   0xDDE78179CF5B1440  1      0.0      1.0     SELECT COUNT(*) FROM film<br>20   0xAC447120122130DE  1      0.0      0.0     SELECT COUNT(*) FROM actor<br>10   0xFE0D5031018F599A  2      0.0      0.0     SET NAMES utf8mb4</pre><p>The ProxySQL admin interface is SQLite underneath, so substr() works and LEFT() does not. One shape accounts for all of the time. It is a customer leaderboard: the top 25 customers by lifetime value, each annotated with the film category they spend the most on.</p><pre>SELECT c.customer_id, c.first_name, c.last_name, ci.city,<br>       SUM(p.amount)       AS lifetime_value,<br>       COUNT(p.payment_id) AS payments,<br>       (SELECT cat.name<br>          FROM payment p2<br>          JOIN rental r2        ON r2.rental_id    = p2.rental_id<br>          JOIN inventory i2     ON i2.inventory_id = r2.inventory_id<br>          JOIN film_category fc ON fc.film_id      = i2.film_id<br>          JOIN category cat     ON cat.category_id = fc.category_id<br>         WHERE p2.customer_id = c.customer_id<br>         GROUP BY cat.name<br>         ORDER BY SUM(p2.amount) DESC<br>         LIMIT 1)           AS top_category<br>  FROM customer c<br>  JOIN address a  ON a.address_id  = c.address_id<br>  JOIN city ci    ON ci.city_id    = a.city_id<br>  JOIN payment p  ON p.customer_id = c.customer_id<br> GROUP BY c.customer_id, c.first_name, c.last_name, ci.city<br> ORDER BY lifetime_value DESC<br> LIMIT 25;</pre><p>This query is not a straw man. EXPLAIN shows every join resolving through an index, and sakila is about 10 MB against a 2 GB buffer pool, so the whole working set is in memory and no read touches a disk. It costs 106 ms anyway, because the correlated subquery is a DEPENDENT SUBQUERY that MySQL evaluates once per customer. That is 599 executions of a five-table join, each building a temporary table and sorting it, on every single call.</p><h3><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql#why-not-just-rewrite-the-slow-query">Why not just rewrite the slow query</a></h3><p>That is the first thing a database engineer asks, so we measured it. We rewrote the correlated subquery as a window function over a common table expression, which is the textbook fix, and ran both forms against the same replica.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zs57tegWBaGBClPJTHzldw.png" /></figure><p><em>Single connection, 8 s warmup and 25 s measured, read directly against the MySQL replica.</em></p><p>The rewrite is worth 1.2x. The work being done is the same either way: aggregate 16,044 payments across a five-table join, then do the whole thing again for the next caller. Routing the query to Readyset is worth 149x on the same hardware, and it gets there by skipping the work rather than by doing it faster.</p><h3><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql#the-minimal-steps-to-add-readyset-behind-proxysql">The minimal steps to add Readyset behind ProxySQL</a></h3><p>Three commands total, and none of them touch MySQL, the application, or the second proxy.</p><p>First, run Readyset on two hosts. Each one is independent of the other, with no shared state and no clustering to configure. AUTO_CACHE=true is the whole configuration: it is shorthand for--cache-mode=shallow --query-caching=inrequestpath, which means every previously unseen SELECT becomes a candidate for a cache that Readyset creates by itself, on the request path, the first time it sees the query.</p><pre>docker run -d --name readyset --restart unless-stopped \<br>  -p 3307:3307 -p 6034:6034 \<br>  -e UPSTREAM_DB_URL=mysql://app:readyset@&lt;replica-ip&gt;:3306/sakila \<br>  -e LISTEN_ADDRESS=0.0.0.0:3307 \<br>  -e AUTO_CACHE=true \<br>  public.ecr.aws/readyset/readyset:latest-nightly</pre><p>We pointed both nodes at the read replica rather than the source, so the queries Readyset runs upstream land on the machine that already serves reads.</p><p>Docker is here because it makes the deployment one command that a reader can run verbatim, which is what a demonstration needs. It is not the only option: the same adapter ships as .deb and .rpm packages for x86_64 and arm64, run under systemd and configured through/etc/readyset/readyset.conf. Both paths are covered at <a href="https://readyset.io/docs/cache/install-rs">Install Readyset Cache</a>. The flags below are the same either way, as environment variables in Docker or as settings in the configuration file.</p><p>Both nodes were accepting connections about 45 seconds after docker run returned, and that is the whole startup. In shallow mode Readyset does not replicate, which the log states plainly:</p><pre>INFO readyset_server::controller::inner: Replication disabled; skipping snapshotting and replication</pre><p>No replication user, no binary-log position, no snapshot of the dataset, no publication and no slot. Readyset connects to the upstream the same way any client does, runs the query on a miss, and keeps the result.</p><p>Second, tell ProxySQL the nodes exist, and point one query shape at them. This is the only change to the existing topology, and in cluster mode it is applied to one proxy rather than to each of them.</p><pre>-- Hostgroup 30 is the new cache tier. The replica joins it at weight 1, which<br>-- makes it an automatic fallback: it takes 1 read in 2001 while Readyset is<br>-- healthy, and all of them when both Readyset nodes are gone.<br>INSERT INTO mysql_servers (hostgroup_id, hostname, port, weight, max_connections, comment)<br>VALUES (30, &#39;&lt;readyset-1-ip&gt;&#39;, 3307, 1000, 500, &#39;readyset-1&#39;),<br>       (30, &#39;&lt;readyset-2-ip&gt;&#39;, 3307, 1000, 500, &#39;readyset-2&#39;),<br>       (30, &#39;&lt;replica-ip&gt;&#39;,    3306,    1, 500, &#39;mysql-replica-fallback&#39;);<br><br>-- Route exactly one query shape at the cache, by the digest ProxySQL reported.<br>INSERT INTO mysql_query_rules (rule_id, active, digest, destination_hostgroup, apply)<br>VALUES (50, 1, &#39;0xAD23E0866F6BDE4B&#39;, 30, 1);<br><br>LOAD MYSQL SERVERS TO RUNTIME;<br>LOAD MYSQL QUERY RULES TO RUNTIME;<br>SAVE MYSQL SERVERS TO DISK;<br>SAVE MYSQL QUERY RULES TO DISK;</pre><p>Routing by digest rather than by table is what keeps the change small. The same tables keep serving hostgroup 20 for every other query. Only this one shape moves.</p><p>We ran that against proxysql-1 and nothing at all against proxysql-2. The second proxy picked the change up on its own in 3.7 seconds:</p><pre>### proxysql-2 BEFORE, nothing applied to it directly<br>hostgroup 30 servers: 0<br>rule 50 present:      0<br><br>### applying the change to proxysql-1 ONLY<br>propagated to proxysql-2 in 3.7s, with no command run against it<br><br>### proxysql-2 AFTER<br>30   172.31.38.52    3306     1  mysql-replica-fallback<br>30   172.31.39.254   3307  1000  readyset-1<br>30   172.31.44.109   3307  1000  readyset-2<br>rule 50:<br>50   1   0xAD23E0866F6BDE4B   30</pre><p>That is what cluster mode buys. Adding a cache tier to a pair of proxies is the same two statements as adding it to one, so the two proxies cannot drift into routing the same query differently.</p><p>One credential detail is easy to miss. Readyset authenticates incoming clients against its own upstream user, so the user ProxySQL uses for its backend connections has to be the same user that appears in UPSTREAM_DB_URL. A separate application user gets an access denied error at the Readyset connection.</p><h3><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql#what-changed-after-routing-one-query-to-readyset">What changed after routing one query to Readyset</a></h3><p>Same driver, same ProxySQL endpoint, same query text. The only difference between the two halves of each pair is whether rule 50 is active.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IslJieCZTrtwFNi8oZPyNQ.png" /></figure><p><em>10 s warmup, 30 s measured, zero errors in every run. Latency ratio is p50 against p50.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Xchx8ud-POqOWwRK.png" /></figure><p><em>The Readyset bars are drawn at a 3 pixel minimum so they are visible at all; at this scale their true widths are half a pixel. These are 2 vCPU demo machines and one query, so read the shape rather than the absolute numbers.</em></p><p>The rows come back identical. We compared the full result set and every field of every column definition between the cached path and the replica read directly: 25 rows, 7 columns, byte for byte the same.</p><h3><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql#how-to-confirm-readyset-served-the-reads">How to confirm Readyset served the reads</a></h3><p>A cache that exists is not a cache that is being used, and the failure mode looks like success: correct rows at a plausible latency, quietly served by the database. So we checked three separate things.</p><p>ProxySQL answers the first two: is the rule firing, and where did the query actually go. Both come from the admin interface on port 6032.</p><pre>-- is the routing rule matching, and how often<br>SELECT s.rule_id, s.hits, r.destination_hostgroup<br>  FROM stats_mysql_query_rules s<br>  JOIN runtime_mysql_query_rules r USING(rule_id)<br> ORDER BY s.rule_id;<br><br>-- and which hostgroup actually served that query shape<br>SELECT hostgroup, digest, count_star<br>  FROM stats_mysql_query_digest<br> WHERE digest = &#39;0xAD23E0866F6BDE4B&#39;;</pre><p>We sent 800 reads at proxysql-1 and 400 at proxysql-2, and each proxy counted only its own share:</p><pre>--- proxysql-1 (drove 800) ---<br>hostgroup  digest              count_star<br>30         0xAD23E0866F6BDE4B  800<br><br>--- proxysql-2 (drove 400) ---<br>rule_id  hits  destination_hostgroup<br>50       400   30<br>100      1     20<br>hostgroup  digest              count_star<br>30         0xAD23E0866F6BDE4B  400</pre><p>Both proxies send the digest to hostgroup 30, and rule 100, the pre-existing catch-all, still handles everything else. One caveat if you run this yourself: stats_mysql_query_rules is flushed asynchronously, so straight after a burst it can lag the digest table by a few seconds before the two agree.</p><p>That proves the query reached Readyset. It does not prove Readyset answered from cache rather than passing the query upstream, which is the failure that looks exactly like success. Readyset’s own per-query counters settle it:</p><pre>curl -s localhost:6034/metrics | grep readyset_shallow_shallow_result</pre><pre>--- readyset-1 ---<br>readyset_shallow_shallow_result_hit{...,query_id=&quot;q_d618b5595f3efeab&quot;}  98347<br>readyset_shallow_shallow_result_miss{...,query_id=&quot;q_d618b5595f3efeab&quot;}     5<br>--- readyset-2 ---<br>readyset_shallow_shallow_result_hit{...,query_id=&quot;q_d618b5595f3efeab&quot;}  93984<br>readyset_shallow_shallow_result_miss{...,query_id=&quot;q_d618b5595f3efeab&quot;}     5</pre><p>Hits move with the traffic and misses stay in single digits. Both nodes built the same cache for the same query identifier on their own, and both carried roughly half the reads. A Readyset node whose hit counter is flat while queries flow through it is serving everything from the database, and this is the counter to alert on.</p><p>Then we measured the instrument itself. A bare SELECT 1 over the same path, from the same driver through the same ProxySQL, costs 0.280 ms at p50. That is the client library, two network hops and the proxy, and it is 39% of Readyset&#39;s 0.716 ms. Readyset&#39;s own contribution to the cached read is closer to 0.44 ms.</p><h3><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql#what-happens-when-a-readyset-node-or-a-proxysql-node-dies">What happens when a Readyset node or a ProxySQL node dies</a></h3><p>Two tiers can now lose a node, so we killed one on each.</p><p>We ran the query continuously with four clients and stopped the Readyset nodes underneath it, checking every second whether the application was still getting correct answers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pthm6bRxytjorj64sN4xuw.png" /></figure><p>Losing one Readyset node did not show up in the latency series at all. The survivor absorbed the traffic and the application had no way to tell.</p><p>Then we did the same to a proxy, with the load running against proxysql-2 and proxysql-1 stopped underneath it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9aVb3tmhEWdb9B5U7HZSEA.png" /></figure><p><em>Zero errors, and no second without completed queries.</em></p><p>The client talking to the surviving proxy did not notice, which is what a second proxy is for. Cluster mode also covers the rebuild. We brought proxysql-1 back from an empty configuration, with its data volume deleted and--initial on the command line, and it pulled the whole Readyset routing from its peer without being told any of it.</p><pre>hostgroup 30 on the rebuilt proxysql-1, which was never told about Readyset:<br>30   172.31.38.52    3306     1  mysql-replica-fallback<br>30   172.31.39.254   3307  1000  readyset-1<br>30   172.31.44.109   3307  1000  readyset-2<br>50   1   0xAD23E0866F6BDE4B   30</pre><p>Losing both Readyset nodes fell through to the replica and the application kept working at the speed it had before Readyset existed. That is the useful property of the weighted fallback: the degraded mode is the system you already had, not an error page. The handful of errors across these runs were connections in flight when a container was killed, and a client with retry logic would have swallowed them.</p><p>We did not measure how much load a pair of Readyset nodes can take, so none of these figures is a capacity claim. Two 2 vCPU machines is a demonstration size, not a sizing exercise.</p><h3><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql#where-the-readyset-and-proxysql-pattern-applies">Where the Readyset and ProxySQL pattern applies</a></h3><p>The topology has to be a proxy that can route by query shape, in front of at least one read query expensive enough that the round trip to Readyset is worth avoiding. On a query that already runs in 2 ms, none of this pays for itself, because the floor we measured is 0.342 ms of driver and network before Readyset does anything at all.</p><p>The two Readyset nodes are independent caches, not a cluster. There is no shared state between them and nothing to configure to pair them up, which is the opposite of the ProxySQL pair, where cluster mode is the whole point. ProxySQL spreads reads across both Readyset nodes and takes either one out of rotation on its own, and that is what makes the Readyset failover simple.</p><p>What kept the change small is that nothing in the application knew about it. The connection string still points at ProxySQL, and the query text, the schema and MySQL’s configuration are all exactly what they were. Cluster mode kept it small in the other direction too: a second proxy did not mean a second copy of the work. One query moved from a hostgroup that costs 106 ms to a Readyset hostgroup that costs 0.716 ms, and the rest of the workload never noticed.</p><p>The documentation for shallow caching, the --auto-cache flag and the other installation methods is at <a href="https://readyset.io/docs/cache/install-rs">Install Readyset Cache</a>.</p><p><em>Originally published at </em><a href="https://readyset.io/blog/readyset-ha-proxysql-mysql"><em>https://readyset.io</em></a><em> on October 22, 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DVWfDYrPQfL3dE4MkMDYmg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b353b1a1b607" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>