<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ryan Kass on Medium]]></title>
        <description><![CDATA[Stories by Ryan Kass on Medium]]></description>
        <link>https://medium.com/@ryan.kass?source=rss-df060feda875------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*t49rRsD8eCTnlsN_KQYWQg.jpeg</url>
            <title>Stories by Ryan Kass on Medium</title>
            <link>https://medium.com/@ryan.kass?source=rss-df060feda875------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 17:36:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ryan.kass/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Introducing Zapa]]></title>
            <link>https://medium.com/@ryan.kass/introducing-zapa-33729e35cc24?source=rss-df060feda875------2</link>
            <guid isPermaLink="false">https://medium.com/p/33729e35cc24</guid>
            <category><![CDATA[zcash]]></category>
            <category><![CDATA[avalanche]]></category>
            <category><![CDATA[privacy]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Ryan Kass]]></dc:creator>
            <pubDate>Wed, 04 May 2022 18:25:50 GMT</pubDate>
            <atom:updated>2022-06-01T13:48:30.664Z</atom:updated>
            <content:encoded><![CDATA[<h3>Zapa: A Better Privacy Cryptocurrency</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/266/1*emC875_qd8mH2X52Hj5n9w.png" /></figure><p><a href="https://github.com/zapalabs/zapa">Zapa</a> is a cryptocurrency that allows users to send and receive funds anonymously. While this is not an entirely novel concept (<a href="https://www.getmonero.org/">Monero</a> and <a href="https://z.cash/">Zcash</a> are two popular currencies that achieve this today), <em>Zapa is the only privacy cryptocurrency that operates on </em><a href="https://docs.avax.network/nodes/validate/staking/#how-does-proof-of-stake-work"><em>Proof of Stake</em></a><em> and offers </em><a href="https://support.avax.network/en/articles/5325234-what-is-transactional-finality#:~:text=Transactional%20finality%20is%20the%20guarantee,t%20be%20changed%2C%20or%20canceled."><em>Transaction Finality</em></a>.</p><p>Zapa is able to offer these features while maintaining high throughput and low latency because it operates as an <a href="https://support.avax.network/en/articles/6158840-subnet-faq">Avalanche Subnet</a>. Most people familiar with the blockchain space know Avalanche as a popular cryptocurrency, but for developers, its most exciting feature is the concept of Subnets. Avalanche Subnets allow blockchains to benefit from the core features of Avalanche, including its state of the art <a href="https://docs.avax.network/overview/getting-started/avalanche-consensus/">Snowman consensus mechanism</a> and optimized networking, while also allowing each chain to build custom logic on top of these primitives. In the case of Zapa, this custom logic is a <a href="https://github.com/zapalabs/zcash/tree/master">Zcash</a> node which has been modified and optimized to work as a virtual machine running as an Avalanche Subnet.</p><p>Today, Zcash faces a problem common among many blockchains: For many years, it has supported a deep economy transacting on top of an established product, while over the same period of time, consensus technology has advanced leaps and bounds. Many chains, like Zcash, <a href="https://electriccoin.co/blog/should-zcash-switch-from-proof-of-work-to-proof-of-stake/">would like to move to Proof of Stake</a>, but doing so as an iteration on current technology is difficult and risky (see Ethereum’s many missed deadlines and struggles migrating to ETH 2.0). Subnets present an avenue for existing blockchains to upgrade their underlying consensus mechanism with little disruption to their underlying technology. Zapa offers this simple path forward for Zcash and its user base.</p><h4>Zapa Under the Hood</h4><p>Zapa is composed of two processes which communicate with each other over RPC, <a href="https://github.com/zapalabs/zapavm">Zapavm</a> and Zcashd. Zapavm is a thin layer which acts as a bridge between the main Avalanche validator code and Zcashd. The Zcash daemon is a fork of Zcash, doing all the heavy lifting in terms of validating anonymous transactions and composing anonymous transactions. The main difference between the Zapa-Zcash fork and Zcash itself is that all of the proof of work and networking elements have been stripped out, as these responsibilities are offloaded to the Avalanche part of the stack (<a href="https://github.com/zapalabs/zcash/blob/master/doc/zapa.md">more details from the Zapa docs</a>). Below is a high level diagram of how these systems interact, annotated with some example API calls. For a more detailed depiction of what happens from the moment a user issues a transaction to the moment it is accepted into the chain, checkout out the in-depth technical diagram on <a href="https://github.com/zapalabs/zapa/blob/master/ZapaDiagram.png">Github</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UwyrzedACZ4JNaveJzZj6A.png" /></figure><h4>What’s next?</h4><p>Zapa is now live on Fuji (Avalanche terminology for testnet). Right now, users can transact ZAPA anonymously by running a validator node and sending requests to this node. That said, there is still more work to be done before Zapa can be released in production and trusted with meaningful value. For example, there is currently no Zapa wallet. What this means in practice is that users must run a validating node in order to transact ZAPA — not a long term solution. We also need a block explorer, heavy stress testing, and other miscellaneous tooling required for running a blockchain in production.</p><h4>Get Involved</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*FoU2XhuEQ9VFrKlbAuEMwg.gif" /></figure><p>We need developers! If you’re interested in pushing this project forward, <a href="https://discord.gg/btgUY5RJma">join our Discord</a>, have a look at our <a href="https://github.com/zapalabs/zapa/">GitHub</a>, reach out with any questions, or jump right in by tackling an <a href="https://github.com/zapalabs/zapa/issues">existing issue on Github</a>.</p><p>Validators also wanted! Help out by validating the chain on testnet and using ZAPA. Instructions <a href="https://github.com/zapalabs/zapa#running-a-validator">here</a>.</p><p>Follow along: <a href="https://discord.gg/btgUY5RJma">Join our Discord</a> to keep up to date with what’s going on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33729e35cc24" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Open Sourcing pg2k4j, Our Change Data Capture Solution]]></title>
            <link>https://medium.com/disney-streaming/open-sourcing-pg2k4j-our-change-data-capture-solution-c2d18dc808a8?source=rss-df060feda875------2</link>
            <guid isPermaLink="false">https://medium.com/p/c2d18dc808a8</guid>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[kinesis]]></category>
            <dc:creator><![CDATA[Ryan Kass]]></dc:creator>
            <pubDate>Wed, 12 Dec 2018 16:43:46 GMT</pubDate>
            <atom:updated>2018-12-12T16:43:46.630Z</atom:updated>
            <content:encoded><![CDATA[<p>Last month, Disney Streaming Services published our first open source project, <a href="https://github.com/disneystreaming/pg2k4j">pg2k4j</a>, which is short for <a href="https://www.postgresql.org/">PostgreSQL</a> to <a href="https://aws.amazon.com/kinesis/">Kinesis</a> For Java. pg2k4j is an app that sits between a PostgreSQL instance and an AWS Kinesis Stream, broadcasting all mutations made to the database onto a Kinesis Stream. It’s a simple Java library (a few hundred lines of code) that accomplishes an essential task within our ecosystem in an extremely performant manner.</p><p><strong>Use Case</strong></p><p>I come home on a Saturday afternoon and turn on <a href="https://plus.espn.com/">ESPN+</a>. Behind the scenes, hundreds of micro-services are responsible for providing data to the client app. On this particular day and time, a college basketball game begins auto-playing, and I see an episode of <em>Earn Everything </em>headlining the popular section. As I keep watching the college basketball game, many of those micro-services want to be kept abreast of my progress in the video:</p><ul><li><em>Recommendation service uses my current place in the stream to make better recommendations about videos I’d want to watch.</em></li><li><em>Content Service needs to know my current place in the stream in order to inform the Continue Watching feature.</em></li><li><em>Most vitally, Playback Service uses this value in order to know where to resume playback if I were to exit and re-enter the stream.</em></li></ul><p>We wanted to build a system that allows all of these downstream services to be notified when a user’s place in a stream has changed. Such a system should have the following guarantees:</p><ul><li><strong>Immediate Propagation of Events</strong>: Downstream clients are made aware of updates within seconds.</li><li><strong>Replayability</strong>: Any client should be able to know the state of user’s bookmarks at any point in time. Similarly, new clients may come on board and glean the entire history of updates in order.</li><li><strong>Scalability</strong>: We have millions of users who may be active at a given time. This system must be able to propagate bookmarks at the rate at which they occur.</li></ul><p><strong>Solution</strong></p><p>A simple approach to satisfying these requirements would be to have the client device (Roku, Apple TV, web browser, etc.) put a user’s latest playhead on a Kinesis stream. Downstream clients like the Playback Service would then be able to consume this event to know the most up to date playhead for a given user, content pair. However, client devices cannot guarantee that these events will get delivered in order. Due to network irregularities, sometimes client devices will put updates on the stream out of order.</p><p>Instead of having every downstream service interested in the latest bookmarks perform logic for ordering these events, we solve this problem in one place using PostgreSQL. Then, when an insert or update occurs in PostgreSQL, that change is propagated to a new stream by pg2k4j. Here’s a visual summary of this architecture:</p><p><strong>Why Postgres?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NBgbbeTZgVv8vILk" /></figure><p>Postgres has two essential features that tipped the scales for us when considering which datastore to use:</p><ul><li><a href="https://hashrocket.com/blog/posts/upsert-records-with-postgresql-9-5">Conditional Upserts</a>: As of Postgres 9.5 we can write statements like the one below which allow us to update our Bookmarks table only if the item we’re inserting occurred <strong>after </strong>the one that exists in our database.</li></ul><pre>INSERT INTO &lt;table_name&gt; (userId, contentId, playhead, occurredOn) VALUES (?, ?, ?, ?) ON CONFLICT (userId, contentId) DO UPDATE SET playhead=excluded.playhead, occurredOn=excluded.occurredOn WHERE excluded.occurredOn &gt; bookmarks.occurredOn</pre><p>This statement allows us to fully offload the recency check to the database. Through load testing we’ve seen our Postgres instance able to handle upwards of 1 million of these upserts per minute.</p><ul><li><a href="https://www.postgresql.org/docs/11/wal-intro.html">Write Ahead Log (WAL)</a>: Postgres’ way of ensuring data integrity while maximizing its write throughput. All changes to data files are first written to this log before changes get flushed to disk. This provides a suitable resource to implement a<a href="https://en.wikipedia.org/wiki/Change_data_capture"> Change Data Capture (CDC)</a> solution against. (CDC refers to the design approach we take here of broadcasting changes to a datastore so that clients may know the state of the datastore at any point in time).</li></ul><p><strong>Why Kinesis?</strong></p><p>As a company that deploys its infrastructure on AWS, there’s a lot of value for our team in choosing an AWS managed service. Furthermore, AWS Kinesis satisfies all of our requirements: records put onto Kinesis are available to consumers within a few seconds; each client may consume or replay events from any point in time (firehosing events to S3 preserves events in perpetuity), and Kinesis can horizontally scale by adding more shards, enabling continuously higher read and write throughputs.</p><p><strong>Making pg2k4j</strong></p><p><a href="https://github.com/eulerto/wal2json">wal2json</a> is a Postgres plugin that decodes the WAL into JSON documents with a predictable schema. This plugin provides a clean interface for reading messages from the WAL. Before we wrote pg2k4j to forward wal2json output to Kinesis, we ran software called<a href="https://github.com/handshake/pg2kinesis"> pg2kinesis</a> to solve this problem. This open source solution is essentially the Python version of pg2k4j. Though we found some minor bugs we needed to correct, pg2kinesis allowed us to quickly get our feet wet with CDC using Postgres.</p><p>It became clear, however, that this Python solution would not be able to keep up with the load we needed it to handle. Though our Postgres instance could easily handle 1 million inserts and updates per minute, pg2kinesis would top out at around 100k. Through profiling, we found two major bottlenecks within pg2kinesis:</p><ol><li><em>Parsing the json output from the</em><a href="https://github.com/eulerto/wal2json"><em> wal2json</em></a><em> plugin</em></li><li><em>Putting the CDC records on the Kinesis Stream</em></li></ol><p>These two bottlenecks immediately stood out as curable via a Java implementation. Even though wal2json outputs well defined JSON models, Python needs to perform type inference in order to convert these strings into dicts. Java does this orders of magnitude quicker, since it knows the exact schema of the models that the wal2json plugin emits (represented as a<a href="https://github.com/disneystreaming/pg2k4j/blob/master/src/main/java/com/disneystreaming/pg2k4j/models/SlotMessage.java"> SlotMessage</a> in pg2k4j).</p><p>As for the put to the Kinesis Stream, this is where the Java implementation really shines. By using the JVM, pg2k4j can make use of<a href="https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-kpl.html"> Amazon’s Kinesis Producer Library (KPL)</a>, which is Amazon’s open source library for publishing high-throughput data to Kinesis. It’s not possible to use this library with a non-JVM language, and while pg2kinesis does make use of an<a href="https://github.com/awslabs/kinesis-aggregation"> aggregation library</a>, that library is significantly less performant than the Amazon’s native KPL.</p><p>These hypotheses turned out to be true: pg2k4j is orders of magnitude faster than pg2kinesis. It has proved itself capable of handling 1 million events per minute, and we are confident that it can keep up with any write speed Postgres can handle. It’s been up and running for four months in production serving many downstream teams without incident.</p><p>Here’s a snapshot of the Replication Slot Lag during one of our load tests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/0*ebOQYBX6t5e-k96x" /></figure><p>The Replication Slot is the abstraction wal2json uses to read the changes to the WAL, and the lag refers to how far behind it is in doing so. When deploying pg2k4j, be sure to watch this number — if you see it consistently increasing, consider devoting more compute power to the app.</p><p>When making pg2k4j public, we wanted to strike a balance between releasing a feature-rich library and a tool that’s extremely simple to use. Internally, we use pg2k4j as a library, extending its behavior to allow us to integrate with<a href="https://newrelic.com/"> NewRelic</a>, add custom fields, and publish records to predictable shards to ensure consistent ordering. We’re optimistic that other teams can find value in this software, both as an out of the box solution that can be run via the command line, and as a library that can be customized and extended. Please reach out with any questions or feedback about pg2k4j, and for details about how to use or contribute to pg2k4j see the <a href="https://github.com/disneystreaming/pg2k4j/blob/master/README.md">README</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JaDOWxZRxwf3-yd0BF9x5A.jpeg" /><figcaption>Photo by <a href="https://pixabay.com/en/users/Comfreak-51581/">Jonny Lindner</a> on <a href="https://pixabay.com/">pixabay</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c2d18dc808a8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/disney-streaming/open-sourcing-pg2k4j-our-change-data-capture-solution-c2d18dc808a8">Open Sourcing pg2k4j, Our Change Data Capture Solution</a> was originally published in <a href="https://medium.com/disney-streaming">disney-streaming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>