<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vyom Sharma | VCrizpy on Medium]]></title>
        <description><![CDATA[Stories by Vyom Sharma | VCrizpy on Medium]]></description>
        <link>https://medium.com/@vyomshm?source=rss-6b48188fd3aa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*_8vZ66BgQtAVGSi1YOX3Jg.jpeg</url>
            <title>Stories by Vyom Sharma | VCrizpy on Medium</title>
            <link>https://medium.com/@vyomshm?source=rss-6b48188fd3aa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 11:08:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vyomshm/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Introducing Syndr Chain]]></title>
            <link>https://medium.com/syndr/introducing-syndr-chain-70d85ddc0119?source=rss-6b48188fd3aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/70d85ddc0119</guid>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[defi]]></category>
            <dc:creator><![CDATA[Vyom Sharma | VCrizpy]]></dc:creator>
            <pubDate>Fri, 23 Jun 2023 02:58:27 GMT</pubDate>
            <atom:updated>2023-06-23T02:58:27.925Z</atom:updated>
            <content:encoded><![CDATA[<h3>Revolutionizing DeFi Derivatives with Syndr Chain: Introducing World’s 1st Layer 3 Decentralized Exchange</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*C-6j4dAjbs2Jr8HXY3JYMA.png" /></figure><p>We’re excited to announce<strong> Syndr Chain</strong>, a <strong>Layer 3</strong> <strong>rollup</strong> (Orbit Chain) built on <strong>Arbitrum</strong> in collaboration with <strong>Caldera,</strong> to <strong>onboard the next 100 million traders to DeFi!</strong></p><h3>Syndr x Arbitrum</h3><p>Syndr Chain will launch as an Orbit Chain, settling to the public Arbitrum layer 2. The Syndr Chain testnet is already live, which settles on the Arbitrum Goerli testnet.</p><p>Leveraging Arbitrum has enabled us to launch a decentralized Orbit chain powered by the Nitro tech stack, leveraging fraud proofs, advanced compression, EVM+ compatibility through Stylus, and ongoing improvements by the team developing Arbitrum.</p><p>With the freedom to govern the data access layer, thanks to Arbitrum, we can explore tailored mechanism designs and effectively capture value opportunities on our Syndr chain.</p><p>We feel confident that Arbitrum’s diverse and vibrant trader community, onboarded already, will aid us in becoming a leading options and futures exchange and help us on our journey to competing with leading CeXs.</p><h4>From Offchain Labs’ team, creators of Arbitrum</h4><blockquote>“At Offchain Labs, we’re excited to see Syndr leverage Arbitrum Orbit with Caldera to create a powerful, institutional-grade DeFi derivatives product for mainstream adoption.”</blockquote><blockquote>— A.J. Warner, Chief Strategy Officer at Offchain Labs</blockquote><h4><strong>From Caldera’s team</strong></h4><blockquote>“At Caldera, we’re so thrilled to partner with Syndr on building the first DeFi L3 on Arbitrum. The Syndr team is developing a forward-thinking product, and we’re confident that Syndr will quickly become the exchange-of-choice for retail and institutional traders alike. We’re hyped to have the opportunity to support their team!”</blockquote><blockquote>— Matt Kattz, Co-founder at Caldera.</blockquote><h3>⛓️ Syndr’s Rollup chain == smooth scaling &amp; performance for the next 100 million traders</h3><h3>High Performance</h3><p>Syndr’s L3 rollup chain is built to be <strong>highly performant and can scale with the industry. Lighting fast transactions</strong> allow traders to escape the costly competition for blockspace on general-purpose blockchains.</p><h3>Battle-tested &amp; Open-source</h3><p>While many Layer 1 and Layer 2 blockchains provide general-purpose networks that developers can build on top of, Syndr’s Layer 3 Rollup is a <strong>customised application-specific chain</strong> powered by Arbitrum’s Nitro stack with proving technology rooted in Ethereum.</p><h3>EVM Compatibility &amp; Interoperability</h3><p>With best-in-class UX and Bridging, Syndr will enable <strong>1-click deposits/withdrawals and easy multichain compatibility</strong>, eliminating friction and enabling composability with other DeFi protocols.</p><h3>Easy Access</h3><p>Syndr abstracts away the underlying architecture for seamless trading experiences. <strong>Traders on Syndr incur zero gas fees</strong>, ensuring that all trades are conducted without requiring gas fees or Metamask popups.</p><h4><strong>About Syndr</strong></h4><h3>World’s 1st high-performance Layer 3 Decentralized exchange</h3><p>Syndr is an institutional-grade, high-performance Options and Futures exchange powered by Arbitrum Orbit, a Layer 3 rollup.</p><ul><li>All-in-one: Trade Options, Perps and Futures with low fees.</li><li>High-performance orderbook: low latency trades</li><li>LPs can Earn by<br>- Providing permissionless liquidity through Syndr LP pools, or<br>- Making markets on the orderbook<br>- Running permissionless MM vaults</li><li>High throughput: powered by layer 3 rollup</li><li>Portfolio/SPAN Margining</li><li>100+ markets with daily/weekly/monthly/quarterly expiries</li><li>RFQs integration + Block-trading</li><li>No Gas Fees</li><li>Easy Onboarding &amp; Simple + familiar UI</li></ul><h3>🎉Sign up Now!</h3><p>👉🏻 <a href="https://form.waitlistpanda.com/go/P7QXlffiz6W4vmDSLODM">Syndr Testnet is in Private Beta. Join our waitlist for early access to our platform.</a></p><p>🐦 Follow Syndr on Twitter: <a href="https://twitter.com/SyndrHQ">@SyndrHQ</a></p><p>📢 Join TG Group: <a href="https://t.me/syndr_official">https://t.me/syndr_official</a></p><p>🤝 Join our team: <a href="https://jobs.ashbyhq.com/syndr">Careers at Syndr</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70d85ddc0119" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syndr/introducing-syndr-chain-70d85ddc0119">Introducing Syndr Chain</a> was originally published in <a href="https://medium.com/syndr">Syndr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing Syndr: Options are not optional!]]></title>
            <link>https://medium.com/syndr/introducing-syndr-options-are-not-optional-ae2af2a94425?source=rss-6b48188fd3aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/ae2af2a94425</guid>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[defi]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[options-trading]]></category>
            <dc:creator><![CDATA[Vyom Sharma | VCrizpy]]></dc:creator>
            <pubDate>Fri, 07 Oct 2022 15:18:00 GMT</pubDate>
            <atom:updated>2023-06-15T19:56:28.434Z</atom:updated>
            <content:encoded><![CDATA[<h3>Options are not optional: Introducing Syndr exchange</h3><h3>Vyom | Syndr 🦇🔊 Token2049 on Twitter: &quot;Options are not optional! This is the reason why even many centralised crypto derivatives exchanges have trouble with them.That&#39;s why we&#39;re building @SyndrHQ / Twitter&quot;</h3><p>Options are not optional! This is the reason why even many centralised crypto derivatives exchanges have trouble with them.That&#39;s why we&#39;re building @SyndrHQ</p><p>Options in DeFi have been a graveyard of protocols where no one has been able to gain any significant traction. Currently, no major liquid markets for trading crypto options exist anywhere in the DeFi space. The only product market fit they’ve found is within structured products i.e. DeFi Option Vaults(DOVs). DOVs, although simple to use &amp; understand, are extremely limited and generally inflexible in terms of what they can be used for.</p><p>We have been working on this since the start of 2022 and are super bullish on the entire Options space in general. We also earlier wrote an entire product thesis about Syndr, crypto Options, and the wider DeFi derivatives space here —</p><p><a href="https://medium.com/syndr/syndr-product-thesis-how-can-defi-derivatives-win-e95c715b4a82">Syndr Product Thesis: How can DeFi derivatives win?</a></p><p>That being said, Options are unique! Unique in the sense that even giant centralized crypto derivative platforms like FTX and Binance have struggled to gain traction in the space. The CeFi market for Options can be summarised very well by the following chart —</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EyKmA3KEkEQP9zKVx1eIPQ.png" /><figcaption>CeFi Notional OI $M (source: Genesis Volatility)</figcaption></figure><p>Enter Syndr!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vmSNq3PwkM5tlXDTmOxSUg.png" /><figcaption>Syndr exchange</figcaption></figure><h3>What is Syndr?</h3><p>Syndr is an institutional-grade options and futures exchange.</p><ul><li>Trade Options + Futures with low fees.</li><li>High-performance orderbook: low latency trades</li><li>High throughput: powered by our custom layer3 rollup</li><li>100+ markets with daily/weekly/monthly/quarterly expiries</li><li>Portfolio/SPAN Margining</li><li>Block-trading integrations</li><li>Easy Onboarding &amp; Simple + familiar UI</li></ul><h3>What makes Syndr different?</h3><p>We aim to become the #1 platform for trading options and perps on-chain by hyper-optimizing for capital efficiency and performance.</p><ul><li><strong>Low latency and high throughput — </strong>Users can trade on our orderbook with 1–30ms of latency which scales to thousands of orders per second. (unprecedented in the DeFi infra landscape!)</li><li><strong>Capital-efficiency for traders &amp; market-makers — </strong>Easy to run delta-hedged strategies with reduced collateral requirements (courtesy of our portfolio/SPAN margining system). As of Sep 2022, no one does portfolio margining in the entire DeFi space.</li><li><strong>Perpetuals + dated futures</strong> — Syndr also enables easy hedging of options exposure via perpetuals &amp; dated futures natively on the same platform.</li><li><strong>Block-trading on Syndr — </strong>Native support for block-trading platforms and easy integrations through RFQ platforms for institutions. One-click trading for spreads, combo products, etc.</li><li><strong>Multi-collateral support — </strong>Syndr will support depositing more than just stablecoins as collateral. Think USDC, USDT, ETH, WBTC, etc.</li><li><strong>Easy onboarding</strong> — easy to use, deposit from any chain using <a href="https://socket.tech/">Socket.tech</a></li><li><strong>Low fees — </strong>with a plan towards NO FEES!!</li><li><strong>Easy integration w/t API endpoints </strong>— We support WebSockets, REST &amp; FIX.</li><li><strong>No dependencies </strong>— We’ve built the entire system from scratch with full-stack vertical integration</li></ul><h3>Easy+Efficient Liquidity Provisioning</h3><p>It has never been this easy to do market-making and liquidity provisioning as it is on Syndr.</p><p>Perpetuals futures are the easiest way to hedge your options exposure. The presence of high-performance perps combined with portfolio margining for both (options + perps), all on the same platform, makes it easy for market makers to hedge their exposure in a capital-efficient manner.</p><p>Market maker testimonials incoming )</p><h3>Rising Vol &amp; stability via crypto Options</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fqZ-DZa5-FuIwZzttFTqgw.png" /><figcaption>source: Deribit</figcaption></figure><p>Crypto markets are notorious for having higher volatility than traditional markets. Although cryptocurrencies are currently a new and hyper-volatile asset class, we expect them to be no more volatile than other TradFi assets in the coming medium-to-long term future. As the entire crypto finance space matures, we expect options to play a big part in providing stability to the new and existing users in the space.</p><p>Right now the current market for crypto options trading is mostly institutional but the retail adoption although slow has been growing. The existing institutional market has also been growing as new, and more established institutions from the traditional finance space enter the market.</p><p>This is the reason why we’re bullish on the entire crypto options space in general and also why we’re building Syndr.</p><h3>Vitalik on sensible layer3 designs</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fYLacNbSW1StGaCWbFrYDA.png" /><figcaption><a href="https://vitalik.ca/general/2022/09/17/layer_3.html">https://vitalik.ca/general/2022/09/17/layer_3.html</a></figcaption></figure><p>Syndr exchange utilizes a hybrid infrastructure design which ultimately inherits all the security guarantees of the Ethereum mainnet. We combine both on-chain and offchain components to give the user an experience that is the best of both worlds — the user still trades with full custody of their funds, but the platform itself is as powerful and scalable as a centralized exchange(meaning low latency and high throughput). We are powered by a custom layer 3 rollup. We’ll release more details about the platform as we move closer to our public testnet.</p><h3>Join us</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QWQsvO0J_wRETxDTb3Dm7g.png" /></figure><p><strong><em>Syndr is an institutional-grade options and futures exchange powered by our very own layer-3 Rollup.</em></strong></p><p>If you’re interested in joining our team, write to us at hi@syndr.com or DM us on Twitter at <a href="https://twitter.com/SyndrHQ">https://twitter.com/SyndrHQ</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae2af2a94425" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syndr/introducing-syndr-options-are-not-optional-ae2af2a94425">Introducing Syndr: Options are not optional!</a> was originally published in <a href="https://medium.com/syndr">Syndr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Syndr Product Thesis: How can DeFi derivatives win?]]></title>
            <link>https://medium.com/syndr/syndr-product-thesis-how-can-defi-derivatives-win-e95c715b4a82?source=rss-6b48188fd3aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/e95c715b4a82</guid>
            <category><![CDATA[derivatives]]></category>
            <category><![CDATA[defi]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[trading]]></category>
            <dc:creator><![CDATA[Vyom Sharma | VCrizpy]]></dc:creator>
            <pubDate>Sun, 17 Jul 2022 16:52:11 GMT</pubDate>
            <atom:updated>2022-08-18T15:51:54.196Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XULIugZ3XpuMpdLtE0Nf4A.png" /><figcaption>Syndr Research: a deep dive into Web3 derivatives</figcaption></figure><p>In this article, we aim to distill our research on the state of derivative products in the cryptocurrency space and present our thesis on the need for a new kind of decentralized derivatives exchange.</p><p><strong>Contents of this paper can be split into 6 major sections —</strong></p><ol><li>Centralized vs. Decentralized Derivatives</li><li>DeFi derivatives</li><li>User Composition</li><li>Volume Composition</li><li>Why are DeFi derivatives lagging?</li><li>How can DeFi derivatives win?</li></ol><h3>Centralized vs. Decentralized Derivatives</h3><p>Currently, Centralized exchanges(CeXs) like Binance, FTX, OkEx, Deribit, Delta exchange, Bybit, etc. dominate the market for crypto derivatives.</p><p>Let’s now look at some usage stats below —</p><p><strong>BTC &amp; ETH Futures</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/954/1*5y7AFVQtjzGAS7y2cmdkDQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*lugyMtfNAEatMLMFE-sKFA.png" /><figcaption>ETH &amp; BTC Futures Daily Volumes as of May 27, 2022 (source: <a href="https://analytics.skew.com/">https://analytics.skew.com/</a>)</figcaption></figure><p><strong>BTC &amp; ETH Options</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*xLuv0fZVor5nWR59luroMw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*r_gydVsw_-7L3WlS957eaw.png" /><figcaption>BTC &amp; ETH Option Volumes as of May 27, 2022 (source: <a href="https://analytics.skew.com/dashboard/ether-options">https://analytics.skew.com/dashboard/ether-options</a>)</figcaption></figure><h3>DeFi derivatives</h3><p>Let’s now look at usage stats from some of the leading decentralized derivatives protocols to compare, namely dYdX, GMX, Perpetual protocol (Perpetual Futures) &amp; Opyn(European Options)</p><p><strong>dYdX</strong></p><p><em>A point of note about </em>dYdX<em> is that it uses a hybrid model i.e. a mix of decentralized + centralized(CLOB + matching engine), with the centralized components having a potential long term path towards complete decentralization.<br>Update: </em>dYdX<em> has recently decided to move to their own Cosmos chain and move away from StarkEX model. The team believes that this is going to enable them to build the best product for their customers.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BDgy3FjIq4tCpb2iEy8o8w.png" /><figcaption>dYdX Trading Volume &amp; Total-Revenue(source: <a href="https://tokenterminal.com/terminal/projects/dydx">tokenterminal</a>)</figcaption></figure><p><strong>Perpetual Protocol</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9VAsc0tWxnttB01VQSwxjg.png" /><figcaption>Perpetual Protocol Trading Volume &amp; Total Revenue(source: <a href="https://tokenterminal.com/terminal/projects/perpetual-protocol">tokenterminal</a>)</figcaption></figure><p><strong>GMX</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kiIBCd0EmhhGKPoVZwZcuQ.png" /><figcaption>GMX Volume + Fees (Arbitrum only)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R4G-vlr2MFs-gAmT6fy9Vw.png" /><figcaption>GMX Unique users + New users (Arbitrum only)</figcaption></figure><p><strong>Opyn-v2</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NzchDC-i4Fs29AFyB0f11Q.png" /><figcaption>Opyn v2 Stats (source: <a href="https://dune.com/aerhy/options-market-overview">https://dune.com/aerhy/options-market-overview</a>)</figcaption></figure><p><strong><em>DOVs/Structured Products</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bvM_kuFVQ_FQHxop8MpY4w.png" /><figcaption>Structured Derivatives Protocol</figcaption></figure><p>After looking at these stats and numbers, without saying anything about the backend design of these products, we can easily make the following observations —</p><ol><li><strong>Centralized Derivative exchanges beat their DeFi counterparts by a wide margin — </strong>in terms of users, volume, liquidity, basically everything!!</li><li><strong>Teams in the DeFi space are segmented by financial construction</strong> —</li></ol><p>We also analyzed usage stats from major derivative protocols from the Solana ecosystem, including Mango Markets, Zeta markets, PsyOptions, Drift Protocol, and 01 exchange. While being different in the way they’re designed and operate, the above observations remain true for them as well</p><h3>Volume Composition</h3><p>Trading volume for cryptocurrency derivatives is relatively easier to break down. BTC and ETH dominate the volumes, as seen on both CeXs and DeXs. These two markets consequently are the main contributors to an exchange’s revenue model as well for the same reason.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-CLLgvgQG-xVBlYG92DOnA.png" /><figcaption>dYdX volume composition</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bWgvTGtdhUZkmHfmDzt-sw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i0HEVuDh6XLrbfge4IakDQ.png" /><figcaption>Perpetual swap mkts vol. on okX(left) + FTX(right)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sh8_wf35atzCpjZBXOb0hg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fmnlLXLBkyKwl2sW6C_C6Q.png" /><figcaption>Perp volumes + mkts available on Delta exchange</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R-91s0uPfcRNjDzg2ARzLw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fx0sQLbPTCtHIHwajwhxVw.png" /><figcaption>ETH, BTC, &amp; SOL = top mkts by vol. on Deribit</figcaption></figure><p>Volume for crypto derivatives roughly follows the same pattern across all exchanges. This fact ties in to another pattern where you’ll see newer exchange platform launching with only one market, usually BTC or ETH. For e.g. GMX only had an ETH market when it launched. Even if there’s user demand, Liquidity provisioning can be a huge bottleneck in terms of how many new markets a platform supports. This changes quickly as protocols try to expand to multiple markets without fragmenting liquidity for existing markets/pairs in order to remain competitive with other players in the ecosystem.</p><h3>User Composition</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Axl4oIKbkpCiOEfC6qPBIA.png" /><figcaption>FTX exchange stats — <a href="https://ftx.com/exchange-stats">https://ftx.com/exchange-stats</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w7CcC9zQ-Ps8p6CEMdMg9w.png" /><figcaption>Institutional mkt share for Options on Deribit — <a href="https://www.paradigm.co/stats">paradigm.co/stats</a></figcaption></figure><p><strong>Some hard truths when it comes to user &amp; volume composition</strong></p><p>It’s hard for people to gauge how much of an exchange’s volume is real vs fake. A <a href="https://ftx.com/volume-report-paper.pdf">research report by FTX &amp; Alameda</a> puts the number at ~ 24%. This is usually no easy way to estimate this for a given platform. The methodology for conducting this research is out of the scope of this paper but we recommend the following references for conducting analysis on DeFi derivative deXs <strong>(source: rage trade team) —</strong></p><ol><li>Perpetual Protocol v1 — <a href="https://medium.com/@ragetrade/the-perpetual-pvp-ponzi-beaff4a0c662">https://medium.com/@ragetrade/the-perpetual-pvp-ponzi-beaff4a0c662</a></li><li>Perpetual Protocol v2 — <a href="https://twitter.com/crypto_noodles/status/1516062279371997188?s=20&amp;t=iVDUk7wbp1zK8vuCytzOog">https://twitter.com/crypto_noodles/status/1516062279371997188?s=20&amp;t=iVDUk7wbp1zK8vuCytzOog</a> + <a href="https://dune.com/ragetrade/PerpV2-Analysis">https://dune.com/ragetrade/PerpV2-Analysis</a></li><li>dYdX — <a href="https://twitter.com/crypto_noodles/status/1521508785164066817?s=20&amp;t=BexO8pvOmzQdKKJhJEZYvA">https://twitter.com/crypto_noodles/status/1521508785164066817?s=20&amp;t=BexO8pvOmzQdKKJhJEZYvA</a></li><li>GMX — <a href="https://github.com/RageTrade/gmx-analysis">https://github.com/RageTrade/gmx-analysis</a> + <a href="https://twitter.com/crypto_noodles/status/1519315563654357001?s=20&amp;t=zzit34nsDxkhafPuOuSu7Q">https://twitter.com/crypto_noodles/status/1519315563654357001?s=20&amp;t=zzit34nsDxkhafPuOuSu7Q</a></li></ol><h3>Why are DeFi derivatives lagging?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EA6w7AlMYvz3Ibwi.png" /><figcaption>Source: Meltem Demirors via Coinshares</figcaption></figure><p><strong>Segmenting by Financial construction is a bad idea ~ kills capital efficiency</strong></p><ul><li>Effectively ensure a lack of composability amongst DeFi derivative protocols, in stark contrast with the rest of the DeFi ecosystem like money markets, AMMs, liquid staking protocols, etc.</li><li>Means no cross-margining &amp; therefore loss of capital-efficiency</li><li>Impossible to delta-hedge — segmented DeFi protocols means one protocol is oblivious of a trader’s positions with other protocols.</li><li>Market makers need to use multiple protocols or even centralized alternatives to hedge their trades, thereby making it harder for them to add liquidity to the exchange.</li><li>No Composability means impossible to implement stuff like Atlantic options (see <a href="https://medium.com/@tztokchad/dopex-atlantic-options-50a2d3b77aa7">dopex papers</a>)</li></ul><p><strong>DeFi derivatives have no/worse cross-margining</strong></p><ul><li>The most important feature of a derivatives exchange is its margining system &amp; risk engine. These collectively dictate the capital efficiency of trading on the exchange.</li><li>Capital efficiency is crucial because a highly capital-efficient exchange will allow market makers to add deeper liquidity with lower capital requirements.</li><li>It’s hard &amp; extremely inefficient to open delta-neutral or delta-one positions in the first place and impossible to have them work together to achieve reduced margin requirements</li></ul><p><strong>Lack of multi-collateral support</strong></p><p>Most DeFi derivative protocols only support USDC as collateral. Though this is slowly changing with <a href="https://twitter.com/perpprotocol/status/1521490021424979968?s=20&amp;t=QXBG_ogAUcKfioRmAQ8etg">Perpetual protocol</a> and GMX adding support for more assets. In comparison, FTX supports multiple assets to be supplied as collateral with each contributing differently to its margining system.</p><p>Collateral weights — <a href="https://help.ftx.com/hc/en-us/articles/360031149632-Non-USD-Collateral">https://help.ftx.com/hc/en-us/articles/360031149632-Non-USD-Collateral</a>)</p><p>Allowing multiple assets to be used as collateral in a derivatives exchange is important primarily because it unlocks liquidity for trading without sacrificing direct exposure to the collateral asset. This feature can even enable other protocols to build delta-neutral basis-trading mechanisms like<a href="https://docs.uxd.fi/uxdprotocol/resources/whitepaper"> UXD Protocol</a> on <a href="https://dashboard.uxd.fi/">Mango Markets</a></p><p><strong>No margin and/or cross-collateral support</strong></p><p>To reduce risk, DeFi option protocols require option writers to post maximum loss as collateral. This is extremely inefficient when compared to TradFi or centralized crypto exchanges.</p><p>Although<a href="https://medium.com/opyn/partially-collateralized-options-now-in-defi-b9d223eb3f4d"> Opyn v2’s gamma protocol added support for undercollateralized options</a> powered by early liquidations and their very own margin calculation system, no one seems to be using it in production. This is probably because DeFi option protocols have found their Product-market-fit with DOVs or Structured product protocols like Ribbon, Friktion, Thetanuts, etc. Undercollateralized options being used in DOVs make such DOVs even riskier as these currently don’t utilize any mechanisms to hedge their tail risk.</p><p><strong>Lack of a scalable settlement layer for DeFi Options &amp; DOVs(DeFi Option Vaults)</strong></p><p>DeFi options(crypto options in general) have struggled to gain traction and currently lag behind as compared to Option volumes in TradFi. As the founder of a leading options exchange told us — no one wants to buy options in the crypto space, yet!!</p><p>According to our research, the issues in the DeFi options and DOVs(DeFi Option Vaults) landscape are as follows —</p><p><strong>Lack of liquidity ~ Low capital efficiency + Fragmentation + Hard-to-use</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_xlDosDQ8lrrVj3TcnhJLg.png" /><figcaption>Crypto Options Market ~ Quote-driven vs. Order driven</figcaption></figure><ul><li>Lack of ideal infrastructure for fast settlement of options → Lack of liquidity</li><li>No liquidity for options → Options are sold via Gnosis Auction or via naive OTC desks (Airswap OTC).</li><li>No market for buying options → only option selling strategies are possible</li><li>Crypto options market is <a href="https://www.paradigm.co/stats">more Quote-driven as opposed to Order-driven</a> whereas settlement in DeFi options is fragmented into a mix of Gnosis auctions, LP pools, Airswap OTC etc. Although as a sign of convergence most DeFi Option Vault(DOV) providers(Ribbon, StakeDAO, Thetanuts, Friktion) have <a href="https://www.paradigm.co/blog/paradigm-announces-the-first-dov-partnership-with-ribbon-finance">partnered with Paradigm</a> to include their RFQ system within their respective settlement processes.</li><li>Lack of interoperability b/w multiple option protocols on diff chains → DOVs restricted to options protocols on one chain. This means options liquidity is fragmented between multiple chains and it’s hard to unify via existing infra i.e. bridges. This issue also exists with multi-chain perpetual exchanges like GMX where liquidity and users are fragmented by chain.</li><li>Options are still not used for hedging stuff in DeFi like LP positions, hedging of options exposure(for Market makers), perpetual futures.</li><li>LPs in the Option protocols &amp; DOVs (the decentralized market-making pools like Hegic, option AMMs, etc) aren’t hedged in a way a true market maker would be. This is a hard problem to solve, especially on-chain.</li><li>All structured product / DOVs running option selling strategies usually sell to the same small group of market makers. Initially, market makers bought these options to hedge their existing positions elsewhere but now these market makers have been hedging this bought option exposure(a portion) by writing the same options contracts on centralized exchanges like Deribit &amp; Delta exchange. Even allowing for open auctions via Gnosis auctions hasn’t made much difference yet as no one apart from market makers is interested in buying them. Another data point to account for is the short-term dampening of premiums observed every week when these DOVs roll over every Friday. All these things suggest that there is not enough organic demand for these options and growth in the TVL of such structured products/DOVs will lead to further dampening of premiums which might consequently disincentivize retail demand.</li><li>Most decentralized exchanges have a much slower UX as compared to CeXs and none offer mobile-based clients(except dYdX on iOS). The ease of use factor is certainly a huge issue when it comes to DeFi in general.</li></ul><p><strong>Pool-Based vs Order-book based mechanics</strong></p><p>DeFi protocols mostly rely on pool-based models due to a lack of scalable order book models and corresponding throughput challenges. dYdX is an exception as it utilizes a custom-built CLOB(Centralized Limit Order Book) and matching engine, in the form of StarkEx. This centralized component which allows scalable throughput gives dYdX a leg up against all its fully decentralized competitors. In contrast, GMX which uses a pool-based model has been growing in terms of users and volume.</p><p><strong>Risk Engines, Liquidations &amp; preventing clawbacks</strong></p><ul><li>Risk engines for derivatives are hard to build on-chain because of their intense computation requirements as well as their need to access real-time data feeds. These are both big bottlenecks in today’s crypto environment.</li><li>Getting an accurate mark price for options is a big problem — For e.g. how Deribit calculates an option’s mark price is not possible to replicate on-chain therefore most DeFi protocols use Black-Scholes for doing this</li><li>Liquidation mechanisms in DeFi differ very much from those used in CeFi and are a big contributor to their bad capital efficiency. CeFi exchanges are able to identify a liquidatable position, issue a margin call &amp; subsequently gradually liquidate it, all in a matter of minutes. This is different from DeFi liquidations which utilize auctions, third-party liquidators, and triggering mechanisms. These also suffer from MeV as well as have to account for network congestion, things that CeFi exchanges don’t have to worry about.</li><li>Due to the highly volatile nature of cryptocurrencies, preventing clawbacks is one of the most important goals of an exchange’s risk management system. A poorly designed risk management system can &amp; does lead to large clawbacks like that of <a href="https://www.coindesk.com/markets/2018/08/03/okex-confirms-9-million-clawback-after-enormous-bitcoin-future-fails/">OkEx</a> &amp; more recently the <a href="https://realms.today/dao/7sf3tcWm58vhtkJMwuw2P3T6UBX7UE5VKxPMnXJUZ1Hn/proposal/HuaL6cDtuNtfnJgvwMnYiZDHVCoLAuDtVFgJD8kYChJ4">Solend incident</a>. Another caveat is that centralized exchanges can more easily iterate on their risk engine as opposed to risk engines implemented as a system of smart contracts which can be hard to upgrade/update.</li></ul><p><strong><em>Some important metrics for reference -</em></strong></p><ul><li>dYdX processes orders at a speed of 1000 orders place/sec and with actual blockchain trades taking place at a speed of 10–30 trades per second</li><li>Solana derivative exchanges can leverage its 60k+ tps with ~ 400ms block times but have <a href="https://status.solana.com/history">frequent network halts due to validators going out of sync</a>. Furthermore, it can be hours before the <a href="https://status.solana.com/incidents/ggym7s4qyzsq">network gets in sync again &amp; restarts</a>. This can be a death blow for a derivatives exchange. (Also sometimes -&gt; Solana network is experiencing degraded performance. Transactions may fail to send or confirm. (TPS: 1238))</li><li>There have been instances where the sequencers of major L2s have also halted tx submissions causing network halts — <a href="https://offchain.medium.com/todays-arbitrum-sequencer-downtime-what-happened-6382a3066fbc">Arbitrum One Beta</a></li><li>Chainlink price feeds only update post every 0.5% movement on mainnet &amp; every 0.05% movement on Arbitrum, every 0.1% on Avax, FTM &amp; BSC</li></ul><h3>How can DeFi derivatives win?</h3><ol><li><strong>Composable derivatives or exert full-stack control</strong></li></ol><p>To enable true cross-margining, different financial derivatives need to interface together efficiently. Take options as an example; for Options to really shine, they need to interface with perpetual swaps.</p><p>Derivatives should integrate with existing DeFi protocols for hedging &amp; leverage seamlessly. Currently, this is not possible due to protocol mechanic limitations. It’s not possible to get Opyn’s oTokens to compose with perpetual swaps on dYdX. This level of composability needs to be pre-planned with functionality built into the codebase itself.</p><p>The other, more easier way, is to have a single protocol which exerts full control and offers a full-stack of derivative products.</p><ul><li>Hedging LP positions</li><li>Hedging lending positions</li><li>Increased leverage i.e. the main reason why options became popular among retail stock traders</li></ul><p><strong>2. Cross-margin everything</strong></p><p>Users on DeFi derivative platforms frequently use other external platforms to hedge their positions. This can be due to the lack of markets, derivative instrument support or plain old lack of liquidity. Users need a single platform for trading multiple derivatives with deep liquidity.</p><p><strong>3. Robust Risk Engine ~ upgradable constraints paired w/t Better Liquidation mechanisms</strong></p><p><strong>4. Reimagine Infrastructure, i.e. Scalable liquidity and settlement layer</strong></p><p><strong>5. Leverage DeFi native instruments</strong></p><p>Power perpetuals — Power perps, first introduced by the teams at Opyn and Paradigm, are a new financial derivative native to DeFi. Amongst other things Power perps can be used to hedge concentrated liquidity AMM LP positions, options, and pretty much any function payoff with a combination of a futures and a quadratic hedge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/904/1*STduX5sX9b9vi_H5CeRGsQ.png" /><figcaption>source: <a href="https://medium.com/opyn/hedging-uniswap-v3-with-squeeth-bcaf1750ea11">https://medium.com/opyn/hedging-uniswap-v3-with-squeeth-bcaf1750ea11</a></figcaption></figure><ul><li>Hedging uni-v3 with Squeeth — <a href="https://medium.com/opyn/hedging-uniswap-v3-with-squeeth-bcaf1750ea11">https://medium.com/opyn/hedging-uniswap-v3-with-squeeth-bcaf1750ea11</a></li><li>Hedging options with Squeeth — <a href="https://medium.com/opyn/how-to-hedge-options-with-squeeth-b5e30d5d83ac">https://medium.com/opyn/how-to-hedge-options-with-squeeth-b5e30d5d83ac</a></li></ul><p>Uniswap v3 LP Positions — it’s possible to build automated hedging strategies using Uniswap v3’s concentrated liquidity LP positions to hedge stuff. <a href="https://lambert-guillaume.medium.com/gamma-transforms-how-to-hedge-squeeth-using-uni-v3-da785cb8b378">It’s theoretically possible to hedge any derivative with a positive gamma using multiple uni-v3 LP positions</a>.</p><h3>Syndr: Product Thesis</h3><p>At Syndr, our goal was to reimagine the derivatives space and build everything from the ground up.</p><p>At Syndr, we’re building a decentralized cryptocurrency derivatives exchange that offers Options, Perpetuals, and Power perpetuals. We’re the very first DeFi platform to pioneer the trading of multiple derivative products on a single platform with high capital efficiency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c2gCia3ey98TCcKiA_fYJw.png" /></figure><p>The core technological innovation behind Syndr is using a custom-built stack that leverages a combination of technologies, including ZK Provers, Syndr TEE, scalable offchain orderbook, Smart contracts, and a robust risk management layer</p><p>Core features of Syndr Exchange are as follows —</p><ol><li>Single platform for multiple derivative products — namely Options, Perpetuals &amp; Power perpetuals</li><li>Cross Margining</li><li>Multi-Collateral support with cross-collateralization</li><li>Portfolio Margining support</li><li>Reimagined infrastructure for high throughput —Decentralized Settlement powered by an Offchain orderbook</li><li>Robust Risk management</li><li>Full Composability ~ Tokenized Sub-accounts</li></ol><p>We’re currently heads down in build mode, and We’re looking forward to sharing more information and updates about our platform in the coming weeks.</p><blockquote><em>Calling all traders, institutions, and market makers, get in touch now for early access!</em></blockquote><p><a href="https://syndr.com/">Website</a> | <a href="https://twitter.com/SyndrHQ">Twitter</a> |<a href="https://www.linkedin.com/company/syndr/"> Linkedin</a> | <a href="https://discord.com/invite/kQYcw8rHTf">Discord</a> |</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e95c715b4a82" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syndr/syndr-product-thesis-how-can-defi-derivatives-win-e95c715b4a82">Syndr Product Thesis: How can DeFi derivatives win?</a> was originally published in <a href="https://medium.com/syndr">Syndr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Announcing our Angel round]]></title>
            <link>https://medium.com/syndr/announcing-our-angel-round-e0677a7c5b8?source=rss-6b48188fd3aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0677a7c5b8</guid>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[investing]]></category>
            <category><![CDATA[defi]]></category>
            <dc:creator><![CDATA[Vyom Sharma | VCrizpy]]></dc:creator>
            <pubDate>Thu, 20 Jan 2022 11:24:00 GMT</pubDate>
            <atom:updated>2023-08-18T11:07:46.749Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ditO8dARwwQHmn2Ry3Ctbw.png" /></figure><p>Like many others in the web3 space, Syndr was envisioned at EthGlobal’s Hackmoney hackathon to solve these problems. The journey so far has been an awesome ride. The DeFi landscape evolves very fast, and so has our vision for Syndr since its inception. We’re delighted to announce that Syndr successfully closed on a small round of pre-seed funding in November, 2021. We have raised $500k in capital from some of the best founders and angels in the space.</p><h3>Round Participants</h3><p>(in no particular order)</p><ul><li><a href="https://twitter.com/k06a">Anton Bukov</a> (Co-Founder, 1inch)</li><li><a href="https://twitter.com/deacix">Sergej Kunz</a> (Co-Founder, 1inch)</li><li><a href="https://twitter.com/bneiluj">Julien Bouteloup</a> (Founder at StakeDAO, Blackpool Finance &amp; Stake Capital + Core team at Curve Finance)</li><li><a href="https://twitter.com/StefanDGeorge">Stefan George</a> (Co-Founder, Gnosis)</li><li><a href="https://twitter.com/tushar307">Tushar Aggarwal</a> (Founder, Persistence.One, pStake)</li><li><a href="https://twitter.com/vaibhavchellani">Vaibhav Chellani</a> (Co-Founder, Socket.tech)</li><li><a href="https://twitter.com/aniket_jindal08">Aniket Jindal</a> (Co-Founder, Biconomy)</li><li><a href="https://twitter.com/harshrajat">Harsh Rajat</a> (Co-Founder, EPNS)</li><li><a href="https://twitter.com/mohakagr">Mohak Agarwal</a> (Founder, Claystack)</li><li><a href="https://twitter.com/HBxCapx">Hitesh Bhardwaj</a> (Co-Founder, CapX)</li><li><a href="https://twitter.com/empAbhishek">Abhishek Singh</a> (Co-Founder, Comdex)</li><li><a href="https://twitter.com/prashant_xyz">Prashant Maurya</a> (Co-Founder, Spheron)</li><li><a href="https://twitter.com/RobertoTalamas">Roberto Talamas</a> (Data Science/Research, Messari)</li><li><a href="https://twitter.com/WilsonWithiam">Wilson Withiam</a> (Research, Messari)</li><li><a href="https://twitter.com/FaridRached">Farid Rached</a> (Angel, Ecosystem growth at Avalabs)</li><li>DCX Ventures (Venture arm of CoinDCX)</li></ul><p>We’d also like to thank <a href="https://twitter.com/PareenL">Pareen Lathia</a> &amp; <a href="https://twitter.com/pythonhulk">Raghu Mohan</a> and everyone at <a href="https://twitter.com/BuidlersTribe">Buidler’s Tribe</a> for supporting us at every turn.</p><p>Funds acquired in this round will be used to expand our team as we build out the platform.</p><h3>Join us: Syndr is Hiring!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PES1wLtRBHHlhMYJ.png" /></figure><p><strong><em>Syndr is an institutional-grade options and futures exchange powered by our very own layer-3 zkRollup.</em></strong></p><p>If you’re interested in joining our team, write to us at hi@syndr.com or DM us on Twitter at <a href="https://twitter.com/SyndrHQ">https://twitter.com/SyndrHQ</a></p><p>For those looking to join/contribute, here are the things you can do</p><ul><li>Follow Syndr on Twitter: <a href="https://twitter.com/SyndrHQ">@SyndrHQ</a></li><li>Follow Syndr on Medium: <a href="https://medium.com/syndr">Syndr</a></li><li>Join the #OctoApe army by adding OctoApe emoji to your Twitter username: 🦍🐙🦍</li></ul><p>Thanks for all your continued support. Stay tuned for more updates.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0677a7c5b8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syndr/announcing-our-angel-round-e0677a7c5b8">Announcing our Angel round</a> was originally published in <a href="https://medium.com/syndr">Syndr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[20DaysBuidL]]></title>
            <link>https://medium.com/series/20daysbuidl-dbc1dd07dc4f?source=rss-6b48188fd3aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/dbc1dd07dc4f</guid>
            <dc:creator><![CDATA[Vyom Sharma | VCrizpy]]></dc:creator>
            <pubDate>Mon, 13 Aug 2018 16:14:26 GMT</pubDate>
            <atom:updated>2018-08-13T16:15:24.284Z</atom:updated>
            <content:encoded><![CDATA[<p>20 async days BuidLing with a new protocol on ethereum. #20DaysWith&lt;insertProtocolName&gt;</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dbc1dd07dc4f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GSoC-2017 : Working on Anomaly Detection at CERN-HSF]]></title>
            <link>https://medium.com/data-science/gsoc-2017-working-on-anomaly-detection-at-cern-hsf-49766ba6a812?source=rss-6b48188fd3aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/49766ba6a812</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[cern]]></category>
            <category><![CDATA[gsoc]]></category>
            <category><![CDATA[lstm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Vyom Sharma | VCrizpy]]></dc:creator>
            <pubDate>Fri, 25 Aug 2017 00:29:33 GMT</pubDate>
            <atom:updated>2017-08-25T00:29:33.955Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sipPlMb-fKx_zvpbaUErOA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p4KmgEiMu2G31nq9oECEpg.png" /><figcaption>Error function a 200,000 points long Time-Series</figcaption></figure><p>This year was my first time working as a student in <a href="https://summerofcode.withgoogle.com/">Google’s Summer of Code program</a>. I was accepted by the <a href="http://hepsoftwarefoundation.org/">CERN-HSF</a> organisation to work on an Anomaly Detection Project under the <a href="http://atlas.cern/">ATLAS</a> sub-organisation. The purpose of this post is to serve as a summary of my efforts, both achievements and failures, over the summer of this massive undertaking.</p><p>I would like to thank my mentors Mario, Alessandro and Tomas from CERN-HSF for guiding me on this journey!!</p><h3><strong>DeepAnomaly — </strong><a href="https://summerofcode.withgoogle.com/projects/#4916590398144512"><strong>Mission Statement</strong></a></h3><p>ATLAS Open Analytics is a platform which collects data generated from the various computing operations between the multiple participating heterogeneous computing centers.</p><p>My project with CERN-HSF was all about using machine-learning for identifying anomalies in this time series data.</p><p>The idea is to design, build and deploy a framework that is able to monitor, classify and cluster anomalous behaviour in this real-time ATLAS Distributed Computing data and then act on this information autonomously.</p><p><strong>The Goal</strong></p><p>To build an Automated Pipeline which triggers anomalies and then analyses them to predict their root cause as well as some potential solutions.</p><p>The idea is for the system to learn from both historical and live data and make accurate predictions about such anomalous events.</p><h3>Data Exploration</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/765/1*MFQPyP67tY1QhCzZeJOSGQ.png" /><figcaption>Recurring Trends for “transfer-done” events within a single day’s index</figcaption></figure><p>Most of the ADC data is stored in two different <a href="https://www.elastic.co/products/elasticsearch">ElasticSearch</a> instances. For people like myself who are new to this totally awesome tool, ElasticSearch is a distributed, RESTful search and analytics engine. The best part about ElasticSearch is that it’s really fast, almost realtime. It allows you to store, index and distribute your data in a really convenient way. Combine that with some really flexible and intuitive visualisation tools like Kibana and it becomes really attractive, especially if you have massive amounts of data.</p><p>I personally used python’s two elasticsearch libraries for dealing with elasticsearch instances : <a href="https://elasticsearch-py.readthedocs.io/">elasticsearch-py</a> and <a href="https://elasticsearch-py.readthedocs.io/">elasticsearch-dsl</a>.</p><p>At any given point of time, the main CERN ES instance contains 31 “<strong>atlas-rucio-events-*</strong>” indices , one for each of the past 31 days. Due to the truly vast nature and wide variety of data available, my mentor , Mario, advised me to start with only ‘<strong>transfer-done</strong>’ events within these “<strong>atlas-rucio-events</strong>” indices.</p><p>I started by querying events of the type — “transfer-done” and saving them into local DataFrames. <strong>Typically every index contains 1,000,000–2,000,000 of such events and every such event has 26 sub-fields which contain various info about that specific transfer like it’s source, destination, size of the file being transferred etc.</strong></p><p>This was my first look at the data and below are some important plots I made while exploring and extracting the data -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*Ln2usX6snXNHDRwU2a5-Sg.png" /><figcaption>File sizes for transfers in bytes</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*9KrIwni-_bJY6OC9Ba_kHg.png" /><figcaption>Actual durations for the transfers to finish(not including the time spent in queue)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*ENMF3O3UqxqPygXURw0UPw.png" /><figcaption>Time that a file spent in queue before it actually got transferred</figcaption></figure><p><strong>It’s common for some files to get queued for an entire week!!( 1 week~=600000 seconds)</strong></p><p>Apart from these continuous variables there were quite a lot of categorical variables for me to deal with. Here are a few histograms depicting their distribution-frequency in a single index:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*av7GT5RfL-_fs7JtCrgUuA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*QwfS9dBBzCZP4G4qNpbcaw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*Z-ySBFL2224SMxcIZFSRMA.png" /></figure><p>Not taking any risks, I selected all the variables that could be meaningfully parsed as inputs for any model and might be influential for deciding the duration of a transfer. I arrived at a total of 10 variables that would serve as my training data for the future models.</p><h3>Machine learning for Anomaly detection</h3><p>After understanding what the different variables in the data actually meant, I began to actually work on identifying anomalies within these transfers. I don’t think I ever really understood what looking for ‘needle in the haystack’ actually meant up-till now.</p><p>As part of my community bonding period I had done quite a bit of research for ML algorithms specific to time-series data. Most of the models used by scientists in various research papers and articles employed some sort of “supervised-learning approach” to such a problem. But here the situation was slightly different. Within these massive dataframes of transfer events, there was <strong>no label separating anomalous transfers form the normal ones</strong>. An unsupervised approach was the probably the way forward and I had mentioned this earlier in this my original proposal too.</p><h3>Choice of ML Model:</h3><h4>Neural Networks</h4><p>One thing that we had going on for me was that there was literally millions of data points for me to train a Machine Learning Model on!! Also I was reliably informed that the <strong>percentage of occurring anomalies was extremely low, about 1 / 1000</strong>. Armed with this knowledge I started by implementing various feedforward architectures in Keras trying to predict the transfer duration for a particular transfer with varying success.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/1*miccQo---ndkADZ_LqAsHw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*ODECHK_hB-r5z8vxndH7tA.png" /></figure><p>Almost all the feedforward architectures I tried, deeper, wider, different/advanced activations …all pretty much demonstrated similar performance. The model learned for a while and then plateaued out at a level that wasn’t very good. I tried different scaling options to see if it made any difference but there wasn’t any significant improvement.</p><h3>Recurrent Neural Networks- Long Short Term Memory Networks</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a5EbLhyxbPR78PhiV5Esjg.png" /></figure><p>After doing some experimentation with feedforward nets, I finally arrived at RNNs. Given the time-series nature of my data, they were my most ideal choice and I was optimistic about them.</p><p>I started with simple LSTM networks with a single LSTM and began training them on transfer-events in time-steps on 10 events. They were slightly better than previous architectures but not much. After hassling with multiple stacked layers, more LSTMS/GRU cells per layer and training the network with data where every datapoint went farther back into it’s history, I finally arrived at the following architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*K7T9-tkKVSniTGHGKCNK2w.png" /></figure><p><strong>This model takes inputs in timesteps of 100 events i.e. for predicting duration for every single transfer, it considers the details of that transfer + details of 99 transfers which came before this one. Every single of these transfers further has 9 inputs — file_size, time_in_queue, source, destination, activity, protocol, transfer-endpoint, src_type, dst_type.</strong></p><p>For an input of dim (100, 9) , the network outputs a single value (the transfer duration of the 100th transfer in the input). We trained this network on a dataset of millions of such events and never more than 2 iterations for every event. This was done so as to ensure that the model doesn’t learn to map any anomalous events in the dataset. Here are some performance plots of the network when evaluated on 200,000 previously unseen transfers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XbPk-XMQBcRhPnYiUSF_fQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Johsb7DYl9zIovk-xYwEw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*YUtTZsGo0KsyuhGHoZsLrw.png" /><figcaption>error distribution</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BJsXflH--hIpR8S4vaz_0Q.png" /></figure><p>This was by far the best performing model that I had gotten.</p><h4>Threshold used:</h4><ul><li>To distinguish anomalies from the normal events I used a threshold of 600 seconds. Transfer events whose actual transfer duration was more than 600 seconds longer than the predicted one were labeled as anomalies.</li></ul><h3>Analysing the detected Anomalies</h3><p>After building the network for detecting anomalies, next step was to verify them and identify their root causes. This proved quite challenging as the number of detected anomalies was still too large. It was at this point I started collaborating with Tomas. With no potential resource to actually verify the detected anomalies, It was still harder to investigate their causes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*L-x4NzdlDdt-69YKs_acRw.png" /><figcaption>correlation plot b/w predicted and actual durations</figcaption></figure><p>After sometime of exploring, and researching my results and multiple discussions with Tomas, we arrived the following conclusions:</p><ul><li>I needed a <strong>better labelling criteria to tag events as anomalies</strong>; something more robust than just a single error threshold.</li><li>The <strong>number of anomalies identified needed to be brought down to a more manageable number</strong>.</li></ul><h3>Using “PerfSonar” Data</h3><p>The ultimate purpose of this project is to fix the problems causing the detected anomalies in the system. This can only be done by<strong> fixing the specific source-destination link over which the transfer-anomaly occurred</strong>.</p><p><strong>CERN has a system which measures some core statistics about performance of such source-destination pair links. These statistics include information like ‘throughput’, ‘latency’ and ‘packet_loss_rate’</strong>. I believe this data can serve as the basis for a better anomaly-triggering system for detecting anomalies based on the LSTM model predictions.</p><p>For Example-</p><ul><li>A transfer event with high deviation value from the predicted-transfer-duration which is being transferred over a src-dst link with an abnormally high packet loss rate, is much more likely to be an anomaly.</li><li>Similarly , the throughput metric can be used to group anomalies being transferred over the same src-dst link.</li></ul><p>This is the state to which my efforts have taken it. As of the day of writing this post, I’m still struggling with the perfSonar data. The road has been tough but I believe that my work has successfully demonstrated that the system to detect and fix anomalies is certainly feasible.</p><h4>Failures and ShortComings:</h4><p>This project was my very first experience with professional coding. I had never worked on a project of this scale, size an complexity. Although I’m proud to have made the progress that I’ve made up-till now, there are some problems where I found myself lacking:</p><ul><li>Implementing a live trigger — the purpose of this was to simultaneously extract data being index in realtime, preprocess it, run it through the model, and detect anomalies within. Due to my lack of experience working with multitasking scripts, I was unable to implement it successfully.</li><li>The current model is sometimes unable to capture high spikes for some non-anomalous transfers with bigger transfer-sizes.</li></ul><p>Although my official work period I would love to keep working on this project and see it through to it’s completion and deployment!!</p><p>Git Repository —<a href="https://github.com/vyomshm/DeepAnomaly"> DeepAnomaly</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49766ba6a812" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/gsoc-2017-working-on-anomaly-detection-at-cern-hsf-49766ba6a812">GSoC-2017 : Working on Anomaly Detection at CERN-HSF</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>