<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Duy Trương on Medium]]></title>
        <description><![CDATA[Stories by Duy Trương on Medium]]></description>
        <link>https://medium.com/@truongb.duy?source=rss-70182490f18a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*q-IJV-ZgMzaWi56v2JhCsg.jpeg</url>
            <title>Stories by Duy Trương on Medium</title>
            <link>https://medium.com/@truongb.duy?source=rss-70182490f18a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 13:56:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@truongb.duy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Now I have Realized My Biggest Trading Enemy Is Myself]]></title>
            <link>https://medium.com/@truongb.duy/now-i-have-realized-my-biggest-trading-enemy-is-myself-084b2b4f921c?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/084b2b4f921c</guid>
            <category><![CDATA[crypto-trading]]></category>
            <category><![CDATA[algorithmic-trading]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[options-trading]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Wed, 08 Oct 2025 05:47:44 GMT</pubDate>
            <atom:updated>2025-10-08T05:49:03.458Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Me vs. My Algorithm: How I am learning to Stop Sabotaging Myself</em></h4><p>Before you begin, I believe most of my audience cares about how to set up their own trading system using <strong>Nautilus Trader</strong>, or how to develop their own cutting-edge algorithm.</p><p>If that’s what you’re looking for, you may not find it here.</p><p>However, if you’re like me — someone who’s already gone past the research phase, found a good-enough algorithm (maybe even several), and decided to put it into the market for testing or live trading — then this article <em>is</em> for you.</p><p>Because chances are, you’ve made (or will make) the same mistake I did: <strong>you interfered with your own algorithm.</strong></p><p>And if you’re making that same mistake right now, see this article as a pat on the back — a reminder that you’re not alone.</p><h3><strong>Where Am I Now?</strong></h3><p>About two months ago, I found an algorithm that could <em>theoretically</em> double my investment every year.</p><p>The catch? Its win rate was only around <strong>35%</strong>.</p><p>That means each winning trade needed to be large enough to not only cover all the previous losses but also generate profit — a strategy that depends entirely on <strong>discipline and probability</strong>, not emotion.</p><p>You might say,</p><blockquote>“Hey, that’s great, Duy. As long as you test it carefully, deploy it, and stick to the plan, you’ll be fine. Losses are part of the game, right?”</blockquote><p>That’s exactly what I thought too.</p><p>I also thought, <em>“I’m not a trader — I’m an engineer. Trading psychology won’t affect me.”</em></p><p><strong>Oh, how wrong I was.</strong></p><p>I’m writing this after losing more than 10<strong>%</strong> of my position — a loss that should have been <strong>under 2%</strong> if I had simply trusted my system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/0*br8g1ygit4JlvCic.jpg" /></figure><p>And this wasn’t the first time I did it.</p><h3>How My Mind Betrayed Me</h3><p>When I look back, I can see now that my mind quietly worked against me — not out of malice, but because it’s wired for <strong>survival</strong>, not <strong>probability</strong>.</p><h4>1. The Worker Mindset</h4><p>Before trading, I was a <strong>worker</strong>.</p><p>I didn’t come from wealth. I worked hard, saved what I could, and eventually built enough of a cushion to stop working and pursue trading full-time.</p><p>As a worker, I was used to <strong>steady income</strong> — money flowing in every month, predictable and safe.</p><p>Trading doesn’t give you that. It gives you <strong>uncertainty</strong>.</p><p>And even though I <em>thought</em> I was mentally prepared for that, I later realized that the worker mindset was buried deep in my subconscious.</p><p>When I saw a losing streak or a drawdown, that “worker voice” inside me screamed:</p><blockquote>“You’re wasting money! You should protect what you’ve earned!”</blockquote><p>It was automatic — a fight-or-flight response.</p><p>And when that voice took over, I interfered. I stopped the algo, cut losses early, or skipped the next signal — all in the name of <em>feeling safe</em>.</p><p>Ironically, that sense of safety cost me more than the system ever would have.</p><h4>2. The Winning Streak Mindset</h4><p>Then came the opposite problem.</p><p>After a few strong wins, I felt unstoppable. I looked at the numbers and thought:</p><blockquote>“I knew it! My algo is amazing — maybe I should increase my size.”</blockquote><p>It wasn’t greed exactly — it was <strong>overconfidence dressed as logic</strong>.</p><p>But the market doesn’t reward ego.</p><p>When the next losing streak came, those oversized positions hit hard.</p><p>My faith in the system cracked again, and once more, I interfered.</p><h4>3. The Losing Streak Mindset</h4><p>This one is the hardest.</p><p>When you see your algo losing again and again — even when you <em>know</em> it’s within statistical expectation — you start doubting everything: your code, your data, yourself.</p><p>I remember staring at my screen, thinking:</p><blockquote>“Maybe the market has changed. Maybe this setup doesn’t work anymore.”</blockquote><p>Each loss felt personal, and I forgot that it was just part of the process.</p><h4>4. The Worst Feeling — When You Increase Your Investment</h4><p>Things get amplified when you scale up.</p><p>Because when you trade with larger size, the numbers stay the same, but the <strong>feelings multiply</strong>.</p><p>A -3% drawdown on $100 feels like nothing.</p><p>Multiply that by 20 — it’s $2,000 — and suddenly it feels like the end of the world.</p><h3><strong>What Did I Take Away from All of This?</strong></h3><h4>1. Realization Is the First Step to Change</h4><p>Our human mind is far more complicated than any machine.</p><p>It’s the result of millions of years of evolution — and what once helped our ancestors survive in the wild may not help us survive in the market.</p><p>When we face what feels like risk or danger, our subconscious mind — shaped by past experiences and social conditioning — takes control. Logic disappears. Emotion takes the wheel.</p><p>I can’t fix that overnight. But realizing it exists is the first step.</p><p>Even if it can’t be changed immediately, awareness gives you distance — it helps you <em>see</em> the pattern instead of being trapped inside it.</p><p>And once you see it, you can begin to move away from it.</p><h4>2. Don’t Take Revenge on the Market — Take Revenge on Your Work</h4><p>This one is hard, but it’s essential.</p><p>When you feel angry or frustrated after a loss, resist the urge to interfere.</p><p>Remember — you’ve tested your algorithm far more rigorously than your emotions are being tested right now. <strong>Trust that work.</strong></p><p>I once came across a quote from <em>Ali Khan</em> that stuck with me (You can find his video <a href="https://www.youtube.com/watch?v=9_5OD6CGKuc">here</a>):</p><blockquote>“If I follow my rules and I lose — <em>then that is a win.</em></blockquote><blockquote>If I don’t follow my rules and I win — <em>then that is a loss.</em>”</blockquote><p>If you really need to take revenge, do it <em>productively</em>:</p><p>review your backtest, improve your data quality, sharpen your strategy logic.</p><p>Turn that energy toward <strong>building</strong>, not <strong>reacting</strong>.</p><h3>End</h3><p>If you’d like to connect or learn more about my work, feel free to reach out.</p><p>I also run a lead trading account on Binance. You can check it out or copy my trades <a href="https://www.binance.com/copy-trading/lead-details/4716905268889080832?ref=1146873597">here</a></p><p><a href="https://www.binance.com/copy-trading/lead-details/4716905268889080832?ref=1146873597">https://www.binance.com/copy-trading/lead-details/4716905268889080832?ref=1146873597</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=084b2b4f921c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pair trading]]></title>
            <link>https://medium.com/@truongb.duy/pair-trading-b663d8a53f94?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b663d8a53f94</guid>
            <category><![CDATA[quant]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Fri, 27 Jun 2025 08:36:40 GMT</pubDate>
            <atom:updated>2025-06-27T08:36:40.601Z</atom:updated>
            <content:encoded><![CDATA[<h4>Why guessing market moves is overrated — and how pair trading can help you win anyway</h4><h3>Introduction to Pair Trading</h3><p>There are already plenty of articles explaining pair trading, and most of them do a decent job. But I figured — why not add one more? Maybe I can make it even clearer.</p><p>When I was new to the markets, I believed a strategy had to predict price movements — either short-term or long-term — to be profitable. Like most traders, I turned to fundamental and technical analysis. Some people make it work.</p><p>But there are some problems with that approach:</p><ul><li>Predicting where the market’s headed is <strong>hard</strong>.</li><li>And what if the market just moves sideways?</li></ul><p>Later on, I came across something different — <strong>market-neutral strategies</strong>.</p><h3>What’s a Market-Neutral Strategy?</h3><p>A market-neutral strategy is basically a way to profit without needing the market to go up or down. You’re not betting on direction — you’re betting on <strong>relative movement</strong>.</p><p><strong>Pair trading</strong> is one of those strategies.</p><p>Here’s the idea: Imagine two assets — let’s say Stock A and Stock B — that usually move in sync. (The technical term is <em>cointegrated</em>, but we’ll talk more about that later.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*WE6yrB6w1YizkKx6Jnb3Zw.png" /><figcaption>Stock A’s price shoots up, but Stock B stays where it is</figcaption></figure><p>Now, suppose Stock A’s price shoots up, but Stock B stays where it is. In most cases — <strong>not all</strong>, and I’ll get to that — the market does one of two things:</p><ul><li><strong>Reversion to the mean</strong>: Stock A drops back in line.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*h6BFEBXobtgtGVYmgvLwGA.png" /><figcaption>Stock A drops back</figcaption></figure><ul><li><strong>Catch-up move</strong>: Stock B rises to catch up.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*VJ7XUsrjv9gLFNdsxas4Xw.png" /><figcaption>Stock B rises</figcaption></figure><p>Rather than guessing which one will happen, pair trading lets you take a smarter path: <strong>short the overpriced one and long the underpriced one</strong> — at the same time.</p><p>That way, it doesn’t matter whether A drops or B rises — you profit either way.</p><h3>Example: How Pair Trading Works</h3><p>Let’s walk through a basic example.</p><ul><li>Stock A = $7</li><li>Stock B = $5</li></ul><p>Then Stock A jumps to $8 while Stock B stays at $5.</p><p>Here’s what you do:</p><ul><li><strong>Sell Stock A</strong> (overpriced) at $8</li><li><strong>Buy Stock B</strong> (undervalued) at $5</li></ul><p>Now, two things could happen:</p><h4>Scenario 1: Reversion to the Mean</h4><ul><li>Stock A falls back to $7</li><li>Stock B stays at $5</li><li>You buy back Stock A at $7 → <strong>Profit: $1/share</strong></li><li>You sell Stock B at $5 → <strong>Break-even</strong></li></ul><h4>Scenario 2: Catch-Up Movement</h4><ul><li>Stock A stays at $8</li><li>Stock B rises to $5.70</li><li>You buy back Stock A at $8 → <strong>Break-even</strong></li><li>You sell Stock B at $5.70 → <strong>Profit: $0.70/share</strong></li></ul><p>You’re not trying to guess direction — you’re playing the <strong>relationship</strong> between the two assets.</p><h4>What If Prices Converge the Other Way?</h4><p>No problem. Just flip the trade.</p><p>Let’s say Stock B rises while Stock A stays flat.</p><ul><li><strong>Buy Stock A</strong> (undervalued)</li><li><strong>Sell Stock B</strong> (overpriced)</li></ul><p>When they come back in line:</p><ul><li>Sell Stock A</li><li>Buy back Stock B</li></ul><p>Same result. You profit from the spread closing, no matter which side moves.</p><h4>So, Why Do Prices Diverge in the First Place?</h4><p>Short answer: <strong>market inefficiency</strong>.</p><p>Some reasons this might happen:</p><ul><li>Insider info (someone knows something you don’t)</li><li>A big fund moves in or out of one stock</li><li>Temporary supply/demand imbalance</li><li>Earnings reports or news events</li></ul><p>Longer-term, prices tend to correct themselves. The market isn’t always efficient — but it usually tries to be.</p><h3>Initial Excitement… Followed by Reality</h3><p>When I first learned about this, I could practically hear the money rolling in. It felt like a cheat code: low risk, no need to guess direction, and solid logic behind it.</p><p>But after the buzz wore off, I started asking harder questions:</p><ul><li>How often do pairs actually revert?</li><li>What if they never realign?</li><li>How do I even find good pairs in the first place?</li></ul><p>Let’s break those down.</p><h4>1. How Often Do Pairs Revert?</h4><p>It depends.</p><ul><li>What pair are you trading?</li><li>What’s your time frame?</li><li>What’s the market doing right now?</li></ul><p>There’s no universal answer. You’ll need to backtest your pairs and get a sense of how often things converge. Some pairs are rock solid. Others just drift forever and wreck your trade. You can update your calculation model based on the latest data to maintain its performance</p><h4>2. How Do I Find Good Pairs?</h4><p>Look for assets that are <strong>fundamentally related</strong>:</p><ul><li>Stocks in the same industry (e.g., Coke and Pepsi)</li><li>ETFs tracking similar assets</li><li>Currency pairs with the same base</li></ul><p>Once you have a candidate, you’ll want to run some <strong>statistical tests</strong> — like checking for cointegration — to confirm they usually move together. Without that, it’s just a guess.</p><h4>3. What If I Can’t Short Sell?</h4><p>Good question.</p><p>In markets that don’t allow shorting, you can try a <strong>long-only version</strong>:</p><ul><li>Hold both assets</li><li>Adjust the weighting so you’re heavier on the undervalued one</li></ul><p>Trade-offs:</p><ul><li>You tie up more capital</li><li>If both prices drop, your account value drops (even if the strategy itself works)</li><li>On the upside, if prices go up, you profit from the strategy <strong>and</strong> the market move</li></ul><p>To manage the risk, try holding multiple pairs across different sectors or asset classes.</p><h3>Let’s Get Real About the Risks</h3><p>Time for the unsexy part — <strong>what can go wrong</strong>?</p><h4>1. Trading Costs Will Eat Your Lunch (Also your Breakfast and Dinner )</h4><p>Remember when I said you profit on one trade and break even on the other? Well, in reality, those tiny trading costs add up. Every time you enter and exit, you’re paying a little. Do this too often and poof — your profits disappear. That’s why you gotta account for these costs in your backtesting.</p><h4>2. Low Liquidity = Big Problems</h4><p>If the assets you’re trading don’t have enough volume, you might not get in or out when you need to. Worse, your trades might move the price. That kills the whole point of hedging. So always check liquidity before jumping in.</p><h4>3. Corporate Events Can Wreck Your Math</h4><p>Dividends, splits, mergers — these can mess up your spread and throw your strategy off. Especially if you’re waiting on a dividend and your capital is stuck. Thus, keep an eye on the calendar and maybe sit out trades around key events.</p><h3>4. Too Many Bots in the Pool</h3><p>This one’s more of a hunch than a proven fact, but in super liquid markets like U.S. equities or crypto, <strong>algos may already be all over pair trading. </strong>And if everyone’s running the same strategy, it gets harder to squeeze out profits.</p><p>One possible fix? Trade baskets instead of just pairs. Adds complexity, but also more opportunity.</p><h3>Final Thoughts</h3><p>I think pair trading flips the usual script — you’re not betting on what the market will do, you’re betting on what <strong>two assets will do relative to each other</strong>. It’s clean, logical, and doesn’t require a crystal ball.</p><p>However, there’s still plenty that can go wrong if you don’t backtest, manage risk, and account for the boring stuff like slippage and liquidity.</p><p>That said, if you’re looking for a strategy that works in all kinds of markets and makes sense intuitively, pair trading is worth exploring.</p><p><strong>Disclaimer</strong>:<br><em>I’m a non-native English speaker and have written this article by myself. That said, I’ve found that LLMs like ChatGPT help make my writing smoother and more natural. I often use them as a final touch to make my work sound more like a native English speaker.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b663d8a53f94" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to use NautilusTrader with Interactive Broker]]></title>
            <link>https://medium.com/@truongb.duy/how-to-use-nautilustrader-with-interactive-broker-a6e3e1be3d31?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6e3e1be3d31</guid>
            <category><![CDATA[quant]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[nautilus-trader]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Sat, 31 May 2025 08:58:10 GMT</pubDate>
            <atom:updated>2025-05-31T08:58:10.260Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*6HtQrNuWs78OauNncn5gwQ.png" /></figure><p><strong>TL;DR:</strong> In under 10 minutes, you can wire up NautilusTrader to trade live through Interactive Brokers — even if you’ve never touched IB before. No lengthy setup, just three quick steps: grab the latest API, enable socket clients, and run a handful of lines of Python.</p><h3>Introduction</h3><ul><li>NautilusTrader (NT) is an all-in-one Python platform for both backtesting and live trading.</li><li>Its power comes at the cost of minimal documentation and a steep learning curve.</li><li>In this article, I’ll walk you through the exact steps I took to connect NT to Interactive Brokers — so you can skip the headaches and dive straight into trading.</li></ul><p>If you’re completely new to NT don’t worry — it’s not as hard as it seems. Just feel free to refer to my earlier blog posts. Those posts document my own journey with NT, so they’ll help you build familiarity quickly. Before long, you’ll see that NT isn’t so intimidating after all.</p><h3>Getting Started</h3><p>Okay, let’s get down to the meat of this article.</p><p>First, let me reassure you: connecting and trading with Interactive Brokers isn’t very complicated. NautilusTrader’s GitHub repository already includes a concise example. Take a look at the file below:</p><p><a href="https://github.com/nautechsystems/nautilus_trader/blob/develop/examples/live/interactive_brokers/connect_with_tws.py">nautilus_trader/examples/live/interactive_brokers/connect_with_tws.py at develop · nautechsystems/nautilus_trader</a></p><p>That example is all you need if you’ve used IB before.</p><p>Sounds like the shortest article. But if you’ve never touched IB, here are a few more tips to help you onboard more quickly.</p><h4>1. <strong>⚠️ Warning, d</strong>on’t start by running pip install ibapi</h4><p>If you simply run the below code and hope for the best, like me, just don’t:</p><pre>pip install ibapi</pre><p>This will get you version 9.81.1, which was released in late 2020 and hasn’t been supported since. Instead, follow IB’s instructions for downloading the latest API bundle:</p><p><a href="https://www.interactivebrokers.com/campus/ibkr-api-page/twsapi-doc/#unix-install">TWS API Documentation | IBKR Campus US</a></p><ol><li>Go to the TWS API documentation on the IB website:</li></ol><p><a href="https://interactivebrokers.github.io/">https://interactivebrokers.github.io</a></p><p>2. Download the “API Latest” package (as of May 28, 2025, that’s version 10.37).</p><p>3. Extract the pythonclient/ibapi folder.</p><p>4. Either install it locally with:</p><pre>cd path/to/ibapi<br>python setup.py bdist_wheel<br>pip install dist/ibapi-10.37-*.whl</pre><p>Or simply copy the ibapi directory into your project so Python sees it directly.</p><p>That way, you’ll have the up-to-date client code that matches IB’s current Trader Workstation (TWS) or IB Gateway.</p><h4>2. Run TWS or IB Gateway first</h4><p>IB provides two ways to connect:</p><ul><li><strong>TWS (Trader Workstation)</strong>: A full-featured desktop trading platform with a graphical UI.</li><li><strong>IB Gateway</strong>: A lighter, resource-efficient application intended for API-only connectivity.</li></ul><p>You can choose either. The one thing to keep in mind is that you must enable API access and use the correct host/port in your Python code.</p><ol><li><strong>Enable API Access</strong></li><li>In both TWS and IB Gateway, go to:</li></ol><pre>Edit → Global Configuration → API → Settings</pre><p>Then check <strong>“Enable ActiveX &amp; Socket Clients.”</strong></p><p><strong>2. Host and Port</strong></p><p><strong>— Host</strong>:</p><pre>127.0.0.1</pre><p>(<em>This assumes you run your Python script on the same machine as TWS/IB Gateway. If you connect from another machine, substitute its IP address and ensure your firewall allows traffic on the chosen port</em>.)</p><p><strong>— Ports</strong>:</p><p><strong>IB Gateway</strong></p><ul><li>Live (production) → 4001</li><li>Paper (demo) → 4002</li></ul><p><strong>TWS</strong></p><ul><li>Live (production) → 7496</li><li>Paper (demo) → 7497</li></ul><p>You can change these defaults in the same API settings screen — just make sure your Python code’s port parameter matches exactly.</p><p><strong>3. Account ID</strong></p><p>Note that account_id in your code is not your login username. Instead, log into TWS, and you’ll see your IB Account ID at the top right of the TWS window. Use that ID in your API calls.</p><h3>What’s Next?</h3><p>NautilusTrader’s native connection to IB lets you get up and running with minimal setup. However, if you ever want to integrate NT with a different broker — perhaps to keep using your own broker’s API — you can build a “glue layer” between NT and that broker’s interface. I haven’t done this myself yet, but if you explore NT’s source code, you’ll find plenty of examples to guide you.</p><p>Happy coding and trading! And don’t forget to let me know what you want me to cover next.</p><p><strong>Disclaimer</strong>:<br><em>I’m a non-native English speaker and have written this article by myself. That said, I’ve found that LLMs like ChatGPT help make my writing smoother and more natural. I often use them as a final touch to make my work sound more like a native English speaker.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6e3e1be3d31" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pair Trading with Databento and NautilusTrader]]></title>
            <link>https://medium.com/@truongb.duy/how-to-fetch-databento-data-in-nautilus-trader-a8287022e2fa?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8287022e2fa</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[quant]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Sun, 18 May 2025 07:46:01 GMT</pubDate>
            <atom:updated>2025-05-31T09:01:45.710Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*zw2JkvwUUkOFrucW0VI16Q.png" /></figure><h3>1. Introduction</h3><p>As I mentioned in the title, this article continues my story about using the <strong>NautilusTrader (NT)</strong> package for backtesting and — hopefully — running a live strategy.</p><p>Before diving into the main content, I want to share what brought me here. If you’re short on time, feel free to skip straight to <strong>Section 2</strong>.</p><h4>1.1. Where It Began</h4><p>More than two weeks ago, <strong>Slobow</strong> commented on one of my articles and asked me to write a tutorial about using <strong>NT with Databento</strong>. This fit perfectly with what I wanted to do, so I decided to create my own project for it.</p><p>I didn’t realize I’d set the bar a bit high. Initially, I wanted my article to have three sections:</p><ol><li><strong>Talk about pair trading in theory</strong> — even though you can easily find articles about it, I wanted to write my own because it’s special to me.</li><li><strong>Show how to use NT with Databento</strong> to build a strategy — because, let’s face it, Yahoo Finance isn’t enough if you want to level up.</li><li><strong>Run my strategy on Databento’s live data</strong> — that’s what drew me to this package in the first place.</li></ol><p>But time is tight, and NT has a steep learning curve (as you know if you’ve read my last articles). So I decided to break it down. <strong>This article covers the second part</strong> — next week, I’ll tackle parts 1 and 3 (if I can make it).</p><h3>2. Article Goal</h3><p>For this article, I’ll focus only on the second part.</p><h4>Some Basic Info</h4><ul><li>Pair trading is a <strong>mean-reversion strategy</strong>. It assumes two assets are <strong>cointegrated</strong> (not just correlated), meaning if they drift apart, they’ll eventually snap back.</li><li>I’m using <strong>Pepsi and Coca-Cola stocks</strong> as my example (a classic pair).</li></ul><h4>New Challenges</h4><ol><li>Getting data from <strong>Databento</strong> instead of Yahoo Finance.</li><li>Handling <strong>two tickers at once</strong> in my strategy. (you can do more once you know the basic)</li></ol><p>The rest should be straightforward.</p><blockquote><strong><em>Note:</em></strong><em> If you’re new to NT, check my old article on creating </em><a href="https://medium.com/@truongb.duy/building-a-short-uvxy-strategy-in-python-using-nautilus-trader-296400d54147"><em>custom data</em></a><em> with example. I won’t cover that here.</em></blockquote><h3>3. Step-by-Step Guide</h3><h4>3.1. Set Up the Strategy</h4><p>It’s obvious, but I realized how important it is to <strong>tell NT what data and instruments to use</strong> in the config.</p><p>For this strategy, I made sure to:</p><ul><li>Define instruments and data types.</li><li>Allow subscriptions to multiple data sources (useful for complex strategies).</li></ul><p><em>Note: I realized that with a complex strategy that requires you to subscribe to multiple data and events, you can declare and subscribe to them in the on_start method. I believe that is one of the advantages of NT.</em></p><pre>class PairTradingConfig(StrategyConfig):<br>    instrument_ids: List[InstrumentId]<br>    bar_types: List[BarType]<br>    enter_threshold: float = 1.2<br>    exit_threshold: float = 0.0<br>    beta: float = 1.1<br>    window: int = 10<br>    trading_size: int = 1</pre><blockquote><strong><em>Note:</em></strong><em> NT can convert tick data to bars, but I’ll skip that here. You can research it yourself — or maybe I’ll write about it later.</em></blockquote><h4>3.2. Custom Data Setup</h4><p>I created a custom data class to track the spread:</p><pre>class PairTradingData(Data):<br>    def __init__(<br>        self,<br>        spread: float,<br>        mean: float,<br>        std: float,<br>        zscore: float,<br>        ts_event: int,<br>        ts_init: int,<br>    ):<br>        self.spread = spread<br>        self.mean = mean<br>        self.std = std<br>        self.zscore = zscore<br>        self._ts_event = ts_event<br>        self._ts_init = ts_init</pre><h4>3.3. Subscribing to Data</h4><p>In on_start, I subscribed to the bars and my custom data:</p><pre>def on_start(self) -&gt; None:<br>      &quot;&quot;&quot;Initialize strategy subscriptions.&quot;&quot;&quot;<br>      for bar_type in self.config.bar_types:<br>          self.subscribe_bars(bar_type)<br>      self.subscribe_data(DataType(PairTradingData))</pre><h4>3.4. Handling Bars</h4><p>You can see that I only trigger calculations when I receive the same number of bars for both instruments. In this approach, I track the bar count for each instrument and calculate the spread when the counts match.</p><p>However, upon reflection, this method might not be ideal when working with tick data or asymmetrical data streams.</p><pre>def on_bar(self, bar) -&gt; None:<br>    &quot;&quot;&quot;Process incoming bar data and calculate pair trading metrics.<br>    <br>    Parameters<br>    ----------<br>    bar : Bar<br>        Price bar data for either instrument.<br>    &quot;&quot;&quot;<br>    # Update bar counts and get prices<br>    if bar.bar_type == self.config.bar_types[0]:<br>        self.log.info(f&quot;First stock bar: {bar}&quot;, LogColor.RED)<br>        self.first_stock_bar_count += 1<br>    elif bar.bar_type == self.config.bar_types[1]:<br>        self.log.info(f&quot;Second stock bar: {bar}&quot;, LogColor.RED)<br>        self.second_stock_bar_count += 1</pre><pre>    # Calculate spread metrics when we have enough data<br>    if self.first_stock_bar_count == self.second_stock_bar_count:<br>        self._calculate_spread_metrics()</pre><p>Ensure your data is published to the message bus — this allows your strategy to retrieve it for calculating entry and exit signals.</p><pre>self.publish_data(DataType(PairTradingData), data)</pre><h4>3.5. Getting Databento Data</h4><p>When you sign up, Databento grants you $125 in free historical‐data credit — plenty to cover this exercise.</p><p>For a fast start, you can follow the exercise written by Chris (the founder of NT) <a href="https://github.com/nautechsystems/nautilus_trader/blob/develop/examples/live/databento/databento_historical_data.py">here</a>. He did an excellent job creating helpful functions that can quickly get you up to speed — provided you know what you’re doing. (This is a big “if,” though; I didn’t know what I was doing until I took a very close look at his code.) Ultimately, I switched to a more comprehensive approach:</p><ol><li><strong>Fetching Data</strong> — I used the Databento package directly to retrieve and store the data locally.</li><li><strong>Loading Data</strong> — Then, I imported it into NT’s native format using the DatabentoDataLoader.</li></ol><pre># Initialize Databento client<br>client = db.Historical(API_KEY)<br><br># Prepare data path<br>path = data_path / &quot;hist.trades.dbn.zst&quot;<br># Download data if not exists<br>if not path.exists():<br>    symbols = [str(id.symbol) for id in instrument_ids]<br>    client.timeseries.get_range(<br>        dataset=&quot;XNAS.ITCH&quot;,<br>        symbols=symbols,<br>        schema=&quot;ohlcv-1d&quot;,<br>        start=start_time,<br>        end=end_time,<br>        path=path,<br>    )<br># Load data into engine<br>from nautilus_trader.adapters.databento.loaders import DatabentoDataLoader<br>loader = DatabentoDataLoader()<br>data = []<br>for instrument_id in instrument_ids:<br>    data.extend(loader.from_dbn_file(<br>        path=path,<br>        instrument_id=instrument_id,<br>        as_legacy_cython=True,<br>    ))<br>engine.add_data(data)</pre><p>The rest should be straightforward if you have prior experience with NT, especially after reviewing my earlier post. For those new to NT, I highly recommend taking a step-by-step approach to fully grasp its concepts.</p><p>As always, you can find my code <a href="https://github.com/truongbduy/articles/tree/main/Nautilus%20Trader%20Exercercise%203">here</a></p><h3>4. What’s Next?</h3><p>In my next article, I’ll try running this strategy on <strong>live Databento data</strong> — if all goes well!</p><p>But I’d love to hear from you:</p><ul><li><strong>What NT topics should I cover next?</strong></li><li>Any specific struggles you’ve faced with NautilusTrader?</li></ul><p>Let me know in the comments!</p><p><strong>Disclaimer</strong>:<br><em>I’m a non-native English speaker and have written this article by myself. That said, I’ve found that LLMs like ChatGPT help make my writing smoother and more natural. I often use them as a final touch to make my work sound more like a native English speaker.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8287022e2fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Short-UVXY Strategy in Python Using Nautilus Trader]]></title>
            <link>https://medium.com/@truongb.duy/building-a-short-uvxy-strategy-in-python-using-nautilus-trader-296400d54147?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/296400d54147</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[quant]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Sun, 04 May 2025 03:24:21 GMT</pubDate>
            <atom:updated>2025-05-21T10:39:08.473Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Building a Short-UVXY Strategy in Python Using NautilusTrader</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*8BIBgSWURrLny_RP8pu9Wg.png" /></figure><h3>1. Introduction</h3><p>In my last <a href="https://medium.com/@truongb.duy/nautilus-trader-beginner-tutorial-how-i-ran-my-first-backtest-a08da1999963">article</a>, I walked you through a “Hello World” project with NautilusTrader to get you comfortable with its basics. In this one, I’d like to tackle a more realistic trading strategy. You’ll see how NT’s framework makes it easy to define signals, manage risk, and run a full backtest — so you can start building your own complex strategies.</p><h3>2. Project Goal</h3><p>In this example, I’ll implement a short-UVXY strategy inspired by Robot Wealth’s Excel model¹. The idea is simple: short UVXY when it’s expensive, and cover when it’s cheap.</p><p>UVXY is a fascinating — and fairly complex — vehicle, but a full deep dive is out of scope here. To stay focused, just remember:</p><ul><li><strong>Long-term decay:</strong> UVXY tends to lose value over time.</li><li><strong>Volatility spikes:</strong> It can surge dramatically when markets brace for turbulence.</li></ul><p>Our approach has two main pillars:</p><h4>Signal generation</h4><ul><li>Compute the 20-day z-score of UVXY’s daily close:</li></ul><p>z = (P_today — μ_20) / σ_20</p><ul><li><strong>Enter short</strong> when z &gt; 2.</li><li><strong>Cover</strong> when z &lt; 0.</li></ul><h4>Rebalancing &amp; Risk Control</h4><ul><li>Every N days (or upon a drawdown/trade-age trigger), adjust position size to lock in gains or cut losses if volatility keeps climbing.</li><li><em>(This part isn’t in the code example below, but it’s straightforward to add — feel free to experiment.)</em></li></ul><p>In the code that follows, I’ll build this short-UVXY strategy using NautilusTrader.</p><h3>3. Step-by-Step Guide</h3><p>Below is a concise walkthrough of the key pieces. (You can find the full code in the repo link <a href="https://github.com/truongbduy/articles/blob/main/Nautilus%20Trader%20Exercise%202.py">here</a>.)</p><h4>3.1. Custom data class</h4><p>Here we define a simple container to carry our 20-bar z-score alongside NautilusTrader’s timestamps:</p><p><strong>Fields:</strong></p><ul><li>ts_event and ts_init (required by NT’s custom-data API)</li><li>z_score (the z-score over the last 20 daily closes)</li></ul><p><strong>Why it matters:</strong> Publishing this as a separate data type decouples your signal calculation from your entry/exit logic.</p><pre>class ZFromNBars(Data):<br>    &quot;&quot;&quot;<br>    Holds mean, std, and z-score of the last N bars.<br>    &quot;&quot;&quot;<br>    def __init__(self, mean, std, z, ts_event, ts_init):<br>        self.mean = mean<br>        self.std = std<br>        self.z = z<br>        self._ts_event = ts_event<br>        self._ts_init = ts_init</pre><h4>3.2. Strategy Config</h4><p>All of your “tunable” parameters live here:</p><ul><li>n_bars (lookback length for the rolling mean/std)</li><li>enter_threshold (z-score above which we short)</li><li>Optional: exit_threshold (z-score below which we cover, I hard-code it in my code base, which is unadvised)</li><li>trade_size, etc.</li></ul><pre>class UVXYConfig(StrategyConfig):<br>    instrument_id: InstrumentId<br>    bar_type_1day: BarType<br>    n_bars: int = 20<br>    threshold: float = 2.0<br>    trading_size: int = 100</pre><h4>3.3. Strategy setup</h4><p>In your UVXYStrategy class:</p><ol><li><strong>Subscriptions</strong></li></ol><ul><li>subscribe_bars(bar_type) to get 1-day closes.</li><li>subscribe_data(DataType(ZFromNBars)) to receive your custom z-score events.</li></ul><p><strong>2. on_bar</strong></p><ul><li>Pull the last n_bars from the cache.</li><li>Compute mean, std, and z-score.</li><li>Publish the ZFromNBars data.</li></ul><p><strong>3. on_data</strong></p><ul><li>When you receive a ZFromNBars data piece, grab its z_score and call your entry/exit checks.</li></ul><pre>def on_start(self):<br>    self.subscribe_bars(self.config.bar_type_1day)<br>    self.subscribe_data(DataType(ZFromNBars))<br><br>def on_bar(self, bar: Bar):<br>    self.bar_counts += 1<br>    # Compute statistics on last N bars<br>    bars = self.cache.bars(self.config.bar_type_1day)[: self.config.n_bars]<br>    closes = np.array([float(b.close) for b in bars])<br>    mean = closes.mean()<br>    std = closes.std()<br>    z = (float(bar.close) - mean) / std if std else 0.0<br><br>    # Publish custom data<br>    data = ZFromNBars(mean, std, z, bar.ts_event, bar.ts_init)<br>    self.publish_data(DataType(ZFromNBars), data)<br><br>def on_data(self, data):<br>    if isinstance(data, ZFromNBars) and self.bar_counts &gt;= self.config.bar_counts:<br>        self.z = data.z<br>        self._try_enter()<br>        self._try_exit()<br><br>def _try_enter(self):<br>    if self.z &gt;= self.config.threshold and (not self.position or self.position.side != PositionSide.SHORT):<br>        order = self.order_factory.market(<br>            instrument_id=self.config.instrument_id,<br>            order_side=OrderSide.SELL,<br>            quantity=Quantity.from_int(self.config.trading_size)<br>        )<br>        self.submit_order(order)<br><br>def _try_exit(self):<br>    if self.z &lt; 0 and self.position and self.position.side == PositionSide.SHORT:<br>        self.close_position(self.position)</pre><h4>3.4. Configure your engine</h4><p>To launch a backtest you need four pieces:</p><ol><li><strong>Strategy config</strong> (your UVXYConfig with thresholds, lookback, etc.)</li><li><strong>Instrument</strong> (the InstrumentId + Equity object for UVXY)</li><li><strong>Venue</strong> (e.g. Venue(“NASDAQ”), account type, starting balances)</li><li><strong>Data feed</strong> (processed bars from BarDataWrangler, fed by your Yahoo Finance download)</li></ol><p>Once those are wired into BacktestEngineConfig, you can call engine.run() and inspect your P&amp;L.</p><pre>instrument = InstrumentId(symbol=Symbol(&quot;UVXY&quot;), venue=Venue(&quot;NASDAQ&quot;))<br>UVXY_ETF_1DAY_BARTYPE = BarType.from_str(f&quot;{instrument}-1-DAY-LAST-EXTERNAL&quot;)<br><br># 4.2 Config &amp; Engine<br>strat_cfg = UVXYConfig(<br>    instrument_id=instrument,<br>    bar_type_1day=UVXY_ETF_1DAY_BARTYPE,<br>    n_bars=20,<br>    threshold=2.0,<br>    trading_size=100<br>)<br>engine_cfg = BacktestEngineConfig(<br>    trader_id=TraderId(&quot;BACKTEST-UVXY1DAY-001&quot;),<br>    strategies=[<br>        ImportableStrategyConfig(<br>            strategy_path=&quot;__main__:UVXYStrategy&quot;,<br>            config_path=&quot;__main__:UVXYConfig&quot;,<br>            config = {<br>                &quot;instrument_id&quot;: instrument,<br>                &quot;bar_type_1day&quot;: UVXY_ETF_1DAY_BARTYPE,<br>                &quot;trading_side&quot;: 100_000,<br>                &quot;threshold&quot;: 2,<br>                &quot;n_bars&quot;: 20<br>            }<br>        )<br>    ],<br>    logging=LoggingConfig(log_level=&quot;DEBUG&quot;)<br>)<br>engine = BacktestEngine(config=engine_cfg)<br><br># 4.3 Venue &amp; Account<br>engine.add_venue(<br>    venue=Venue(&quot;NASDAQ&quot;),<br>    oms_type=OmsType.NETTING,<br>    account_type=AccountType.MARGIN,<br>    starting_balances=[Money(1_000_000, USD)],<br>    base_currency=USD,<br>    default_leverage=Decimal(1)<br>)<br><br># 4.4 Instrument<br>from nautilus_trader.model.instruments import Equity<br>eq = Equity(<br>    instrument_id=instrument,<br>    raw_symbol=Symbol(&quot;UVXY&quot;),<br>    currency=USD,<br>    price_precision=2,<br>    price_increment=Price.from_str(&quot;0.01&quot;),<br>    lot_size=Quantity.from_int(1),<br>    ts_event=0,<br>    ts_init=0<br>)<br>engine.add_instrument(eq)<br><br># 4.5 Data (example using yfinance)<br>import yfinance as yf<br>df = yf.download(&quot;UVXY&quot;, start=&quot;2020-01-01&quot;, end=&quot;2025-01-01&quot;)<br>df = df.rename(columns={&quot;Open&quot;:&quot;open&quot;, &quot;High&quot;:&quot;high&quot;, &quot;Low&quot;:&quot;low&quot;, &quot;Close&quot;:&quot;close&quot;, &quot;Volume&quot;:&quot;volume&quot;})<br>wrangler = BarDataWrangler(UVXY_ETF_1DAY_BARTYPE, eq)<br>bars = wrangler.process(df)<br>engine.add_data(bars)<br><br># 4.6 Run<br>engine.run()</pre><h3>4. Limitations</h3><p>There are a few areas I’ve intentionally left out (or haven’t solved) in this article:</p><ol><li><strong>Rebalancing:</strong> This is the essence of the strategy if you want to lock in profits and avoid margin calls. Due to time constraints, I’ve left it as an exercise for you.</li><li><strong>Price data &amp; fee calculation:</strong> Yahoo Finance supplies adjusted prices, so you can’t use them to calculate actual position sizes or simulate fees accurately. I haven’t found a clean workaround — if you have experience sourcing raw futures prices or estimating trading fees, please let me know.</li><li><strong>Short-position financing costs:</strong> Borrowing and margin fees can materially affect a short strategy’s P&amp;L but aren’t modeled here. Handling financing costs properly is left as an exercise for the reader.</li></ol><p><strong>Disclaimer</strong>:<br><em>I’m a non-native English speaker and have written this article by myself. That said, I’ve found that LLMs like ChatGPT help make my writing smoother and more natural. I often use them as a final touch to make my work sound more like a native English speaker.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=296400d54147" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NautilusTrader Beginner Tutorial: How I Ran My First Backtest]]></title>
            <link>https://medium.com/@truongb.duy/nautilus-trader-beginner-tutorial-how-i-ran-my-first-backtest-a08da1999963?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a08da1999963</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[nautilus-trader]]></category>
            <category><![CDATA[quant]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Mon, 28 Apr 2025 02:38:39 GMT</pubDate>
            <atom:updated>2025-05-21T10:37:27.935Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*mvtEUDrUK-d76KsgrZtsHw.jpeg" /></figure><h3>1. Introduction</h3><h3>Why NautilusTrader?</h3><p>I’m working on a project that’s almost open-ended:</p><p>It needs to <strong>ingest external signals</strong> (like Tweets), <strong>generate our own signals</strong>, and <strong>trade across multiple exchanges</strong> (stocks, options, crypto).</p><p>The project demands <strong>speed</strong> — and getting everything from data collection, formatting, backtesting, and live trading setup can be extremely time-consuming.</p><p>That’s when I was introduced to <strong>NautilusTrader (NT)</strong>.</p><p>At first glance, NT feels like the <strong>Holy Grail</strong> for my needs:</p><ul><li>It can easily <strong>convert external data</strong> into its internal format.</li><li>It lets you <strong>backtest and switch to live trading</strong> with minimal changes.</li><li>It is designed for <strong>high-performance</strong>, even capable of <strong>High-Frequency Trading</strong> (not our goal, but still an impressive feature).</li></ul><p><strong>Important Note:</strong></p><p>NT is <strong>not really for casual day traders</strong> with simple strategies.</p><p>Instead, it’s <strong>powerful</strong> for <strong>small to mid-sized teams</strong> who want to find and deploy strategies fast.</p><p><strong>Biggest Challenges?</strong></p><ul><li><strong>Steep learning curve</strong>.</li><li><strong>Limited tutorials and documentation</strong> available online. (If you’re hoping GPT can magically write your first NT code… let’s just say it gets frustrating fast.)</li></ul><p>Because of that, I decided to <strong>document my own journey</strong> learning NautilusTrader —</p><p>hopefully, it’ll make it easier for you if you decide to explore NT too.</p><p><strong>Official Resources:</strong></p><ul><li><a href="https://nautilustrader.io/docs/latest/getting_started/">Getting Started Documentation</a></li><li><a href="https://github.com/nautechsystems/nautilus_trader/tree/develop/examples">GitHub Examples</a></li></ul><h3>2. Goal</h3><p>In this project, my goal is simple:</p><ul><li><strong>Get NVDA (NVIDIA) stock data</strong> from Yahoo Finance.</li><li><strong>Convert the data into NautilusTrader’s format</strong>.</li><li><strong>Use MACD</strong> to generate entry/exit signals.</li><li><strong>Run a backtest</strong> using those signals.</li></ul><h3>3. Step-by-Step Guide</h3><h3>3.1 Install the Package</h3><p>The installation process is pretty straightforward.</p><p><strong>Important:</strong></p><p>NautilusTrader officially supports <strong>Python 3.11, 3.12, and 3.13</strong>.</p><p>Make sure you’re using the right Python version.</p><p>Then, just run:</p><pre>pip install -U nautilus_trader</pre><p>Full installation guide here:</p><p><a href="https://nautilustrader.io/docs/latest/getting_started/installation">Installation Guide</a></p><h3>3.2 Core Components Overview</h3><p>NautilusTrader has many core components. For this beginner project, I’ll only focus on the essentials:</p><p><strong>Strategy</strong>:</p><ul><li>A class where your trading logic lives — handles data events, order events, and position state changes.</li><li>(When to enter? Exit? What to do if an order succeeds or fails?)</li></ul><p><strong>BacktestEngine</strong>: The engine to <strong>run backtests</strong> or <strong>live trading</strong>. It needs:</p><ul><li><strong>Venue</strong>: Venue <strong>simulates</strong> an exchange during backtests, and represents a real exchange for live trading.</li><li><strong>Data</strong>: The historical or live data feed that the strategy consumes.</li><li><strong>Strategy</strong>: Your trading logic.</li></ul><h3>3.3 Build a Strategy</h3><p>Let’s dive into action!</p><p>First, <strong>create a Strategy Config</strong>.</p><pre>import datetime as dt<br><br>from nautilus_trader.config import StrategyConfig<br><br>from nautilus_trader.indicators.macd import MovingAverageConvergenceDivergence<br><br>from nautilus_trader.trading.strategy import Strategy<br><br>from nautilus_trader.model import BarType<br>from nautilus_trader.model import Bar<br>from nautilus_trader.model import Position<br>from nautilus_trader.model import InstrumentId<br>from nautilus_trader.model.enums import PriceType<br>from nautilus_trader.model.enums import OrderSide<br>from nautilus_trader.model.enums import PositionSide<br><br>class MACDStrategyConfig(StrategyConfig):<br><br>    instrument_id: InstrumentId<br>    bar_type_1day: BarType<br>    fast_period: int = 4<br>    slow_period: int = 8<br>    trading_side = 10_000<br>    enter_threshold: float = 0.00010</pre><p>Since I’m building a <strong>MACD strategy</strong>, and since <strong>MACD is built into NT</strong>, it makes life easier.</p><pre># Create a subclass of Strategy class<br>class MACDStrategy(Strategy):<br>    def __init__(self, config: MACDStrategyConfig):<br>        super().__init__()<br>        self.bar_type_1day = config.bar_type_1day<br><br>        self.macd = MovingAverageConvergenceDivergence(<br>            fast_period = config.fast_period,<br>            slow_period = config.slow_period,<br>            price_type = PriceType.MID<br>        )<br>        self.instrument_id = config.instrument_id<br>        self.trade_size = Quantity.from_int(config.trading_side)<br>        <br>        self.position: Position | None = None<br>        self.enter_threshold = config.enter_threshold<br>        <br>        self.count_processed_bars = 0<br>        <br>        self.start_time = None<br>        self.end_time = None<br><br>    def on_start(self):<br>        self.start_time = dt.datetime.now()<br><br>        self.subscribe_bars(self.bar_type_1day)<br><br>        self.log.info(f&quot;My MACD strategy started at {self.start_time}&quot;)<br><br>    def on_bar(self, bar: Bar):<br>    <br>        self.count_processed_bars += 1<br><br>        self.macd.handle_bar(bar)<br>        if not self.macd.initialized:<br>            return<br><br>        self.check_for_entry()<br>        self.check_for_exit()<br><br>    def check_for_entry(self):<br>        if self.macd.value &gt;= self.enter_threshold:<br>            # If already in Long position do nothing<br>            if self.position and self.position.side == PositionSide.LONG:<br>                return<br>            # Else make order<br>            order = self.order_factory.market(<br>                instrument_id = self.instrument_id,<br>                order_side = OrderSide.BUY,<br>                quantity = self.trade_size<br>            )<br>            self.submit_order(order)<br><br>        elif self.macd.value &lt; -self.enter_threshold:<br>            # If already in Short poisition do nothing<br>            if self.position and self.position.side == PositionSide.SHORT:<br>                return<br>            # Else make order<br>            order = self.order_factory.market(<br>                instrument_id = self.instrument_id,<br>                order_side = OrderSide.SELL,<br>                quantity = self.trade_size<br>            )<br>            self.submit_order(order)<br><br>    def check_for_exit(self):<br>        # If we are in a Short position, exit when MACD value is positive<br>        if self.macd.value &gt;= 0:<br>            if self.position and self.position.side == PositionSide.SHORT:<br>                self.close_position(self.position)<br>        <br>        # If we are in a Long position, exit when MACD value is negative<br>        else:<br>            if self.position and self.position.side == PositionSide.LONG:<br>                self.close_position(self.position)<br>        <br>    <br>    def on_end(self):<br>        <br>        self.end_time = dt.datetime.now()<br>        self.close_all_positions(instrument_id = self.config.instrument_id)<br>        self.unsubscribe_bars()<br>        <br>        self.log.info(f&quot;My MACD strategy finnished at {self.end_time}&quot;)<br>        self.log.info(f&quot;Total count of 1 day bars: {self.count_processed_bars} &quot;)</pre><p>(In another article, I’ll show you how to build your own custom indicators.)</p><h3>3.4 Setting Up the Backtest Engine</h3><h4>3.4.1 Create the Engine &amp; Add the Strategy</h4><p>Before creating the engine:</p><ul><li>We define the <strong>instrument</strong> (NVDA) — so NT knows what asset we’re working with.</li></ul><pre>from decimal import Decimal<br>from nautilus_trader.model.identifiers import InstrumentId<br>from nautilus_trader.model.identifiers import Symbol<br>from nautilus_trader.model.identifiers import Venue<br>from nautilus_trader.model.instruments import Equity<br>from nautilus_trader.model.currencies import USD<br>from nautilus_trader.model.objects import Price, Quantity<br><br><br>instrument_id = InstrumentId(<br>        symbol=Symbol(&quot;NVDA&quot;),<br>        venue=Venue(&quot;NASDAQ&quot;),<br>    )<br><br><br># Create the NVIDIA equity instrument<br>NVDA_STOCKS_INSTRUMENT = Equity(<br>    instrument_id=instrument_id,<br>    raw_symbol=Symbol(&quot;NVDA&quot;),<br>    currency=USD,<br>    price_precision=2,  # Prices to 2 decimal places<br>    price_increment=Price.from_str(&quot;0.01&quot;),<br>    lot_size=Quantity.from_int(1),  # Trading in whole shares<br>    isin=&quot;US67066G1040&quot;,  # NVIDIA&#39;s ISIN identifier<br>    ts_event=0,  # Timestamp when the instrument was created/updated<br>    ts_init=0,   # Timestamp when this object was initialized<br>)</pre><ul><li>We specify the <strong>data resolution</strong> (in this case, <strong>1 Day</strong> candles).</li></ul><pre>NVDA_STOCKS_1DAY_BARTYPE = BarType.from_str(<br>    f&quot;{NVDA_STOCKS_INSTRUMENT.id}-1-DAY-LAST-EXTERNAL&quot;<br>)</pre><p>Then, we create the engine and add the strategy to it.</p><pre>from nautilus_trader.backtest.engine import BacktestEngine<br><br>from nautilus_trader.config import BacktestEngineConfig<br>from nautilus_trader.config import LoggingConfig<br>from nautilus_trader.config import ImportableStrategyConfig<br><br>from nautilus_trader.model import TraderId<br>from nautilus_trader.model import Bar<br><br>engine_config = BacktestEngineConfig(<br>    trader_id = TraderId(&quot;BACKTEST-NVDA1DAY-001&quot;),<br>    strategies = [<br>        ImportableStrategyConfig(<br>        strategy_path = &#39;__main__:MACDStrategy&#39;,<br>        config_path = &#39;__main__:MACDStrategyConfig&#39;,<br>        config = {<br>            &quot;instrument_id&quot;: instrument_id,<br>            &quot;bar_type_1day&quot;: NVDA_STOCKS_1DAY_BARTYPE,<br>            &quot;fast_period&quot;: 10,<br>            &quot;slow_period&quot;: 20,<br>            &quot;enter_threshold&quot;: 0.00001<br>        }<br>        )<br>    ],<br>    logging = LoggingConfig(log_level = &quot;DEBUG&quot;),<br>    <br>)<br><br>engine = BacktestEngine(config = engine_config)<br><br>engine.add_instrument(NVDA_STOCKS_INSTRUMENT)</pre><h4>3.4.2 Add a Venue</h4><p>We create a <strong>Simulated Venue</strong> and plug it into the engine.</p><pre>from nautilus_trader.model.identifiers import Venue<br>from nautilus_trader.model.enums import AccountType<br>from nautilus_trader.model.enums import OmsType<br>from nautilus_trader.model.currencies import USD<br>from nautilus_trader.model.objects import Money<br><br>engine.add_venue(<br>        venue=Venue(&quot;NASDAQ&quot;),<br>        oms_type=OmsType.NETTING,  # Order Management System type<br>        account_type=AccountType.MARGIN,  # Type of trading account<br>        starting_balances=[Money(1_000_000, USD)],  # Initial account balance<br>        base_currency=USD,  # Base currency for account<br>        default_leverage=Decimal(1),  # No leverage used for account<br>    )</pre><p>(Think of this as setting up your virtual exchange.)</p><h4>3.4.3 Add the Data</h4><p>Next:</p><ul><li>Download NVDA historical data from <strong>Yahoo Finance</strong>.</li></ul><pre>ydata_df = yfinance.download(tickers=[&#39;NVDA&#39;], start=&#39;2000-01-01&#39;, end=&#39;2025-01-01&#39;)<br><br># Turn yahoo data into Nautilus Data type<br><br>ydata_df.head()</pre><ul><li>Convert it into NT’s internal BarData format.</li></ul><pre>from nautilus_trader.persistence.wranglers import BarDataWrangler<br><br>wrangler = BarDataWrangler(<br>    NVDA_STOCKS_1DAY_BARTYPE,<br>    NVDA_STOCKS_INSTRUMENT<br>)<br><br>nvda_1day_bars_list: list[Bar] = wrangler.process(ydata_df)</pre><ul><li>Feed it into the engine.</li></ul><pre>engine.add_data(nvda_1day_bars_list)</pre><h3>3.5 Run the Backtest</h3><p>Now, everything is ready!</p><p>To run the backtest, simply call:</p><pre>engine.run()</pre><p>To see the results:</p><pre>engine.trader.generate_orders_report()<br>engine.trader.generate_positions_report()</pre><h3>4. Conclusion</h3><p><strong>NautilusTrader is a strong and promising trading framework.</strong></p><p>It may solve many of the problems we face in building a multi-asset trading system without excessive effort.</p><p>But — <strong>be warned</strong> — the steep learning curve and the limited number of beginner-friendly tutorials can be very discouraging.</p><p>I hope this article makes it a little easier for you to get started!</p><p>If you’re looking for the <strong>full code</strong> example I used, you can find it here:</p><p><a href="https://github.com/truongbduy/articles/blob/main/Nautilus%20Trader%20Exercise%201.ipynb">[GitHub Link ]</a></p><p><strong>Disclaimer</strong>:<br><em>I’m a non-native English speaker and have written this article by myself. That said, I’ve found that LLMs like ChatGPT help make my writing smoother and more natural. I often use them as a final touch to make my work sound more like a native English speaker.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a08da1999963" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Log return]]></title>
            <link>https://medium.com/@truongb.duy/log-return-ed1021b5e16d?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed1021b5e16d</guid>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[quant-trading]]></category>
            <category><![CDATA[quant]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Sat, 19 Apr 2025 07:47:03 GMT</pubDate>
            <atom:updated>2025-04-19T07:58:32.777Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’ve ever wondered why quants and academics obsess over log returns while your broker shows simple returns, you’re not alone.</p><h3>1. Introduction</h3><p>When I tried to draft my first backtesting strategy, one of the things I stumbled on — and found counterintuitive — was the concept of log return.</p><p>I mean, why use log returns when simple returns seem good enough, right? Simple returns are easier to communicate and easier to comprehend.</p><p>Yet, the more I learned, the more the beauty of log returns revealed itself. In short:</p><p><strong>Simple returns are great for eyeballing daily gains, but log returns reveal hidden patterns</strong> — like how compounding works under the hood.</p><p>Hence, I’m writing this article to retrace my journey with log returns.</p><h3>2. Simple Return</h3><p>Let’s start with the humble simple return, which all of us can understand.</p><h4>Formula:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*LO3tfQkwh0vSGCgVBJZ6gQ.png" /><figcaption>Simple return</figcaption></figure><h4>Example:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/77790fa8e40677cc45f850ec69f586be/href">https://medium.com/media/77790fa8e40677cc45f850ec69f586be/href</a></iframe><ul><li>At <strong>t0</strong>, my asset price is <strong>$100</strong>.</li><li>At <strong>t1</strong>, it rises to <strong>$110</strong>. The return is <strong>10%</strong>:</li></ul><p><em>R₁ = (110–100)/100 = 10%</em></p><ul><li>At <strong>t2</strong>, the asset drops back to <strong>$100</strong>. The return is <strong>-9.09%</strong>:</li></ul><p><em>R₂ = (100–110)/110 = -9.09%</em></p><p>At <strong>t2</strong>, my asset value returns to the initial <strong>$100</strong>, so my <strong>total portfolio return is 0%</strong>. Easy-peasy, right?</p><p>Yet, two things stand out:</p><ol><li><strong>R₁ and R₂ are not symmetric</strong> even though the gain and loss amounts are equal. <strong>Notice how a 10% gain and a 9.09% loss don’t cancel out? That’s because simple returns are ‘path-dependent’ — the order of gains/losses changes the outcome 🤔.</strong></li><li>To find the <strong>total return over the period</strong>, I can’t just add R₁ and R₂ 🤔.</li></ol><p>This is where <strong>log returns</strong> come to my rescue.</p><h3>3. Log Return</h3><h4>Formula:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/290/1*jByr5BuFoweyrvyy2_FKgw.png" /><figcaption>log return</figcaption></figure><p>Using the same example:</p><ul><li><em>r₁ (log return at t1) = ln(110/100) ≈ 9.53%</em></li><li><em>r₂ (log return at t2) = ln(100/110) ≈ -9.53%</em></li></ul><p><strong>Wow!</strong> The values are identical (except for the sign). Log returns solve both problems:</p><ul><li><strong>Symmetry between gains/losses</strong> (a 10% gain and 10% loss in log terms perfectly offset).</li><li><strong>Additivity over time</strong> (just sum the log returns for total growth).</li></ul><p><strong>Why this matters</strong>:</p><ul><li><strong>Financial models</strong> (like Geometric Brownian Motion) use log returns to predict stock prices.</li><li><strong>Volatility</strong> is calculated as the standard deviation of log returns.</li><li><strong>Comparing assets</strong> (e.g., Bitcoin vs. S&amp;P 500) becomes apples-to-apples.</li></ul><p><strong>Bonus</strong>: To convert log return (<em>r</em>) to simple return (<em>R</em>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*FGtD5PwZ9gsEcuHy3_VULw.png" /></figure><p>For our example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/1*21uUd1UmlUQVPr0M5B1Kog.png" /></figure><p><strong>This ‘inflates’ log returns back to simple terms, like translating a whisper into a shout.</strong></p><h3>4. Conclusion</h3><p>To me, log returns are a <strong>cousin</strong> of the familiar simple return. Their values are close (especially for small changes), but they’re <strong>built for different purposes</strong>:</p><ul><li><strong>Simple returns</strong>: Intuitive for single-period communication (<em>“talking to your grandma about her stock gains”</em>).</li><li><strong>Log returns</strong>: Essential for modeling (<em>“building a trading algorithm or analyzing decade-long trends”</em>).</li></ul><p><strong>Disclaimer</strong>:<br><em>I’m a non-native English speaker and have written this article by myself. That said, I’ve found that LLMs like ChatGPT help make my writing smoother and more natural. I often use them as a final touch to make my work sound more like a native English speaker.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed1021b5e16d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Giới thiệu về Data Contract CLI]]></title>
            <link>https://medium.com/vn-digital-outliers/gi%E1%BB%9Bi-thi%E1%BB%87u-v%E1%BB%81-data-contract-cli-0cb0a112e8ce?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/0cb0a112e8ce</guid>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[data-contract]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Sun, 04 Aug 2024 02:38:47 GMT</pubDate>
            <atom:updated>2024-08-04T02:38:47.575Z</atom:updated>
            <content:encoded><![CDATA[<h4>Bài viết này là bài thứ 2 trong chuỗi 2 bài viết liên quan tới Data Contract</h4><p>Dùng Data Contract CLI để quản lý Data Contract như thế nào</p><h4>1. Giới thiệu</h4><p>Trong bài viết trước, mình có giới thiệu qua về Data Contract và cách mình thấy nó thay đổi cách quản lý Data Product. Bạn có thể đọc nó thêm về nó ở <a href="https://medium.com/vn-digital-outliers/gi%E1%BB%9Bi-thi%E1%BB%87u-v%E1%BB%81-data-contract-%C4%91%E1%BB%83-qu%E1%BA%A3n-l%C3%BD-chia-s%E1%BA%BB-d%E1%BB%AF-li%E1%BB%87u-bf6320e95e1d">đây</a> nếu chưa biết về khái niệm Data Contract.</p><p>Trong bài viết này, mình sẽ giới thiệu về package Python Data Contract CLI. Package này cho thấy rằng Data Contract không chỉ dừng lại ở khái niệm mà hoàn toàn có thể áp dụng vào thực tế để cải thiện việc quản lý Data Product hiệu quả hơn.</p><p>Bài viết này mình sẽ dựa hoàn toàn vào trang web chính thức, tuy nhiên mình sẽ tập trung vào những điểm mà mình cảm thấy phù hợp và đang áp dụng vào tổ chức mà mình đang làm việc. Chi tiết bài viết gốc bằng tiếng Anh, các bạn có thể tìm hiểu ở <a href="https://cli.datacontract.com/">đây</a>.</p><h4>2. Hello world!</h4><p>Bước đầu để bạn có thể trải nghiệm về package này là tải về với lệnh</p><pre>$ python3 -m pip install datacontract-cli[all]</pre><p>Và tạo ra kiểm tra Hợp Đồng dữ liệu (Data Contract) đầu tiên với lệnh</p><pre>datacontract test https://datacontract.com/examples/orders-latest/datacontract.yaml</pre><p>Lệnh này sẽ thực hiện việc kiểm tra dữ liệu mẫu đang được lưu ở S3 có khớp với Data Contract đã được khai báo ở <a href="https://datacontract.com/examples/orders-latest/datacontract.yaml">đây</a> hay không.</p><p>Các bạn cũng có thể dùng lệnh sau để biến Data Contract mẫu từ file Yaml thành file có định dạng html để có thể dễ đọc hơn. Bạn có thể xem mẫu html đã được generate ở <a href="https://datacontract.com/examples/orders-latest/datacontract.html">đây</a>.</p><pre>datacontract export --format html https://datacontract.com/examples/orders-latest/datacontract.yaml &gt; datacontract.html</pre><h4>3. Các phần nội dung chính trong Data Contract</h4><p>Nếu bạn làm thử các bước ở mục (2). Bạn đã nhận ra Data Contract thực tế là 1 file yaml. Và việc lưu trữ nó dưới dạng file Yaml sẽ rất hữu ích trong việc sử dụng version control để quản lý các phiên bản của Data Contract theo thời gian.</p><p>Giờ mình sẽ đi qua các phần nội dung chính:</p><h4>3.1. Info</h4><p>Phần Info này cho phép bạn ghi nhận các nội dung mô tả lại về contract như: Mục đích sử dụng, thông tin mô tả, thông tin liên lạc.</p><h4>3.2. Servers</h4><p>Ở đây bạn phải cung cấp những thông tin để có thể truy vấn được các Data Product mà các bạn đang cung cấp. Thông tin này cũng được sử dụng để xác thực Data Product của bạn đáp ứng được các yêu cầu đã được nêu trong Contract.</p><p>Hiện tại Package đang hỗ trợ các nền tảng thường được sử dụng làm DWH chính như Google BigQuery, Snowflake, Databricks, Microsoft SQL Server</p><h4>3.3. Terms</h4><p>Các điều kiện trong Contracts đã được thoả thuận giữa 2 bên có thể được mô tả ở đây. VD dữ liệu phục vụ cho mục đích gì, các giới hạn trong quá trình sử dụng, chi phí sử dụng tối đa. Rất tiếc là chưa có cơ chế nào để ràng buộc trực tiếp các điều kiện này, tuy nhiên nếu đây là những tiêu chí quan trọng, mình nghĩ bạn có thể code thêm để có những cảnh báo, thậm chí hạn chế truy cập nếu cần thiết.</p><h4>3.4. Models</h4><p>Ở khu vực này bạn có thể khai báo các bảng, cột và loại dữ liệu trong Data Product mà bạn đang cung cấp. Khi bạn chạy lệnh <em>datacontract test </em>nó sẽ kiểm tra xem bảng có tồn tại hay không, cột có đầy đủ hay không và loại dữ liệu có khớp với mô tả hay không. Theo mình đây là một tính năng khá hữu hiệu để tránh tình trạng gãy pipeline.</p><h4>3.5. Service levels</h4><p>Đây là những thoả thuận về mức độ dịch vụ của 2 bên cung cấp và sử dụng dữ liệu. Ở đây bạn có thể mô tả về tần suất load dữ liệu, thời điểm load dữ liệu, mức độ sẵn sàng của dữ liệu. Cũng như phần terms ở trên, rất tiếc là chưa có cách nào để kiểm tra mức độ đáp ứng dịch vụ đã được đưa ra. Tuy nhiên đây cũng là những thông tin hữu ích nên được quy định rõ giữa 2 bên.</p><h4>3.6. Quality</h4><p>Đây là phần mà theo mình là rất hữu ích nếu tổ chức của bạn biết cách tận dụng. Phần Quality cho phép bạn khai báo các Quality Check mà bên cung cấp và sử dụng dữ liệu xác định cùng với nhau. Nếu khai báo tốt nó sẽ giúp cho bạn tránh được rủi ro dữ liệu mà bạn có thể sẽ gặp phải.</p><h4>4. Flow</h4><p>Theo mình có 2 hướng mà bạn có thể áp dụng.</p><h4>4.1. Chạy lệnh test Data contract vào cuối pipeline</h4><p>Cách chạy này giúp bạn không phải chạy thêm 1 luôn xử lý dữ liệu mới mà chỉ cần tận dụng luôn pipeline có sẵn. Tuy nhiên bên mình đã gặp 1 trường hợp là luồn pipeline chính không chạy, dẫn tới toàn bộ các nội dung test sau đó cũng không chạy.</p><h4>4.2. Tạo ra 1 luồn chạy test Data Contract song song</h4><p>Việc tạo ra 1 luồn chạy song song chỉ để test dữ liệu sẽ giống như bạn có 1 bên độc lập để chỉ ra các điểm sai của bạn. Tuy nhiên nó có thể khiến cho việc triển khai và vận hành tốn thêm chút thời gian. Dẫu vậy nếu việc đảm bảo vận hành là yêu cầu bắt buộc thì mình nghĩ phương thức này sẽ giúp chất lượng dữ liệu của bạn lúc nào cũng đạt mức cao.</p><h4>5. Kết</h4><p>Nếu tổ chức của bạn đang gặp phải các vấn đề liên quan tới:</p><ul><li>Chất lượng dữ liệu không đảm bảo</li><li>Phối hợp giữa bên cung cấp và sử dụng dữ liệu chưa hiệu quả</li><li>Số lượng Data Product tăng theo cấp số nhân</li></ul><p>Thì mình cho rằng Data Contract có thể sẽ là 1 ứng dụng cực kỳ hữu dụng, dễ tích hợp và hoàn toàn miễn phí mà bạn có thể thử.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0cb0a112e8ce" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vn-digital-outliers/gi%E1%BB%9Bi-thi%E1%BB%87u-v%E1%BB%81-data-contract-cli-0cb0a112e8ce">Giới thiệu về Data Contract CLI</a> was originally published in <a href="https://medium.com/vn-digital-outliers">VN Digital Outliers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Contract để quản lý chia sẻ Dữ liệu dễ dàng hơn]]></title>
            <link>https://medium.com/vn-digital-outliers/gi%E1%BB%9Bi-thi%E1%BB%87u-v%E1%BB%81-data-contract-%C4%91%E1%BB%83-qu%E1%BA%A3n-l%C3%BD-chia-s%E1%BA%BB-d%E1%BB%AF-li%E1%BB%87u-bf6320e95e1d?source=rss-70182490f18a------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf6320e95e1d</guid>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[data-contract]]></category>
            <category><![CDATA[data-management]]></category>
            <dc:creator><![CDATA[Duy Trương]]></dc:creator>
            <pubDate>Wed, 17 Jul 2024 05:46:18 GMT</pubDate>
            <atom:updated>2024-07-17T05:49:21.337Z</atom:updated>
            <content:encoded><![CDATA[<h4>Bài viết này là bài viết đầu tiên trong chuỗi 2 bài viết liên quan tới Data Contract</h4><h4>Data contract là gì? Có thể ứng dụng Data contract vào việc quản lý chia sẻ dữ liệu như thế nào?</h4><blockquote>Data contract thật sự mang người cung cấp dữ liệu (data providers) và bên sử dụng dữ liệu (data consumers) đến gần nhau hơn.</blockquote><h4>1. Giới thiệu</h4><p>Cũng như các tổ chức khác, nơi mình làm việc cũng sẽ có yêu cầu lấy dữ liệu từ hệ thống Data Analytics Platform để cung cấp cho các hệ thống Dữ liệu vận hành. Để đảm bảo cho việc vận hành hiệu quả nhất, thì các yêu cầu sau đây sẽ cần phải làm rõ và đảm bảo:</p><ul><li><strong>Hình dạng dữ liệu (Data Shape)</strong>: Loại dữ liệu, kiểu dữ liệu, số lượng cột, mô tả dữ liệu.</li><li><strong>Kỳ vọng (Expectations)</strong>: mức độ sẵn sàng và chất lượng của dữ liệu.</li><li><strong>Quy định (Regulations)</strong>: Mô tả các điều khoản có thể dẫn tới vi phạm thoả thuận và các biện pháp đảm bảo tuân thủ.</li><li><strong>Cách thức (How)</strong>: Quy định/hướng dẫn cách lấy dữ liệu và thời gian sử dụng dữ liệu.</li></ul><p>Đội quản lý dữ liệu thường có thể ghi nhận các thông tin này trong các định dạng file văn bảng như Word/Excel hoặc trong các hệ thống quản lý dự án/phần mềm như Jira. Các thông tin này có thể được coi như là 1 thoả thuận (<em>Agreement</em>) giữa 2 bên (bên cung cấp và bên sử dụng dữ liệu). Sau khi có những thoả thuận này, đội triển khai sẽ bắt đầu triển khai các yêu cầu này vào thực tế.</p><p>Hình dưới sẽ mô tả về cách mình nhìn thấy về mô hình quản lý hiện tại.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*vuKi2M-5ZLAe6OhZIgpDYg.png" /><figcaption>Quy trình quản lý của bên Sử Dụng và bên Cung cấp dữ liệu bị tách biệt</figcaption></figure><p>Tuy nhiên, phương pháp lưu trữ những mô tả này đi kèm với những hạn chế nhất định mà đội quản lý dữ liệu phải đối mặt.</p><ol><li><strong>Cập nhật thông tin</strong>: Đội quản lý dữ liệu (provider) và đội sử dụng (consumer) thường sẽ rất tích cực trong thời gian đầu trong việc phối hợp với nhau để mô tả yêu cầu. Sau giai đoạn này khi các nhu cầu mới phát sinh, để thuận tiện, cả 2 bên thường hỗ trợ triển khai ngay mà bỏ qua phần cập nhật lại các thoả thuận ban đầu. Điều này dẫn tới việc các thoả thuận cung cấp dữ liệu thường bị lạc hậu và không phản ánh đúng thực tế hiện tại.</li><li><strong>Quản lý phiên bản</strong> (version control): Trong quá trình điều chỉnh, các thay đổi có thể không được phân ra từng phiên bản cụ thể. Bên cạnh đó việc xem lại các phiên bản cũ hơn có thể bị hạn chế, tuỳ thuộc vào loại công cụ sử dụng để quản lý.</li><li><strong>Tận dụng cho các mục đích khác</strong>: Đối với những yêu cầu về đảm bảo chất lượng dữ liệu, bên cạnh việc ghi nhận trong văn bản thì còn phải được triển khai trên 1 công cụ quản lý chất lượng dữ liệu khác như DBT hay Great Expectations. Điều này dẫn tới rủi ro có 1 độ vênh (gap) nhất định giữa thoả thuận và thực tế sau này.</li></ol><p><em>Bạn có tưởng tượng được vấn đề này sẽ bị nhân lên như thế nào khi số lượng dữ liệu mà bạn phải cung cấp nhân lên gấp 10 hay 100 lần với phương pháp hiện tại?</em></p><p>Data contract là một giải pháp mà các bạn có thể sử dụng để giải quyết các vấn đề trên. Vì Data contract không chỉ dừng lại ở một khái niệm, mà giờ đây nó còn là một công cụ mà các bạn có thể ứng dụng vào thực tế. Data contract thật sự mang người cung cấp dữ liệu và bên sử dụng dữ liệu đến gần nhau hơn.</p><p>Đến đây, hi vọng các bạn đã cảm thấy hứng thú hơn. Phần còn lại của bài viết này sẽ tiếp tục mô tả một cái nhìn khái quát của người viết về Data Contract và cách người viết áp dụng nó trong thực tế.</p><h4>2. Vậy Data contract là gì?</h4><p>Dựa theo mô tả của <a href="https://datacontract.com/">https://datacontract.com/</a> chúng ta có một định nghĩa như bên sau</p><blockquote>Hợp đồng dữ liệu là tài liệu xác định cấu trúc, định dạng, ngữ nghĩa, chất lượng và điều khoản sử dụng để trao đổi dữ liệu giữa bên cung cấp dữ liệu và bên sử dụng dữ liệu. Hãy nghĩ về một API, nhưng dành cho dữ liệu. Hợp đồng dữ liệu được triển khai bởi một sản phẩm dữ liệu (Data product) hoặc các công nghệ dữ liệu khác, thậm chí cả kho dữ liệu (Data Warehouse). Hợp đồng dữ liệu cũng có thể được sử dụng để mô tả các kỳ vọng về sự phụ thuộc dữ liệu và xác thực các điều kiện được đưa ra. — <a href="https://datacontract.com/">https://datacontract.com/</a></blockquote><p>Các bạn có thể vào trực tiếp trang Data contract để xem thêm thông tin. Mình sẽ gỉai thích một cách đơn giản về sự khác biệt của việc sử dụng Data Contract với phương pháp lưu trữ cũ như sau. Data Contract là một file <strong>yaml</strong>, trong đó nó mô tả các thoả thuận giữa bên cung cấp và bên sử dụng dữ liệu. Bằng cách này, nó có thể được lưu trữ, chia sẻ thông qua các công cụ version control. Nhưng quan trọng hơn, nó có thể được đưa vào các hệ thống Data Platform hiện đại để đảm bảo các điều kiện được quy định trong Data Contract được thực thi.</p><p>Mô hình mình vẽ dưới đây thể hiện cách mình thấy về việc áp dụng Data contract vào thực tế.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*V3sxlvrZ1KEmuO9_WuW6RQ.png" /><figcaption>Quy trình quản lý của bên Sử Dụng và bên Cung cấp dữ liệu được gắn kết qua Data Contract</figcaption></figure><p>Thay vì phải tách biệt giữa 2 hệ thống/quy trình riêng biệt, trong đó 1 hệ thống chỉ tập trung vào việc đưa thống nhất yêu cầu/thoả thuận, và 1 hệ thống chỉ tập trung vào việc triển khai. Data Contract sẽ đóng vai trò cầu nối, trong đó vừa thể hiện được các yêu cầu về dữ liệu, và đồng thời nó còn đảm bảo việc thực thi các quy định đã được mô tả.</p><p><em>Bạn đã bắt đầu cảm nhận được sự tuyệt vời của mô hình này? Nó giúp cho việc quản lý các sản phẩm dữ liệu trở nên tinh tế và hiệu quả hơn bao giờ hết. Vì nó buộc các bên phải tích cực tham gia vào việc cập nhật Data contract thường xuyên nếu muốn chất lượng dữ liệu được đảm bảo và thực thi.</em></p><h4>3. Kết luận</h4><p>Tới đây mình tin rằng các bạn đã rõ hơn về Data contract là gì và lợi ích của nó ra sao. Bước tiếp theo bạn có thể :</p><ol><li>Bắt đầu tìm hiểu thêm về Data Contract thông qua trang web <a href="https://datacontract.com/">https://datacontract.com/</a></li><li>Bắt đầu sử dụng <a href="https://pypi.org/project/datacontract-cli/0.9.7/#:~:text=The%20datacontract%20CLI%20is%20an,tool%20is%20written%20in%20Python.">Data Contract CLI</a>, một Python package, để lập trình Data contract vào hệ thống dữ liệu mà bạn đang quản lý</li><li>Đọc thêm bài viết số 2 của mình để hiểu thêm về Data Contract CLI và cách mình đã áp dụng nó trong thực tế.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf6320e95e1d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vn-digital-outliers/gi%E1%BB%9Bi-thi%E1%BB%87u-v%E1%BB%81-data-contract-%C4%91%E1%BB%83-qu%E1%BA%A3n-l%C3%BD-chia-s%E1%BA%BB-d%E1%BB%AF-li%E1%BB%87u-bf6320e95e1d">Data Contract để quản lý chia sẻ Dữ liệu dễ dàng hơn</a> was originally published in <a href="https://medium.com/vn-digital-outliers">VN Digital Outliers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>