<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Amirali Omidvar on Medium]]></title>
        <description><![CDATA[Stories by Amirali Omidvar on Medium]]></description>
        <link>https://medium.com/@amirali.omidvar80?source=rss-ffe8da74e0db------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dZJw5kXJHcgZ263p-wYGHQ.jpeg</url>
            <title>Stories by Amirali Omidvar on Medium</title>
            <link>https://medium.com/@amirali.omidvar80?source=rss-ffe8da74e0db------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 01:32:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@amirali.omidvar80/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Case Against Technical Analysis]]></title>
            <link>https://medium.com/coinmonks/the-case-against-technical-analysis-a513d0629536?source=rss-ffe8da74e0db------2</link>
            <guid isPermaLink="false">https://medium.com/p/a513d0629536</guid>
            <category><![CDATA[market]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[technical-analysis]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[finance]]></category>
            <dc:creator><![CDATA[Amirali Omidvar]]></dc:creator>
            <pubDate>Mon, 04 Mar 2024 15:15:25 GMT</pubDate>
            <atom:updated>2024-03-04T15:15:25.384Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="technical analysis down trend" src="https://cdn-images-1.medium.com/max/1024/0*5utUhEz3t4ght4iU" /></figure><h3>Intro</h3><p>Unlike one or two decades ago, many people are involved with financial markets. either because of increasing living costs or the accessibility that technology has brought, these markets have been busier than ever. and the tool most people use to navigate these markets is technical analysis. but how effective is technical analysis? in this article, I give my opinion on why I don’t believe technical analysis is useful in market prediction.</p><h3>Definition of Technical Analysis</h3><p>Let&#39;s start with some basics so we are on the same page, technical analysis is a form of modeling the markets using historical and current price and volume data. technical analysis employs charts, graphs, and indicators to predict market movements.</p><p>Now that we covered that, let me explain why I don&#39;t use or even believe in technical analysis. I form my argument into five points, one of them being the most important which I will later explain.</p><h4>1. Broad and Personal Definition of Analysis</h4><p>Technical analysis is the most used type of modeling market among traders. mostly taking the form of support resistance, trends, patterns, and indicators like RSI or MACD.</p><p>This vast implementation of TA has led to two major problems. One is many people develop their definitions or implementation of TA concepts.</p><figure><img alt="Order Block Trading Definition Example 1" src="https://cdn-images-1.medium.com/max/1024/1*Bu8lyrsykkajoJ_DVeQu_Q.png" /><figcaption>Order Block Definition Example 1</figcaption></figure><figure><img alt="Order Block Trading Definition Example 2" src="https://cdn-images-1.medium.com/max/846/1*b1aKPlbyHuANoY7ldijtHw.png" /><figcaption>Order Block Definition Example 2</figcaption></figure><p>Here are two examples of how to find an order block that is completely different from each other. Since no community goes through a scientific process to validate claims in the field of technical analysis, there is no one definition of TA concepts.</p><p>The other problem that exists is the broad definition of concepts. for example consider one of the building blocks of TA, support resistance. if a resistance line breaks and the price line starts trending, traders who use technical analysis assume TA has worked. but when it doesn&#39;t and it returns down, by definition it&#39;s called a false break.</p><blockquote>By integrating false predictions into the system, we will have a predicting system that is never wrong by definition, with no guarantee of correct predictions. and with loose rules on how we define concepts in this system, we reach pseudoscience territory.</blockquote><h4>2. Number of parameters, Degrees of freedom, and Overfitting.</h4><figure><img alt="complicated trading strategy" src="https://cdn-images-1.medium.com/max/1024/0*atqeywDoNhhpS3T9.jpg" /></figure><p>If you used technical indicators, you have seen that they have some parameters, for example, RSI has length parameters that define how many candles should be used for RSI calculation from historical data. although people can use the default value which is 14. but this might not be the best value and won’t result in the best performance. so this creates a question what is the best number for RSI length, or what is the best RSI value to exit or enter the trade, or even outside of RSI, how long is the resistance support zone, or what should be the depth of ZigZag?</p><p>A trading strategy usually uses multiple technical analysis concepts. usually multiple as primary trade opportunity detection and multiple as confirmation. this leads to having many parameters that can be changed. I already have an article on how to find the best parameters for a trading bot, <a href="https://medium.com/me/stats/post/99a5c37483a2">here</a>. although you can find the parameters that have the best performance, you lose generalization.</p><p>By tuning these parameters, you create a predictive system that can work on data that has been seen, but necessarily not other types of data. this can be as insignificant as “the strategy works on cryptocurrencies but not on stocks” if the number of variables is low, or as dangerous as “this trading strategy only works for a week after the historic data it has been fed” if the number parameters are high.</p><blockquote>Since technical trading strategies have many parameters to be worked with and tuned, its best version would lack generalization and might fail on providing the performance it showed in testing.</blockquote><h4>3. Random Walk and Market Efficiency</h4><figure><img alt="random walk" src="https://cdn-images-1.medium.com/max/1024/0*GO4mzEH4o95bzpxp.jpeg" /></figure><p>This is in my opinion the most important reason why technical analysis is not effective. first, let&#39;s define two concepts. they might be a little complicated at first but are fairly simple to understand.</p><p><strong>Market Efficiency: </strong>This concept refers to how well the price of a security represents all the information about it. this leads to all market securities being rightly priced, and not lower or higher than their true value. there is a lot of debate about whether this hypothesis is true or not, but most experts agree on a middle ground where markets exhibit both efficient and inefficient behaviors with changing ratios.</p><p><strong>Random Walk: </strong>A random walk is a hypothetical behavior that describes how prices move in the market. according to random walk, the next instance of price is a random up or down movement. so the price of the security in the future will be a random variable with a normal distribution (see <a href="https://www.youtube.com/watch?v=HV0Tqcg9MtU">here</a> for more info).</p><p>But where does this randomness come from? well in reality calling anything random in real life is wrong, anything can be predicted with enough precision. the randomness we perceive is a lack of precision. in physics, the movement of particles is affected by many other particles and other factors that are impossible to measure, so they are considered random (see <a href="https://en.wikipedia.org/wiki/Brownian_motion">Brownian Motion</a>). In markets same things happen, there are so many factors like weather, politics, social events, or even the mood of a fund manager that can affect the price. so to us, since we lack precision, these events seem random.</p><p>Both of these concepts are not 100% true in my opinion. for example, bubbles form because of the FOMO mentality of traders can lead to overvalued prices, or in the long run successful businesses have stocks with trending prices. but because of the randomness in the markets we described, is hard to detect when the market is inefficient. and since the only way to profit from the market is to detect when a price is under or overvalued, (aka inefficiencies) we need a tool that can detect them.</p><p>This brings us back to technical analysis. does technical analysis help us detect these market inefficiencies? well, let’s imagine a scenario. you found out that in market there is a stock that is undervalued using technical analysis. since TA is the most popular tool for analyzing the market many other people will find this too, so all of these people start buying, now in a short time the price of the stock will reach what it should be and you are now left behind or didn’t have the time to analyze this inefficiency since many people use TA.</p><blockquote>The act of finding the market inefficiency and predicting price has affected the price itself. a predictive system for the financial market has to be innovative and be out of the majority of trader&#39;s access, to be able to detect the market efficiency without affecting it.</blockquote><p>There is also the problem that no technical analysis concept takes the randomness factor into account. let&#39;s try something, I will give you two pictures of 5-minute candles both are normalized and have the same date. you have to guess which one is from real data and which one is randomly generated using technical analysis.</p><figure><img alt="an example of synthetically created candle price data" src="https://cdn-images-1.medium.com/max/1024/1*jyDEIylxFbuyn5xftHTK6w.png" /></figure><figure><img alt="an example of synthetically created candle price data" src="https://cdn-images-1.medium.com/max/1024/1*qbyNsmisRvmred1027JNVg.png" /></figure><p>Their candle data is also available <a href="https://github.com/AmiraliOmidvar/candle_data">here</a> if you want to take a closer look.</p><p>If you have decided on your decision go down and see the answer.</p><p>Well, this might be a cheap trick but both of these are generated by random processes with the same characteristics.</p><blockquote>I&#39;ve done this test so many times even with more data samples and real data, and people aren&#39;t able to guess correctly most of the time, enough to statically say the technical analysis these people use cannot differentiate between random data and real data, so there is not predictive capability in technical analysis.</blockquote><h4>4. On the Should of Giants</h4><figure><img alt="Isaac Newton on the shoulder of giants quote" src="https://cdn-images-1.medium.com/max/602/0*HuN51vCfk-ULBBwv" /></figure><p>To find the best approach to any work that we want to do, is important to take a look at how predecessors and successful individuals have walked and paved the path. so let&#39;s take a look at the methods of some of the most successful traders in their methods.</p><ol><li>Jim Simons: Simons is a mathematician and founder of Renaissance Technologies, a highly successful hedge fund known for its quantitative approach. Simons and his team use mathematical models and algorithms to identify trading opportunities, focusing on patterns and trends in market data.</li><li>Warren Buffett: Buffett is a renowned value investor and the chairman of Berkshire Hathaway. His method involves analyzing companies based on their intrinsic value, economic moats, and long-term potential. Buffett is not known for using technical analysis but instead emphasizes understanding the fundamentals of a business.</li><li>George Soros: Soros is known for his macroeconomic approach. He analyzes and anticipates large-scale economic trends, making substantial bets on currency and commodity markets. Soros combines fundamental analysis with market sentiment, rather than relying on technical indicators.</li><li>Paul Tudor Jones: Jones is a hedge fund manager who gained fame for predicting the 1987 market crash. His trading method involves trend following and macroeconomic analysis. While he uses charts and technical analysis to some extent, he also incorporates fundamental factors into his decision-making.</li><li>Ray Dalio: Dalio is the founder of Bridgewater Associates. His approach involves risk parity, which aims to balance and diversify risks across different asset classes. While Dalio uses quantitative models, his strategy is rooted in economic principles rather than traditional technical analysis.</li><li>John Paulson: Paulson gained fame for betting against the subprime mortgage market in 2007. He is an event-driven investor, focusing on special situations and corporate events. Paulson combines fundamental analysis with a keen understanding of market dynamics.</li><li>Stanley Druckenmiller: Druckenmiller is known for his success as a hedge fund manager and his role as George Soros’s right-hand man. He adopts a macroeconomic approach, analyzing global trends and making bets based on his economic outlook. Druckenmiller combines fundamental and technical analysis.</li><li>David Tepper: Tepper is a hedge fund manager known for his success in distressed debt investing. He focuses on companies facing financial difficulties and reaps profits as they recover. While he may use some technical analysis, Tepper’s strategy is predominantly based on fundamental analysis.</li><li>Richard Dennis: Dennis, along with William Eckhardt, developed the Turtle Trading experiment. They taught a group of individuals with little or no trading experience to follow trend-following strategies. The approach involved a combination of technical indicators and disciplined risk management.</li><li>Ed Seykota: Seykota is a trend-following trader known for his success in the commodities market. His approach involves identifying and riding trends while using technical analysis, such as moving averages, to make trading decisions. Seykota emphasizes the importance of psychology and discipline in trading.</li></ol><blockquote>As you can see, most of these traders either don&#39;t use technical analysis for trading or use it as a secondary tool for a primary tool like fundamental analysis. as time has passed, less successful people are using technical analysis, due to majority of people using it.</blockquote><h3>My Experience With Techincal Analysis</h3><p>I’ve been an Algo trader from the start, turning technical analysis concepts into code and optimizing the parameters to create a bot that can trade for me, but with my input as well. when I started cryptocurrency was booming and it was the most accessible market to algotrading. I’ve made a good profit using this bot, around 15% monthly for 7 months. I used a high-frequency strategy that made 10–15 trades daily. but after the cryptocurrency market crashed the profit stopped too.</p><p>At that time I thought I had beaten the market, but I was just riding the wave up, and I was lucky to trade in a low timeframe and also understand that I have to stop. after this, I made many more attempts at trying to create another bot to be able to work in any market condition on lower timeframes, but I wasn’t able to. no matter what I did or how much research went into bot development, I couldn&#39;t beat the market, and couldn&#39;t achieve more than the average return.</p><p>When doing research, I concluded that technical analysis can&#39;t achieve what I want from it, due to the reasons I mentioned. currently, I&#39;m not trading, I&#39;m working in fintech because one, I don&#39;t see the ability in myself to beat the market, and trading puts a lot of strain on traders&#39; psyche.</p><h3>Conclusion</h3><p>In summary, the widespread use of technical analysis in financial markets raises questions about its reliability. The diverse interpretations of its concepts, the challenges of finding optimal parameters, and the unpredictable nature of markets challenge its effectiveness. Personal experiences in algorithmic trading underscore these limitations. In a world of market efficiency and randomness, a more varied approach may be necessary.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a513d0629536" width="1" height="1" alt=""><hr><p><a href="https://medium.com/coinmonks/the-case-against-technical-analysis-a513d0629536">The Case Against Technical Analysis</a> was originally published in <a href="https://medium.com/coinmonks">Coinmonks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Celery Features That Can Improve Your System Design]]></title>
            <link>https://levelup.gitconnected.com/celery-features-that-can-improve-your-system-design-ea5ab10e5ca1?source=rss-ffe8da74e0db------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea5ab10e5ca1</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[system-architecture]]></category>
            <category><![CDATA[celery]]></category>
            <dc:creator><![CDATA[Amirali Omidvar]]></dc:creator>
            <pubDate>Tue, 27 Feb 2024 21:21:53 GMT</pubDate>
            <atom:updated>2024-02-27T21:21:53.013Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="celery for python icon" src="https://cdn-images-1.medium.com/max/1024/0*4IPZaXtMRQf_nCBD.jpg" /></figure><p>In the discussion of system design, one concept is usually ever-present and that&#39;s queuing. There are a lot of technologies that enable us to perform this concept and the most well-known for Python programming is Celery. Most Backend Developers are familiar with some basic concepts like Celery Beat and Celery Queue, but some really important features can go unnoticed. in this article, I try to introduce 3 celery features that are less known and can improve your system design.</p><h3>Celery Canvas</h3><p>Most people will use the default celery workflow, by putting tasks in a queue as FIFO and consuming it by workers. but the celery canvas enables developers to create much more diverse workflows. Here are the Celery Canvas features:</p><h4>Celery Chord</h4><p>In a basic queue, every task is independent of the other one and will be consumed in order. this can cause problems when a task depends on one or multiple other tasks. for example, imagine you have a large data set and you want to perform some operation on it. each task operates on a subset of this dataset and the final task is an aggression of the partial results. in this case, one task is dependent on others and cannot be handled.</p><p>In these kinds of designs, Celery Chord can help a ton. A Chord has two parts, <strong>parallel tasks,</strong> and a <strong>callback task</strong>. parallel tasks are the tasks that are independent of each other or the callback task. they do similar processes on different data or different subsets of data. the callback task is dependent on the parallel tasks and is only called when all parallel tasks are finished. this task also has access to all of the parallel tasks.</p><p>Here’s a simple distributed example for Celery Chord:</p><pre><br>from celery import Celery<br>from celery.result import AsyncResult<br>app = Celery(&#39;chord_example&#39;, broker=&#39;redis://redis_host:port&#39;)<br><br># parallel tasks<br>@app.task<br>def compute_task(data):<br>    # a simple time consuming operation<br>    result = data ** 2<br>    return result<br><br># callback task<br>@app.task<br>def aggregate_results(results):<br>    # aggregating the results of the paraller tasks<br>    total_result = sum(results)<br>    print(f&quot;Final result: {total_result}&quot;)<br><br># list of data for parallel computation<br>data_list = [1, 2, 3, 4, 5]<br><br># applying Chord to orchestrate parallel tasks and callback task<br>chord_callback = aggregate_results.s()<br>parallel_tasks = [compute_task.s(data) for data in data_list]<br><br># distributing parallel tasks to worker machines (e.g., through a message queue)<br>task_results = [task.apply_async() for task in parallel_tasks]<br><br># check if all tasks are completed<br>if all(result.ready() for result in task_results):<br>    # retrieve results and aggregate using Chord callback<br>    results = [AsyncResult(result.id).get() for result in task_results]<br>    final_result = chord(results)(chord_callback).get()<br>    print(f&quot;Final result obtained from distributed machines: {final_result}&quot;)<br>else:<br>    print(&quot;Not all tasks have completed yet.&quot;)</pre><p>In this example, we define two tasks parallel and callback. the parallel just returns the power of two of the data. the callback returns the sum of these powers of two. using apply_async we add the parallel tasks to the queue, check when they are done, and then call the callback.</p><p>If there are a dynamic number of chords we can just do this.</p><pre>chords = []<br>for index, dataset in enumerate(datasets):<br>    chord_callback = aggregate_results.s()<br>    parallel_tasks = [process_data_subset.s(data_subset) for data_subset in dataset]<br>    chord_task = chord(parallel_tasks)(chord_callback)<br>    chords.append(chord_task)<br><br># Execute dynamic Chords<br>final_results = [chord_result.get() for chord_result in chords]<br><br>print(&quot;Final results from dynamic Chords:&quot;)<br>for index, result in enumerate(final_results):<br>    print(f&quot;Chord {index + 1}: {result}&quot;)</pre><h4>Celery Chain</h4><p>Imagine a scenario where you want to do some tasks one after the other but also comply with the Separation of concerns and not use a giant task. for example, an e-commerce system where there are multiple tasks like verification, shipment, and updating status… that have to be done back to back. here we can use task chains to put the tasks in one of the other</p><pre>from celery import Celery, chain<br><br>app = Celery(&#39;order_processing&#39;, broker=&#39;redis://redis_host:port&#39;)<br><br># task: Verify Payment<br>@app.task<br>def verify_payment(order_id):<br>    # Perform payment verification logic<br>    print(f&quot;Payment verification for order {order_id} completed.&quot;)<br>    return order_id<br><br># task: Prepare Items for Shipment<br>@app.task<br>def prepare_items_for_shipment(order_id):<br>    # Prepare items for shipment logic<br>    print(f&quot;Items prepared for shipment for order {order_id}.&quot;)<br>    return order_id<br><br># task: Ship Items<br>@app.task<br>def ship_items(order_id):<br>    # Ship items logic<br>    print(f&quot;Items shipped for order {order_id}.&quot;)<br>    return order_id<br><br># task: Send Confirmation Email<br>@app.task<br>def send_confirmation_email(order_id):<br>    # Send confirmation email logic<br>    print(f&quot;Confirmation email sent for order {order_id}.&quot;)<br><br># order processing chain<br>order_processing_chain = chain(<br>    verify_payment.s(order_id) |<br>    prepare_items_for_shipment.s() |<br>    ship_items.s() |<br>    send_confirmation_email.s()<br>)</pre><p>In this example, we apply four tasks one after the other, and we can get an update after each task is done while complying separation of concerns.</p><h4>Celery Chunks</h4><p>When trying to design celery tasks, there is always the question of how much the task should be broken down into small parts. if the task is too large, it causes maintainability issues or even issues at the message broker level. If it is too small, it causes some overhead for task retrieval, although there is pre_fetch setting that can help this issue, the is another way, and is chunking. chunking is putting similar small tasks into a chunk of tasks that will be consumed by one worker as a single task.</p><pre># main_and_tasks.py<br>from celery import Celery<br><br>app = Celery(&#39;main_and_tasks&#39;, broker=&#39;pyamqp://guest@localhost//&#39;)<br><br>@app.task<br>def process_email_chunk(email_chunk):<br>    valid_emails = []<br><br>    for email in email_chunk:<br>        if is_valid_email(email):<br>            valid_emails.append(email)<br><br>    return valid_emails<br><br>def is_valid_email(email):<br>    return True<br><br>if __name__ == &quot;__main__&quot;:<br>    all_emails = [&quot;user1@example.com&quot;, &quot;user2@example.com&quot;, ...]<br>    chunk_size = 100<br>    email_chunks = [all_emails[i:i+chunk_size] for i in range(0, len(all_emails), chunk_size)]<br><br>    # use .s() to create signatures for each chunk<br>    signatures = [process_email_chunk.s(chunk) for chunk in email_chunks]<br><br>    # chain the signatures together and apply asynchronously<br>    result = signatures[0].chain(*signatures[1:]).apply_async()<br><br>    # collect the final result<br>    all_valid_emails = result.get()<br><br>    print(&quot;All valid emails:&quot;, all_valid_emails)</pre><p>In this example, some smaller tasks, such as email verification are put in a chunk of tasks and then the chunk is applied as a task.</p><h3>Task Hooks</h3><p>Tasks hooks are special methods that can be defined to perform specified actions at certain points of the task life cycle. The are ten stages of a celery task. some stages are optional hooks, that will perform a static action on the tasks. Here are the stages</p><ol><li>Creation: A task is created from celery.Task class, some initial information is given to the object as arguments.</li><li>Task Initialization: When the task is applied (sync or async)</li><li>Before task publish (Hook): If the hook is defined using before_task_publish method, an action will be performed on all tasks before publishing such as logging.</li><li>Task Publish: When tasks are serialized and published to the message broker.</li><li>After Task Publish (Hook): If the hook is defined using after_task_publish method, the specified action will be performed on all tasks that are published. this would be very useful for system control and by counting tasks and making sure statistically everything is working as intended.</li><li>Task Execution: When the task is received by the worker and executed.</li><li>Tasks Success (Hook): When tasks are done successfully. will be called if on_success method is called.</li><li>Task Failure (Hook): When tasks fail. will be called if on_failure method is defined.</li><li>Task Return (Hook): Will be called regardless of the success state of the task. defined in the task. defined in after_return method.</li><li>Task Acknowledgment: The Celery worker acknowledges the successful completion of the task, removing it from the message broker.</li></ol><p>Here is a simple example of defining and using task hooks in celery.</p><pre>app = Celery(&#39;email_tasks&#39;, broker=&#39;pyamqp://guest:guest@localhost//&#39;)<br>class EmailTask(Task):<br>    def before_task_publish(self, *args, **kwargs):<br>        # Log information before the task is published<br>        print(f&quot;Task {self.name} is about to be published with args: {args} and kwargs: {kwargs}&quot;)<br><br>    def after_task_publish(self, *args, **kwargs):<br>        # Log information after the task is published<br>        print(f&quot;Task {self.name} has been published with args: {args} and kwargs: {kwargs}&quot;)<br><br>    def run(self, to_email, subject, message):<br>        # Simulate sending an email<br>        print(f&quot;Sending email to {to_email} with subject: &#39;{subject}&#39; and message: &#39;{message}&#39;&quot;)<br><br>app.tasks.register(EmailTask())<br><br># Use the task<br>result = EmailTask().apply_async(args=(&#39;user@example.com&#39;, &#39;Hello&#39;, &#39;This is a test email&#39;))</pre><h3>Task Routing</h3><p>If you ever had to deal with multiple kinds of tasks, you might’ve had to deal with the problem of balancing the load between them. for example, some tasks are more important or take longer to complete and need more resources dedicated to them. in cases like this Task Routing can help a lot.</p><p>Task routing is assigning queues are workers to a task. this way a worker only consumes from single or multiple assigned queues.</p><p>Here’s a guide on how to enable routing in celery.</p><ol><li>Define Queues: First, we need to define the queues that we want our system to have.</li></ol><pre>from celery import Celery<br>from kombu import Exchange, Queue<br><br>app = Celery(&#39;myapp&#39;, broker=&#39;pyamqp://guest:guest@localhost//&#39;)<br><br># Define exchanges<br>high_priority_exchange = Exchange(&#39;high_priority&#39;, type=&#39;direct&#39;)<br>low_priority_exchange = Exchange(&#39;low_priority&#39;, type=&#39;direct&#39;)<br><br># Define queues and associate them with exchanges<br>CELERY_QUEUES = (<br>    Queue(&#39;high_priority_queue&#39;, high_priority_exchange, routing_key=&#39;high_priority&#39;),<br>    Queue(&#39;low_priority_queue&#39;, low_priority_exchange, routing_key=&#39;low_priority&#39;),<br>)<br><br># Configure the Celery app with the queues<br>app.conf.task_queues = CELERY_QUEUES</pre><p>Let&#39;s first see what is kombu library. kombu is the underlying library that enables Celery to communicate with the message brokers. The queue concept is defined in kombu and celery simply interacts with it. The exchange class is an object that defines the distribution mechanism of the tasks between queues and workers. for example, the direct type is the default behavior of the queues, you can also set the type to topic for different behavior. I might do a dive deep into Kombu later but I leave it at that now.</p><p>َThe rest of the code defines the queues. creating two different exchanges allows us to set different behaviors. the routing key is what Celery will use later to attach queues and tasks to provide the behavior we want.</p><pre>CELERY_ROUTES = {<br>    &#39;app.high_priority_task&#39;: {&#39;queue&#39;: &#39;high_priority&#39;},<br>    &#39;app.low_priority_task&#39;: {&#39;queue&#39;: &#39;low_priority&#39;},<br>}<br>app.conf.task_queues = task_queues</pre><p>Here we define the routing.</p><pre>from celery import Celery, Task<br>app = Celery(&#39;myapp&#39;, broker=&#39;redis://redishost&#39;)<br><br>@app.task<br>def high_priority_task():<br>    # Task logic for high-priority task<br>    pass<br><br>@app.task<br>def low_priority_task():<br>    # Task logic for low-priority task<br>    pass</pre><p>Now we define the tasks as normal.</p><pre>high_priority_task.apply_async(args, kwargs, routing_key=&#39;high_priority&#39;)</pre><p>Now when we want to apply the task we also give the routing key as an argument.</p><pre>celery -A myapp worker -Q high_priority</pre><p>And we can also attach a worker to a queue to only consume from that queue. this enables us to distribute the resources more efficiently.</p><h3>Conclusion</h3><p>In summary, Celery provides robust features to enhance system design and scalability. We explored Celery Chord, Celery Chain, and Celery Chunks for advanced workflow orchestration and task management. Task Hooks offer control and insight, while Task Routing optimizes resource allocation. By incorporating these features, you can build a scalable and maintainable distributed system tailored to your application’s specific needs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea5ab10e5ca1" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/celery-features-that-can-improve-your-system-design-ea5ab10e5ca1">Celery Features That Can Improve Your System Design</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Glimpse Into AlgoTrading: Making Self-Tuning Trader Bot Using Python]]></title>
            <link>https://medium.com/coinmonks/a-glimpse-into-algotrading-making-self-tuning-trader-bot-using-python-99a5c37483a2?source=rss-ffe8da74e0db------2</link>
            <guid isPermaLink="false">https://medium.com/p/99a5c37483a2</guid>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[trading-bot]]></category>
            <dc:creator><![CDATA[Amirali Omidvar]]></dc:creator>
            <pubDate>Fri, 23 Feb 2024 11:42:34 GMT</pubDate>
            <atom:updated>2024-02-23T11:42:34.972Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="a trading bot visual representation. a robot standing in front of python code." src="https://cdn-images-1.medium.com/max/1024/0*gzs10g_qvYHwU7xq.jpeg" /></figure><h3><strong>Intro</strong></h3><p>The vast expanse of financial markets leads to scalp traders missing a lot of profitable opportunities. Not only humans can only track a limited number of profitable situations, but it’s much harder for us to adapt ourselves fast enough to different market phases. Herein comes the automation. a trading bot not only can survey a much wider angle of the financial market, but it also can tune itself to the everchanging behavior of the market.</p><p>In this article, I will implement a really simple trading bot using Python and differential evolution (I have a story on this here) to create a self-tuning trader bot, but before we start I have to clear some things.</p><h4><strong>What is a Self-Tuning Trading Bot?</strong></h4><p>A self-tuning but does not create a trading strategy on itself. it optimizes it. so let&#39;s imagine you have a trading strategy based on RSI, if it is below 30 you buy, if it is above 70 you sell. A self-tuning bot won&#39;t suddenly decide to use a fundamental approach by analyzing the sentiment of the market, it just tunes the strategy parameters (our two points, 30 and 70). so it might decide to use numbers 27.5 and 68.9.</p><h4><strong>Will the bot be profitable?</strong></h4><p>This depends on the trading strategy. the trading bot ensures that the strategy is using the best parameters, this doesn&#39;t mean profitability.</p><p>Ok with these definitions out of the way let&#39;s start. There are five steps to creating a trading bot:</p><h3><strong>Developing a Trader Bot</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*4E7w35wmhezN7Qsececydw.png" /></figure><h4><strong>1. Retrieving Data</strong></h4><p>For a trading bot to tune itself with the market behavior, it needs the market data. this data can include price or volume data, fundamental data, or sentimental data gathered from social media. this data will be processed into</p><p>For our example, we just need price data to calculate the RSI. many sources offer price data for free. for this example, I’m going to get crypto data from Binance. It’s fast and reliable and you don’t even need a Binance account to get the data from it and also.</p><pre>import pandas as pd<br>from binance.client import Client<br><br># Replace &#39;your_api_key&#39; and &#39;your_api_secret&#39; with your Binance API key and secret<br>api_key = &#39;your_api_key&#39;<br>api_secret = &#39;your_api_secret&#39;<br><br># Initialize the Binance client<br>client = Client(api_key, api_secret)<br><br># Define the number of candles to fetch<br>num_candles = 10000<br><br># Define the symbol pairs for the 50 coins you&#39;re interested in<br>symbol_pairs = [&#39;BTCUSDT&#39;, &#39;ETHUSDT&#39;, &#39;BNBUSDT&#39;, &#39;XRPUSDT&#39;, &#39;LTCUSDT&#39;,  # Add more symbols as needed<br>                &#39;ADAUSDT&#39;, &#39;DOTUSDT&#39;, &#39;LINKUSDT&#39;, &#39;XLMUSDT&#39;, &#39;DOGEUSDT&#39;,  # Add more symbols as needed<br>                &#39;UNIUSDT&#39;, &#39;BCHUSDT&#39;, &#39;SOLUSDT&#39;, &#39;MATICUSDT&#39;, &#39;ICPUSDT&#39;,   # Add more symbols as needed<br>                &#39;AVAXUSDT&#39;, &#39;ETCUSDT&#39;, &#39;FILUSDT&#39;, &#39;TRXUSDT&#39;, &#39;LUNAUSDT&#39;,   # Add more symbols as needed<br>                &#39;VETUSDT&#39;, &#39;AAVEUSDT&#39;, &#39;ATOMUSDT&#39;, &#39;XTZUSDT&#39;, &#39;EOSUSDT&#39;,   # Add more symbols as needed<br>                &#39;MKRUSDT&#39;, &#39;THETAUSDT&#39;, &#39;ALGOUSDT&#39;, &#39;XMRUSDT&#39;, &#39;NEOUSDT&#39;,   # Add more symbols as needed<br>                &#39;DASHUSDT&#39;, &#39;TFUELUSDT&#39;, &#39;SNXUSDT&#39;, &#39;ATOMUSDT&#39;, &#39;GRTUSDT&#39;,  # Add more symbols as needed<br>                &#39;BATUSDT&#39;, &#39;NEARUSDT&#39;, &#39;ENJUSDT&#39;, &#39;DODOUSDT&#39;, &#39;ZRXUSDT&#39;]   # Add more symbols as needed<br><br># Create a dictionary to store DataFrames for each symbol<br>candles_data = {}<br><br># Loop through each symbol pair and fetch candles<br>for symbol_pair in symbol_pairs:<br>    # Fetch candles for the symbol pair<br>    candles = client.get_klines(symbol=symbol_pair, interval=Client.KLINE_INTERVAL_5MINUTE, limit=num_candles)<br>    <br>    # Extract relevant data from candles<br>    candle_data = {&#39;timestamp&#39;: [candle[0] for candle in candles],<br>                   &#39;open&#39;: [float(candle[1]) for candle in candles],<br>                   &#39;high&#39;: [float(candle[2]) for candle in candles],<br>                   &#39;low&#39;: [float(candle[3]) for candle in candles],<br>                   &#39;close&#39;: [float(candle[4]) for candle in candles],<br>                   &#39;volume&#39;: [float(candle[5]) for candle in candles]}<br><br>    # Create a DataFrame for the symbol<br>    symbol_df = pd.DataFrame(candle_data)<br>    <br>    # Convert the timestamp to a readable datetime format<br>    symbol_df[&#39;timestamp&#39;] = pd.to_datetime(symbol_df[&#39;timestamp&#39;], unit=&#39;ms&#39;)<br><br>    # Add the DataFrame to the dictionary<br>    candles_data[symbol_pair] = symbol_df</pre><p>This code uses Binance python package code to get the last 10000 5-minute candles for 50 coins and each of them to pandas data frame.</p><h4><strong>2. Calculations</strong></h4><p>Now that we have the raw data we want, we process them into a form that is usable for our Python bot and trading strategy. in our case, we process raw candle price data into an RSI value. if our strategy needed sentimental data, our raw data would’ve been raw news or social media text, and using different text processing methods we would turn them into usable form as sentiment metrics.</p><pre>import pandas as pd<br>import talib<br><br># Calculate RSI for each symbol in the dictionary<br>for symbol_pair, symbol_df in symbol_data.items():<br>    # Extract the closing prices for the specific symbol<br>    close_prices = symbol_df[&#39;close&#39;].values<br>    <br>    # Calculate RSI using talib<br>    rsi_values = talib.RSI(close_prices, timeperiod=14)<br>    <br>    # Add RSI values to the DataFrame<br>    symbol_df[&#39;rsi&#39;] = rsi_values</pre><p>This Python code uses TALib (A python technical indicator package) to calculate the RSI for each coin data frame.</p><h4><strong>3. Testing</strong></h4><p>Okay, now we are reaching the interesting part. Now we have to create a Python function that tests our trading strategy given different parameters (the RSI strategy would have two parameters, sell RSI and buy RSI). You might be familiar with the term backtesting. this concept means that we can test our trading strategy on historical data in a short time and see how it performs. Of course, the result of the backtesting won&#39;t reflect the real performance of a trading strategy but if done right, it gives us an understanding of strategy behavior.</p><pre>class Trade:<br>    def __init__(self, symbol, entry_price, entry_time):<br>        self.symbol = symbol<br>        self.entry_price = entry_price<br>        self.entry_time = entry_time<br>        self.exit_price = None<br>        self.exit_time = None<br><br><br>def backtest_rsi_strategy(symbol_data, buy_rsi, sell_rsi):<br>    &quot;&quot;&quot;<br>    Backtests an RSI-based trading strategy.<br><br>    Parameters:<br>    - symbol_data: Dictionary of DataFrames where each key is a symbol pair, and the DataFrame<br>                   contains historical price and RSI data.<br>    - buy_rsi: The RSI threshold to trigger a buy signal.<br>    - sell_rsi: The RSI threshold to trigger a sell signal.<br><br>    Returns:<br>    - List of Trade objects representing the executed trades during the backtest.<br>    &quot;&quot;&quot;<br>    trades = []<br><br>    # Iterate over each symbol&#39;s DataFrame<br>    for symbol_pair, symbol_df in symbol_data.items():<br>        in_trade = False<br>        trade = None<br><br>        # Iterate over each candle in the DataFrame<br>        for index, row in symbol_df.iterrows():<br>            if not in_trade:<br>                # Check for a buy signal<br>                if row[&#39;rsi&#39;] &lt; buy_rsi:<br>                    # Enter a trade<br>                    trade = Trade(symbol_pair, row[&#39;close&#39;], row[&#39;timestamp&#39;])<br>                    in_trade = True<br>            else:<br>                # Check for a sell signal or exit condition after 20 candles<br>                if row[&#39;rsi&#39;] &gt; sell_rsi or len(trades[-1].symbol) &gt;= 20:<br>                    # Close the trade<br>                    trades[-1].exit_price = row[&#39;close&#39;]<br>                    trades[-1].exit_time = row[&#39;timestamp&#39;]<br>                    in_trade = False<br><br>        # Close any remaining open trade at the end of the data<br>        if in_trade:<br>            trades[-1].exit_price = symbol_df.iloc[-1][&#39;close&#39;]<br>            trades[-1].exit_time = symbol_df.iloc[-1][&#39;timestamp&#39;]<br><br>    return trades<br><br><br># Example usage:<br># Define buy and sell RSI thresholds<br>buy_rsi_threshold = 30<br>sell_rsi_threshold = 70<br><br># Perform backtesting<br>trades_result = backtest_rsi_strategy(symbol_data, buy_rsi_threshold, sell_rsi_threshold)</pre><p>This Python function simply iterates over the price candles of each crypto coin, at each candle it checks if is already in a trade (for simplicity, I assume the trading bot can only handle a single trade at each given time) using the Trade object. if not it checks the current candle RSI. if it fits into the given buy RSI condition, we enter a trade by buying some imaginary crypto coins. if we are already in a trade and our current RSI value fits in the sell condition, we close the trade and add the trade to a list to analyze them. also, I’ve added a condition to exit the trade after 20 candles to avoid getting stuck in a trade.</p><p>Now we have to measure the success of the trading strategy in the backtest. the simplest way to do this is to calculate the profit it achieved but this is not always the best metric since it doesn&#39;t take into account the risk, exposure, lost potential, etc.</p><pre>def analyze_trades(trades):<br>    &quot;&quot;&quot;<br>    Analyzes the trading results and calculates various performance metrics.<br><br>    Parameters:<br>    - trades: List of Trade objects representing the executed trades.<br><br>    Returns:<br>    - Dictionary containing performance metrics.<br>    &quot;&quot;&quot;<br>    total_profit = 0<br>    total_return = 0<br>    winning_trades = 0<br>    losing_trades = 0<br><br>    for trade in trades:<br>        # Calculate trade profit<br>        trade_profit = (trade.exit_price / trade.entry_price - 1) * 100<br><br>        # Update total profit and return<br>        total_profit += trade_profit<br>        total_return += trade_profit<br><br>        # Update winning and losing trades count<br>        if trade_profit &gt; 0:<br>            winning_trades += 1<br>        else:<br>            losing_trades += 1<br><br>    # Calculate average return per trade<br>    avg_return_per_trade = total_return / len(trades) if len(trades) &gt; 0 else 0<br><br>    # Create a dictionary to store the performance metrics<br>    performance_metrics = {<br>        &#39;total_profit&#39;: total_profit,<br>        &#39;avg_return_per_trade&#39;: avg_return_per_trade,<br>        &#39;winning_trades&#39;: winning_trades,<br>        &#39;losing_trades&#39;: losing_trades<br>    }<br><br>    return performance_metrics<br><br><br># Example usage:<br># Assuming you already have the list of trades (trades_result)<br>performance_metrics_result = analyze_trades(trades_result)</pre><p>This Python code snippet calculates the profit and some other metrics for all the trades. As you can see I’ve taken a certain percentage from each trade as commission of the broker. this is one of the most important steps analyze the trading strategy’s performance.</p><p>Different buy or sell RSIs will result in different profits. although all of them are probably negative less negative is better :) so what is the best pair of parameters for our strategy? to answer this we have to go to the next step.</p><h4><strong>4. Tuning</strong></h4><p>In this step, we use the differential evolution algorithm to find the best parameters. I recommend reading the post I wrote about differential evolution. using test function, the DE algorithm will use this test function to test different parameters and find the best pair.</p><pre># Define the bounds for the parameters you want to optimize<br>parameter_bounds = {<br>    &quot;buy_rsi&quot;: (20, 50),  # Adjust the bounds as needed<br>    &quot;sell_rsi&quot;: (50, 80)   # Adjust the bounds as needed<br>}<br><br># Define the DE population parameters<br>pop_size = 1000<br>direction = Population.MINIMIZE<br>parameters = [&quot;buy_rsi&quot;, &quot;sell_rsi&quot;]<br>generations = 20<br><br># Create the DE population<br>population = Population(<br>    pop_size=pop_size,<br>    direction=direction,<br>    bounds=parameter_bounds,<br>    parameters=parameters<br>)<br>population.initiate()<br><br># Create the Optimization instance<br>optimization = Optimization(<br>    f=0.5,  # Scaling factor<br>    cp=0.7, # Crossover probability<br>    population=population<br>)<br><br># Run the DE algorithm<br>optimization.main(generations)<br><br># After running the DE algorithm, you can retrieve the best individual<br>best_individual = population.get_best_individual()<br>best_parameters = best_individual.parameter_values<br><br># Use the best parameters to evaluate your trading strategy<br># For example:<br>trades_result = backtest_rsi_strategy(symbol_data, best_parameters[&#39;buy_rsi&#39;], best_parameters[&#39;sell_rsi&#39;])<br>performance_metrics = analyze_trades(trades_result)<br><br># Print the best parameters and performance metrics<br>print(&quot;Best Parameters:&quot;, best_parameters)<br>print(&quot;Performance Metrics:&quot;, performance_metrics)</pre><p>This code snippet runs the DE algorithm (<a href="https://github.com/AmiraliOmidvar/de_python/blob/master/de.py">provided here</a>) and gives the best result. As you can see it improved the profit by a lot. But this is an illusion. There is a concept called overfitting in the data analysis field. It means the algorithm has done such a good job at finding the best parameters for data we provided to it that it found parameters that only work on current and might not work on future data. To see the real performance of the trading strategy we have to split data into two parts. Training data and test data. We run the algorithm on the training data to find the best parameters and do a final test to find the profit of the parameters on unseen data.</p><h4><strong>5. Trading</strong></h4><p>Now that we have the trading strategy parameters figured out, we can start trading. many platforms offer API and SDK to enable users to trade on their platform. I&#39;m still gonna use the Binance Python package as an example for now. all we need to trade is to get data, calculate RSI, and decide whether to enter a trade or not.</p><pre>from binance.client import Client<br>import time<br><br># Replace &#39;your_api_key&#39; and &#39;your_api_secret&#39; with your Binance API key and secret<br>api_key = &#39;your_api_key&#39;<br>api_secret = &#39;your_api_secret&#39;<br><br># Initialize the Binance client<br>client = Client(api_key, api_secret)<br><br># Define the trading parameters<br>symbol_pair = &#39;BTCUSDT&#39;  # Replace with the symbol pair you want to trade<br>buy_rsi_threshold = 30  # Use the optimized buy RSI threshold<br>sell_rsi_threshold = 70  # Use the optimized sell RSI threshold<br><br># Function to check if a buy signal is triggered<br>def is_buy_signal():<br>    # Fetch the latest candle data for the symbol<br>    candles = client.get_klines(symbol=symbol_pair, interval=Client.KLINE_INTERVAL_5MINUTE, limit=1)<br><br>    # Extract the closing price and calculate RSI<br>    close_price = float(candles[0][4])<br>    rsi = calculate_rsi(symbol_pair, close_price)<br><br>    # Check if the RSI is below the buy threshold<br>    return rsi &lt; buy_rsi_threshold<br><br># Function to check if a sell signal is triggered<br>def is_sell_signal():<br>    # Fetch the latest candle data for the symbol<br>    candles = client.get_klines(symbol=symbol_pair, interval=Client.KLINE_INTERVAL_5MINUTE, limit=1)<br><br>    # Extract the closing price and calculate RSI<br>    close_price = float(candles[0][4])<br>    rsi = calculate_rsi(symbol_pair, close_price)<br><br>    # Check if the RSI is above the sell threshold<br>    return rsi &gt; sell_rsi_threshold<br><br># Function to execute a buy order<br>def execute_buy_order():<br>    # Replace this with your actual buy order execution logic<br>    print(f&quot;Executing BUY order for {symbol_pair}&quot;)<br><br># Function to execute a sell order<br>def execute_sell_order():<br>    # Replace this with your actual sell order execution logic<br>    print(f&quot;Executing SELL order for {symbol_pair}&quot;)<br><br># Function to calculate RSI for a given symbol and closing price<br>def calculate_rsi(symbol, close_price):<br>    # Replace this with your RSI calculation logic<br>    # You may use the talib library or any other method<br>    # For simplicity, I&#39;ll just return a placeholder value<br>    return 50<br><br># Main trading loop<br>while True:<br>    try:<br>        # Check for buy signal<br>        if is_buy_signal():<br>            execute_buy_order()<br><br>        # Check for sell signal<br>        elif is_sell_signal():<br>            execute_sell_order()<br><br>        # Sleep for a specified interval before checking signals again<br>        time.sleep(300)  # Sleep for 5 minutes (adjust as needed)<br><br>    except Exception as e:<br>        print(f&quot;Error in trading loop: {e}&quot;)<br>        # Handle the error as needed, maybe log it or take corrective actions<br>        time.sleep(60)  # Sleep for 1 minute before retrying</pre><p>This Python code snippet does exactly that. but the market behavior doesn&#39;t stay the same for long. So the trading bot has to update itself periodically. To achieve this we can add a function to the bot to update the bot after a given amount of time. all the trading bot needs to do is run the tuning step again and after it&#39;s updated it starts trading again.</p><pre>import time<br><br># Assume you have the necessary functions and classes defined (backtest_rsi_strategy, analyze_trades, etc.)<br><br>class TradingBot:<br>    def __init__(self, symbol_data):<br>        self.symbol_data = symbol_data<br>        self.update_interval = 86400  # Update every 24 hours (adjust as needed)<br>        self.bot_parameters = self.run_tuning_step()<br><br>    def run_tuning_step(self):<br>        # Define the bounds for the parameters you want to optimize<br>        parameter_bounds = {<br>            &quot;buy_rsi&quot;: (20, 50),  # Adjust the bounds as needed<br>            &quot;sell_rsi&quot;: (50, 80)   # Adjust the bounds as needed<br>        }<br><br>        # Define the DE population parameters<br>        pop_size = 1000<br>        direction = Population.MINIMIZE<br>        parameters = [&quot;buy_rsi&quot;, &quot;sell_rsi&quot;]<br>        generations = 20<br><br>        # Create the DE population<br>        population = Population(<br>            pop_size=pop_size,<br>            direction=direction,<br>            bounds=parameter_bounds,<br>            parameters=parameters<br>        )<br>        population.initiate()<br><br>        # Create the Optimization instance<br>        optimization = Optimization(<br>            f=0.5,  # Scaling factor<br>            cp=0.7,  # Crossover probability<br>            population=population<br>        )<br><br>        # Run the DE algorithm<br>        optimization.main(generations)<br><br>        # After running the DE algorithm, retrieve the best individual (parameters)<br>        best_individual = population.get_best_individual()<br>        return best_individual.parameter_values<br><br>    def trade(self):<br>        while True:<br>            # Get the current time<br>            current_time = time.time()<br><br>            # Run the trading strategy with the current parameters<br>            trades_result = backtest_rsi_strategy(self.symbol_data, self.bot_parameters[&#39;buy_rsi&#39;],<br>                                                  self.bot_parameters[&#39;sell_rsi&#39;])<br>            performance_metrics = analyze_trades(trades_result)<br><br>            # Print or log performance metrics<br>            print(&quot;Performance Metrics:&quot;, performance_metrics)<br><br>            # Sleep until the next update interval<br>            time.sleep(self.update_interval)<br><br>            # Update the trading bot parameters<br>            self.bot_parameters = self.run_tuning_step()<br><br># Example usage:<br># Assuming you have symbol_data defined<br>trading_bot = TradingBot(symbol_data)<br><br># Start the trading bot<br>trading_bot.trade()</pre><h3><strong>Conclusion</strong></h3><p>Although it does not guarantee earning profits, self-tuning can help creative traders up their game by controlling and analyzing many more aspects of the market than a human could. its ability to adapt to market behavior allows traders to not fall behind the market. it takes some time to find a trading style that fits you, but having tools such as a trading bot would make things easier.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=99a5c37483a2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/coinmonks/a-glimpse-into-algotrading-making-self-tuning-trader-bot-using-python-99a5c37483a2">A Glimpse Into AlgoTrading: Making Self-Tuning Trader Bot Using Python</a> was originally published in <a href="https://medium.com/coinmonks">Coinmonks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Full Guide to Implementing Differential Evolution In Python]]></title>
            <link>https://levelup.gitconnected.com/full-guide-to-implementing-differential-evolution-in-python-9fc7fdeccaa8?source=rss-ffe8da74e0db------2</link>
            <guid isPermaLink="false">https://medium.com/p/9fc7fdeccaa8</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[genetic-algorithm]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[Amirali Omidvar]]></dc:creator>
            <pubDate>Tue, 06 Feb 2024 16:01:43 GMT</pubDate>
            <atom:updated>2024-02-20T23:12:12.455Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Differential evolution implemented in python visual." src="https://cdn-images-1.medium.com/max/741/1*IUyUsC1Ukx2r0Do_Guj1bg.png" /></figure><h3>Intro</h3><p>While most people’s attention is turning to machine learning for problem-solving, Metaheuristic algorithms still provide the capability to optimize and solve smaller problems. One subset of these Algorithms is Genetic Algorithms (GA) which simulate a Natural Evolution for some really simple individuals. Most people in the field are familiar with GA, but today I want to talk about Differential Evolution, a similar but more useful sibling to GA, and implement it in Python.</p><h3>Differential Evolution</h3><p>I&#39;m just going to give a simple explanation of DE, and then I&#39;m jumping into coding it in Python.</p><p>Differential Evolution is an algorithm that tries to solve global optimization problems by creating some random individuals, comparing them together with a predefined measurement, keeping the best ones, creating some other individuals by mixing the remaining ones, and repeating the process. I will explain more in coding.</p><h3>Individuals</h3><p>Individuals are the objects that hold the current state of the algorithm. this information includes the parameters and score.</p><p>Suppose we want to minimize x^2 + y^2 function. each individual would have parameters : (x,y) and score of x^2 + y^2 .</p><p>so let&#39;s create an object for individuals in Python.</p><pre>from typing import List, Dict, Tuple, Union<br><br>class Individual:<br>    def __init__(<br>            self,<br>            parameters: Dict[str, float],<br>            score: Union[float, None] = None<br>    ) -&gt; None:<br>        super().__init__()<br>        self.parameter_values = parameters<br>        self.score = score</pre><p>This object holds the parameters and scores for a single individual. an individual with parameters with x=2, y=3 would have</p><pre>parameter_value = {&quot;x&quot;:2, &quot;y&quot;:3}<br>score = 13</pre><h3>Scoring</h3><p>Since we want to compare individuals to each other, we need to have a measurement and what makes an individual better. we define an individual as better when it&#39;s closer to the optimized point, or in other words as a higher score.</p><p>for our scenario, a simple Python function like below would be enough for scoring.</p><pre>def scoring(ind: Individual) -&gt; float:<br>    return ind.parameter_values[&quot;x&quot;] ** 2 + ind.parameter_values[&quot;y&quot;] ** 2</pre><h3>Population</h3><p>To take individuals closer we need to create a population of them and evolve them. this is the fundamental process of GAs.</p><pre>class Population:<br>    MAXIMIZE = 1<br>    MINIMIZE = -1<br><br>    def __init__(<br>            self,<br>            pop_size,<br>            direction: int,<br>            bounds: Dict[str, Tuple[float, float]],<br>            parameters: List[str],<br><br>    ) -&gt; None:<br>        self.pop_size = pop_size<br>        self.direction = direction<br>        self.bounds = bounds<br>        self.parameters = parameters<br>        self.individuals: List[Individual] = []<br><br>    def _create_individual(self) -&gt; Individual:<br>        dict_parameters = {}<br>        for parameter in self.parameters:<br>            param_value = random.uniform(self.bounds[parameter][0], self.bounds[parameter][1])<br>            dict_parameters[parameter] = param_value<br>        individual = Individual(parameters=dict_parameters)<br>        individual.score = scoring(individual)<br>        return individual<br><br>    def initiate(self):<br>        for i in range(self.pop_size):<br>            individual = self._create_individual()<br>            self.individuals.append(individual)<br><br>    def get_best_individual(self) -&gt; Individual:<br>        self.best_individual = max(self.individuals, key=lambda x: x.score * self.direction)<br>        return self.best_individual<br><br>    def get_worst_individual(self) -&gt; Individual:<br>        self.best_individual = min(self.individuals, key=lambda x: x.score * self.direction)<br>        return self.best_individual<br><br>    def get_mean_score(self) -&gt; float:<br>        self.mean_score = sum([ind.score for ind in self.individuals]) / len(self.individuals)<br>        return self.mean_score<br><br>    def get_std_score(self) -&gt; float:<br>        self.std_score = sum([(ind.score - self.mean_score) ** 2 for ind in self.individuals]) / len(self.individuals)<br>        return self.std_score</pre><p>This is an object that holds the population of individuals for us. let&#39;s go through its functionality step by step.</p><h4>Arguments</h4><ol><li>Pop Size: Number of individuals in our population. I will explain how this number can affect the optimization process later on.</li><li>Direction: Maximize or Minimize.</li><li>Bounds: What are the boundaries of the parameters? for our example, I will assume both x,y have range of [-100, 100] .</li><li>Parameters: List of parameters</li></ol><h4>Methods</h4><p>There are four methods for monitoring the population, that gets the mean, standard variation of scores, and also best and worst individuals.</p><p>Two other methods are four initiating and populating the first individuals. To create an individual we make random parameters in the range of boundaries and calculate the score for it.</p><h3>Optimization</h3><p>Now that we have the initial population we can begin to optimize it. the optimization is iteration-based, meaning each iteration will do the same process until the population reaches where we want it.</p><p>Here are the steps for each iteration.</p><h4>Mutation</h4><p>Like natural selection in the real world, the evolution progress happens with random mutations. we choose three semi-parents from the population randomly and combine their parameters to create a trial individual.</p><p>We use this equation to create the parameters for the trial individual: trial = semi_parent1 + F * (semi_parent2-semi_parent3)</p><p>In this equation F if is a scaling factor that can affect the process of the optimization which I explain later on. for now, assume F is 0.5. For example, if parent1,parent2, and parent3, have x parameters of 4,4,3. The trial x parameter would be 4.5.</p><p>After calculating the parameter, we have to make sure it&#39;s within the bounds of the parameters we defined, if it&#39;s bigger we simply change its value to the upper bound and do the same for the lower band.</p><p>This Python code snippet handles the mutation functionality.</p><pre> class Optimization:<br>    def __init__(self, f: float, cp: float, population: Population):<br>        self.f = f<br>        self.cp = cp<br>        self.population = population<br><br>    @staticmethod<br>    def _mutant_choose(population: Population) -&gt; Tuple[Individual, Individual, Individual]:<br>        # chooses 3 random individuals<br>        individuals = population.individuals<br>        random_individuals = random.sample(individuals, 3)<br>        return tuple(random_individuals)<br><br>    @staticmethod<br>    def _handle_boundaries(parameter_value: float, bounds: Tuple[float, float]) -&gt; float:<br>        # handles the bounds in case calculated parameter goes beyond bounds<br>        if parameter_value &gt; bounds[1]:<br>            parameter_value = bounds[1]<br>        elif parameter_value &lt; bounds[0]:<br>            parameter_value = bounds[0]<br>        return parameter_value<br><br>    @staticmethod<br>    def _calculate_parameter(<br>            semi_parents: Tuple[Individual, Individual, Individual],<br>            parameter_name: str,<br>            bounds: Tuple[float, float],<br>            f: float<br>    ) -&gt; float:<br>        # calculates the parameter for trial individual based on semi parents<br>        trial_parameter = semi_parents[0].parameter_values[parameter_name] + \<br>                          f * (semi_parents[1].parameter_values[parameter_name] -<br>                               semi_parents[2].parameter_values[parameter_name])<br>        trial_parameter = Optimization._handle_boundaries(trial_parameter, bounds)<br>        return trial_parameter<br><br>    def _mutation(self, population: Population) -&gt; Individual:<br>        # create the trial individual<br>        # choose semi parents<br>        semi_parents = Optimization._mutant_choose(population)<br>        trial_parameters = {}<br>        # for each parameter name<br>        for parameter in population.parameters:<br>            # calculate parameter for trail individual<br>            trial_parameter = self._calculate_parameter(<br>                semi_parents, parameter,<br>                population.bounds[parameter],<br>                self.f<br>            )<br>            trial_parameters[parameter] = trial_parameter<br>        # create trial individual<br>        trial_individual = Individual(parameters=trial_parameters)<br>        return trial_individual</pre><h4>Crossover</h4><p>After we create the trial individual, we choose the actual parent and combine the trial and the parent parameters, this combination process is different from the mutation. let’s define a paramter as cp (crossover probability) it shows the probability that the child parameter would get its parameter value from the parent, otherwise, it would inherit the parameter from the trial individual, so the child would be a mix of parent and trial.</p><p>Here’s the Python code to handle the Crossover functionality.</p><pre>def _crossover(self, parent: Individual, trial: Individual, parameters):<br>    child_parameters = {}<br>    # choose the parameters<br>    for parameter in parameters:<br>        prob = random.random()<br>        if prob &lt; self.cp:<br>            child_parameters[parameter] = parent.parameter_values[parameter]<br>        else:<br>            child_parameters[parameter] = trial.parameter_values[parameter]<br>    # create child individual<br>    child = Individual(parameters=child_parameters)<br>    return child</pre><h4>Selection</h4><p>Another principle of natural selection is the survival of the fittest, and we have to simulate this principle into our algorithm. As I talked about before, how we define which individual is better or fitter is their score. by creating a score comparison between the child and its parent and keeping the better individual, we simulate the natural selection in the DE algorithm with this Pytho function.</p><pre>@staticmethod<br>def _selection(child: Individual, parent: Individual, direction: int) -&gt; bool:<br>    child.score = scoring(child)<br>    # preferring child to parent in case of equal scores<br>    if direction == Population.MAXIMIZE:<br>        return child.score &gt;= parent.score<br>    else:<br>        return child.score &lt;= parent.score</pre><h4>Optimization Process</h4><p>Now we have all the pieces to start optimizing our population. by repeating these steps: Mutation-Crossover-Selection for each iteration, we get closer and closer to an optimized point.</p><pre>def main(self, generations: int):<br>    population = self.population<br><br>    for gen in range(generations):<br>        new_individuals = []<br><br>        for i in range(population.pop_size):<br>            # Mutation<br>            trial_individual = self._mutation(population)<br><br>            # Crossover<br>            parent = population.individuals[i]<br>            child = self._crossover(parent, trial_individual, population.parameters)<br><br>            # Selection<br>            if self._selection(child, parent, self.population.direction):<br>                new_individuals.append(child)<br>            else:<br>                new_individuals.append(parent)<br><br>        # Update the population with the new individuals<br>        population.individuals = new_individuals<br><br>        # Update statistics or perform any other necessary tasks at the end of each generation<br>        best_individual = population.get_best_individual()<br>        worst_individual = population.get_worst_individual()<br>        mean_score = population.get_mean_score()<br>        std_score = population.get_std_score()<br><br>        # Print or store relevant information about the generation<br>        print(<br>            f&quot;Generation {gen + 1}: Best Score - {best_individual.score}, Worst Score - {worst_individual.score}, Mean Score - {mean_score}, Std Score - {std_score}&quot;)<br><br>    # After completing all generations, you can return or perform any final actions<br>    final_best_individual = population.get_best_individual()<br>    print(<br>        f&quot;Optimization complete. Best individual: {final_best_individual.parameter_values}, Score: {final_best_individual.score}&quot;)</pre><p>The generation number represents how many times the population goes through evolution.</p><p>running the optimization process with the Python code below:</p><pre>population = Population(<br>    pop_size=50,<br>    direction=Population.MINIMIZE,<br>    bounds={&quot;x&quot;: (-100, 100), &quot;y&quot;: (-100, 100)},<br>    parameters=[&quot;x&quot;, &quot;y&quot;]<br>)<br>population.initiate()<br><br>optimization = Optimization(<br>    f=0.5,<br>    cp=0.7,<br>    population=population<br>)<br><br>optimization.main(20)</pre><p>we get this result:</p><pre>Generation 1: Best Score - 227.4416252070246, Worst Score - 18497.300895123102, Mean Score - 5761.345579501305, Std Score - 16915729.974560272<br>Generation 2: Best Score - 227.4416252070246, Worst Score - 14545.297070243176, Mean Score - 5010.059755381193, Std Score - 12978175.931191077<br>Generation 3: Best Score - 227.4416252070246, Worst Score - 14545.297070243176, Mean Score - 4473.585301514974, Std Score - 11216682.724672504<br>Generation 4: Best Score - 3.4385946097222364, Worst Score - 14545.297070243176, Mean Score - 3950.3945813343807, Std Score - 11242005.143788697<br>Generation 5: Best Score - 3.4385946097222364, Worst Score - 14545.297070243176, Mean Score - 3369.7055827463582, Std Score - 10143067.833940094<br>Generation 6: Best Score - 3.4385946097222364, Worst Score - 14545.297070243176, Mean Score - 2871.3199498197205, Std Score - 9194412.495692344<br>Generation 7: Best Score - 3.4385946097222364, Worst Score - 9949.198425181936, Mean Score - 2384.9214875265598, Std Score - 6514105.135908899<br>Generation 8: Best Score - 1.6733434770501088, Worst Score - 9949.198425181936, Mean Score - 2071.6357418586012, Std Score - 5399238.725073254<br>Generation 9: Best Score - 1.6733434770501088, Worst Score - 8260.637081944824, Mean Score - 1596.8035021308708, Std Score - 3469563.4938868876<br>Generation 10: Best Score - 1.6733434770501088, Worst Score - 8260.637081944824, Mean Score - 1403.866997742772, Std Score - 2450540.443889007<br>Generation 11: Best Score - 1.6733434770501088, Worst Score - 5113.94166108131, Mean Score - 1246.1031813288662, Std Score - 1765757.2773411989<br>Generation 12: Best Score - 1.6733434770501088, Worst Score - 4919.398639971296, Mean Score - 1064.4393893023082, Std Score - 1385279.1173340352<br>Generation 13: Best Score - 1.6733434770501088, Worst Score - 4919.398639971296, Mean Score - 926.0436566291788, Std Score - 1341259.57531087<br>Generation 14: Best Score - 1.6733434770501088, Worst Score - 4919.398639971296, Mean Score - 721.2504885162537, Std Score - 1070756.965907924<br>Generation 15: Best Score - 1.6733434770501088, Worst Score - 4919.398639971296, Mean Score - 659.7735826255573, Std Score - 934308.2425962642<br>Generation 16: Best Score - 1.1420718073236673, Worst Score - 2956.582262433441, Mean Score - 450.1419729517197, Std Score - 364084.9567184075<br>Generation 17: Best Score - 1.1420718073236673, Worst Score - 1668.3179220396407, Mean Score - 304.0106624909399, Std Score - 161652.01459746764<br>Generation 18: Best Score - 1.1420718073236673, Worst Score - 1668.3179220396407, Mean Score - 230.8485798028454, Std Score - 115926.7643605015<br>Generation 19: Best Score - 1.1420718073236673, Worst Score - 1668.3179220396407, Mean Score - 205.84797668815438, Std Score - 103705.73859060678<br>Generation 20: Best Score - 1.1420718073236673, Worst Score - 1668.3179220396407, Mean Score - 165.35446095979322, Std Score - 75340.32141876635<br>Generation 21: Best Score - 1.1420718073236673, Worst Score - 1668.3179220396407, Mean Score - 146.28945048328515, Std Score - 72251.1327179642<br>Generation 22: Best Score - 1.1420718073236673, Worst Score - 1668.3179220396407, Mean Score - 124.87657983687312, Std Score - 70563.04433179043<br>Generation 23: Best Score - 1.1420718073236673, Worst Score - 818.9781293511609, Mean Score - 78.24691009188221, Std Score - 18528.66145307911<br>Generation 24: Best Score - 1.1420718073236673, Worst Score - 655.096226749938, Mean Score - 57.96478952496863, Std Score - 11726.46753692781<br>Generation 25: Best Score - 1.1420718073236673, Worst Score - 655.096226749938, Mean Score - 54.745142638814215, Std Score - 11608.354314664644<br>Generation 26: Best Score - 0.28816246385958383, Worst Score - 655.096226749938, Mean Score - 49.9874913375001, Std Score - 11626.132202770172<br>Generation 27: Best Score - 0.28816246385958383, Worst Score - 371.2961546593531, Mean Score - 34.86479386647983, Std Score - 3934.8457782976016<br>Generation 28: Best Score - 0.28816246385958383, Worst Score - 371.2961546593531, Mean Score - 33.71715629036727, Std Score - 3960.483702630012<br>Generation 29: Best Score - 0.050228488153683026, Worst Score - 371.2961546593531, Mean Score - 28.987869849411986, Std Score - 3551.5620086912777<br>Generation 30: Best Score - 0.050228488153683026, Worst Score - 371.2961546593531, Mean Score - 26.31746339997663, Std Score - 3419.355860737079<br>Generation 31: Best Score - 0.050228488153683026, Worst Score - 371.2961546593531, Mean Score - 21.173148385457285, Std Score - 3029.142999591831<br>Generation 32: Best Score - 0.050228488153683026, Worst Score - 371.2961546593531, Mean Score - 19.210748561457013, Std Score - 2989.191829209833<br>Generation 33: Best Score - 0.050228488153683026, Worst Score - 371.2961546593531, Mean Score - 17.723684516222406, Std Score - 2980.329214223267<br>Generation 34: Best Score - 0.050228488153683026, Worst Score - 363.399480151379, Mean Score - 17.202092127877634, Std Score - 2878.827551971876<br>Generation 35: Best Score - 0.050228488153683026, Worst Score - 363.399480151379, Mean Score - 13.51589268666954, Std Score - 2668.982923099746<br>Generation 36: Best Score - 0.0007007720750009951, Worst Score - 31.96289847175133, Mean Score - 4.440223546680103, Std Score - 40.71837774279357<br>Generation 37: Best Score - 0.0007007720750009951, Worst Score - 31.96289847175133, Mean Score - 4.244486633964991, Std Score - 36.72586873553191<br>Generation 38: Best Score - 0.0007007720750009951, Worst Score - 31.96289847175133, Mean Score - 3.666887762954041, Std Score - 35.61327804909445<br>Generation 39: Best Score - 0.0007007720750009951, Worst Score - 31.96289847175133, Mean Score - 2.9835077535822436, Std Score - 30.459759881542368<br>Generation 40: Best Score - 0.0007007720750009951, Worst Score - 31.96289847175133, Mean Score - 2.538531720605209, Std Score - 28.91712049847726<br>Generation 41: Best Score - 0.0007007720750009951, Worst Score - 31.96289847175133, Mean Score - 2.146804878714528, Std Score - 25.668607634274952<br>Generation 42: Best Score - 0.0007007720750009951, Worst Score - 17.781224823935247, Mean Score - 1.3544183225874413, Std Score - 7.5969355485482915<br>Generation 43: Best Score - 0.0007007720750009951, Worst Score - 12.834997664758449, Mean Score - 1.1400443570021603, Std Score - 4.628625559570187<br>Generation 44: Best Score - 0.0007007720750009951, Worst Score - 12.684122684361961, Mean Score - 0.9686790213034325, Std Score - 4.495668337986387<br>Generation 45: Best Score - 0.0007007720750009951, Worst Score - 12.684122684361961, Mean Score - 0.7947732376732511, Std Score - 3.6800436454094974<br>Generation 46: Best Score - 0.0007007720750009951, Worst Score - 3.5640726476803826, Mean Score - 0.40080370174745267, Std Score - 0.43629980156007536<br>Generation 47: Best Score - 0.0007007720750009951, Worst Score - 3.5640726476803826, Mean Score - 0.2899601479812607, Std Score - 0.26633009670254837<br>Generation 48: Best Score - 0.0001428228544170628, Worst Score - 0.8784379562783969, Mean Score - 0.19375082120496742, Std Score - 0.04701712878984084<br>Generation 49: Best Score - 0.0001428228544170628, Worst Score - 0.7484442352921914, Mean Score - 0.15595454588228222, Std Score - 0.03200110310816078<br>Generation 50: Best Score - 0.0001428228544170628, Worst Score - 0.5970257508310275, Mean Score - 0.12252512381569314, Std Score - 0.017435255963598142<br>Optimization complete. Best individual: {&#39;x&#39;: 0.011544641583982218, &#39;y&#39;: 0.003089353510761006}, Score: 0.0001428228544170628</pre><h3>Parameters</h3><p>The are multiple parameters in the optimization process that affect how the algorithm behaves:</p><ol><li>Pop size: This parameter defines how many individuals are in a population. the higher the number, the more diversity is in the population. for example with a population of 300 in our example, with 20 generations we reach almost the same best score of the 50 generation with 50 pop size.</li></ol><pre>Generation 1: Best Score - 0.2892874791176888, Worst Score - 18914.374098406344, Mean Score - 5689.310533336302, Std Score - 14894133.237002514<br>Generation 2: Best Score - 0.2892874791176888, Worst Score - 18914.374098406344, Mean Score - 4922.99633622035, Std Score - 13106964.003723977<br>Generation 3: Best Score - 0.2892874791176888, Worst Score - 17219.779229774478, Mean Score - 4294.081579241987, Std Score - 12087284.639528418<br>Generation 4: Best Score - 0.2892874791176888, Worst Score - 17219.779229774478, Mean Score - 3621.0222832553095, Std Score - 10344623.546601914<br>Generation 5: Best Score - 0.2892874791176888, Worst Score - 17219.779229774478, Mean Score - 3133.692751757101, Std Score - 9062755.39869737<br>Generation 6: Best Score - 0.2892874791176888, Worst Score - 17219.779229774478, Mean Score - 2668.6483147782137, Std Score - 7288448.517254785<br>Generation 7: Best Score - 0.2892874791176888, Worst Score - 17219.779229774478, Mean Score - 2329.1488967865716, Std Score - 6345571.25118263<br>Generation 8: Best Score - 0.2892874791176888, Worst Score - 17219.779229774478, Mean Score - 1926.3816837858058, Std Score - 4927659.771709917<br>Generation 9: Best Score - 0.2892874791176888, Worst Score - 17219.779229774478, Mean Score - 1584.015511200321, Std Score - 4048322.516441089<br>Generation 10: Best Score - 0.2892874791176888, Worst Score - 10361.769549957036, Mean Score - 1292.1005154701083, Std Score - 2891337.605140347<br>Generation 11: Best Score - 0.2892874791176888, Worst Score - 10361.769549957036, Mean Score - 1080.8838899262594, Std Score - 2330956.7240146315<br>Generation 12: Best Score - 0.2892874791176888, Worst Score - 10361.769549957036, Mean Score - 886.2253253651165, Std Score - 1840761.8707774358<br>Generation 13: Best Score - 0.2892874791176888, Worst Score - 10361.769549957036, Mean Score - 721.5118327570327, Std Score - 1453387.5194910818<br>Generation 14: Best Score - 0.06669605475338797, Worst Score - 10001.504898512676, Mean Score - 576.636486394219, Std Score - 978365.5759541079<br>Generation 15: Best Score - 0.06669605475338797, Worst Score - 10001.504898512676, Mean Score - 448.73368507382327, Std Score - 613672.8675742977<br>Generation 16: Best Score - 0.06669605475338797, Worst Score - 10001.504898512676, Mean Score - 366.54482538790626, Std Score - 439610.1843520182<br>Generation 17: Best Score - 0.06669605475338797, Worst Score - 5838.05492245703, Mean Score - 295.1125814959884, Std Score - 273950.97557010216<br>Generation 18: Best Score - 0.005650696311271542, Worst Score - 5838.05492245703, Mean Score - 237.7494031568629, Std Score - 197875.78725714132<br>Generation 19: Best Score - 0.005650696311271542, Worst Score - 5838.05492245703, Mean Score - 192.78271137807434, Std Score - 171609.95485872566<br>Generation 20: Best Score - 0.005650696311271542, Worst Score - 5838.05492245703, Mean Score - 161.17070891878603, Std Score - 142303.14898040536<br>Optimization complete. Best individual: {&#39;x&#39;: 0.03549696454744389, &#39;y&#39;: -0.06626206923413314}, Score: 0.005650696311271542</pre><p>ass you can see even though the best scores are close, but Mean and Std of scores are much higher meaning a more diverse population. the diversity can help avoid <strong>Local Optimized Points </strong>which I will explain later on. but it comes with a performance cost.</p><p>2. F parameter: The F parameter dictates how big are parameter changes, if F is too low, the individuals are closer to their generation and don&#39;t move much, and vice versa. what the optimal number of F value should be is reliant on the type of the problem but 0.5 in my experience is a good starting point.</p><p>3. CP: the probability that the child inherits from the parent and not the trial individual. If CP is high, children will be more similar to their parents and the population will move slower and vice versa.</p><h3>Limitations</h3><p>As versatile Differential Evolution Algorithm is, it&#39;s not without its shortcomings. the most apparent of these limitations is the Local Optima problem that many metaheuristic algorithms suffer from. the DE algorithm or any genetic algorithm cannot guarantee that the best parameters they find are the best solution. Think of a deep valley, that our population wants to move through, the population wants to lower ground, so they all might just go in the valley in get stuck there while there is a deeper valley somewhere else. changing the parameters to make it more diverse and taking bigger steps around solution space can help but comes with performance costs.</p><h3>Conclusion</h3><p>In this guide, we covered the core components of the Differential Evolution algorithm and its Python implementation. The example that we went through and the Python code provided showed the simple process of optimization with this algorithm, its advantages, and its shortcomings. if you have any questions please leave a comment.</p><p>Here’s the repo <a href="https://github.com/AmiraliOmidvar/de_python">link</a> for the full code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9fc7fdeccaa8" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/full-guide-to-implementing-differential-evolution-in-python-9fc7fdeccaa8">Full Guide to Implementing Differential Evolution In Python</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dive Deeper into Redis | Five Underutilized Redis Features]]></title>
            <link>https://medium.com/@amirali.omidvar80/dive-deeper-into-redis-five-underutilized-redis-features-4789eeb6d05d?source=rss-ffe8da74e0db------2</link>
            <guid isPermaLink="false">https://medium.com/p/4789eeb6d05d</guid>
            <category><![CDATA[redis]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[databas]]></category>
            <category><![CDATA[performance]]></category>
            <dc:creator><![CDATA[Amirali Omidvar]]></dc:creator>
            <pubDate>Thu, 01 Feb 2024 05:21:52 GMT</pubDate>
            <atom:updated>2024-02-01T06:15:22.594Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*aJDKlWI7VxQmWZNynOIV_Q.png" /></figure><h3>Intro</h3><p>In my last projects, I’ve had a lot of run-ins with Redis. From implementing distributed systems to creating high-performance price caches. Throughout all of that, I mostly used some basic and important features of Redis like Sorted Sets, Lists, Pipelining, etc. but sometimes I needed a really specific functionality and I was surprised that Redis provides it and I didn’t know about it until now. so I decided to share these 5 features that Redis provides, which proved to be game-changers in my projects.</p><h3>Lua Scripts</h3><p>One of the struggles I face every day is Python’s low performance. although packages like Numpy and Pandas can speed things up with vectorized operations, sometimes there are better solutions, for me, one of them was Lua Scripting in Redis. Not only we can do multiple operations on data on Redis in a single ATOMIC command, but we can also put the calculation burden on the Redis server side itself, which can be faster in some cases.</p><p>Let’s assume a scenario where you have multiple Sorted Sets for storing candles with timestamps as a score. (if you don’t know what are candles don’t worry it’s not important, just imagine a JSON with four floating number fields and timestamp as a score of the set). Here’s what it looks like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*8BM31alPwPJPOQtkGGNKHw.png" /></figure><p>There is a sorted set for every symbol. let’s say we have three symbols: symbol1, symbol2, and symbol3, and we want to get to get from timestamp start_timestampto end_timestamp, for all symbols, and also do some operations and the candle value.</p><pre>def retrieve_candles_in_range(start_time, end_time, redis_host=&#39;localhost&#39;, redis_port=6379):<br>    # Connect to Redis<br>    redis_client = redis.Redis(host=redis_host, port=redis_port, db=0)<br>    # Define the sorted set keys<br>    sorted_set_keys = [&#39;symbol1&#39;, &#39;symbol2&#39;, &#39;symbol3&#39;]<br>    # Define the Lua script that retrieves the candles<br>    lua = &quot;&quot;&quot;<br>    local results = {}<br>    for _, key in ipairs(KEYS) do<br>        local candles = redis.call(&#39;ZRANGEBYSCORE&#39;, key, ARGV[1], ARGV[2])<br>        for i, candle in ipairs(candles) do<br>            local candle_table = cjson.decode(candle)<br>            local scale = (candle_table[&#39;high&#39;] + candle_table[&#39;low&#39;] + candle_table[&#39;close&#39;]) / 3<br>            candle_table[&#39;open&#39;] = candle_table[&#39;open&#39;] / scale<br>            candle_table[&#39;high&#39;] = candle_table[&#39;high&#39;] / scale<br>            candle_table[&#39;low&#39;] = candle_table[&#39;low&#39;] / scale<br>            candle_table[&#39;close&#39;] = candle_table[&#39;close&#39;] / scale<br>            candles[i] = cjson.encode(candle_table)<br>        end<br>        table.insert(results, candles)<br>    end<br>    return results<br>    &quot;&quot;&quot;<br>    # Register the Lua script in redis client<br>    script = redis_client.register_script(lua)<br>    # Run the Lua script, passing the sorted set keys as the keys and the start and end times as the arguments<br>    results = script(keys=sorted_set_keys, args=[start_time, end_time])<br>    # Convert the results from bytes to strings and map them to their corresponding symbols<br>    results = {sorted_set_keys[i]: [candle.decode(&#39;utf-8&#39;) for candle in symbol_candles] for i, symbol_candles in enumerate(results)}<br>    return results</pre><p>This Python code connects to Redis and runs a Lua script that does the functionality we want, and returns the results. let’s inspect the Lua script.</p><pre>--create a variable to store results<br>local results = {}<br>--iterate over given keys<br>for _, key in ipairs(KEYS) do<br>    --call the redis command to get the needed data, ARGVs are given in python code<br>    local candles = redis.call(&#39;ZRANGEBYSCORE&#39;, key, ARGV[1], ARGV[2])<br>    for i, candle in ipairs(candles) do<br>        --iterate over candles, deserialize candles<br>        --and do the desired operaions<br>        local candle_table = cjson.decode(candle)<br>        local scale = (candle_table[&#39;high&#39;] + candle_table[&#39;low&#39;] + candle_table[&#39;close&#39;]) / 3<br>        candle_table[&#39;open&#39;] = candle_table[&#39;open&#39;] / scale<br>        candle_table[&#39;high&#39;] = candle_table[&#39;high&#39;] / scale<br>        candle_table[&#39;low&#39;] = candle_table[&#39;low&#39;] / scale<br>        candle_table[&#39;close&#39;] = candle_table[&#39;close&#39;] / scale<br>        --serialize the candle and add to result candles<br>        candles[i] = cjson.encode(candle_table)<br>    end<br>    --add the candles of the symbol to result<br>    table.insert(results, candles)<br>end<br>--return result<br>return results</pre><p>The Lua script calls the redis command, iterates over the results does some arbitrary operations on them, and puts the result in the same format as it came. The result is given as we want it and we don’t have to worry about decoding, formatting, pipelining, etc.</p><h3>Geospatial Indexing</h3><p>One of the rarely used features of the redis is Geospatial indexing. it’s a powerful tool for storing geolocation data and efficiently querying them from the database. Here is the basic implementation of this feature:</p><h4>Adding a Location</h4><pre>GEOADD user_locations 19.161989 79.229812 &quot;user1&quot;</pre><p>The GEOADDcommand adds a new location to a collection of locations with the key user_locations. The numbers are the longitude and latitude of the location we are storing and “user1” is the key.</p><h4>Retrieving Coordinates</h4><pre>GEOPOS user_locations user1<br><br>Output:<br>1) 1) &quot;19.161989&quot;<br>   2) &quot;79.229812&quot;</pre><p>The GEOPOS command returns the coordinates of a key in the collection.</p><h4>Find Distance</h4><pre>GEODIST user_locations user1 user2</pre><p>With the GEODISTcommand, we can find the distance between two keys.</p><h4>Find Keys In a Given Radius From a Point</h4><pre>GEORADIUS user_locations 13 79 200 km<br><br>output:<br>1) &quot;user1&quot;</pre><p>Last but not least is GEORADIUS. this command allows us to find the keys that are a certain distance from a given point. It is insanely efficient and enables us to find users, locations, etc in a given radius easily.</p><p>The GEORADIUS the command came to my aid when I wanted to send a notification for a location-based event to users nearby. Since geoindexing in the Redis database is implemented with <a href="https://medium.com/outer-labs/spatial-relationships-with-z-order-curves-6b1175d1ad9c">Z-order curve</a> it is extremely efficient in querying coordinates data.</p><h3>Keyspace Notifications</h3><p>Keyspace notifications tool provides a powerful mechanism to overview and monitor all changes being applied to keys in a Redis database. By enabling keyspace notifications, we will receive notifications for specific changes related to keyspace.</p><p>I think a demonstration is the best way to introduce this tool. first, we need to enable keyspace notifications.</p><pre>CONFIG SET notify-keyspace-events &quot;KEA&quot;</pre><p>This command enables the keyspace notification with the KEA flag. we’re subscribing to events related to keyspace changes (K), keyevents (E).</p><pre>K     Keyspace events, published with __keyspace@&lt;db&gt;__ prefix.<br>E     Keyevent events, published with __keyevent@&lt;db&gt;__ prefix.<br>g     Generic commands (non-type specific) like DEL, EXPIRE, RENAME, ...<br>$     String commands<br>l     List commands<br>s     Set commands<br>h     Hash commands<br>z     Sorted set commands<br>t     Stream commands<br>d     Module key type events<br>x     Expired events (events generated every time a key expires)<br>e     Evicted events (events generated when a key is evicted for maxmemory)<br>m     Key miss events (events generated when a key that doesn&#39;t exist is accessed)<br>n     New key events (Note: not included in the &#39;A&#39; class)<br>A     Alias for &quot;g$lshztxed&quot;, so that the &quot;AKE&quot; string means all the events except &quot;m&quot; and &quot;n&quot;.</pre><p>Here is the full list of flags provided by Redis.</p><p>There are two types of notifications we can receive. <strong>Keyspace</strong> and <strong>Keyevent. </strong>the Keyspace events are related to the keys, for example, let’s say we set a key named key1with 2 second expiry. after two seconds, two <a href="https://redis.io/docs/interact/pubsub/">pubsub</a> messages will be sent in two channel patterns:</p><pre>1) channel: &quot;__keyspace@0__:key1&quot; value: &quot;expired&quot;<br>2) channel: &quot;__keyevent@0__:expired&quot; value &quot;key1&quot;</pre><p>The two-channel patterns give distinct information to the __keyspace__ pattern channel returns events that happened to keys. so for example by <a href="https://redis.io/commands/psubscribe/"><strong>PSUBSCRIBE</strong></a> to __keyspace we get every event that happens on all keys. (@0 means database 0 or Redis instance)</p><p>On the other hand, by <a href="https://redis.io/commands/subscribe/"><strong>SUBSCRIBE</strong></a> to __keyevent@0__:expired we get names of the keys that went through expiration.</p><p>To get every event that happens in Keyspace, we can simply <strong>PSUBSCRIBE</strong> to __key*</p><h3>Bitfield</h3><p>Bitfields in Redis provide a compact and efficient way to store and manipulate sequences of bits. It allows you to perform atomic bitwise operations on binary data, making it useful for applications that require compact data representation, such as flags, permissions, or any scenario where individual bits need to be manipulated.</p><h4>Setting Bits</h4><p>To get started with Bit we need to set a bit key. here’s how to do it.</p><pre>SETBIT permissions 10 1</pre><p>This command sets the bit at position 10 to “1”.</p><h4>Getting Bits</h4><pre>GETBIT permissions 10</pre><p>With the <strong>GETBIT</strong> command, we can get a bit value at a given position.</p><h4>Bitwise operations</h4><pre>BITOP AND result_key key1 key2<br>BITOP OR result_key key1 key2<br>BITOP XOR result_key key1 key2<br>BITNOT result_key key</pre><p>These commands allow you to perform bitwise AND, OR, XOR, and NOT operations on the specified keys, storing the result in a new key.</p><h3>Streams</h3><p>Redis Streams provide a log-like data structure that allows you to model and process real-time data. It is particularly useful for scenarios where you need to handle event streams or task queues in a scalable way. Here’s a basic implementation of Redis Streams.</p><p>Suppose we want to capture temperature data from multiple sensors and process this data in a distributed architecture in multiple machines (Consumers of data). Redis makes this really with the Stream feature:</p><h4>Creating a Stream</h4><pre>XADD stream * field1 value1 field2 value2</pre><p>This command adds a new entry to the stream “stream” with specified fields and values. The * indicates that a new entry should be appended to the stream.</p><pre>XADD stream_temp * sensor_id 123 temperature 25.5<br>XADD stream_temp * sensor_id 456 temperature 28.2<br>XADD stream_temp * sensor_id 789 temperature 22.8</pre><p>This command adds three readings from three sensors into the stream.</p><h4>Reading from a Stream</h4><p>To read a stream from a stream key we use this command.</p><pre>XRANGE stream_temp - +<br><br>Output:<br>1) 1) &quot;1609798077442-0&quot;<br>   2) 1) &quot;sensor_id&quot;<br>      2) &quot;123&quot;<br>      3) &quot;temperature&quot;<br>      4) &quot;25.5&quot;<br>2) 1) &quot;1609798077443-0&quot;<br>   2) 1) &quot;sensor_id&quot;<br>      2) &quot;456&quot;<br>      3) &quot;temperature&quot;<br>      4) &quot;28.2&quot;<br>3) 1) &quot;1609798077444-0&quot;<br>   2) 1) &quot;sensor_id&quot;<br>      2) &quot;789&quot;<br>      3) &quot;temperature&quot;<br>      4) &quot;22.8&quot;</pre><p>his command returns all entries from the stream “mystream.” The - and + are range arguments that specify the start and end of the range, respectively.</p><h4>Consumer Groups</h4><p>Consumer groups allow multiple consumers to read from a stream concurrently, ensuring that each message is processed by only one consumer in the group. To create a consumer group, we can use the XGROUP command.</p><pre>XGROUP CREATE stream_temp gropu1 $</pre><p>This command creates a consumer group named “group1” for the stream “stream_temp,” and the $ the symbol represents the last delivered message.</p><h4>Reading from a Consumer Group</h4><p>To read messages from a consumer group, we can use the XREADGROUP command.</p><pre>XREADGROUP GROUP group1 consumer1 COUNT 10 STREAMS mystream &gt;</pre><p>This command reads up to 10 messages from the stream “mystream” for the consumer group “group1.” The &gt; symbol specifies that messages should be read starting from the last delivered message.</p><h4>Acknowledge Messages</h4><p>After processing a message, a consumer should acknowledge it to remove it from the pending messages list. Use the XACK the command for acknowledgment:</p><pre>XACK stream_temp group1 1609798077442-0</pre><p>This command acknowledges the message with the given ID in the stream “stream_temp” for the consumer group “group1.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4789eeb6d05d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Glimpse into Quantitative Trading: Price Distribution Prediction with the Heston Model and Python]]></title>
            <link>https://medium.com/coinmonks/a-glimpse-into-quantitative-trading-future-price-distribution-with-the-heston-model-and-python-45e429f0039f?source=rss-ffe8da74e0db------2</link>
            <guid isPermaLink="false">https://medium.com/p/45e429f0039f</guid>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[stochastic-process]]></category>
            <dc:creator><![CDATA[Amirali Omidvar]]></dc:creator>
            <pubDate>Mon, 22 Jan 2024 00:46:52 GMT</pubDate>
            <atom:updated>2024-01-24T09:42:41.682Z</atom:updated>
            <content:encoded><![CDATA[<p>Quantitative trading relies on mathematical models, and the Heston Model is a powerful tool for understanding asset price dynamics, especially in stochastic volatility contexts. This article explores key model parameters, utilizing them to generate the price-log probability distribution function (PDF). The accompanying Python code implementation breaks down the mathematical underpinnings and numerical techniques for computing <br>Probability density function(PDF) of future price-log, offering insights into potential price movements and risk assessment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*69dcIblPCrHmC274eqcBlg.jpeg" /></figure><h3>Heston Model</h3><p>The Heston model is a mathematical model used in finance to describe the dynamics of asset prices, particularly for options pricing. Developed by Steven Heston in 1993, it extends the Black-Scholes model by incorporating stochastic volatility, allowing the volatility of the underlying asset to vary over time.</p><p>In the Heston model, the volatility follows a square root process, resulting in a more realistic representation of market behavior. This is most apparent during periods of high and low volatility compared to binomial or constant volatility models. The model is widely employed in option pricing and risk management to better capture the complex dynamics observed in financial markets.</p><p><strong>Here’s the basic Heston model formulation:</strong></p><p>Like most Price prediction models, the Heston model assumes asset price is determined by a stochastic process:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/247/1*scpfVZuf6usjuHgFZb1QbQ.png" /></figure><p>What differentiates the Heston model from simpler models like Black-Scholes, is non-constant volatility. Heston assumes volatility to be determined from a stochastic process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/317/1*oTOOlgwYxiAK5xcPItYkYg.png" /></figure><p>Where:</p><ul><li><em>St</em> is the asset price at time t.</li><li><em>vt</em> is the volatility of the asset at time t.</li><li><em>r</em> is the risk-free interest rate.</li><li>κ is the speed of mean reversion for the volatility. A high <em>κ</em> means the volatility quickly returns to its average. A low <em>κ</em> means it takes more time.</li><li>θ is the long-term average or mean level of volatility (price variance). If <em>θ</em> is 0.2, it means that over the long term, the volatility of the asset likes to stay around 20%.</li><li><em>σ</em> is the volatility of volatility. If <em>σ</em> is 0.1, it means the volatility’s fluctuations are relatively small. If <em>σ</em> is 0.5, the volatility can change a lot.</li><li><em>W</em>1,<em>t</em>​ and <em>W</em>2,<em>t</em>​ are independent Brownian motions.</li><li><em>ρ</em> is the correlation between the Brownian motions.</li></ul><h3>Numerical Approach to compute Heston model</h3><p>Ok with the Heston model itself out of the way, we need to define its <strong>Characteristic function. </strong>The characteristic function (CF) provides a way to summarize the probability distribution of a random variable in a concise mathematical form. Financial markets often involve complex processes, and the characteristic function allows us to handle these complexities more efficiently by providing a clear and compact representation of the probability distribution.</p><p>Since Heston CF is complicated we are going to use numerical methods to calculate price distribution I&#39;m only going to write the CF in Python.</p><pre>import numpy as np<br>import math<br><br>r=0.05<br>k=2<br>theta = 0.2<br>sigma = 0.1<br>rho = -0.7<br>s0 = np.log(100) # we are generating price-log distribution.<br>v0 = 0.2<br>t=1/12 # time of the future price we want the PDF of (1 month)<br><br>def heston_cf(x):<br>    gamma = np.sqrt(sigma**2*(x**2+1j*x)+(k-1j*rho*sigma*x)**2)<br>    power1 = 1j*x*s0<br>    power2 = 1j*x*t*r<br>    power3 = k*theta*t*(k-1j*rho*sigma*x)/(sigma**2)<br>    power4_face = (x**2+1j*x)*v0<br>    power4_deduction = gamma*(1/np.tanh(gamma*t/2))+k-1j*rho*sigma*x<br>    power4 = power4_face/power4_deduction<br>    deduction_base = np.cosh(gamma*t/2)+(k-1j*rho*sigma*x)/gamma*np.sinh(gamma*t/2)<br>    deduction_pow = 2*k*theta/(sigma**2)<br>    deduction = np.power(deduction_base, deduction_pow)<br>    face = np.exp(power1+power2+power3-power4)<br>    cf = face/deduction<br>    return cf</pre><p>Given the parameters and the code, we can calculate Heston CF for any given value of random variable X.</p><p>We can derive the Probability density function (PDF) from CF by implementing the Inverse Fourier Transform (IFT) based on the Probability and Statistics Theorem. if you are interested in the mathematics behind this I recommend reading this <a href="https://www.valpo.edu/mathematics-statistics/files/2015/07/Estimating-Option-Prices-with-Heston%E2%80%99s-Stochastic-Volatility-Model.pdf">article</a>.</p><p>Using numerical methods we can compute an approximation for the IFT value (We will never reach the true values of PDF since we can do finite amount calculation for integration). Here’s the Python code for this transformation implementing numerical methods.</p><pre>from scipy.integrate import quad<br># prices with constant interval to numerically integrate over<br>s_vals = np.linspace(np.log(70), np.log(140), num=1000)<br>def cf_to_pdf(cf, s_vals):<br>    # Define the inverse Fourier transform<br>    def ift(t, cf, x):<br>        return np.real(np.exp(-1j * t * x) * heston_cf(t)) / (2 * np.pi)<br>    <br>    # Calculate the PDF using the inverse Fourier transform<br>    pdf_vals = [0]*len(s_vals)<br>    for i, s in enumerate(s_vals):<br>        pdf_vals[i], _ = quad(ift, -100, 100, args=(cf,s), limit = 1000)<br>    return pdf_vals</pre><p>Implementing the quad function we can integrate the IFT and numerically get the PDF values for the price-log. Calculating the PDF estimation and plotting these values we get this curve that looks like a bell curve from a Normal distribution.</p><pre>import matplotlib.pyplot as plt<br>pdf = cf_to_pdf(heston_cf, s_vals)<br>plt.plot(s_vals, pdf)<br>plt.title(&#39;PDF of the log-price process&#39;)<br>plt.xlabel(&#39;x&#39;)<br>plt.ylabel(&#39;f(x)&#39;)<br>plt.axvline(s0)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*VkBpwP9mt6nwAgA7CPKwYQ.png" /></figure><p>Let&#39;s see what this plot means. the plot depicts PDF of the price log of 1 month into the future, The middle line indicates the starting price (S0). as you can see the price around this line is the highest, meaning the price is more likely to be close to the current price than the further prices. we can see this fact reflected in derivative pricing such as options.</p><p>playing with the values of Heston parameters we can see their effect on price. Given the risk-free rate of 50% (Exaggerated for visuals), we get this curve. we can see the probabilities of higher prices increase since the expected value of the price in a given time interval increases with a risk-free rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*k-_dCkp2akz_WhwuX5Pzxw.png" /></figure><p>For a low average volatility (θ) and high reversion rate (κ) we get a much more compact curve.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*oRZUMrmBkbZx4lrN5ZpiAg.png" /></figure><p>If we want to get the Cumulative Distribution Function (CDF) from PDF we can use this Python code.</p><pre>def pdf_to_cdf(x, pdf):<br>    cdf = np.cumsum(pdf) * np.diff(x)[0]<br>    cdf = cdf / cdf[-1]<br>    return cdf</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*1EVoxb8XzNqCdJjd09Yz_A.png" /></figure><h3><strong>Conclusion</strong></h3><p>Diving into quantitative trading, we explored the Heston Model — a powerful tool for understanding asset price dynamics in financial markets. By incorporating stochastic volatility, Heston goes beyond Black-Scholes, capturing the nuanced dance of prices. With Python and numerical finesse, we reached the PDF and CDF of price-log, offering insights into potential price movements for risk management.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=45e429f0039f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/coinmonks/a-glimpse-into-quantitative-trading-future-price-distribution-with-the-heston-model-and-python-45e429f0039f">A Glimpse into Quantitative Trading: Price Distribution Prediction with the Heston Model and Python</a> was originally published in <a href="https://medium.com/coinmonks">Coinmonks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimize Machine Learning Models with GA_HyperTuner: A Genetic Algorithm Python Package]]></title>
            <link>https://levelup.gitconnected.com/optimize-machine-learning-models-with-ga-hypertuner-a-genetic-algorithm-python-package-6669189ae8bf?source=rss-ffe8da74e0db------2</link>
            <guid isPermaLink="false">https://medium.com/p/6669189ae8bf</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[genetic-algorithm]]></category>
            <dc:creator><![CDATA[Amirali Omidvar]]></dc:creator>
            <pubDate>Sat, 20 Jan 2024 09:06:46 GMT</pubDate>
            <atom:updated>2024-02-29T02:36:49.569Z</atom:updated>
            <content:encoded><![CDATA[<p>In the fast-paced world of machine learning, finding the right set of hyperparameters for your models can be a challenging and time-consuming task. Enter GA_HyperTuner, a Python package designed to simplify and expedite the hyperparameter tuning process using a genetic algorithm approach. In this article, we’ll explore the features, installation, and usage of GA_HyperTuner, demonstrating how it can save you valuable time while enhancing the performance of your machine-learning models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*NC4qM7YsAwDaa5GnTfyAQA.jpeg" /></figure><h3>Features</h3><p>GA_HyperTuner offers a range of features to make hyperparameter tuning a breeze:</p><ol><li><strong>Easy Configuration</strong>: Users can effortlessly specify hyperparameters, define their ranges and constraints, and set up genetic algorithm parameters such as population size, combination probability, and mutation rates.</li><li><strong>Intuitive Interface</strong>: The package provides an intuitive interface that allows users to monitor the optimization process and visualize results easily. This simplifies the hyperparameter tuning workflow, saving users time and effort.</li><li><strong>Flexible Optimization</strong>: Leveraging the power of genetic algorithms, specifically Differential Evolution, GA_HyperTuner dynamically explores hyperparameter spaces to find optimal values that lead to improved model accuracy and performance.</li></ol><h3>Usage</h3><p>Let’s dive into a couple of examples showcasing how GA_HyperTuner can be seamlessly integrated into your machine-learning workflow:</p><h4>Example 1: Logistic Regression on Iris Dataset</h4><pre># Import necessary libraries and modules<br>from ga_hypertuner.tuner import Tuner<br>from sklearn import datasets<br>from sklearn.linear_model import LogisticRegression as lr<br><br># Load data<br>x_train, y_train = datasets.load_iris(return_X_y=True, as_frame=True)<br><br># Define model parameters, boundaries, and tuning criteria<br>model_parameters = {&quot;solver&quot;: &quot;liblinear&quot;, &quot;class_weight&quot;: &quot;balanced&quot;, &quot;C&quot;: [None, float]}<br>boundaries = {&quot;C&quot;: [0, 1]}<br>best_params = Tuner.tune(x_train, y_train, lr, Tuner.default_ga_parameters, model_parameters, boundaries, &#39;accuracy&#39;, stratified=True, k=3, verbosity=1)</pre><h4>Example 2: XGBoost Regression on Diabetes Dataset</h4><pre># Import necessary libraries and modules<br>from ga_hypertuner.tuner import Tuner<br>from sklearn import datasets<br>from xgboost import XGBRegressor as xgbr<br><br># Load data<br>x_train, y_train = datasets.load_diabetes(return_X_y=True, as_frame=True)<br><br># Set genetic algorithm parameters and XGBoost model parameters<br>ga_parameters = {&quot;pop_size&quot;: 15, &quot;fscale&quot;: 0.6, &quot;gmax&quot;: 200, &quot;direction&quot;: &quot;max&quot;, &quot;cp&quot;: 0.6}<br>model_parameters = {&quot;eta&quot;: [None, float], &quot;min_child_weight&quot;: [None, float], &quot;colsample_bytree&quot;: [None, float], &quot;n_estimators&quot;: 100, &quot;alpha&quot;: [None, float], &quot;gamma&quot;: [None, float]}<br>boundaries = {&quot;eta&quot;: [0, 1], &quot;min_child_weight&quot;: [0, 5], &quot;colsample_bytree&quot;: [0, 1], &quot;alpha&quot;: [0, 1], &quot;gamma&quot;: [0, 1]}<br><br># Tune parameters, display progress plot every 10 generations<br>best_params = Tuner.tune(x_train, y_train, xgbr, ga_parameters, model_parameters, boundaries, &#39;r2&#39;, k=3, verbosity=3, show_progress_plot=True, plot_step=10)</pre><h3>Installation</h3><p>Getting started with GA_HyperTuner is a one-liner. Simply use pip to install the package:</p><pre>pip install ga_hypertuner</pre><h3>Documentation</h3><p>For detailed information and more examples, refer to the official GitHub <a href="https://github.com/AmiraliOmidvar/ga_hypertuner">repo</a>.</p><p>In summary, GA_HyperTuner stands out as a valuable tool for machine learning practitioners. It provides an organized and efficient approach to hyperparameter tuning, allowing users to focus on the creative aspects of model development rather than getting bogged down by manual tuning. Install GA_HyperTuner today to leverage the potential of genetic algorithms and enhance the performance of your machine-learning models.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6669189ae8bf" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/optimize-machine-learning-models-with-ga-hypertuner-a-genetic-algorithm-python-package-6669189ae8bf">Optimize Machine Learning Models with GA_HyperTuner: A Genetic Algorithm Python Package</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>