<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Marie Riviere on Medium]]></title>
        <description><![CDATA[Stories by Marie Riviere on Medium]]></description>
        <link>https://medium.com/@marie.riviere?source=rss-d17b4c68260e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*WCLxd_rYV71tD3ztgjPyng.jpeg</url>
            <title>Stories by Marie Riviere on Medium</title>
            <link>https://medium.com/@marie.riviere?source=rss-d17b4c68260e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 19:34:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@marie.riviere/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Getting Started with Alpaca Trading APIs (Python Tutorial)]]></title>
            <link>https://medium.com/@marie.riviere/getting-started-with-alpaca-trading-apis-python-tutorial-6b304aec6c32?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b304aec6c32</guid>
            <category><![CDATA[trade]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[cryptocurrency-investment]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Sun, 18 Sep 2022 07:47:15 GMT</pubDate>
            <atom:updated>2022-09-18T07:47:15.971Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*XM-98N9NulCnbkf2" /><figcaption><em>Photo by </em><a href="https://unsplash.com/@jhonkasalo"><em>Joakim Honkasalo on Unsplash.com</em></a></figcaption></figure><p>In this tutorial I would like to introduce the <a href="https://alpaca.markets/docs/api-references/">Alpaca Trading APIs</a> which can be used for algorithmic trading to get you started with the <a href="https://alpaca.markets/algotrading">Alpaca Stock and Crypto Exchange</a>. Since there are a large variety APIs, we will only focus on the most important ones.</p><blockquote><strong>If you found this article helpful, please consider leaving a tip. Thanks for reading and best of luck on your job search !</strong></blockquote><blockquote><strong>Buy me a coffee so I can do surgery for my cat : </strong><a href="https://www.buymeacoffee.com/botservices"><strong>https://www.buymeacoffee.com/botservices</strong></a></blockquote><h3>What is Alpaca?</h3><p>Alpaca is an exchange for stock- and crypto trading founded in 2015 and headquartered in Silicon Valley, US.</p><p>According to their website <a href="https://alpaca.markets/docs/introduction/">their vision</a> is:</p><blockquote><em>To allow the 7 billion people on the planet to access financial markets. We are committed to providing a secure, reliable and compliant platform for anyone who wants to build their own trading strategies, asset management automation, trading and robo-advisor apps, new brokerage services, investment advisory services and investment products.</em></blockquote><p>The company currently provides commission-free trading for stocks and ETFs and trading for cryptocurrencies.</p><p>In the future, the company also plans to offer other instruments such as options, futures, FX, private equities, and international equities.</p><h3>Why use Alpaca?</h3><p>A compelling reason for using Alpaca is that they offer <strong>commission-free trading</strong> for stock trading. Sweet!</p><p>Here are a couple of other reasons:</p><ul><li>Convenient <a href="https://alpaca.markets/docs/introduction/">API </a>access and ‘developer first’ mindset</li><li>Offers crypto trading of more than 52 currency pairs.<br>(Not<strong> </strong>commission free)</li><li>ETF trading</li><li>Market Data access for stock prices, crypto prices and news</li><li>Margin- and short selling</li><li>Paper trading environment</li><li>Two-factor authentication</li><li>Access for US customers.</li></ul><h3>Implementation</h3><p>There are two Alpaca Python SDK’s available: <a href="https://github.com/alpacahq/alpaca-trade-api-python">alpaca-trade-api</a>, which is deprecated and <a href="https://github.com/alpacahq/alpaca-py">alpaca-py</a>, which is the new official Alpaca Python SDK.</p><p>In this tutorial we will use alpaca-py.</p><p>You can install the Alpaca Python SDK using this statement:</p><pre>pip install alpaca-py</pre><h3>Sign Up</h3><p>In order to use some of the Alpaca APIs, you need to first sign up and get your API keys <a href="https://app.alpaca.markets/signup">here</a>. If you only want to tinker around and do some paper trading, you don’t have to link your account to a funding method.</p><h3>Market APIs</h3><p>Market APIs are used to retrieve market price information for the different instruments like stocks, ETFs or crypto currencies as well as news.</p><p><a href="https://alpaca.markets/docs/api-references/market-data-api/">Here the link</a> for the Alpaca REST Market Data API.</p><p><a href="https://alpaca.markets/docs/python-sdk/api_reference/data_api.html">Here the link</a> for the alpaca-py SDK Market Data API.</p><h3>Get last ask price for Tesla stock</h3><pre>from alpaca.data.historical import StockHistoricalDataClient<br>from alpaca.data.requests import StockLatestQuoteRequest# Create stock historical data client<br>api_key = &quot;&lt;Your API key&gt;&quot;<br>api_secret = &quot;&lt;Your API secret&gt;&quot;<br>client = StockHistoricalDataClient(api_key, api_secret)# Create request<br>multisymbol_request_params = StockLatestQuoteRequest(symbol_or_symbols=[&quot;TSLA&quot;])<br>latest_multisymbol_quotes = client.get_stock_latest_quote(multisymbol_request_params)<br>latest_ask_price = latest_multisymbol_quotes[&quot;TSLA&quot;].ask_price<br>print(latest_ask_price)</pre><h3>Get historic prices for Tesla stock</h3><pre>from alpaca.data.timeframe import TimeFrame<br>from alpaca.data.requests import StockBarsRequest<br>from alpaca.data.historical import StockHistoricalDataClient# Create stock historical data client<br>client = StockHistoricalDataClient(api_key, api_secret)<br>request_params = StockBarsRequest(<br>                        symbol_or_symbols=[&quot;TSLA&quot;],<br>                        timeframe=TimeFrame.Day,<br>                        start=&quot;2022-01-01 00:00:00&quot;<br>                 )bars = client.get_stock_bars(request_params)<br>bars_df = bars.df<br>print(bars_df)</pre><h3>Get the last ask price for Bitcoin</h3><p>Note that the Market API for crypto does not require an API key.</p><pre>from alpaca.data.historical import CryptoHistoricalDataClient<br>from alpaca.data.requests import CryptoLatestQuoteRequest# Get last Ask price for Bitcoin<br>client = CryptoHistoricalDataClient()<br>request_params = CryptoLatestQuoteRequest(symbol_or_symbols=&quot;BTC/USD&quot;)latest_quote = client.get_crypto_latest_quote(request_params)<br>print(latest_quote[&quot;BTC/USD&quot;].ask_price)</pre><h3>Get historical Ethereum prices</h3><pre>from alpaca.data.historical import CryptoHistoricalDataClient<br>from alpaca.data.requests import CryptoBarsRequest<br>from alpaca.data.timeframe import TimeFrame#  Get historical Ethereum prices<br>request_params = CryptoBarsRequest(<br>                        symbol_or_symbols=[&quot;ETH/USD&quot;],<br>                        timeframe=TimeFrame.Day,<br>                        start=&quot;2022-01-01 00:00:00&quot;<br>                 )bars = client.get_crypto_bars(request_params)<br>bars_df = bars.df<br>print(bars_df)</pre><h3>Fetching News</h3><p>The Alpaca API suite also provides market news information: Historical news that can be searched by date and real-time news.</p><p>Unfortunately, the news API is not yet implemented in the Python SDK alpaca-py. However, you can still access it using Python requests and calling the Alpaca news REST service directly.</p><p>This requests gets historical news for Tesla for the month of September:</p><pre>import requests<br>import jsonsymbols = &#39;TSLA&#39;<br>start_date = &quot;2021-09-01T00:00:00Z&quot;<br>end_date = &quot;2021-09-30T11:59:59Z&quot;<br>url = f&#39;https://data.alpaca.markets/v1beta1/news?start={start_date}&amp;end={end_date}&amp;symbols={symbols}&#39;<br>headers = {&#39;content-type&#39;: &#39;application/json&#39;, &#39;Apca-Api-Key-Id&#39;: api_key, &#39;Apca-Api-Secret-Key&#39;: api_secret}response = requests.get(url, headers=headers)response_dict = json.loads(response.text)for i in response_dict:<br>    print(&quot;key: &quot;, i, &quot;val: &quot;, response_dict[i])</pre><h3>Trading API</h3><p>The Trading API offers access to a variety of trading actions such as getting asset and position information, placing orders.</p><p>There are other APIs for secondary actions like clock synchronization, a market-day calendar, corporate announcements and a watchlist API.</p><p><a href="https://alpaca.markets/docs/api-references/trading-api/">Here the link</a> to the Alpaca REST Trading API.</p><p><a href="https://alpaca.markets/docs/python-sdk/api_reference/trading_api.html">Here the link</a> to the alpaca-py SDK Trading API.</p><h3>Get Account Info</h3><p>Note that we are using the paper trading flag.</p><pre>from alpaca.trading.client import TradingClienttrading_client = TradingClient(api_key, api_secret, paper=True)account = trading_client.get_account()<br>print(account.account_number)</pre><h3>Get all Assets</h3><pre>from alpaca.trading.client import TradingClient<br>from alpaca.trading.requests import GetAssetsRequest<br>from alpaca.trading.enums import AssetClasstrading_client = TradingClient(api_key, api_secret, paper=True)# search for crypto assets<br>search_params = GetAssetsRequest(asset_class=AssetClass.CRYPTO)assets = trading_client.get_all_assets(search_params)<br>for asset in assets:<br>    print(asset)</pre><h3>Place a Market Order</h3><p>These statements can be used to place a market order for Tesla stock in the paper trading environment:</p><pre>from alpaca.trading.client import TradingClient<br>from alpaca.trading.requests import MarketOrderRequest<br>from alpaca.trading.enums import OrderSide, TimeInForcetrading_client = TradingClient(api_key, api_secret, paper=True)# preparing orders<br>market_order_data = MarketOrderRequest(<br>                   symbol=&quot;TSLA&quot;,<br>                   qty=0.001,<br>                   side=OrderSide.BUY,<br>                   time_in_force=TimeInForce.DAY<br>                   )# Market order<br>market_order = trading_client.submit_order(<br>               order_data=market_order_data<br>              )<br>print(market_order)</pre><h3>Get all orders</h3><p>Note that you can also filter order by setting the status to ‘open’ or ‘closed’.</p><pre>from alpaca.trading.client import TradingClient<br>from alpaca.trading.requests import GetOrdersRequest<br>from alpaca.trading.enums import OrderSide, OrderStatustrading_client = TradingClient(api_key, api_secret, paper=True)# params to filter orders by<br>request_params = GetOrdersRequest(<br>                    status=&#39;all&#39;,<br>                    side=OrderSide.SELL<br>                 )# orders that satisfy params<br>orders = trading_client.get_orders(filter=request_params)<br>for order in orders:<br>    print(order)</pre><h3>Cancel order by id</h3><p>Canceling orders is not guaranteed.</p><pre>from alpaca.trading.client import TradingClienttrading_client = TradingClient(api_key, api_secret, paper=True)# Attempt to cancel order<br>cancel_status = trading_client.cancel_order_by_id(&#39;12345&#39;)<br>print(cancel_status)</pre><h3>Get all positions</h3><pre>from alpaca.trading.client import TradingClienttrading_client = TradingClient(api_key, api_secret, paper=True)positions = trading_client.get_all_positions()<br>for position in positions:<br>    print(position)</pre><h3>Close a position by symbol or position id</h3><pre>from alpaca.trading.client import TradingClienttrading_client = TradingClient(api_key, api_secret, paper=True)trading_client.close_position(&#39;TSLA&#39;) # or position id</pre><h3>Broker API</h3><p>The Broker API is used for account management services and performs actions such as account and funding services. It is geared towards building web applications or bots that manage investment services for other parties and is not needed for the individual retail trader.</p><p>These services extend past the scope of this tutorial but you are welcome to explore the Broker API on your own.</p><p>The Broker API requires that you sign up on a separate Alpaca website <a href="https://broker-app.alpaca.markets/sign-up">here</a>.</p><p><a href="https://alpaca.markets/docs/api-references/broker-api/">Here the link</a> for the Alpaca REST Broker API.</p><p><a href="https://alpaca.markets/docs/python-sdk/broker.html">Here the link</a> for the alpaca-py SDK Broker API.</p><h3>Wrapping Up</h3><p>In this post we reviewed the most important APIs of the Alpaca Exchange used for getting market data, news info and trading and went over samples of how to implement them in Python.</p><p>I hope you found this post worth your time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b304aec6c32" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create a Simple Crypto Dashboard Using Python]]></title>
            <link>https://medium.com/@marie.riviere/create-a-simple-crypto-dashboard-using-python-dedacf433f91?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/dedacf433f91</guid>
            <category><![CDATA[cryptocurrency-investment]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[investing]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Sun, 11 Sep 2022 13:17:48 GMT</pubDate>
            <atom:updated>2022-09-11T13:17:48.390Z</atom:updated>
            <content:encoded><![CDATA[<h3>A tutorial on web scraping, time series analysis, and web app deployment using Python and Streamlit.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6gvxAWXsR7xfFMeB.png" /><figcaption>Crypto Dashboard</figcaption></figure><blockquote><strong>If you found this article helpful, please consider leaving a tip. Thanks for reading and best of luck on your job search !</strong></blockquote><blockquote><strong>Buy me a coffee so I can do surgery for my cat : </strong><a href="https://www.buymeacoffee.com/botservices"><strong>https://www.buymeacoffee.com/botservices</strong></a></blockquote><p>Dashboard is an information management tool (or a Data Scientist’s play tool) where you can keep track of the important metrics, performance indicators, etc relating to a business or a project.</p><p>Here, our aim is to build a tool that visualizes the time series price fluctuations of various cryptocurrencies over a span of a few years to daily price fluctuations with all the essential information.</p><p>A dashboard can be created in a number of different ways depending upon the use case. Since I want this to be a simple dashboard, I included the following aspects:</p><ol><li>A subset of cryptocurrencies — as there are thousands of them listed on the trading sites. I picked the top 25 cryptos.</li><li>Financial charts like Candlestick charts — to display price features like open, close, high, and low.</li><li>Line chart to show intraday price fluctuations.</li><li>And a table showing all cryptocurrencies with their latest price and stats.</li></ol><p>This is everything that we will include in the dashboard.</p><h3>Data Collection</h3><p>After we are set with the plan for the dashboard structure, the next step naturally is the data collection. From my research, I found several ways to collect crypto price data:</p><ol><li>Crypto APIs — offered by different websites such as Binance, CoinMarketCap, Nomics, CoinGecko, etc.</li><li>Python libraries like YFinance, Pandas DataReader, Fastquant to name a few.</li><li>Good old web scraping using BeautifulSoup4.</li></ol><p>I explored YFinance and web scraping to collect data for my dashboard.</p><p>Web scraping can be tricky as some websites make it difficult to scrape the data and these techniques should be used more for educational purposes.</p><p>I got some help on doing web scraping on <a href="https://finance.yahoo.com/cryptocurrencies/">Yahoo Finance</a> from this <a href="https://www.blog.datahut.co/post/scraping-yahoo-finance-data-using-python">site</a>. Although, I modified the code to better suit my purpose.</p><pre>#-------------------<br># Web scraping Yahoo Finance<br>#-------------------<br>from bs4 import BeautifulSoup<br>import requests<br>import pandas as pd</pre><pre>dic = {}<br>url = &#39;<a href="https://finance.yahoo.com/cryptocurrencies?offset=0&amp;count=100&#39;">https://finance.yahoo.com/cryptocurrencies?offset=0&amp;count=100&#39;</a><br>soup = BeautifulSoup(requests.get(url).text)</pre><pre># store values in separate lists and then in a dictionary<br>for listing in soup.find_all(&#39;div&#39;, attrs={&#39;id&#39;:&#39;fin-scr-res-table&#39;}):<br>    symbol_list = []<br>    name_list = []<br>    price_list = []<br>    change_list = []<br>    mcap_list = []<br>for symbol in listing.find_all(&#39;td&#39;, attrs={&#39;aria-label&#39;:&#39;Symbol&#39;}):<br>    symbol_list.append(symbol.text)<br>    dic[&#39;Symbol&#39;] = symbol_list<br>for name in listing.find_all(&#39;td&#39;, attrs={&#39;aria-label&#39;:&#39;Name&#39;}):<br>    name_list.append(name.text)<br>    dic[&#39;Name&#39;] = name_list<br>for price in listing.find_all(&#39;td&#39;, attrs={&#39;aria-label&#39;:&#39;Price (Intraday)&#39;}):<br>    price_list.append(price.text)<br>    dic[&#39;Price&#39;] = price_list<br>for change in listing.find_all(&#39;td&#39;, attrs={&#39;aria-label&#39;:&#39;% Change&#39;}):<br>    change_list.append(change.text)<br>    dic[&#39;% Change&#39;] = change_list<br>for mcap in listing.find_all(&#39;td&#39;, attrs={&#39;aria-label&#39;:&#39;Market Cap&#39;}):<br>    mcap_list.append(mcap.text)<br>    dic[&#39;Market Cap&#39;] = mcap_list<br>      <br># create a dataframe from dictionary <br>df_scrape = pd.DataFrame(dic)<br>df_scrape.Symbol = df_scrape.Symbol.str.replace(&#39;-USD&#39;,&#39;&#39;)<br>df_scrape.Name = df_scrape.Name.str.replace(&#39; USD&#39;,&#39;&#39;)</pre><p>The scraped data looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/0*EQX2uLfnW7n6qsGV.png" /><figcaption>DataFrame scraped from Yahoo Finance</figcaption></figure><h3>Time series data</h3><p>The web scraped data shows the real-time price of each cryptocurrency but not the historical data. So, I used YFinance python library to collect the historical price data. The code is simple.</p><pre>import yfinance as yf</pre><pre>df = yf.download(tickers=&#39;BTC-USD&#39;, period = &#39;5y&#39;, interval = &#39;1d&#39;)</pre><p>Period is the total duration of time series (5 years in the above code) and interval is the frequency of each value in the dataset (1 day). Below shown is a screenshot of the dataset obtained:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*naRS70rfRwDy7qX7.png" /></figure><p>Historical Bitcoin price data collected from YFinance python library</p><h3>Data Visualization</h3><p>The trickiest part of data collection is done and now we move to data visualization. As we are dealing with a financial time series dataset, a candlestick chart would be a good idea. We can also add simple moving averages over the candlestick chart to make it more informative.</p><p>I included 3 different visualizations in the dashboard:</p><h3>1. Candlestick chart</h3><pre>import yfinance as yf<br>import pandas as pd<br>import plotly.graph_objs as go</pre><pre># download 5 year crypto prices from Yahoo Finance<br>df = yf.download(tickers=&#39;BTC-USD&#39;, period = &#39;5y&#39;, interval = &#39;1d&#39;)</pre><pre># compute moving averages<br>df[&#39;MA100&#39;] = df.Close.rolling(100).mean()<br>df[&#39;MA50&#39;] = df.Close.rolling(50).mean()<br>df[&#39;MA20&#39;] = df.Close.rolling(20).mean()</pre><pre># Plotly candlestick chart<br>fig = go.Figure(data=<br>                [go.Candlestick(x=df.index,<br>                                open=df.Open, <br>                                high=df.High,<br>                                low=df.Low,<br>                                close=df.Close,<br>                                name=f&#39;{select_token}&#39;), <br>                go.Scatter(x=df.index, y=df.MA20, <br>                            line=dict(color=&#39;yellow&#39;,width=1),name=&#39;MA20&#39;),<br>                go.Scatter(x=df.index, y=df.MA50, <br>                            line=dict(color=&#39;green&#39;,width=1),name=&#39;MA50&#39;),<br>                go.Scatter(x=df.index, y=df.MA100, <br>                            line=dict(color=&#39;red&#39;,width=1),name=&#39;MA100&#39;)])<br>    <br>fig.update_layout(go.Layout(xaxis = {&#39;showgrid&#39;: False},<br>                  yaxis = {&#39;showgrid&#39;: False}),<br>                  title=f&#39;{dic1[select_token]} Price Fluctuation with Moving Averages&#39;,<br>                    yaxis_title=f&#39;Price ({select_fiat})&#39;, <br>                    xaxis_rangeslider_visible=False)</pre><pre>fig.show()</pre><h3>2. Daily Trends Line Chart</h3><p>This line graph shows the up and down trends of the price fluctuation compared to the open price.</p><pre># download daily crypto prices from Yahoo Finance<br>df = yf.download(tickers=f&#39;{select_token}-{select_fiat}&#39;, period = &#39;1d&#39;, interval = &#39;1m&#39;)</pre><pre># Plotly line chart<br>fig = go.Figure()<br>fig.add_scattergl(x=df.index, y=df.Close, <br>                  line={&#39;color&#39;: &#39;green&#39;},name=&#39;Up trend&#39;)<br>fig.add_scattergl(x=df.index, y=df.Close.where(df.Close &lt;= df.Open[0]), <br>                  line={&#39;color&#39;: &#39;red&#39;},name=&#39;Down trend&#39;)<br>fig.add_hline(y=df.Open[0])<br>fig.update_layout(go.Layout(xaxis = {&#39;showgrid&#39;: False},<br>                  yaxis = {&#39;showgrid&#39;: False}),<br>                  title=f&#39;{dic1[select_token]} Daily Trends in Comparison to Open Price&#39;,<br>                    yaxis_title=f&#39;Price ({select_fiat})&#39;,template=&#39;plotly_dark&#39;,<br>                    xaxis_rangeslider_visible=False)<br>fig.show()</pre><h3>3. Table with a color-coded column</h3><p>The color coding is set to green when the change is positive and red when the change is negative.</p><pre>df_scrape[&quot;color&quot;] = df_scrape[&quot;% Change&quot;].map(lambda x:&#39;red&#39; if x&lt;0 else &#39;green&#39;)<br>cols_to_show = [&#39;Name&#39;,&#39;Token&#39;, &#39;Price&#39;, &#39;% Change&#39;, &#39;Market Cap&#39;]</pre><pre># to change color of &quot;% change&quot; column<br>fill_color = []<br>n = len(df_scrape)<br>for col in cols_to_show:<br>    if col!=&#39;% Change&#39;:<br>        fill_color.append([&#39;black&#39;]*n)<br>    else:<br>        fill_color.append(df_scrape[&quot;color&quot;].to_list())</pre><pre># Plotly Table<br>data=[go.Table(columnwidth = [20,15,15,15,15],<br>                header=dict(values=[f&quot;&lt;b&gt;{col}&lt;/b&gt;&quot; for col in cols_to_show],<br>                font=dict(color=&#39;white&#39;, size=20),<br>                height=30,<br>                line_color=&#39;black&#39;,<br>                fill_color=&#39;dimgrey&#39;,<br>                align=[&#39;left&#39;,&#39;left&#39;, &#39;right&#39;,&#39;right&#39;,&#39;right&#39;]),<br>                cells=dict(values=df_scrape[cols_to_show].values.T,<br>               fill_color=fill_color,<br>               font=dict(color=&#39;white&#39;, size=20),<br>               height=30,<br>               line_color=&#39;black&#39;,<br>               align=[&#39;left&#39;,&#39;left&#39;, &#39;right&#39;,&#39;right&#39;,&#39;right&#39;]))]</pre><pre>fig = go.Figure(data=data)<br>fig.update_layout(go.Layout(xaxis = {&#39;showgrid&#39;: False},<br>                  yaxis = {&#39;showgrid&#39;: False}))<br>fig.show()</pre><h3>Streamlit Integration</h3><p>So, we have all the essential components to build a dashboard and now, the job is to put the code together and integrate it with streamlit.</p><p>Streamlit is a framework to create beautiful and functional web apps using Python. It is simple to use compared to the Dash framework. The dashboard I created uses a sidebar on the left that lists the filters (enabling users to input the choice of tokens or fiat) and some explanations about the dashboard.</p><pre>import streamlit as st</pre><pre>st.set_page_config(layout=&quot;wide&quot;)</pre><pre>#-------------------<br># Streamlit Sidebar<br>#-------------------<br>fiat = [&#39;USD&#39;,&#39;EUR&#39;,&#39;GBP&#39;]<br>tokens = df_scrape.Symbol.values</pre><pre># filters selectbox<br>st.sidebar.title(&quot;Filters&quot;)<br>select_token = st.sidebar.selectbox(&#39;Tokens&#39;, tokens)<br>select_fiat = st.sidebar.selectbox(&#39;Fiat&#39;, fiat)</pre><pre># special expander objects<br>st.sidebar.markdown(&#39;***&#39;)<br>with st.sidebar.expander(&#39;Help&#39;):<br>    st.markdown(&#39;&#39;&#39;<br>                    - Select token and fiat of your choice.<br>                    - Interactive plots can be zoomed or hovered to retrieve more info.<br>                    - Plots can be downloaded using Plotly tools.&#39;&#39;&#39;)</pre><pre>#-------------------<br># Title Image<br>#-------------------<br>col1, col2, col3 = st.columns([1,6,1])<br>with col1:<br>    st.write(&quot;&quot;)<br>with col2:<br>    st.image(&#39;title.png&#39;,width=600)<br>with col3:<br>    st.write(&quot;&quot;)<br>    <br>st.markdown(&#39;***&#39;)</pre><pre>#-------------------<br># Add crypto logo and name<br>#-------------------<br>col1, col2 = st.columns([1,10])<br>with col1:<br>    try:<br>        st.image(f&#39;logos/{select_token}.png&#39;,width=70)<br>    except:<br>        pass<br>with col2:<br>    st.markdown(f&#39;&#39;&#39;## {dic1[select_token]}&#39;&#39;&#39;)<br>    <br>#-------------------<br># Candlestick chart with moving averages<br>#-------------------  <br>st.plotly_chart(fig, use_container_width=True)</pre><pre>#-------------------<br># Line Chart with daily trends<br>#-------------------<br>st.plotly_chart(fig, use_container_width=True)</pre><pre>#-------------------<br># Table showing top 25 cryptos<br>#-------------------<br>st.plotly_chart(fig, use_container_width=True)</pre><p>Instead of dumping the entire code, I made a code skeleton of the Streamlit framework that I used to develop the dashboard.</p><p>If you are new to Streamlit, these quick tips might help you get started:</p><ol><li>Everything in the sidebar can be controlled using st.sidebar method. I used selectbox to input the selections and expander to wrap additional information like a help guide.</li><li>To display text, you can use st.title , st.writeor st.markdown. It is simple to create unordered or ordered lists using markdown. There is a possibility to add HTML scripts inside the markdown method too.</li><li>Add images using st.image and Plotly charts using st.plotly_chart method.</li><li>If you want to place text and image side by side, horizontally, it can be done using st.columns</li></ol><p>Streamlit gives you a wide list of tools and methods which you can use to fancify the dashboard.</p><h3>Additional Tools</h3><p>I used Canva to generate the title image for free and this <a href="https://cryptologos.cc/">website</a> to download good-quality logos for cryptocurrencies which added a bit of bling to the dashboard.</p><h3>Deployment</h3><p>After the Streamlit integration, all we need is a .py file (with our code) and a requirements.txt file on a GitHub repository. You can now choose any deployment service to deploy the web app.</p><p>Some of the options to deploy web apps for free are Streamlit Cloud, Heroku, and Netlify. They also offer premium services but to deploy a simple app, the free tier works just fine. I used Streamlit Cloud to deploy my Crypto Dashboard.</p><p>Congrats! you’ve learned how to deploy a simple crypto dashboard.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dedacf433f91" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Example: How to Improve a Machine Learning Model’s Trading Strategy]]></title>
            <link>https://medium.com/@marie.riviere/how-to-improve-a-machine-learning-models-trading-strategy-f6efcb44d2b2?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/f6efcb44d2b2</guid>
            <category><![CDATA[trade]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Wed, 07 Sep 2022 19:53:13 GMT</pubDate>
            <atom:updated>2022-09-08T20:39:06.223Z</atom:updated>
            <content:encoded><![CDATA[<h3>Automatically enhance trading backtests using parameter tuning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/0*ybwn2WRWLRvEpsAQ.jpeg" /></figure><p>If you have been reading along with my previous articles, you can probably see that I have been hard at work trying to build a machine learning powered trading strategy. The goal of this strategy is to beat the market without much input from myself (besides the initial code development), which is easier said than done. Many of my previous attempts have centered around backtesting a trading strategy that utilizes a specific time-series model called Facebook Prophet.</p><p>This time around I’ll be backtesting it against the good-old stock market. The goal of this trading strategy is to simply outperform a basic buy and hold strategy of the same stock and still end up profitable. But this time, I’ll be enhancing it by tuning the many custom parameters I made for this AI powered trading strategy. Often times I have found that these parameters need to be adjusted differently for different stocks. However, I shouldn’t need to manually change them <em>every time</em> I backtest a new stock.</p><p>In order to automate this task, I’ll need to accomplish the following:</p><ol><li>Make sure my custom functions integrate seamlessly with one another.</li><li>Loop through the various combinations of different parameters I have created.</li><li>Develop an analysis function that automatically evaluates each backtest to determine whether or not it needs to run the backtest again but with different parameters.</li></ol><p>In the following sections, you may notice some familiar functions used from my previous articles, but with some slight alterations here and there. I’ll point out any significant changes but they are largely the same as before. Now let’s get started!</p><h3>Python Libraries to Import</h3><pre># Libraries<br>import pandas as pd<br>from datetime import datetime, timedelta<br>from tqdm.notebook import tqdm<br>import numpy as np<br>import plotly.express as px<br>from prophet import Prophet<br>import yfinance as yf<br>import itertools<br>import time<br>import random<br>from scipy import stats<br>from statsmodels.stats.weightstats import ztest</pre><h3>Data and Facebook Prophet</h3><p>The following functions below are used to: first, grab price data from Yahoo finance, then instantiate Facebook Prophet with some default parameters, and finally, run it over a specific time frame:</p><pre>def getStockPrices(stock, n_days, training_days, mov_avg):<br>    &quot;&quot;&quot;<br>    Gets stock prices from now to N days ago and training amount will be in addition <br>    to the number of days to train.<br>    &quot;&quot;&quot;<br>    <br>    # Designating the Ticker<br>    ticker = yf.Ticker(stock)</pre><pre># Getting all price history<br>    price_history = ticker.history(period=&quot;max&quot;)<br>    <br>    # Check on length<br>    if len(price_history)&lt;n_days+training_days+mov_avg:<br>        return pd.DataFrame(), price_history<br>    <br>    # Getting relevant length<br>    prices = price_history.tail(n_days+training_days+mov_avg)<br>        <br>    # Filling NaNs with the most recent values for any missing data<br>    prices = prices.fillna(method=&#39;ffill&#39;)<br>    <br>    # Getting the N Day Moving Average and rounding the values for some light data preprocessing<br>    prices[&#39;MA&#39;] = prices[[&#39;Close&#39;]].rolling(<br>        window=mov_avg<br>    ).mean().apply(lambda x: round(x, 2))</pre><pre># Resetting format for FBP<br>    prices = prices.reset_index().rename(<br>        columns={&quot;Date&quot;: &quot;ds&quot;, &quot;MA&quot;: &quot;y&quot;}<br>    )<br>    <br>    # Dropping the Nans<br>    prices.dropna(inplace=True, subset=[&#39;y&#39;])<br>    <br>    return prices, price_history<br>  <br>  <br>def fbpTrainPredict(df, forecast_period, interval_width=0.80):<br>    &quot;&quot;&quot;<br>    Uses FB Prophet and fits to a appropriately formatted DF. Makes a prediction N days into <br>    the future based on given forecast period. Returns predicted values as a DF.<br>    &quot;&quot;&quot;<br>    # Setting up prophet<br>    m = Prophet(<br>        daily_seasonality=True, <br>        yearly_seasonality=True, <br>        weekly_seasonality=True,<br>        interval_width=interval_width<br>    )<br>    <br>    # Fitting to the prices<br>    m.fit(df[[&#39;ds&#39;, &#39;y&#39;]])<br>    <br>    # Future DF<br>    future = m.make_future_dataframe(<br>        periods=forecast_period,<br>        freq=&#39;B&#39;,<br>        include_history=False<br>    )<br>            <br>    # Predicting values<br>    forecast = m.predict(future)</pre><pre># Returning a set of predicted values<br>    return forecast[[&#39;ds&#39;, &#39;yhat&#39;, &#39;yhat_lower&#39;, &#39;yhat_upper&#39;]]<br>  <br>  <br>def runningFBP(prices, forecast_period, training_days, interval_width):<br>    &quot;&quot;&quot;<br>    Runs Facebook Prophet to get predictions over a set period <br>    of time. Uses FBP to train and predict every N days and gets the <br>    price forecasts.<br>    &quot;&quot;&quot;<br>    # DF for the predicted values<br>    pred_df = pd.DataFrame()</pre><pre># Running the model each day<br>    for i in tqdm(range(training_days, len(prices)+1), leave=False):<br>        <br>        # Training then Predicting the last day of the forecast<br>        forecast = fbpTrainPredict(<br>            prices[i-training_days:i], <br>            forecast_period,<br>            interval_width=interval_width<br>        ).tail(1)<br>                <br>        # Adding the forecast predicted (last day)<br>        pred_df = pred_df.append(forecast, ignore_index=True)<br>        <br>    # Prepping for merge by converting date values to be the same type<br>    pred_df[&#39;ds&#39;] = pred_df[&#39;ds&#39;].apply(lambda x: str(x)[:10])</pre><pre>prices[&#39;ds&#39;] = prices[&#39;ds&#39;].apply(lambda x: str(x)[:10])<br>    <br>    # Shifting the forecasts back in order to compare it to the &#39;current&#39; open values<br>    pred_df[[&#39;yhat&#39;, &#39;yhat_lower&#39;, &#39;yhat_upper&#39;]] = pred_df[[&#39;yhat&#39;, &#39;yhat_lower&#39;, &#39;yhat_upper&#39;]].shift(-forecast_period)<br>    <br>    # Merging with the prices DF in order to compare values for positions later<br>    merge_df = prices[[&#39;ds&#39;, &#39;Open&#39;]].merge(<br>        pred_df,<br>        on=&#39;ds&#39;,<br>        how=&#39;outer&#39;<br>    ).dropna().set_index(&#39;ds&#39;)<br>    <br>    return merge_df</pre><h3>Function Descriptions</h3><p>These 3 functions are very similar to if not the same as the functions I’ve used in my previous articles. Here’s a quick rundown on each one:</p><ul><li>getStockPrices() — Retrieves the price history for a given stock and formats the returned Pandas DataFrame to fit Facebook Prophet seamlessly.</li><li>fbpTrainPredict() — Instantiates Prophet with a set of default parameters and make predictions based on the provided DF.</li><li>runningFBP() — Uses an instantiated Facebook Prophet repeatedly over a specific length of time, makes and collects its predictions, then returns a DF of those predictions.</li></ul><h3>Establishing Prophet Trading Positions</h3><p>This function establishes the logic of the trading strategy. It determines whether or not a position should be a buy, hold/exit. You may be familiar with this position function based on my previous articles.</p><pre>def fbpPositions(pred_df, short=True):<br>    &quot;&quot;&quot;<br>    Gets positions based on the predictions and the actual values.     This<br>    is the logic of the trading strategy.<br>    &quot;&quot;&quot;<br>    if pred_df[&#39;open&#39;] &lt; pred_df[&#39;yhat_lower&#39;]:<br>        return 1<br>    elif pred_df[&#39;open&#39;] &gt; pred_df[&#39;yhat_upper&#39;] and short:<br>        return -1<br>    else:<br>        return 0</pre><p>Essentially, what this function does is observe the price forecasts from FBP and compares them to the “Open” prices. Since FBP forecasts upper bound, middle bound, and lower bound prices, these values will be compared to the “Open” prices to determine the strategy’s recommended trade position.</p><h3>Analyzing Risk and Performance</h3><p>Next, I’ll need a function capable of automatically evaluating the performance of the trading strategy’s backtest without my input. In every one of my related articles, I had to examine a graph of the backtest results in order to determine its performance. This method is fine when evaluating a single backtest with specific parameters. However, in order to iterate through the various combinations of parameters I wish to test, I would need to develop a function that can examine a backtest’s performance without visualizations and my personal assessments.</p><pre>def riskAnalysis(performance, prices, price_history, interval_width):<br>    &quot;&quot;&quot;<br>    Analyzes the performance DataFrame to calculate various<br>    evaluation metrics on the backtest to determine if<br>    the backtest performance was favorable.<br>    &quot;&quot;&quot;<br>    ### Hypothesis testing average returns<br>    <br>    # Weekly returns for fb prophet<br>    rets = performance[&#39;fbp_positions&#39;].pct_change(5).dropna()</pre><pre># Buy and hold for about the last two years for the stock<br>    hold = price_history[&#39;Close&#39;].tail(500).apply(np.log).diff().cumsum().apply(np.exp).dropna()</pre><pre># Weekly returns in those years<br>    hold_ret = hold.pct_change(5).mean()</pre><pre># Average returns<br>    if len(performance)&lt;=30:<br>        # T-testing<br>        stat_test = stats.ttest_1samp(<br>            rets, <br>            popmean=hold_ret<br>        )<br>    else:<br>        # Z-testing<br>        stat_test = ztest(<br>            rets, <br>            value=hold_ret<br>        )</pre><pre># Ending portfolio balance<br>    bal = performance.tail(1)<br>    <br>    # Moving Average returns<br>    ma_ret = performance.rolling(window=5).mean().dropna()<br>    <br>    # How often fbp beats holding<br>    ma_ret[&#39;diff&#39;] = ma_ret[&#39;fbp_positions&#39;] &gt; ma_ret[&#39;buy_hold&#39;]<br>    <br>    diff = ma_ret[&#39;diff&#39;].mean()</pre><pre># How often the fbp portfolio had a balance greater than its initial balance<br>    ma_ret[&#39;beat_bal&#39;] = ma_ret[&#39;fbp_positions&#39;] &gt; 1<br>    <br>    beat_bal = ma_ret[&#39;beat_bal&#39;].mean()<br>    <br>    # How often fbp MA returns were positive<br>    ma_ret[&#39;uptrend&#39;] = ma_ret[&#39;fbp_positions&#39;].diff().dropna()&gt;=0<br>    <br>    uptrend = ma_ret[&#39;uptrend&#39;].mean()<br>    <br>    # Performance score<br>    score = 0<br>    <br>    # P-value check<br>    if stat_test[1]&lt;0.05:<br>        score += 1<br>    <br>    # Checking ending portfolio balance<br>    if bal[&#39;fbp_positions&#39;][0]&gt;bal[&#39;buy_hold&#39;][0] and bal[&#39;fbp_positions&#39;][0]&gt;1: <br>        score += 1<br>    <br>    # How often fbp outperformed buy and hold<br>    if diff&gt;.8: <br>        score += 1<br>        <br>    # How often fbp had returns greater than the initial portfolio balance<br>    if beat_bal&gt;.6: <br>        score += 1<br>        <br>    # How often fbp had positive upward trend<br>    if uptrend&gt;.55:<br>        score += 1<br>        <br>    # Dictionary containing values<br>    score_res = {<br>        &quot;result&quot;: True,<br>        &quot;score&quot;: score,<br>        &quot;endingBalance&quot;: {<br>            &quot;prophet&quot;: bal[&#39;fbp_positions&#39;][0],<br>            &quot;buyHold&quot;: bal[&#39;buy_hold&#39;][0]<br>        },<br>        &quot;betterThanBuyHold&quot;: diff,<br>        &quot;greaterPortfolioBalance&quot;: beat_bal,<br>        &quot;upwardTrend&quot;: uptrend,<br>        &quot;pValue&quot;: stat_test[1],<br>        &quot;interval_width&quot;: interval_width<br>    }<br>    <br>    if score&gt;=5:<br>                <br>        return score_res<br>    <br>    else:<br>        # Backtest result is bad<br>        score_res[&#39;result&#39;] = False<br>        <br>        return score_res</pre><p>In this function, I chose to create custom evaluation metrics for assessing the backtest results. Here are the 5 things I looked for:</p><ol><li>The backtest’s ending portfolio balance.</li><li>How often it outperformed a buy and hold strategy in the same time frame.</li><li>How often the portfolio balance exceeded its initial balance.</li><li>How often the daily returns were positive.</li><li>If the average weekly returns of this trading strategy outperformed the average weekly returns of a buy and hold strategy (of the last 2ish years) in a statistically significant way.</li></ol><p>In order to be considered a successful backtest performance, the trading strategy has to pass <em>every one of these evaluation metrics</em>. You may notice that many of the evaluation metrics have hard coded values. These values are arbitrarily set by myself, however, I believe they are still significant enough to warrant a pass. (<em>But, if you think that they should be different, feel free to play around with the code yourself, which I will provide at the end of this article</em>).</p><h3>Creating a Backtesting Function</h3><p>With the risk analysis function complete, I can now create a backtesting function chaining together all of my recently created functions.</p><pre>def backtestStock(stock, pred_df, prices, price_history, interval_width):<br>    <br>    # Adding positions to the forecast DF<br>    positions = pred_df</pre><pre># Getting forecast prophet positions <br>    positions[&#39;fbp_positions&#39;] = positions.apply(<br>        lambda x: fbpPositions(x, short=True), <br>        axis=1<br>    )</pre><pre># Buy and hold position<br>    positions[&#39;buy_hold&#39;] = 1<br>    <br>    # Getting daily returns<br>    log_returns = prices[[&#39;ds&#39;, &#39;Close&#39;]].set_index(<br>        &#39;ds&#39;<br>    ).loc[positions.index].apply(np.log).diff()<br>    <br>    # The positions to backtest (shifted ahead by 1 to prevent lookahead bias)<br>    bt_positions = positions[[<br>        &#39;buy_hold&#39;, <br>        &#39;fbp_positions&#39;<br>    ]].shift(1)</pre><pre># The returns during the backtest<br>    returns = bt_positions.multiply(<br>        log_returns[&#39;Close&#39;], <br>        axis=0<br>    )</pre><pre># Inversing the log returns to get daily portfolio balance<br>    performance = returns.cumsum().apply(<br>        np.exp<br>    ).dropna().fillna(<br>        method=&#39;ffill&#39;<br>    )<br>    <br>    # Performing risk analysis<br>    risk = riskAnalysis(performance, prices, price_history, interval_width)<br>                <br>    return risk, performance</pre><p>You will notice that I am sticking to the same vectorized backtesting method I’ve used many times before. Anyways, after the risk analysis function runs on the backtest results, a simple analysis report of the backtest and its performance is returned by the function.</p><h3>Tuning Parameters</h3><p>As you may have noticed throughout each function I’ve created, I’ve included numerous adjustable variables for each one:</p><ul><li>Training days for FBP — the amount of days required to train the model.</li><li>Moving average amount — how many days to average in order to smooth out the price data (usually set to 3 or 5).</li><li>Forecast period — how many days into the future the forecast model will predict (usually set to 3 or 5).</li><li>Interval width — how far the range is for the upper and lower bound price predictions (default is 0.8).</li><li>Length of Backtest — how many days to backtest the model (the longer this is, the longer it will take to backtest multiple parameters).</li></ul><p>Each of these variables can have a significant effect on the outcome of the performance. Each stock I’ve tried requires adjustments to at least one of these variables.</p><p>Previously, I would have been content with hard coding in the parameters of the backtest and leaving it as is. Now, I believe the next step is to find the optimum combination of parameters in order to achieve the best performing backtest.</p><p>This function utilizes an array of different values for each variable listed above. It then iterates through the many parameter combinations that can be made until a successful backtest is found for that specific stock.</p><pre>def parameterTuning(stock, n_days_lst, training_days_lst, mov_avg_lst, <br>                    forecast_period_lst, fbp_intervals, stop_early=True):<br>    &quot;&quot;&quot;<br>    Given a list of parameters for a specific stock. Iterates through<br>    different combination of parameters until a successful backtest<br>    performance is found.<br>    <br>    Optional stop_early variable for stopping tuning immediately <br>    when a positive backtest result is found<br>    &quot;&quot;&quot;<br>    <br>    # Tuning the stock with FB Prophet<br>    print(f&quot;Tuning FBP parameters for {stock}. . .&quot;)<br>    <br>    # All combinations of the parameters<br>    params = [n_days_lst, training_days_lst, mov_avg_lst, forecast_period_lst, fbp_intervals]<br>    <br>    lst_params = list(itertools.product(*params))<br>    <br>    # Randomizing order of params<br>    random.shuffle(lst_params)<br>    <br>    # List of tested params<br>    param_lst = []<br>    <br>    # Iterating through combos<br>    for param in tqdm(lst_params):</pre><pre># Retrieving prices with the given parameters<br>        prices, price_history = getStockPrices(<br>            stock, <br>            n_days=param[0],<br>            training_days=param[1], <br>            mov_avg=param[2]<br>        )</pre><pre># Checking if the prices retrieved are empty<br>        if prices.empty:<br>            print(f&quot;Not enough price history for {stock}; skipping backtest...&quot;)<br>            continue</pre><pre># Running Facebook Prophet with the set parameters<br>        pred_df = runningFBP(<br>            prices, <br>            forecast_period=param[3], <br>            training_days=param[1],<br>            interval_width=param[4]<br>        )</pre><pre># Running backtest<br>        backtest, performance = backtestStock(<br>            stock, <br>            pred_df, <br>            prices, <br>            price_history, <br>            interval_width=param[4]<br>        )<br>        <br>        # Creating param dictionary to record results<br>        res = {<br>            &quot;n_days&quot;: param[0],<br>            &quot;training_days&quot;: param[1],<br>            &quot;mov_avg&quot;: param[2],<br>            &quot;forecast_period&quot;: param[3],<br>            &quot;interval_width&quot;: param[4],<br>            &quot;backtestAnalysis&quot;: backtest,<br>            &quot;bt_performance&quot;: performance<br>        }<br>        <br>        # Appending the results<br>        param_lst.append(res)<br>                <br>        # Checking backtest result<br>        if backtest[&#39;result&#39;]==True and stop_early==True:<br>                        <br>            return {<br>                &quot;optimumParamLst&quot;: param_lst,<br>                &quot;optimumResultFound&quot;: True<br>            }<br>        <br>    # Dictionary containing sorted parameter results list; best result is last<br>    param_d = {<br>        &quot;optimumParamLst&quot;: sorted(<br>            param_lst,<br>            key=lambda x: (<br>                x[&#39;backtestAnalysis&#39;][&#39;score&#39;], <br>                x[&#39;backtestAnalysis&#39;][&#39;pValue&#39;]*-1,<br>                x[&#39;backtestAnalysis&#39;][&#39;endingBalance&#39;][&#39;prophet&#39;]<br>            )<br>        ),<br>        &quot;optimumResultFound&quot;: True<br>    }</pre><pre># Returning parameter tuning results<br>    if backtest[&#39;result&#39;]==True:<br>        <br>        return param_d<br>    <br>    else:<br>        <br>        # Poor backtest performances<br>        param_d[&#39;optimumResultFound&#39;] = False<br>        <br>        return param_d</pre><p>The stop_early parameter is used to stop searching for the best combination of parameters as soon as one is found. There can possibly be many optimum parameter sets, but if you just want to find the first one that passes all the analysis checks ASAP, then the option is provided. If no optimum combination is found, then the “best” set of parameters (<em>passing the most analysis checks, lowest P-value, highest portfolio ending balance, etc.</em>) will be returned.</p><h3>Backtesting Stocks with Current Data</h3><p>Assuming that the parameter tuning function is able to provide a successful backtest, what then will be the next? Use these functions on a list of stocks and give a trade position recommendation. Finally, I can use this strategy on today’s stock price!</p><pre>def backtestStockAndPredict(stock, n_days_lst, training_days_lst, mov_avg_lst, <br>                            forecast_period_lst, fbp_intervals,stop_early=True, visualize=True):<br>    <br>    # Printing the stock<br>    print(f&quot;\nBacktesting {stock}. . .&quot;)</pre><pre># Tuning parameters for stock<br>    results = parameterTuning(<br>        stock, <br>        n_days_lst, <br>        training_days_lst, <br>        mov_avg_lst, <br>        forecast_period_lst, <br>        fbp_intervals,<br>        stop_early=stop_early<br>    )<br>    <br>    if results[&#39;optimumResultFound&#39;]==False:<br>        print(f&quot;\t***No optimum params found for {stock}***&quot;)<br>    <br>    # Optimum Parameters<br>    opt_params = results[&#39;optimumParamLst&#39;][-1]<br>    <br>    # Visualizing last (best) result<br>    if visualize:<br>        <br>        # Getting the performance DF<br>        performance = opt_params[&#39;bt_performance&#39;]<br>        <br>        # Visual of the backtest<br>        fig = px.line(<br>            performance,<br>            x=performance.index,<br>            y=performance.columns,<br>            title=f&#39;FBProphet vs Buy&amp;Hold for {stock}&#39;,<br>            labels={&quot;value&quot;: &quot;Portfolio Balance&quot;,<br>                    &quot;index&quot;: &quot;Date&quot;}<br>        )</pre><pre>fig.show()<br>    <br>    # Retrieving prices with the given parameters<br>    prices, price_history = getStockPrices(<br>        stock, <br>        n_days=opt_params[&#39;n_days&#39;],<br>        training_days=opt_params[&#39;training_days&#39;], <br>        mov_avg=opt_params[&#39;mov_avg&#39;]<br>    )<br>            <br>    # Run Prophet for current prediction<br>    preds = fbpTrainPredict(<br>        prices.tail(opt_params[&#39;training_days&#39;]), <br>        opt_params[&#39;forecast_period&#39;],<br>        opt_params[&#39;interval_width&#39;]<br>    ).tail(1)</pre><pre>preds[&#39;Open&#39;] = prices.tail(1)[&#39;Open&#39;].values</pre><pre># Getting forecast prophet positions<br>    trade_decision = fbpPositions(preds.to_dict(&#39;records&#39;)[0], short=True)</pre><pre>trade_dict = {<br>        1 : f&quot;Buy {stock}&quot;,<br>        0 : f&quot;Exit {stock}/Do nothing&quot;,<br>        -1: f&quot;Short {stock}&quot;<br>    }</pre><pre># Printing trade decision<br>    print(trade_dict[trade_decision])<br>    <br>    # Printing the optimum params<br>    print(&quot;Best Optimum Parameters Found:\n&quot;, opt_params)</pre><pre>return</pre><p>In this function, a specific stock goes through the usual backtest process, risk analysis, and parameter tuning. Afterwards, if optimum parameters are found, then the function uses those parameters to make another forecast but based on today’s price data. In the end, I should have the position recommendation printed out current day!</p><h3>Backtesting a List of Stocks</h3><p>In the following code below, I set a list of parameter values to test out and perhaps find which set of parameters performs best. There’s nothing special about the list of stocks that I chose. I wanted to try and see how this trading strategy performs against different types of industries and their respective stock tickers.</p><pre># Lists of different params to test<br>n_days_lst = [100]training_days_lst = [50,100,200,400]mov_avg_lst = [3,5]forecast_period_lst = [3,5]fbp_intervals = [0.80, 0.90, 0.99]# List of stocks to backtest<br>tickers = [&quot;MSFT&quot;, &quot;JNJ&quot;, &quot;DIS&quot;, &quot;LMT&quot;, &quot;GOOG&quot;]# Shuffling the stocks<br>random.shuffle(tickers)# Iterate through the stocks<br>for i in tickers:<br>    <br>    # Backtesting each one<br>    backtestStockAndPredict(<br>        stock=i, <br>        n_days_lst=n_days_lst, <br>        training_days_lst=training_days_lst, <br>        mov_avg_lst=mov_avg_lst,<br>        forecast_period_lst=forecast_period_lst,<br>        fbp_intervals=fbp_intervals,<br>        stop_early=True,<br>        visualize=True<br>    )</pre><p>With this code block, I can iterate through the list and use my custom functions and parameter values to find a successful backtest result…</p><h3>Optimized Backtest Results and Analysis</h3><p>Below are 4 of the 5 most optimum backtest results found for the stocks listed above while stop_early is set to True:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/0*LORBUJsTocKsOVmK.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/0*6joacT6GhZpPgHe9.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/0*oT2tDEkt4tN5dl22.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/0*IW373meNoS9ZlD4O.png" /></figure><p>The trading of Google and Microsoft with FB Prophet (top 2 images) resulted in a significantly better performance compared to their buy and hold counterparts. Even when Google’s stock price was fairly stagnant, the FBP trading strategy was still capable of showing gains. You’ll notice a similar outcome in Microsoft as well.</p><p>The bottom two (Johnson &amp; Johnson and Disney) didn’t outperform their buy and hold counterparts nearly as well as I hoped or had just one lucky trade differentiating the two strategies. Specifically, in the case of Disney, just one trade seemed to separate it from the same fate as the buy and hold strategy.</p><p>The last optimized backtest was LMT and it heavily underperformed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*aHF2LpC-jO-3_mdk.png" /></figure><p>It’s not the worst performance in the world as its portfolio balance still ended positively but compared to a simple buy and hold strategy — the FBP trading strategy couldn’t quite cut it. In this case, I wouldn’t even bother using the FBP strategy. However, maybe if I were to include more parameter values to test out it might have resulted in a positive performance?</p><h3>Closing Thoughts</h3><p>It should be noted that running this backtest optimization function can be very costly. In my case, this whole process of optimization for just 5 stocks took about an hour and a half. Even with the stop_early parameter set to True.</p><p>There’s no real way of knowing if this trading strategy is truly successful without trying it out myself. What this means is that I would need to perform some forward testing methods such as paper trading. If the paper trading is successful as well, then the next step would be trading/investing with real money!</p><blockquote><strong><em>If you found this article helpful, please consider leaving a tip. Thanks for reading and best of luck on your job search !</em></strong></blockquote><blockquote><strong><em>Buy me a coffee so I can do surgery for my cat : </em></strong><a href="https://www.buymeacoffee.com/botservices"><strong><em>https://www.buymeacoffee.com/botservices</em></strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f6efcb44d2b2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Machine Learning to Build Trading Strategies]]></title>
            <link>https://medium.com/@marie.riviere/using-machine-learning-to-build-trading-strategies-ff67de35d108?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff67de35d108</guid>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[cryptocurrency-investment]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Sun, 04 Sep 2022 07:21:56 GMT</pubDate>
            <atom:updated>2022-09-04T07:21:56.229Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/680/0*Q0vEXCyqNAXD8bLo.jpeg" /></figure><p>Some matches are made in heaven. Cereal and milk. Peanut butter and jelly. Netflix and chill. My crush and I — still waiting for her to text me back. Any day now. Or what about machine learning and trading? Pair really fast machines that are good at math with a free market where you can make a ton of money. Sounds like a dream come true. What could possibly go wrong? Or actually, the better question is, what color trim do you want on the seats of your new Lamborghini?</p><p>You think I’m kidding, but my head truly was in the clouds a few months ago. Back when my team and I started building <a href="https://youtu.be/qNl_sRUB8vA">Chartmaster</a>, an advanced trading strategy backtesting tool. It’s just a Go backend hooked up to some JavaScript charts. We use it to simulate a trading strategy’s performance over historical data. Just to see where it would’ve gone wrong. Hopefully to avoid similar mishaps in the future. Which sounds great, until it shows you things you don’t want to see.</p><p>Here’s what went down: We quickly tuned our strategies. Found settings that worked over different periods. But the manual process of dialing in took too long. So we thought, “Why not train an ML model to generate settings on the fly?” Perfect. Several dozen lines of disgraceful Python code later, we had the model. But after some training and testing, we found a problem.</p><p>Dynamically setting strategy parameters works just fine provided you use the right training data. But the problem is when used in real life, the predictions are less reliable. The minute you introduce live, current data into the equation, things fall apart. We could get into training errors and limitations of machine learning due to its inherent pattern-recognizing nature. But to save us all the headache, just take my word for it: It didn’t work. We never managed to integrate machine learning into a core role in any of our strategies. But that’s the key word: “Core role”.</p><p>Because I suspect what machine learning can actually be used for is a <em>supporting</em> role. Because many strategies, ours included, require plenty of data collection. You need to find the best settings for it. You know, things like the stop loss percentage, account size to risk, and more. That takes time. But it’s not very intelligent business. Tedious, methodical, and straightforward. Even a computer could do it.</p><p>And that’s exactly what machine learning could do: Assist in data collection. Find optimal settings. Point out patterns in sets of data. Which is still useful, but it’s not what we think about when we imagine machine learning in trading. Guess those Alcantara seats will have to wait.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff67de35d108" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Built a Simple Trading Algorithm in Python and You Can Too]]></title>
            <link>https://medium.com/@marie.riviere/i-built-a-simple-trading-algorithm-in-python-and-you-can-too-a621b6d1d3f6?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/a621b6d1d3f6</guid>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Thu, 01 Sep 2022 19:23:50 GMT</pubDate>
            <atom:updated>2022-09-01T19:23:50.296Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*IdoSTU6LFe5WSf-L.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@nick604?utm_source=medium&amp;utm_medium=referral">Nick Chong</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Money makes the world go ‘round. Or at least that’s what the world will tell you. For the past few years, jobs around the world have been automated or replaced by machines in order to maximize profits. Workers on Wall Street, such as stock brokers, have also experienced this. The implementation of computers and access to the internet have made the traditional stock broker largely obsolete. But stock trading still requires some human element to it.</p><p>Some people might trade based on feeling or intuition, which a machine cannot replicate at this time. Others trade based on a strict set of rules or guidelines which can be closely, if not completely, replicated by a computer. So today, let’s try and replicate a simple set of rules by coding them in Python. The code we will write will enable us to trade stocks automatically with no need to observe stock prices or stock charts. Isn’t that amazing?</p><p>Now this attempt at programming a set of rules or guidelines to trade stocks is called — <strong>Algorithmic Trading</strong>. Traders who use this method are classified as Quantitative (<em>Quant</em>) Traders operating in Quantitative finance. Algorithmic trading requires some understanding of the stock market in order to better comprehend how some strategies are applied. It is not necessary to understand the market like Warren Buffet in order to make a trading algorithm, but it definitely doesn’t hurt. With that in mind, let’s quickly explain a few market concepts and definitions.</p><h3>Stock Trading Strategies</h3><p>First, let’s define the two main methods of stock market trading: buying<em> (longing) </em>and selling (<em>shorting</em>). You buy stocks when you expect them to go up and short stocks when you expect them to go down. This activity of buying and shorting is distinguished as either ‘<em>bullish</em>’ (stock price will increase) or ‘<em>bearish</em>’ (stock price will decrease). When it comes to stock market trading, there are many different strategies implemented. One strategy, called <strong>Technical Analysis</strong>, is based on trading signals and patterns derived from multiple calculations of a stock’s price, volume, highs, lows, etc. Here’s an example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*RZEhr1QirG6x2gUx.png" /></figure><p>Technical Analysis of TWMJF using signal crossing, and trend breakouts.</p><p>Some criticize this strategy as they believe that technical analysis is just as effective as Zodiac signs in determining the future. But others swear by it and have lead successful trading careers.</p><p>Another strategy implemented in trading is called — <strong>Fundamental Analysis</strong>. Simply put, this strategy observes a stock’s financial statements and current value compared to its competitors. A trader will either be bullish or bearish depending on whether they believe a stock is undervalued or overvalued relative to their current sector and market.</p><p>That is about all we are going to cover in regards to the stock market. Studying the market and trading strategies takes months, if not years, of review. If you are a beginner looking to trade then click <a href="https://www.thebalance.com/stock-trading-101-358115">here</a> for a simple starting point. For now we will be moving on to automating a stock trading strategy in Python code.</p><h3>Quantopian</h3><p>In order to begin, we will need to go to this site: <a href="https://www.quantopian.com/home"><strong>Quantopian.com</strong></a> to use their platform which helps streamline the process. There you will be able to test out different strategies and practice some of your programming skills for the market without the risk of using actual money. You must create a free account in order to use their own unique IDE specifically calibrated for backtesting different market strategies. After you create an account, please navigate to creating a new algorithm by clicking <strong><em>Research — Algorithms — New Algorithm. </em></strong>Once you find yourself at a page that looks like the one below, then we can begin.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*pYOSq5oP3-yxouPt.png" /><figcaption>The starting template for creating a new algorithm.</figcaption></figure><p>To keep this simple, we will be coding a basic trading strategy involving the crossing of two simple moving averages: the 50 day and 200 day moving average. A simple moving average is calculated by averaging the most recent <em>N </em>days closing prices. We will be buying stocks when the 50 day moving average crosses above the 200 day moving average and shorting when the opposite occurs. The strategy looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/0*ESUQV0x9qP5KGQi1.gif" /><figcaption>Simple Moving Average Stock Strategy</figcaption></figure><p>Now this specific strategy is not endorsed as a legitimate strategy and you are primarily responsible for any capital loss incurred if you choose to implement this strategy for yourself. We will just be using this strategy as an example of algorithmic trading. Feel free to use any other strategy you may know or discover later on once you feel comfortable coding in Quantopian.</p><p>Okay, so now that we have a strategy, we can begin the coding aspect of the algorithm. Quantopian uses many different libraries unique to its IDE, so we will explain the differences when necessary. However, we will not be able to cover every question you may have about the platform, so please explore their <a href="https://www.quantopian.com/help">help page</a> when needed.</p><h3>The Trading Algorithm</h3><p>Let’s start by importing the libraries we will need:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/0*MZgAwIwAHxqVBSme.png" /></figure><p>A major difference with Quantopian’s IDE is how some functions are run. You are still able to call functions like normal but others need to be scheduled to run at specific times during the market.</p><h3>Initialize Function</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/0*HaTvDQBep539QMwf.png" /><figcaption>The Initialize function</figcaption></figure><p>Here we have the initialize() function. Within this function we will be using the algo.schedule_function() to schedule three functions:</p><ul><li>trading()</li><li>exiting_trades()</li><li>record_vars()</li></ul><p>The other arguments:</p><ul><li>algo.date_rules.every_day()</li><li>algo.time_rules.market_close()</li></ul><p>specify when the functions will run. The reason these functions are scheduled for these times is to fix any potential leverage issues. Leverage plays an important role in our algorithm which will be explained later on.</p><h3>Trading Function</h3><p>So now that we have an initialize() function that will run our algorithm, we can move on to the trading() function which will automatically trade the stocks we want.</p><p>But first we must explain how we are retrieving the stocks we want. In Quantopian, stocks are assigned unique id values. To access the stocks, we call upon the sid() function, type in the stock ticker, and a drop down menu will appear. See below to see which id number is assigned to TSLA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/0*JXBru8eewmrqccxi.png" /><figcaption>TSLA and its sid #</figcaption></figure><p>For our trading function, we will be making a list of stocks that we would like to pick from. Those stocks are: TSLA, DIS, AAPL, and SPY. Each with their own sid #. There are ways to grab more stocks and filter out selected ones but that requires another function.</p><p>With our list of stocks, we will iterate through them with a ‘for’ loop. The steps taken in the ‘for’ loop are:</p><ol><li>Fetching each stock’s closing price history.</li><li>Calculating their 50 day and 200 day moving averages.</li><li>Creating a True/False statement to determine crossing points.</li><li>Calculating leverage allowance and setting it as another True/False statement.</li><li>Creating conditional statements using open_lev, and bullish_cross or bearish_cross.</li><li>Once all conditions are satisfied, we place an order for a stock using order_target_percent(stock, 0.25) to fill 25% of our portfolio with that stock. (We set the percentage as negative or positive to buy or short the stock).</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*qGDpXqcF-3cGvW4e.png" /><figcaption>The Trading function</figcaption></figure><p>Now each step requires specific Quantopian methods and functions. So please refer to the <a href="https://www.quantopian.com/help">help page</a> once again for a detailed explanation. Next, we’ll be explaining the conceptual reasoning behind the crossing signals and open leverage.</p><p>To determine if the 50 day MA crosses the 200 day MA in a bullish fashion, we set the 50 day MA to be less than the 200 day MA, both calculated from one day before. Then, we set the current 50 day MA to be greater than the current 200 day MA. By setting this as a conditional statement, we can capture the crossing point of the moving averages.</p><p>Now regarding open leverage, its importance is taken into account because leverage determines whether we are using our own money or borrowing money. The results from borrowing money can drastically alter our trading outcomes. The formula and schedule coded before allow us to make trades without exceeding our own leverage or cash limit.</p><h3>Exiting Trades Function</h3><p>Now the next function, exiting_trades(), is very similar to our trading function. The only difference is checking if we have any positions open in the first place and what kind of position that is (long or short).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*CJRbH-8oHb8nY7Ne.png" /><figcaption>The exiting trade function</figcaption></figure><p>As you can see, there is not much of difference between exiting_trades() and trading(). This exiting trades function repeats most of the calculations and conditions from before in order to close positions whenever the moving averages crosses against us.</p><p>Next, let’s finish the code up with a record_vars() function to keep track of a few variables that we believe are necessary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/0*Du_xPA37wh71Fw6a.png" /></figure><p>Here we will record() our leverage and how many positions we have open. We want leverage to not drastically exceed 1. Open positions should not exceed 4 because we devoted only 25% of our portfolio for each stock we trade.</p><h3>Running a Backtest</h3><p>Finally, set the dates and starting capital to match the one below. We won’t be running a full backtest because it is not necessary. A full backtest will provide more information about the coded strategy but, as of now, it is not needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/0*LLmmlAnVEJFH28ao.png" /><figcaption>Set to 7 years of backtesting with a starting amount of $10,000</figcaption></figure><p>Click the button <strong>Build Algorithm,</strong> which will run the code, and then you will end up with a page like so:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/0*-0sGeg72eXslsNRf.png" /><figcaption>Results of our coded strategy</figcaption></figure><p>Great! We have successfully backtested our first trading algorithm. We were successful in limiting the leverage to 1 (albeit with a little leeway, which is fine) and the trading positions never exceeded 4. Feel free to alter some of the code to see if it would significantly affect the results.</p><h3>Closing Thoughts</h3><p>This trading strategy is very simple and very common and can be used in conjunction with other strategies to increase its overall effectiveness. It may appear that this strategy is extremely successful but be warned that we are trading stocks that are inherently stable and profitable compared to others. Also, as a general rule in determining its effectiveness, a trading strategy is noteworthy if it is able to beat SPY’s returns. However, this strategy was not able to significantly beat the SPY returns. If we were to trade stocks that were not as stable then we would probably see drastically different results.</p><p>By now you should have a better understanding of the inner workings of a trading algorithm, even if this one was based on a simple strategy. If you felt like this was too difficult, then you can also explore Quantopian’s <a href="https://www.quantopian.com/tutorials">tutorial page</a>. But if you felt like this was too easy, then keep exploring <a href="https://www.quantopian.com/help">Quantopian’s help page</a> and try to create a new and improved algorithm. Maybe then you can beat Wall Street with your new trading algorithm!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a621b6d1d3f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What you need to start crypto Trading: A Four-Step Guide]]></title>
            <link>https://medium.com/@marie.riviere/what-you-need-to-start-crypto-trading-a-four-step-guide-7bf89fb68d4f?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/7bf89fb68d4f</guid>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[cryptocurrency-investment]]></category>
            <category><![CDATA[trading]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Wed, 31 Aug 2022 19:55:28 GMT</pubDate>
            <atom:updated>2022-08-31T19:55:28.472Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*t_PxrATy3bWGqyMfiyx_bg.jpeg" /></figure><p>The crypto market cap is now about $300 bn with more than 1500 crypto coins. It comes as no surprise that more and more people worldwide are deciding to get into this industry and starting to invest in crypto. However, at the beginning trading crypto can seem confusing as the process itself differs from trading on fiat exchanges and what many investors and traders are used to.</p><p>Today we will outline the four essential steps that will help you kick off your crypto trading.</p><h3>1. Pick an exchange to trade on</h3><p>Just like fiat finance, to invest in cryptocurrencies you need a place where these coins are traded. To commence you will need to register and put some funds into your account, and then start buying and selling crypto.</p><p>There are a number of popular crypto exchanges, for example:</p><ul><li><a href="https://www.coinbase.com/">Coinbase</a></li><li><a href="https://poloniex.com/">Poloniex</a></li><li><a href="https://cex.io/">CEX</a></li><li><a href="https://bittrex.com/">Bittrex</a></li><li><a href="https://www.bitfinex.com/">Bitfinex</a></li><li><a href="https://www.binance.com/">Binance</a></li><li><a href="https://www.kucoin.com/">KuCoin</a></li></ul><p>Such websites can have different terms of service and their trading platforms may differ a lot, so it is best to research and go through several reviews of exchanges (here are two: <a href="https://www.finder.com/cryptocurrency/trading">one</a>, <a href="https://blockonomi.com/exchanges-trading-screens/">two</a>). This will help you to choose a venue that suits you best.</p><p>According to CryptocoinCharts’ <a href="https://cryptocoincharts.info/markets/info">statistics</a>, there are currently almost 200 cryptocoin exchanges with a total daily trade volume of $3.24 bn though not all of these marketplaces are reliable. Situations where crypto traders lose their money due to the exchange being hacked happen on a regular basis (here are some examples: <a href="https://uk.finance.yahoo.com/news/digital-currency-ethereum-cratering-amid-093418142.html">one</a>, <a href="https://finance.yahoo.com/news/bitcoin-hack-bitfinex-cold-storage-bitgo-cftc-225434534.html">two</a>, <a href="https://finance.yahoo.com/news/coincheck-cryptocurrency-hack-everything-know-111140809.html">three</a>). There are also some <a href="https://nulltx.com/top-4-bitcoin-exchanges-which-disappeared-under-suspicious-circumstances/">cases</a> where exchanges suddenly terminate their operations with little explanation, leading to the complete loss of funds by investors. So it is crucial to choose a trusted exchange. In the beginning, it is a good idea to go with the biggest and most well-known options.</p><p>Important note: not all exchanges accept fiat money, so you may have to buy cryptocurrency first and then deposit coins into the exchange to start investing (however, there are also exchange-broker-wallet hybrids like Coinbase where you can do everything in one place).</p><p>This leads us to the next step of obtaining a cryptocurrency wallet.</p><h3>2. Chose a cryptocurrency wallet</h3><p>As there are lots of exchanges, investors have to transfer funds between them somehow and also have a place to store their assets. Cryptocurrency wallets are tools that allow you to store and send crypto coins back and forth.</p><p>Currently, there are several types of crypto wallets (mobile, desktop, hardware, paper ones). To get started, you will need a mobile or desktop (web) hot wallet, connected to the internet. Read more on the topic in <a href="https://blockgeeks.com/guides/cryptocurrency-wallet-guide/">this</a> guide.</p><p>Almost all cryptocurrencies nowadays have their official wallets such as the Bitcoin Core Wallet, Ethereum Wallet or MyEtherWallet, Litecoin-QT, and Dash Core. You will find a link to their official wallet on most cryptocurrency websites.</p><p>Multi-currency wallets are also available, here are several examples:</p><ul><li><a href="https://jaxx.io/">Jaxx Wallet</a>: allows transactions with Bitcoin, Bitcoin Cash Ethereum, Ethereum Classic, Litecoin, Dash, Zcash, and other coins;</li><li><a href="https://coinomi.com/">Coinomi</a>: supports 64 cryptocurrencies;</li><li><a href="https://www.exodus.io/">Exodus</a>: a desktop wallet which can be used for storing and sending Bitcoin, Bitcoin Cash, Ethereum, Ethereum Classic, Litecoin, OmiseGo, Qtum, Dash, Bat, Aragorn, and other currencies.</li></ul><p>As mentioned above, there are some opportunities to use exchanges with built-in wallets, such as Poloniex or Coinbase.</p><p>When you’ve chosen the wallets you’d like to use, you are all set and ready to start investing.</p><h3>3. Find sources of information</h3><p>Cryptocurrencies are highly volatile assets whose prices can go up and down in large percentages in a few short hours. Such moves can be triggered by multiple events, so a crypto trader must constantly monitor market information.</p><p>Before putting your money into a coin it is a good idea to study the asset. One of the best tools for this research is <a href="https://coinmarketcap.com/">Coinmarketcap</a>, a website where you can find information on every coin and token available. The data includes capitalization, supply, trade volume etc. So the site can be used for cryptocurrency comparisons. <a href="https://www.cryptocompare.com/">Cryptocompare</a> is another useful site for this sort of data.</p><p>The best way to track crypto-related news is to follow major publications and blogs in the industry: Coindesk, News Bitcoin, The Merkle, Hackernoon. Even more useful information can be found on social media — you should dig into crypto-related subreddits, Telegram and Discord channels, and Quora threads, for example.</p><p>Another great way of sourcing data is a <a href="https://coinmarketcal.com/">CoinMarketCal</a> which tracks crypto-related events which can potentially affect coins traded on exchanges.</p><h3>4. Use technology to enhance productivity and mitigate risks</h3><p>By now you are equipped with some knowledge and instruments for trading. However, compared with the fiat investment process there are still some elements missing.</p><p>First of all, fiat investors can use stock indices for benchmarking when building their portfolio — S&amp;P500 and Nasdaq Composite are perfect examples of such benchmarks. They allow investors to trade whole sectors easily and manage complicated portfolios in a straightforward investment, reducing the risks and volatility of the portfolio.</p><p>You need to find similar tools for the crypto market — for example, <a href="http://cryptoindex.io/">Cryptoindex 100</a> (CIX100) is an automated index calculated by a machine learning algorithm which analyzes cryptocurrencies. This tool allows traders to reduce volatility and risks for their portfolios as well as the cost and effort inherent in investing in multiple coins, resulting in substantial savings in trading.</p><p>After the portfolio is built you can track coins via specialized apps such as Blockfolio or Delta, which provide valuable insights (real-time exchange prices, the order book for every coin etc). Using such tools also enhances productivity.</p><h3>Conclusion</h3><p>Cryptocurrency trading has a range of differences when compared with the traditional fiat exchanges. To succeed in crypto, you will need to find trustworthy information sources, perform your research carefully, pick the most suitable exchange and wallet, and find benchmarks and tools for portfolio management.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7bf89fb68d4f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Coding a Simple Trading Strategy in TradingView Step-By-Step]]></title>
            <link>https://medium.com/@marie.riviere/coding-a-simple-trading-strategy-in-tradingview-step-by-step-35d21459c498?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/35d21459c498</guid>
            <category><![CDATA[strategy]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Tue, 30 Aug 2022 18:33:36 GMT</pubDate>
            <atom:updated>2022-08-30T18:33:36.830Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Vtfwj5FitwCitFWC" /><figcaption>Photo by <a href="http://prod-upp-image-read.ft.com/f766b018-014b-11ea-a530-16c6c29e70ca">intelligenthq</a></figcaption></figure><p>Without a doubt the Stochastic Oscillator is a universally used indicator. It uses such a simple yet powerful formula in order to detect rapid oversold and overbought conditions. This article discusses a certain strategy based on this indicator.</p><h3>Coding the Stochastic Oscillator in Pine Script</h3><p>The stochastic oscillator is a bounded technical indicator seeks to find oversold and overbought zones by incorporating the highs and lows using the normalization formula as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/0*h7UC-RYxxrsQKYbh.png" /></figure><p>This means that at any time, it must be bounded between 0 and 100 as dictated by the mathematical formula above. We generally use the oscillator in a contrarian way where we use the concepts of oversold and overbought to describe certain extreme events.</p><p>An <strong>overbought</strong> level is an area where the market is perceived to be extremely bullish and is bound to consolidate, therefore giving us a bearish bias. An <strong>oversold </strong>level is an area where market is perceived to be extremely bearish and is bound to bounce, therefore giving us a bullish bias. Hence, the stochastic oscillator is a contrarian indicator that seeks to signal the expected reactions of extreme movements.</p><p><em>Why is this section called “Creating the Stochastic Oscillator in Pine Script” when it is already a built-in script in TradingView and does not require coding?</em></p><p>The answer to that is because we want to learn how to code all types of indicators from scratch in Pine Script so that we become fluent and comfortable back-testing and tweaking any strategy. By understanding how to code it, we are also able to code variations on the indicator such as the stochastic smoothing oscillator, an indicator discussed in previous articles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*6o-zpCbx2Cz6ZMT8.png" /><figcaption><strong>Hourly USDJPY values with the 13-period stochastic oscillator.</strong></figcaption></figure><p>Generally, we smooth the oscillator with a 3-period moving average applied onto it. This can be seen in the above chart with the orange line following the stochastic in blue.</p><p>To code the oscillator, we first start by choosing the version of Pine Script, the official language of TradingView. In our case, we will choose the latest version. Then as we will create an indicator, we have to tell the program that we will do that by using the <strong>indicator()</strong> function. We will put two arguments inside, the title which we will call<em> “Our Stochastic Oscillator”</em> and the Boolean argument overlay. By putting false, we are telling the script that the oscillator will be in a different panel. If we were to create a moving average, we must put true.</p><pre><strong>// This source code is subject to the terms of the Mozilla Public License 2.0 at </strong><a href="https://mozilla.org/MPL/2.0/"><strong>https://mozilla.org/MPL/2.0/</strong></a><strong><br>// © Sofien-Kaabar//@version = 5</strong><br>indicator(&quot;Our Stochastic Oscillator&quot;, overlay = false)</pre><p>Next, we define the input values such as the lookback period, the oversold level, and the overbought level. We use the <strong>input()</strong> function to give us the opportunity to tweak these parameters on the settings directly from the chart. In other words, we do not have to code new parameters if we want to change them. The title is the name of the variable and the defval is the default value. In our case, it is 13, 20, and 80 as discussed previously.</p><pre>lookback = input(title = &#39;Period&#39;, defval = 13)<br>oversold_level = input(title = &#39;Oversold&#39;, defval = 20)<br>overbought_level = input(title = &#39;Overbought&#39;, defval = 80)</pre><p>Now, we will calculate the highest highs and the lowest lows in the lookback period so that we construct the stochastic formula. This is done by using the <strong>ta.highest()</strong> and <strong>ta.lowest() </strong>functions. Each require a lookback which in our case is the defined variable with a default value of 13.</p><pre><strong>// highest high</strong><br>highest = ta.highest(high, lookback)<strong>// lowest low</strong><br>lowest = ta.lowest(low, lookback)</pre><p>The last step is applying the formula. This is simply done below where we define the stochastic_k (%K) followed by the stochastic_D (%D) which is the 3-period simple moving average of stochastic_k. The simple moving average is calculated using the <strong>ta.sma()</strong> function.</p><pre><strong>// stochastic oscillator</strong><br>stochastic_K = ((close - lowest) / (highest - lowest)) * 100<br>stochastic_D = ta.sma(stochastic_K, 3)</pre><p>Let us plot what we have so far and compare it to the built-in stochastic oscillator script.</p><pre>plot(stochastic_K)<br>plot(stochastic_D, color = color.orange)<br>hline(oversold_level)<br>hline(overbought_level)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*WEEFf5cmbcbmn_ba.png" /><figcaption><strong>Hourly USDJPY values with the 13-period built-in stochastic oscillator and the 13-period coded stochastic oscillator.</strong></figcaption></figure><pre><strong>// This source code is subject to the terms of the Mozilla Public License 2.0 at </strong><a href="https://mozilla.org/MPL/2.0/"><strong>https://mozilla.org/MPL/2.0/</strong></a><strong><br>// © Sofien-Kaabar//@version = 5</strong><br>indicator(&quot;Our Stochastic Oscillator&quot;, overlay = false)lookback = input(title = &#39;Period&#39;, defval = 13)<br>oversold_level = input(title = &#39;Oversold&#39;, defval = 20)<br>overbought_level = input(title = &#39;Overbought&#39;, defval = 80)<strong>// highest high</strong><br>highest = ta.highest(high, lookback)<strong>// lowest low</strong><br>lowest = ta.lowest(low, lookback)<strong>// stochastic oscillator</strong><br>stochastic_K = ((close - lowest) / (highest - lowest)) * 100<br>stochastic_D = ta.sma(stochastic_K, 3)plot(stochastic_K)<br>plot(stochastic_D, color = color.orange)<br>hline(oversold_level)<br>hline(overbought_level)</pre><h3>Coding and Analyzing the Strategy in Pine Script</h3><p>The strategy is pretty clear, we will initiate position based on the exit of extremes. Therefore, the trading conditions are:</p><ul><li><strong>A buy (Long) signal is generated whenever the current 13-period stochastic oscillator is above 20 while the previous reading is below 20.</strong></li><li><strong>A sell (Short) signal is generated whenever the current 13-period stochastic oscillator is below 80 while the previous reading is above 80.</strong></li></ul><p>Let us code the conditions of the signals first. When we use the stochastic_K[1] statement, it refers to the previous value of wherever we are now.</p><pre>buy_signal  = stochastic_K &gt; oversold_level and stochastic_K[1] &lt; oversold_level<br>sell_signal = stochastic_K &lt; overbought_level and stochastic_K[1] &gt; overbought_level</pre><p>Now, to simulate the orders, we use <strong>strategy.entry()</strong> built-in function which allows us to either go long or short whenever the signal is generated. The position is initiated at the opening price after the signal (Which itself is generated at the close).</p><pre>strategy.entry(&#39;Buy&#39;,  strategy.long,  when = buy_signal)<br>strategy.entry(&#39;Sell&#39;, strategy.short, when = sell_signal)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*anl8qGqlQs1pfDpQ.png" /><figcaption><strong>Signal chart following the strategy.</strong></figcaption></figure><pre><strong>// This source code is subject to the terms of the Mozilla Public License 2.0 at </strong><a href="https://mozilla.org/MPL/2.0/"><strong>https://mozilla.org/MPL/2.0/</strong></a><strong><br>// © Sofien-Kaabar</strong>//@version=5<br>strategy(&quot;Our Stochastic Oscillator&quot;, overlay = false)lookback = input(title = &#39;Period&#39;, defval = 13)<br>oversold_level = input(title = &#39;Oversold&#39;, defval = 20)<br>overbought_level = input(title = &#39;Overbought&#39;, defval = 80)<strong>// highest high</strong><br>highest = ta.highest(high, lookback)<strong>// lowest low</strong><br>lowest = ta.lowest(low, lookback)<strong>// stochastic oscillator</strong><br>stochastic_K = ((close - lowest) / (highest - lowest)) * 100<br>stochastic_D = ta.sma(stochastic_K, 3)plot(stochastic_K)<br>plot(stochastic_D, color = color.orange)<br>hline(oversold_level)<br>hline(overbought_level)buy_signal  = stochastic_K &gt; oversold_level and stochastic_K[1] &lt; oversold_level<br>sell_signal = stochastic_K &lt; overbought_level and stochastic_K[1] &gt; overbought_levelstrategy.entry(&#39;Buy&#39;,  strategy.long,  when = buy_signal)<br>strategy.entry(&#39;Sell&#39;, strategy.short, when = sell_signal)</pre><h3>Summary</h3><p>To sum up, what I am trying to do is to simply contribute to the world of objective technical analysis which is promoting more transparent techniques and strategies that need to be back-tested before being implemented. This way, technical analysis will get rid of the bad reputation of being subjective and scientifically unfounded.</p><p>Medium is a hub to interesting reads. I read a lot of articles before I decided to start writing.</p><blockquote><strong><em>If you found this article helpful, please consider leaving a tip. Thanks for reading and best of luck on your job search !</em></strong></blockquote><blockquote><strong><em>Buy me a coffee so I can do surgery for my cat : </em></strong><a href="https://www.buymeacoffee.com/botservices"><strong><em>https://www.buymeacoffee.com/botservices</em></strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35d21459c498" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating a Random forest algorithm for financial trading decision-making]]></title>
            <link>https://medium.com/@marie.riviere/creating-a-random-forest-algorithm-for-financial-trading-decision-making-f9fc259fa0a3?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9fc259fa0a3</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[trade]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[finance]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Tue, 30 Aug 2022 18:26:38 GMT</pubDate>
            <atom:updated>2022-08-30T18:33:46.780Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LK2JjClOv1Y3g2Sy" /><figcaption>Photo by <a href="http://prod-upp-image-read.ft.com/f766b018-014b-11ea-a530-16c6c29e70ca">intelligenthq</a></figcaption></figure><p>Coding a random forest algorithm to predict the S&amp;P500’s daily direction through the Put/Call ratio.</p><blockquote><strong><em>Main idea of the article</em></strong><em>: We will create a random forest algorithm that predicts the Put/Call ratio’s direction for tomorrow. Using that information, we will try to predict tomorrow’s return for the S&amp;P500. Hence, we will not predict the direction of the equity market, </em><strong><em>rather we will try to predict the direction of a time series that is correlated to the S&amp;P500.</em></strong></blockquote><p>first things first, a call option is the right to <strong>buy </strong>a certain asset in the future at a determined price while a put option is the right to <strong>sell </strong>a certain asset in the future at a pre-determined price.</p><p>Hence, when you buy a call you get to buy something later and when you buy a put you get to sell something later. Every transaction has two sides, therefore, when you’re buying a call or a put, someone else is selling them to you. This brings two other positions that can be taken on options, selling calls and selling puts. <strong>The put/call indicator deals with the buyers of options and measures the number of put buyers divided by the number of call buyers</strong>. That gives us an idea on the sentiment of market participants around the specified equity (in our case it will be the US stock market).</p><p>A <strong>higher </strong>put/call ratio means that there are <strong>more put</strong> buyers (traders are betting on the asset going lower) and a lower put/call ratio signifies <strong>more call buyers</strong> (traders are betting on the rise of the asset). A known way of using this ratio in analyzing market sentiment is by evaluating the following scenarios:</p><ul><li><strong>A rising ratio signifies a bearish sentiment. Professionals “feel” that the market will go lower.</strong></li><li><strong>A falling ratio signifies a bullish sentiment. Professionals “feel” that the market will go up.</strong></li></ul><p>Do option markets offer insights to profitable spot/futures trading? this is what we will see in this back-test. By taking a look at the put/call ratio, <em>we want to translate the sentiment of market participants into trading opportunities.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*2mZl1Coz3mrGXsd8.png" /><figcaption>The Put/Call ratio. We can notice the more stationary characteristic compared to the S&amp;P500 index prices.</figcaption></figure><h3>Plan of attack</h3><p><strong>How will we do that?</strong> We will first begin by a simple known strategy where we buy the market when the Put/Call reaches <strong>1.2</strong> and sell (short) the market when the Put/Call reaches <strong>0.65</strong>. The holding period will be 20.</p><p>After having performed and analyzed this strategy, we will turn to a more advanced one which is the core of this study. <em>Is it possible to use a machine learning algorithm to predict tomorrow’s value of the Put/Call ratio and use that information to trade the market?</em> We’ll see, but first, let us try out the simpler (first) strategy. Note that the historical correlation between the changes in the S&amp;P500 index and the changes in the Put/Call ratio is between <strong>-0.35</strong> and <strong>-0.45</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*sTalUA4h7NMbw46L.png" /></figure><h3>Strategy #1</h3><blockquote><em>The intuition of this trading strategy is that 0.65 and 1.2 are statistical extremes and therefore, the extreme bearish sentiment around 1.2 is bound to reverse and a bottoming of the market out might occur. And vice versa.</em></blockquote><p>We’ll start by importing the necessary libraries. The usual ones necessary for every time series researcher.</p><pre># Importing libraries<br>import matplotlib.pyplot as plt<br>import numpy             as np<br>import pandas            as pd</pre><p>Let’s define the necessary variables. The <strong>holding_period</strong> is self-explanatory and is simply how long will we hold the position during initiation. The <strong>investment </strong>variable is our starting balance. And finally, the support and resistance levels are simply the implied statistical extremes that are used as triggers.</p><pre>holding_period = 20                    <br>investment     = 1000<br>support        = 1.2<br>resistance     = 0.65</pre><p>Now, by having an excel file called PCR we will import it to Python in order to be read and further cleaned.</p><pre># Importing data and calculating<br>Data = pd.read_excel(&#39;PCR.xlsx&#39;)<br>Data = np.array(Data)</pre><p>The excel file will be composed of two columns, the first one is the Put/Call ratio while the second one is the closing price of the SPX.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/180/0*lf1v5Eo_JvuqVthR.png" /><figcaption>Sample of the Dataframe PCR.</figcaption></figure><p>Simply put, we will buy whenever the PCR reaches 1.20 and sell whenever it reaches 0.65</p><pre># Buy/Sell conditions<br>for i in range(len(Data)):<br>    try:<br>        if Data[i, 0] &gt;= support and Data[i - 1, 0] &lt; support:<br>            Data[i + 1, 2] = 1 # We have added a new column<br>        <br>        elif Data[i, 0] &lt;= resistance and Data[i - 1, 0] &gt; resistance:<br>            Data[i + 1, 3] = -1 # We have added a new column<br>            <br>        else:<br>            continue<br>    except IndexError:<br>        pass</pre><p>If we do a quick check on the number of positions we have and whether they are longs (Buy orders) or shorts (Sell orders), then we can use the following code:</p><pre>np.count_nonzero(Data[:, 2])<br>np.count_nonzero(Data[:, 3])</pre><p>Giving us <strong>108 </strong>long positions, <strong>23 </strong>short positions, and <strong>131 </strong>positions taken in total between 2006 and 2019 (this is where the free PCR data stops).</p><p>Now, to simply calculate the returns over the trades while keeping in mind the holding period of 20:</p><pre># Returns<br>for i in range(len(Data)):<br>    try:<br>        if Data[i, 2] == 1:<br>            Data[i + holding_period, 4] = (Data[i + holding_period, 1] - Data[i, 1])<br>            <br>        if Data[i, 3] == -1:<br>            Data[i + holding_period, 5] = (Data[i, 1] - Data[i + holding_period, 1]) <br>            <br>    except IndexError:<br>        pass</pre><p>Having done all of this, we can chart the signals and the equity curve. Note, that the back-test was simplistic and more of an approximation as we didn’t include fees, commissions, contract changes, or risk management tools.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*AMvV2YxXCv1uINNt.png" /><figcaption>PCR Strategy — Signal chart. Green arrows are buy orders while red arros are sell orders.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*XiFX9VLbmQrGG8S7.png" /><figcaption>Equityve curve following the PCR strategy.</figcaption></figure><p>With the performance statistics:</p><pre>Profit factor        =  3.069Hit ratio            =  70.77 %Gross return         =  418.0 %Expectancy           =  32.17Realized RR          =  1.27Buy:Sell ratio       =  108 : 23Win% Buy:Sell        =  74.0 % : 52.0 %</pre><p><strong>Some observations that need to be said by simply looking at the signal chart</strong>: The number of long signals is noticeably higher than the number of short signals and the signal quality seems to be better. Long signals seem to capture well the botoms of corrections and consolidations. The equity curve of the strategy is somewhat satisfying but still not 100% reflecting the reality (did we really know that this strategy will work back in 2006 if we hadn’t had the results already?), but still, it is interesting to see if it will continue to behave as such in the future. We also note that with the 20-Day holding period used as well as the tendancy of markets to go up, we could have been pushed by this fact to be more profitable. Still, it is rare to see such beautiful upward sloping equity curve.</p><p>Will this strategy keep working for the forseeable future? That is hard to say but what is for sure, that it will take a long time to know. From the data above, in 13 years, we’ve only had 131 trades, amounting to 10 trades per year. Relaxing some conditions will give us 1 trade per month. It may not be bad for a sub-allocation of a portfolio to actively trade on this strategy assuming more optimization is done.</p><h3>Strategy #2</h3><blockquote><em>The intuition of this trading strategy is that by using a well-liked and well-known algorithm, we want to see if the Put/Call ratio is stationary and predictable enough to give us accurate signals to trade on. We will then compare both strategies and see which one outperformed and which one underperformed.</em></blockquote><p>Before we start by defining our algorithm and going through with the back-test, it is interesting to know how well our model will do in a Utopian world where we can predict tomorrow’s Put/Call ratio with <strong>100% accuracy</strong>? <strong>In other words, knowing the negative correlation and being sure of tomorrow’s change of the ratio, how good will we be at trading the US market?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*9zxb4eH0lA4taDm6.png" /><figcaption>Equity curve following a strategy that knows in advance the change in the Put/Call ratio.</figcaption></figure><p>So, it becomes mouth-watering to see whether we can predict the Put/Call ratio or not after seeing this almost perfect equity curve (Gross return around <strong>1,200%</strong> since 2006 and by trading everyday the open and close out at the end of the day). Surely, to put things into perspective, we’re not going to obtain such an equity curve and we’re not even sure the algorithm can even predict the PCR that well. <strong>But, the excitement of research is what makes us move forward</strong>. <em>Fact: I’ve started writing this article as I am creating the algorithm, therefore, most of what I have written so far has been before commencing the back-tests. Now, let’s start with the serious stuff.</em></p><p><strong>The Random Forest</strong> algorithm is part of a broader type of algorithms called ensemble methods. In layman’s terms, many decision trees form a random forest which is a type of ensemble methods.<strong> But, what is the intuition of a decision tree without going too much into technical details?</strong></p><p>A decision tree is a supervised learning algorithm that is both suitable for classification and regression problems due to its linear and non-linear suitability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*N9dVyduE0Fwb6Spi.png" /><figcaption>Source: <a href="https://www.researchgate.net/figure/Basic-structure-of-a-decision-tree-All-decision-trees-are-built-through-recursion_fig3_295860754">https://www.researchgate.net/figure/Basic-structure-of-a-decision-tree-All-decision-trees-are-built-through-recursion_fig3_295860754</a></figcaption></figure><p><em>Decision trees break down the dataset into smaller subsets using conditions to finally arrive at a probability estimate</em>. The first tree is called the root node while the ones following it (coming out of it) are referred to as decision nodes. The algorithm used to iterate through the trees to calculate entropy is called <strong>Iterative Dichotomiser 3</strong> (ID3) and was invented by Ross Quinlan. The idea here is that when you average out a lot of models, you will get a smoother and more accurate forecast. That is why random forest algorithms are well-suited for both classification and regression problems. The random forest algorithm is but a <strong>collection of decision trees</strong> to reduce overfitting and average out the results giving us supposedly better accuracy. Hence, a decision tree is just a random forest with one decision tree.</p><p>We’ll start by importing the necessary libraries. The first line imports the RandomForestRegressor function from sklearn which we will use below to predict the PCR.</p><pre># Importing libraries<br>from sklearn.ensemble    import RandomForestRegressor<br>import matplotlib.pyplot as plt<br>import numpy             as np<br>import pandas            as pd</pre><p>The <strong>trees </strong>variable is the key to the Random Forest algorithm, the more there are, the more complex the model will be. We will choose 20 trees in our example. The lag variable is how far in the past will we look to include in the prediction formula. A value of three means that the last 3 PCR change-in-values will be used to predict tomorrow’s change.</p><pre># Parameters<br>investment     = 1000                  <br>trees          = 20<br>lag            = 2</pre><blockquote><em>Now, what variables will the Random Forest algorithm use to predict tomorrow’s PCR? As said just above, we will use a technique called Autoregression in which lagged values are used as explanatory variables. This means that today and yesterday’s change in the PCR can help us predict tomorrow’s change (i.e. is it going to go up or down from today’s value).</em></blockquote><p>So, given the below data structure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/0*QtKfa_Nbu_iNvL0C.png" /></figure><p>Autoregression Matrix on the PCR. On the far left is the most recent observation. Going to the left, we take yesterday’s observations.</p><p>The above matrix means that if we take index 3 (the first row), then the -0.02 can be explained by 0.01, -0.07, and 0.06 given a calculated relationship. In other words, the first three columns are the explanatory variables that should possess some information on the last column.</p><p>We will call the first three columns <strong>X </strong>(the explanatory variables) and the last column <strong>Y</strong> (the independent variable).</p><pre># Separating the data<br>Determinant = len(Data.columns)<br>X = Data.iloc[:, 0:Determinant-1].values<br>Y = Data.iloc[:, -1].values<br>Y = np.reshape(Y, (-1, 1))</pre><p>So, now let’s take the X and Y datasets and divide them into training set and test set:</p><pre># Splitting the dataset into the Training set and Test set<br>X_train = X[:-projection,] #Training explanatory variables<br>Y_train = Y[:-projection,] #Training predictions<br>X_test  = X[-projection:,] #Test explanatory variables<br>Y_test  = Y[-projection:,] #Test predictions</pre><p>The below code defines the Random Forest function, chooses the number of trees (through the n_estimators variable) and fits the training set to obtain the implied relationship that we hope to continue towards the future.</p><pre># Fitting the model<br>regressor = RandomForestRegressor(n_estimators = trees)<br>regressor.fit(X_train, Y_train.ravel())</pre><p>What we want to do now, is take that relationship, give it a dataset similar to X_train and apply the forecasts. Luckily we have created our X_test dataset that is the out-of-sample structure. It should look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/335/0*HzFElSlvu1BiNA_D.png" /><figcaption>An example of X_test</figcaption></figure><pre># Predicting the Test set results<br>Prediction = regressor.predict(X_test) #Predict over X_test<br>Prediction = np.reshape(Prediction, (-1, 1))<br>Real = Y_test</pre><p>Doing that will give us two datasets, the real changes in the PCR and our predictions. Let’s concatenate them together and calculate the correlation coefficient to give us an idea whether we’ve been doing well or not. Although, the correlation coefficient doesn’t say anything about the accuracy (as the magnitudes may be large enough to distort it)</p><pre>Comparison = np.concatenate((Prediction_LR, RealLR), axis = 1)<br>np.corrcoef(Comparison[:,0], Comparison[:,1])</pre><p>The output will give us</p><pre># 0.343</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*xFVxU3EuG5IXiHfH.png" /><figcaption>PCR prediction in orange versus PCR real values.</figcaption></figure><p>Which is not bad for a start. But how well would that do if we try trading the S&amp;P500? Let’s continue. The below code is simply applying the strategy which is taking our PCR predictions, putting them in parallel of changes in the SPX while accounting for time bias, and <strong>then checking whether a negative prediction before the start of the day is associated with a positive SPX change in the end of the day or not?</strong></p><pre>DataSPX = pd.read_excel(&#39;PCR.xlsx&#39;)<br>    DataSPX = np.array(DataSPX)<br>    DataSPX = DataSPX[:, 1]<br>    DataSPX = np.reshape(DataSPX, (-1, 1))<br>    DataSPX = pd.DataFrame(DataSPX)<br>    DataSPX = DataSPX.diff()<br>    DataSPX = DataSPX.iloc[1:,].values<br>    <br>    DataSPX  = DataSPX[-projection:,]<br>    <br>    Combined = np.concatenate((Prediction_LR, DataSPX), axis = 1)<br>    Combined = adder(Combined, 4) #Function to add 4 columns (Check end of article)<br>    <br>    for i in range(len(Combined)):<br>        if Combined[i, 0] &lt; 0:<br>            Combined[i, 2] = 1<br>        if Combined[i, 0] &gt; 0:<br>            Combined[i, 3] = -1<br>    <br>    for i in range(len(Combined)):<br>        if Combined[i, 2] == 1 and Combined[i, 1] &gt; 0:<br>            Combined[i, 4] = Combined[i, 1]<br>        if Combined[i, 2] == 1 and Combined[i, 1] &lt; 0:<br>            Combined[i, 4] = Combined[i, 1]<br>        if Combined[i, 3] == -1 and Combined[i, 1] &gt; 0:<br>            Combined[i, 5] = -Combined[i, 1]<br>        if Combined[i, 3] == -1 and Combined[i, 1] &lt; 0:<br>            Combined[i, 5] = abs(Combined[i, 1])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*RbnjUmyWJegJ-pMA.png" /><figcaption>PCR Strategy using a Random Forest regressor.</figcaption></figure><pre>Profit factor        =  1.209Hit ratio            =  51.4 %Gross return         =  83.0 %Expectancy           =  1.67Realized RR          =  1.14</pre><p>The good news about the algorithm is that it doesn’t suffer from any bias. The long-to-short ratio is around 1.0 (Meaning there are as many buy orders as there are sell orders).</p><h3>Conclusion</h3><p>After performing the back-tests, we can make the following observations:</p><ul><li>Perfectly predicting the Put/Call ratio will give us a utopian index strategy.</li><li><strong>The first strategy’s results</strong> showed that the PCR might add some value in trading if it’s near the extreme. The signals as well as the performance show some timing capacities. The long predominance however makes the basic PCR strategy biased. I would recommend keeping it as a tool to aid in decision-making. For example, a trader wanting to go long the US stock market and the PCR is showing upper extreme values. This could be a confirmation to the existing conviction.</li><li><strong>The second strategy’s results </strong>showed that optimization is necessary to improve the model as a pure Random Forest with only 10 trees does not do much in predicting the direction of the stock market using the PCR.</li><li>What else? Sklearn library has many regression algorithms. We are not bounded by only using the Random Forest algorithm. We can try as many as we want and average out their predictions to create an even better model. The possibilities are limitless.</li></ul><pre># The adder function seen above. It adds a number of desired columns to a datasetdef adder(Data, times):<br>    <br>    for i in range(1, times + 1):<br>    <br>        z = np.zeros((len(Data), 1), dtype = float)<br>        Data = np.append(Data, z, axis = 1)return Data</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*3k7vzO2MVpwcs0Zi.jpeg" /><figcaption><a href="https://pixabay.com/photos/blur-chart-computer-data-finance-1853262/">https://pixabay.com/photos/blur-chart-computer-data-finance-1853262/</a></figcaption></figure><blockquote><strong><em>If you found this article helpful, please consider leaving a tip. Thanks for reading and best of luck on your job search !</em></strong></blockquote><blockquote><strong><em>Buy me a coffee so I can do surgery for my cat : </em></strong><a href="https://www.buymeacoffee.com/botservices"><strong><em>https://www.buymeacoffee.com/botservices</em></strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9fc259fa0a3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Trading with Python Library Cloudcraftz]]></title>
            <link>https://medium.com/@marie.riviere/trading-with-python-library-cloudcraftz-f4d58d1bced2?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4d58d1bced2</guid>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Sun, 28 Aug 2022 17:29:15 GMT</pubDate>
            <atom:updated>2022-08-28T17:29:15.469Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*1IlkTXa4sEwfUodbfK37TA.jpeg" /><figcaption>Trading with Indicators</figcaption></figure><p>Hello , readers , I’m back again with my finance and trading series and sorry for not posting in the last week , got a little too caught up with the deadlines .Today we are going to use a python library that we use internally at <strong>Cloudcraftz Solutions Pvt. Ltd.</strong> for various experiments . Today we are going to see how to put that library to use for trading . This is a very simple strategy that we’ll be using to trade . So let’s start .</p><blockquote><strong><em>Note</em></strong><em>: If you’re wondering how to get this library , simply install using </em><em>pip install cloudcraftz .</em></blockquote><pre>import pandas as pd<br>import matplotlib.pyplot as plt<br>import numpy as np <br>import yfinance as yf</pre><pre>from cloudcraftz.technical_indicators import simple_moving_average, exponential_moving_average<br>from cloudcraftz.optimizers import sma_optimizer, ema_optimizer<br>from cloudcraftz.plots import cc_plot<br>from cloudcraftz.utils import split_func, get_signal_strategy, analyze_strategy, get_benchmark_result</pre><h3>Loading the price data from yahoo finance</h3><pre>df = yf.download(&quot;TCS&quot;, &quot;2007-01-01&quot;, &quot;2022-03-31&quot;)[*********************100%***********************]  1 of 1 completeddf.tail()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/0*IFiMkcZ_-tqorxh0.png" /></figure><p>So we have gathered the data , now we want to invest in TCS . So to replicate a real life trading scenario we will at the very beginning hold out a portion of the data which will serve as the unseen data for us .</p><p>Today we will be mainly using two technical indicators namely SMA and EMA . (Simple and Exponential Moving Averages (MA — abbreviated)).</p><pre># We&#39;ll be taking the close price as our price , assuming that we buy or sell at close .</pre><pre>train_X, test_X = split_func(df[&#39;Close&#39;], test_size=0.2)train_X.head()Date<br>2013-11-01    36.200001<br>2013-11-04    35.349998<br>2013-11-05    35.349998<br>2013-11-06    35.900002<br>2013-11-07    35.310001<br>Name: Close, dtype: float64</pre><p>There are many rules that are used by traders such as 5-day SMA signal or 8-day EMA signal, but rather relying on those we will let our library function decide which is best for us , here we will use the crossover strategy , Crossovers are one of the main moving average strategies. The first type is a price crossover, which is when the price crosses above or below a moving average to signal a potential change in trend. <strong>Say for example a 5-day moving average crosses over a 20-day moving average, then it signifies a strong trend of the presence of a bull market</strong>, because the 5 day moving average is giving us a very recent smoothened price of the stock as compared to a 20 day MA .</p><pre># Let&#39;s get the optimum time periods for our stock , we&#39;ll be using both SMA and EMA signals to trade .<br>t1_sma, t2_sma, sma = sma_optimizer(train_X)<br>t1_ema, t2_ema, ema = ema_optimizer(train_X)</pre><p>So we have got the optimal signal along with the optimal periods, we don’t have to do anything we just pass the prices and it does all the optimisations internally and spits out the results. Now comes the fun part , to simulate a real trading scenario we will take one row of the held out set at a time which will serve as each passing day .</p><pre>old_data = list(train_X.values)def signals(close):<br>    old_data.append(close)<br>    sma_short = simple_moving_average(old_data, {&#39;periods&#39;: t1_sma})<br>    sma_long = simple_moving_average(old_data, {&#39;periods&#39;: t2_sma})<br>    sig_today_sma = get_signal_strategy(sma_short, sma_long)<br>    <br>    ema_short = exponential_moving_average(old_data, {&#39;periods&#39;: t1_ema})<br>    ema_long = exponential_moving_average(old_data, {&#39;periods&#39;: t2_ema})<br>    sig_today_ema = get_signal_strategy(ema_short, ema_long)<br>    <br>    one, two = sig_today_sma[-1], sig_today_ema[-1]<br>    <br>    sig = 0.6 * two + 0.4 * one<br>    <br>    if sig &gt;= 0.5:<br>        return 1<br>    elif sig &lt;= -0.5:<br>        return -1<br>    else:<br>        return 0</pre><p>We got our function ready which gives us the SMA and EMA signals for that particular day , given the closing price or the buying price of that day . Now we know , that EMA or Exponential MA is very sensitive to price changes, so we give a 60% weight to that signal and 40% to our SMA signal . We have used <strong>0.5</strong> as our threshold for deciding <strong>whether to buy or not to buy or to hold (1, -1 or 0)</strong></p><p>What we are doing below is passing through each day (used a for loop for that) we get the price each day and pass it through our function signals to recieve the action that we must take on that particular day .</p><pre># We keep a list to keep track of our action , that is buy sell or hold<br>actions = []</pre><pre>for prices in test_X.values:<br>    action = signals(prices)<br>    actions.append(action)</pre><h3>Evaluation of our strategy</h3><p>For that we will be using the analyze_strategy function in the <strong>cloudcraftz&#39;s utils module .</strong></p><blockquote><strong><em>Note</em></strong><em>: Below before passing our data to </em><em>analyze_strategy function , we did a reset to make the date index as column and also renamed the </em><em>Close column to </em><em>prices or else it’ll give an error . (Read the doc-string before using the library functions).</em></blockquote><pre># pass the list of actions taken and the prices of each day with their dates.<br>prices = test_X.reset_index()<br>prices.rename(columns={&quot;Close&quot;: &quot;prices&quot;}, inplace=True)</pre><pre>analyze_strategy(actions, prices, plot=True)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/0*IiL19nvuBCUHf3-2.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*ARU2KRugtG126Awn.png" /></figure><p>WoW ! we got a return of 165% , so had we used this strategy to trade for the given period we would have made 165 times of what we invested . That actually makes sense as well since TCS is a IT company , for the period we are testing our strategy on, the prices actually had an upward trend for all IT based companies . But the question is did we actually did better as compared to the index ? I mean we need to have something to compare our strategy with right ? <strong>Let&#39;s compare it with </strong><strong>NIFTY50 to see how it performed .</strong></p><pre>get_benchmark_result(&quot;^NSEI&quot;, &quot;2020-07-27&quot;, &quot;2022-03-31&quot;)[*********************100%***********************]  1 of 1 completed</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/0*qvZi6c3M7LJcZIY9.png" /></figure><p>Amazing so we are beating the market by 8 times, the NIFTY50 index had an Annual return of 23.59% whereas we with our strategy have 165% , which is extremely good .</p><p>So the result that we got is too good , you may try this out on other stocks as well , though I’m kind of not sure if it’ll produce such good results , since it is a very simple strategy , try it out on various stocks from different industries to see how well it performs before you start investing in real time .</p><p>Thank you geeks , I hope you enjoyed this article , do let me know in the comments if you want to see more such posts , till then see ya geeks !!!</p><blockquote><strong><em>If you found this article helpful, please consider leaving a tip. Thanks for reading and best of luck on your job search !</em></strong></blockquote><blockquote><strong><em>Buy me a coffee so I can do surgery for my cat : </em></strong><a href="https://www.buymeacoffee.com/botservices"><strong><em>https://www.buymeacoffee.com/botservices</em></strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4d58d1bced2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Full Stack Data Science Project (from Data Collection to Predictive Modelling) for an E-commerce…]]></title>
            <link>https://medium.com/@marie.riviere/a-full-stack-data-science-project-from-data-collection-to-predictive-modelling-for-an-e-commerce-35f18b54a90a?source=rss-d17b4c68260e------2</link>
            <guid isPermaLink="false">https://medium.com/p/35f18b54a90a</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[ecommerce]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Marie Riviere]]></dc:creator>
            <pubDate>Sun, 28 Aug 2022 17:24:23 GMT</pubDate>
            <atom:updated>2022-08-28T17:24:23.473Z</atom:updated>
            <content:encoded><![CDATA[<h3>A Full Stack Data Science Project (from Data Collection to Predictive Modelling) for an E-commerce startup</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XpdUcqzlblXZewMvNiZNAw.jpeg" /></figure><p>Hello readers, today is Saturday and I’m back with another medium story. This one is a bit different , this was a P.O.C (Proof of Concept) I was asked to perform by the owner of an E-commerce startup before I started working as a full time Data Scientist at <strong>Cloudcraftz Solutions Pvt. Ltd.</strong></p><p>One of my friends started an E-commerce business few months back, so one fine day he called me up in the morning and asked me — “Is it possible to know what people in our area or in nearby areas may be interested in buying online ? what kind of people likes to buy what kind of stuff ?” . That is when I got my first commissioned Data Science project .</p><p>Since it was a new startup they didn’t have much data with them, the task of collecting the data was also a part of my job , according to the demand of the owner I created a simple google form and circulated it across various cities throughout India (mostly West Bengal since it was a very new startup their main focus was on the home state.)</p><blockquote><em>Note : The name of the startup and their product has been changed in this post due to privacy issues .</em></blockquote><p>So, here it begins .</p><h3>Notebook Imports</h3><pre>%matplotlib inline<br>import pandas as pd<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>from sklearn.model_selection import train_test_split<br>from sklearn.ensemble import RandomForestClassifier<br>from sklearn.metrics import accuracy_score</pre><p>Below is the Dataset of 200 people who responded to our survey form (The Data is too less for any Data Related or ML Project, but I still didn’t want to give up).</p><h3>Loading the Dataset</h3><pre>data = pd.read_csv(&#39;Survey.csv&#39;)</pre><h3>Exploratory Data Analysis (EDA)</h3><pre>data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*t1z6SAdZJH1kSk7f.png" /></figure><pre>data.tail()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2NuYB_X686yfWzdu.png" /></figure><pre>data.info()&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>RangeIndex: 200 entries, 0 to 199<br>Data columns (total 9 columns):<br> #   Column                                   Non-Null Count  Dtype <br>---  ------                                   --------------  ----- <br> 0   Age                                      200 non-null    int64 <br> 1   Gender                                   200 non-null    object<br> 2   State/ Union Territory                   200 non-null    object<br> 3   City                                     200 non-null    object<br> 4   Monthly Family Income                    200 non-null    object<br> 5   Product Interested in Buying Online      200 non-null    object<br> 6   Employment Status                        200 non-null    object<br> 7   You preferably use  for shopping online  200 non-null    object<br> 8   Have you heard about XPay ?              200 non-null    object<br>dtypes: int64(1), object(8)<br>memory usage: 14.2+ KB# Checking for null values if any <br>pd.isnull(data).sum()Age                                        0<br>Gender                                     0<br>State/ Union Territory                     0<br>City                                       0<br>Monthly Family Income                      0<br>Product Interested in Buying Online        0<br>Employment Status                          0<br>You preferably use  for shopping online    0<br>Have you heard about XPay ?                0<br>dtype: int64</pre><h3>Monthly Family Income vs Online Commodity Preferred</h3><p>Here we are grouping the dataset based on the monthly income of the family to see how financial status affects the choice of an individual in an online shopping landscape.</p><pre>group_data = data.groupby([&#39;Monthly Family Income&#39;, &#39;Product Interested in Buying Online&#39;])group_data.count()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*miT6K-LLd7Qonbzf.png" /></figure><p>From the above grouped data analysis it can be figured out that people falling in the income range of <strong>10K and Below</strong> to <strong>70K</strong> usually prefer buying books online , whereas people falling in the income range of <strong>50–70K and Above</strong> prefer buying Electronic Gadgets such as Laptops, Mobiles, Watches etc from online stores .</p><h3>Online Commodity preferred v/s Age</h3><p>Here we are grouping the dataset based on the Age to see how age plays a role in deciding the kind of product one buys online.</p><pre>group_data_age = data.groupby([&#39;Product Interested in Buying Online&#39;])<br>mean_age = group_data_age[&#39;Age&#39;].mean()colors = {&#39;Male&#39;:&#39;tab:blue&#39;, &#39;Female&#39;:&#39;tab:orange&#39;}</pre><pre>plt.figure(figsize=(8, 5))<br>plt.plot(mean_age, &#39;gv--&#39;)<br>plt.xticks(rotation=90, fontsize=14)<br>plt.title(&#39;Mean Age vs Products Frequently bought online&#39;, fontsize=15)<br>plt.xlabel(&#39;Products&#39;, fontsize=14)<br>plt.ylabel(&#39;Mean Ages&#39;, fontsize=14)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*z-Zi6H6pNWbTaPw5.png" /></figure><h3>Now let’s see how the gender is influencing the market alongside the age</h3><p>Is gender also playing some role ? let’s explore that as well, how age together with gender plays a role here .</p><pre>new_data = pd.concat([data[&#39;Gender&#39;], data[&#39;Age&#39;], data[&#39;Product Interested in Buying Online&#39;]], axis=1)<br>new_group_male = new_data[new_data.Gender==&#39;Male&#39;].groupby(&#39;Product Interested in Buying Online&#39;)<br>new_group_female = new_data[new_data.Gender==&#39;Female&#39;].groupby(&#39;Product Interested in Buying Online&#39;)age_male = new_group_male[&#39;Age&#39;].mean()<br>age_female = new_group_female[&#39;Age&#39;].mean()plt.figure(figsize=(8, 6))<br>plt.plot(age_male, &#39;bo--&#39;)<br>plt.plot(age_female, &#39;rv--&#39;)<br>plt.xticks(rotation=90)<br>plt.ylabel(&#39;Mean Age&#39;, fontsize=14)<br>plt.legend(labels=(&#39;Male&#39;, &#39;Female&#39;))<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*qdKrUqnCxsknjhpu.png" /></figure><h3>Few Notable Conclusion from the above data analysis :</h3><ol><li>Fashion Items are mostly bought online by women having a mean age greater than that of men , and online bill payments have a similar trend as well .</li><li>Mobile Accessories, Books and Shoes are bought online by men and women having almost the same mean age .</li><li>Other than these two , in all other categories the average or mean age of men is greater than that of women .</li></ol><h3>City Tier vs Online Products</h3><p>We all know that the lifestyle of people vary from one place to the other , a person living in a village will have a lifestyle completely different from a person living in a town or a city , that may also play a major role in deciding what people prefer buying online .</p><pre>pie_chart = []</pre><pre>num = data[data.City==&#39;Tier 1&#39;].count()<br>pie_chart.append(num.Age)</pre><pre>num = data[data.City==&#39;Tier 2&#39;].count()<br>pie_chart.append(num.Age)</pre><pre>num = data[data.City==&#39;Tier 3&#39;].count()<br>pie_chart.append(num.Age)</pre><p><strong>This is the percentage of Tier 1 , Tier 2 and Tier 3 readings in the data</strong></p><p>A lot of entry in our dataset is from Tier 1 cities , though we tried our best to get data from all kinds of cities , towns and villages , but with the limited time and resources that was really difficult .</p><pre>labels = &#39;Tier 1&#39;, &#39;Tier 2&#39;, &#39;Tier 3&#39;</pre><pre>plt.figure(figsize=(5, 5))<br>plt.pie(pie_chart, labels=labels, autopct=&#39;%1.1f%%&#39;, shadow=True, startangle=140)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/286/0*R2rpl0KK7rv-Vbym.png" /></figure><pre>group_tier1 = data[data.City==&#39;Tier 1&#39;].groupby(&#39;Product Interested in Buying Online&#39;)<br>group_tier2 = data[data.City==&#39;Tier 2&#39;].groupby(&#39;Product Interested in Buying Online&#39;)<br>group_tier3 = data[data.City==&#39;Tier 3&#39;].groupby(&#39;Product Interested in Buying Online&#39;)new_list = data[&#39;Product Interested in Buying Online&#39;].unique()</pre><pre>Tier_1, Tier_2, Tier_3 = [], [], []<br>labels2, labels3 = [], []</pre><pre>for i in new_list:<br>    Tier_1.append(group_tier1.get_group(i).Age.count())<br>    <br>for i in new_list:<br>    if i==&#39;Shoes&#39;:<br>        pass<br>    else:<br>        labels2.append(i)<br>        Tier_2.append(group_tier2.get_group(i).Age.count())<br>        <br>for i in new_list:<br>    if i==&#39;Shoes&#39; or i==&#39;Electrical Appliances (Eg: AC, TV, Mixer Grinder etc)&#39; or i==&#39;Watches&#39; or i==&#39;Home Accessories (Eg: Chair, Tables)&#39;:<br>        pass<br>    else:<br>        labels3.append(i)<br>        Tier_3.append(group_tier3.get_group(i).Age.count())</pre><h3>Now we’ll see how the items are distributed in these 3 tiers of Cities (separately) .</h3><pre># pie chart for Tier 1</pre><pre>labels = new_list</pre><pre>plt.figure(figsize=(10, 8))<br>plt.pie(Tier_1, labels=labels, autopct=&#39;%1.1f%%&#39;, shadow=True, startangle=140)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/0*X4vC8tvPIvFf8GmE.png" /></figure><pre># pie chart for Tier 2</pre><pre>labels = labels2</pre><pre>plt.figure(figsize=(10, 8))<br>plt.pie(Tier_2, labels=labels, autopct=&#39;%1.1f%%&#39;, shadow=True, startangle=140)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/0*FGX1HV0Hr8UMqx-4.png" /></figure><pre># pie chart for Tier 3</pre><pre>labels = labels3</pre><pre>plt.figure(figsize=(10, 8))<br>plt.pie(Tier_3, labels=labels, autopct=&#39;%1.1f%%&#39;, shadow=True, startangle=140)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/0*Q76C8Up_EP9yT8qA.png" /></figure><p>One notable fact is , books are something that people mostly prefers buying online irrespective of where they live .</p><h3>Lastly we would love to see the distribution of things with respect to Employment Status</h3><pre>group_emp = data.groupby([&#39;Product Interested in Buying Online&#39;, &#39;Employment Status&#39;])group_emp.count()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zoTZkNLyOHoNjMj1.png" /></figure><p><strong>The demand of books, Mobile Accessories and Gadgets are quite high among the students . We will not be exploring the features ‘You preferably use for shopping online’ and ‘Have you heard about XPay ?’ because there’s hardly any variance in the those two columns and therefore doesn’t seem to affect the decisions much .</strong></p><p>But let’s not leave those two features mentioned above unexplored .</p><pre># preferably used platform for shopping online<br>data.iloc[:, 7].value_counts()Amazon                 116<br>Flipkart                73<br>Myntra                   3<br>Auto. Com                2<br>Nykaa                    1<br>Jio Mart                 1<br>Groccer                  1<br>Do not shop online       1<br>TataCLIQ                 1<br>Ajio                     1<br>Name: You preferably use  for shopping online, dtype: int64</pre><p>We may conclude that most of the people use either <strong>Amazon</strong> or <strong>Flipkart</strong> for online shopping .</p><h3>Modelling the data for predicting the future outcomes</h3><pre># here we are taking just the categorical variable to get the dummy variables </pre><pre>features = [&#39;Gender&#39;, &#39;City&#39;, &#39;Monthly Family Income&#39;, &#39;Employment Status&#39;]<br>target = [&#39;Product Interested in Buying Online&#39;]model_data = pd.get_dummies(data[features])<br>model_target = pd.get_dummies(data[target])# Since we found that age is an important factor in our analysis so we&#39;ll include that as well .</pre><pre>model_data = pd.concat([model_data, data[&#39;Age&#39;]], axis=1)train_X, test_X, train_y, test_y = train_test_split(model_data, model_target, test_size=0.2, random_state=1)model = RandomForestClassifier(n_estimators=100, max_depth=11, random_state=1)<br>model.fit(train_X, train_y)</pre><pre>accuracy_score(test_y, model.predict(test_X)) * 10030.0</pre><p>So, the result is not at all good , but I was not surprised , why ? There are mainly two reasons why :<br>1. The amount of data was too less as we already stated.<br>2. The dataset is a skewed one , we were not able to capture the real scenario , the dataset was somewhat biased .</p><p>But we still tried to make it better with some statistical tests such as chi-square test , but which again didn’t help much .</p><p><strong>On performing the chi-square test we found that the Gender and Employment Status property can be easily rejected .</strong></p><pre># here we are taking just the categorical variable to get the dummy variables <br>features = [&#39;City&#39;, &#39;Monthly Family Income&#39;]<br>target = [&#39;Product Interested in Buying Online&#39;]</pre><pre>model_data = pd.get_dummies(data[features])<br>model_target = pd.get_dummies(data[target])</pre><pre>model_data = pd.concat([model_data, data[&#39;Age&#39;]], axis=1)</pre><pre>train_X, test_X, train_y, test_y = train_test_split(model_data, model_target, test_size=0.2, random_state=1)</pre><pre>model = RandomForestClassifier(n_estimators=100, max_depth=11, random_state=1)<br>model.fit(train_X, train_y)</pre><pre>accuracy_score(test_y, model.predict(test_X)) * 10037.5</pre><h3>Conclusion</h3><ol><li><strong>Age , City and Monthly Family Income</strong> turns out to be the most important factors in determining what ones buy from online stores .</li><li>Amount of data plays a very significant role that decides the success of a data science project .</li><li>Data must be not be biased , we should try to collect or include data that captures the real picture of the whole scenario .</li></ol><p>So, thank you readers, I hope you liked this article , though the results were not up to the mark but it brought about a whole lot of learning , and that is the harsh reality . If you liked this post do share it your friends and colleagues . See ya geeks !</p><blockquote><strong><em>If you found this article helpful, please consider leaving a tip. Thanks for reading and best of luck on your job search !</em></strong></blockquote><blockquote><strong><em>Buy me a coffee so I can do surgery for my cat : </em></strong><a href="https://www.buymeacoffee.com/botservices"><strong><em>https://www.buymeacoffee.com/botservices</em></strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35f18b54a90a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>