<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mohd Sadiq on Medium]]></title>
        <description><![CDATA[Stories by Mohd Sadiq on Medium]]></description>
        <link>https://medium.com/@msadiq074?source=rss-f802ef7375fa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZcwvQI4vZl9TW295F8LvRA.jpeg</url>
            <title>Stories by Mohd Sadiq on Medium</title>
            <link>https://medium.com/@msadiq074?source=rss-f802ef7375fa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 05:00:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@msadiq074/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a Quantitative Finance Project IV — Got Bored]]></title>
            <link>https://medium.com/@msadiq074/building-a-quantitative-finance-project-iv-got-bored-615e7defb713?source=rss-f802ef7375fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/615e7defb713</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[fintech]]></category>
            <dc:creator><![CDATA[Mohd Sadiq]]></dc:creator>
            <pubDate>Mon, 04 Mar 2024 01:01:49 GMT</pubDate>
            <atom:updated>2024-03-04T01:01:49.370Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Quantitative Finance Project IV — Got Bored</h3><h3>How to maintain code readability?</h3><p>One of the main issues that most developers face is not being able to understand other people’s code easily. There can be multiple factors why one has issues understanding it but one factor can be having poor coding standards. Since my primary coding language is Python, I would try my best to maintain my code to the upmost quality. The coding standard followed by most python developers is the PEP8 standards. A list of the PEP8 standards are documented <a href="https://peps.python.org/pep-0008/">here</a>.</p><p>Since I am human I am bound to make mistakes. There will be a few lines here and there that might not be properly formatted or some variable types improperly declared. I’m not the only one limited to these errors. Since the repository I’m building is public, anyone across the globe can feel free to participate or contribute. Hence, there are a few guidelines that need to be in place.</p><ol><li>The main branch will have branch protection. These include</li></ol><ul><li>Pull Request before merging: With an added ‘require approval’ condition</li><li>Require status checks to pass before merging: I would prefer that the custom tests I have set up to pass before anyone can merge into main</li><li>Require conflict resolution before merging: I don’t want a PR to go through without the developers being able to properly justify their changes/improvements/bugfixes.</li></ul><p>2. Setup workflows for static tests for maintaining code readability.</p><p>3. Setup workflows for unit tests as well.</p><h3>Setting up Static Tests</h3><p>Making formatting changes like indentation and line length limits can be set up a few python libraries like isort and black . To make sure that your code follows proper indentation and line length limits simply run these commands in the terminal.</p><pre>$ isort {path_to_python_files}<br>$ black {path_to_python_files}</pre><p>I generally use both together while formatting my code.</p><p>Since the formatting aspect of the code has been looked after, the next main thing to look after is type checking, type hints etc. I generally follow three python libraries while checking my scripts. These include pylint, flake8 , pycodestyle and mypy . All four of them in tandem have generally provided me a robust static type checking environment in my previous experience. To test out the code these can be called like</p><pre>$ pylint {path_to_python_files}<br>$ flake8 {path_to_python_files}<br>$ pycodestyle {path_to_python_files}<br>$ mypy {path_to_python_files}</pre><p>If there are any errors or any missing return type annotations or line length issues, one of these libraries is sure to point it out.</p><pre># Sample flake8 error<br>scripts\stock_downloader.py:62:13: F841 local variable &#39;df3&#39; is assigned to but never used<br><br># Sample mypy error<br>scripts\stock_downloader.py:38: error: Missing return statement  [return]</pre><p>Now, I wouldn’t want to go about and download someone’s code from GitHub, switch to their branch or PR and then run these tests manually. That would not be ideal. I would like this test to happen automatically every time someone tries to make a PR to my main branch. GitHub luckily gives us the freedom to implement workflows on different events within GitHub. One such event is a On Pull Request. Hence, we can have our static tests run every time someone makes a PR.</p><p>One way to visualize the functioning behind it is that GitHub will start an instance with a given amount of RAM, processing power and disk space. Within environment (ideally Ubuntu), we need to have our code present and have our tests run. If all the steps ran without any errors, then the status check for that particular test passes. Else if it fails, the merge to PR would be blocked unless someone fixes their code. Having these tests run in a docker container also abstracts the developer from relying on GitHub’s instance environment for proper installation of libraries.</p><p>Hence we define our docker containers in a file called Dockerfile.dev . This file will contain blueprints of our docker and what we would like to have in the container.</p><pre># Dockerfile.dev<br><br>FROM python:latest<br><br>RUN mkdir -p /usr/src/app<br><br>COPY scripts /usr/src/app/scripts<br>COPY tables /usr/src/app/tables<br>COPY Makefile /usr/src/app/Makefile<br>COPY requirements.txt /usr/src/app/requirements.txt<br>COPY requirements-dev.txt /usr/src/app/requirements-dev.txt<br><br>WORKDIR /usr/src/app<br><br>RUN pip install --no-cache-dir -r requirements.txt<br>RUN pip install --no-cache-dir -r requirements-dev.txt</pre><p>The Dockerfile is usually run in the parent folder of the repository. The container uses the python container as a base layer and adds our particular commands on top of it. This includes having copying our code onto the container and also installing the given libraries. Alongside the rest of the code there are 3 more files that have been created to make my life a build easier. requirements.txt contains a list of all the libraries that would be necessary to run the final project and the requirements-dev.txt contains libraries that would be helpful in diagnosing or testing the code.</p><p>Makefile contains shortened commands that would be easier to refer to.</p><pre>static-tests:<br> pylint scripts tables<br> flake8 scripts tables<br> pycodestyle scripts tables<br> mypy scripts tables<br><br>static-tests-docker:<br> docker compose build qffun-db-tests<br> docker compose run --rm qffun-db-tests make static-tests</pre><p>Hence a list of commands can simply be called using the make command.</p><pre>make static-tests-docker # To build a docker and run static tests within it</pre><p>Alongside these files the docker-compose.yml file will also have a few changes made to it. It will include a new service called qffun-db-tests that is meant to create us a docker container to run all our tests within it.</p><pre>services:<br>  qffun-static-test:<br>    build: <br>      context: .<br>      dockerfile: Dockerfile.dev<br>    image: qffun-postgres-dev</pre><p>Now we have everything that we need to set up static tests on our PRs. The way we define steps to run our PRs is in a yml file stored in the .github/workflows folder. To run static tests, the workflow configuration is defined as</p><pre>name: Pull Request Tests<br>on:<br>  pull_request:<br>    branches:<br>      - development<br>      - main<br>      - integration<br><br>jobs:<br>  tests:<br>    runs-on: ubuntu-latest<br>    steps:<br>      - uses: actions/checkout@v4<br>      - name: Static Tests<br>        run: |<br>            make static-tests-docker</pre><p>Once we set these up, we can see the code in action on GitHub every time anyone wants to create a PR to main.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*EBTqNIpgSE-hTUgxLIJmag.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*NWOvTAFnpCA84kAUeIMRvQ.png" /><figcaption>A failed and successful static test run</figcaption></figure><p>If a person wishes to check their code quality locally and don’t wish to rely on constantly waiting for static tests to be completed on a PR, they can run the following command and get the same report.</p><pre>make static-tests-docker</pre><p>If there are any errors on the PR, developers can easily refer to the respective action for it and get the logs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/919/1*D3GSxUw0Pw0SBHLmzJ598A.png" /><figcaption>Example logs of a failed static test run</figcaption></figure><h3>Why so much effort?</h3><p>One must wonder why so much effort is being put into this one repository? Why don’t just have the ML code download and save the data at runtime? There are advantages to having the entire code in one place like easier end-to-end testing, quick deployment, faster (in general) etc. Such kind of a way of architecting a system falls under the category of a Monolith architecture. In it every component is tightly knit to each other and the entire system is usually deployed at the same time.</p><p>Unfortunately having tightly knit modules also leads to lesser flexibility. A single bug can break the entire system and is generally harder to maintain. If there is a single change that needs to be made then the entire application needs to be deployed again.</p><p>Having components separated helps me break up each component and design them individually. Such a system is called a microservice architecture. Each component is independent of each other and communicate through endpoints/APIs. This makes the underlying functioning of each component agnostic to different components. The ML model doesn’t need to know how I store the data, whether its in an AWS RDS service or a PostgreSQL database or a NoSQL database. All it cares about is receiving the training dataset in a particular format. Similarly the ML model doesn’t need to define benchmarks and training dataset splits. It is meant to train on data and validate its results on the benchmarks. As time goes on the benchmarks can be updated and the training dataset can be replaced with an online learning system in which it constantly gets a stream of data from our data source component and just trains on it. Similarly if I wish to upgrade my model, all I need to do would be to upgrade my ML component. My database remains untouched.</p><p>Each component can be scaled up individually without affecting other components and the best part about it is that I can assign people tasks much more efficiently and review different components much more thoroughly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=615e7defb713" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Quantitative Finance Project III — Got Bored.]]></title>
            <link>https://medium.com/@msadiq074/building-a-quantitative-finance-project-iii-got-bored-c72d6d1566d8?source=rss-f802ef7375fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/c72d6d1566d8</guid>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[sqlalchemy]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mohd Sadiq]]></dc:creator>
            <pubDate>Sun, 25 Feb 2024 02:35:38 GMT</pubDate>
            <atom:updated>2024-02-25T02:35:38.447Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Quantitative Finance Project III — Got Bored.</h3><h3>How do I fill my Database?</h3><p>Well since we are using sqlalchemy, it is fairly simple to connect to the database and use a python script to fill it up. For now to maintain simplicity, it seems like an easy choice to have a scripts folder that shall contain all our python scripts to test out our functions that we run locally. Hence to initialize all the tables in the database we can use the following script.</p><pre># scripts/create_database.py<br><br>import os<br>from sqlalchemy import create_engine<br>from tables import Base<br>from tables.benchmark_dataset import BenchmarkDataset<br>from tables.distribution_table import DistributionDataset<br>from tables.stock_price import StockPriceDataset<br>from tables.training_dataset import TrainingDataset<br><br><br>def main() -&gt; None:<br>    engine = create_engine(str(os.getenv(&quot;DB_URL&quot;)), echo=True)<br>    Base.metadata.create_all(engine)<br><br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><p>Also before running the script it is important to set the Database URL as an environment variable. If you have set it up as a docker the general format of the environment variable would likepostgresql+psycopg2://{username}:{password}@localhost:5432/{database} . Depending on where the database is hosted the URL will change accordingly. To set it as an environment variable one can use the following commands.</p><pre># Windows<br>$env:DB_URL = &#39;...&#39;<br><br>#Linux<br>export DB_URL=&#39;...&#39;</pre><p>To run this script I ideally call it using the following command from within the parent directory.</p><pre>$ python -m scripts.create_database</pre><p>This should help in creating all the tables that are necessary in the database. To populate the stock price table, we can have a script that gets the entire history of a particular stock, create individual entries for the table and then upload it to the database. This can be looped over all the assets present in the S&amp;P 500.</p><pre># scripts/populate_database.py<br><br>import logging<br>from typing import List<br>import pandas as pd<br>from scripts import add_entry_to_database<br>from scripts.stock_downloader import StockDownloader<br>from tables.stock_price import StockPriceDataset<br><br><br>def get_SNP500() -&gt; List[str]:<br>    &quot;&quot;&quot;Scrapes the Wikipedia page to get the S&amp;P 500 Assets&quot;&quot;&quot;<br><br>    tickers = pd.read_html(<br>        (&quot;https://en.wikipedia.org/wiki/&quot; &quot;List_of_S%26P_500_companies&quot;)<br>    )[0]<br><br>    # Conver the dataframe into a list of tickers<br>    return list(tickers[&quot;Symbol&quot;].iloc[:].values)<br><br><br>def fill_database(tickers: List[str]) -&gt; None:<br>    &quot;&quot;&quot;Script used to fill the database&quot;&quot;&quot;<br><br>    downloader = StockDownloader()<br><br>    for ticker in tickers:<br>        # Download the entire stock history of an asset<br>        stock_price_history = downloader.download_stocks(ticker)<br>        asset = ticker<br>        try:<br>            # Try creating entries for the table <br>            stock_history_objects = []<br>            for index, row in stock_price_history.iterrows():<br><br>                # represent the entry in a dictionary<br>                entry_dict = {<br>                    &quot;date&quot;: index,<br>                    &quot;asset&quot;: asset,<br>                    &quot;open&quot;: float(row[&quot;Open&quot;]),<br>                    &quot;high&quot;: float(row[&quot;High&quot;]),<br>                    &quot;low&quot;: float(row[&quot;Low&quot;]),<br>                    &quot;close&quot;: float(row[&quot;Close&quot;]),<br>                    &quot;volume&quot;: int(row[&quot;Volume&quot;]),<br>                    &quot;dividends&quot;: float(row[&quot;Dividends&quot;]),<br>                    &quot;stock_split&quot;: float(row[&quot;Stock Splits&quot;]),<br>                }<br>                stock_price_entry = StockPriceDataset(**entry_dict)<br>                stock_history_objects.append(stock_price_entry)<br><br>            # Add all the entries to the table of a particular stock<br>            add_entry_to_database(stock_history_objects)<br>        except Exception as e:<br>            <br>            # If it errors then log it.<br>            logging.error(f&quot;{e}, {ticker}&quot;)<br>        <br>        # A means to keep track of which assets were processed.<br>        print(f&quot;Finished Populating {asset}&quot;)<br><br><br>if __name__ == &quot;__main__&quot;:<br>    fill_database(get_SNP500())</pre><p>To facilitate the downloading process and adding entries to the Database, I further abstracted it and defined those functionality in their own respective modules.</p><pre># scritps/stock_downloader.py<br><br>import logging<br>import pandas as pd<br>import yfinance as yf<br><br><br>class StockDownloader:<br>    &quot;&quot;&quot;<br>    Class<br>        Downloads stock history<br><br>    Methods:<br>        download_stocks:<br>            Args:<br>                tickers (str): Ticker of the asset in question<br><br>    &quot;&quot;&quot;<br><br>    def download_stocks(self, ticker: str) -&gt; pd.DataFrame:<br><br>        &quot;&quot;&quot;<br>        Method<br>            Download Stock History<br><br>        Args:<br>            ticker (str): ticker code<br><br>        Output:<br>            (pd.DataFrame) : pandas dataframe that has the<br>                            full history of the stock<br><br>        &quot;&quot;&quot;<br><br>        # Try downloading<br>        try:<br>            tickerObj = yf.Ticker(ticker)<br>            tickerObjHist = tickerObj.history(period=&quot;max&quot;)<br>            return tickerObjHist<br><br>        # If failed then log it<br>        except Exception as e:<br>            logger.error(f&quot;{e}, {ticker}&quot;)<br><br>        # If failed then return an emtpty DataFrame<br>        df3 = pd.DataFrame(<br>            columns=[<br>                &quot;Date&quot;,<br>                &quot;Open&quot;,<br>                &quot;High&quot;,<br>                &quot;Low&quot;,<br>                &quot;Close&quot;,<br>                &quot;Volume&quot;,<br>                &quot;Dividends&quot;,<br>                &quot;Stock Splits&quot;,<br>            ]<br>        )<br>        return df3</pre><pre># scripts/__init__.py<br><br>import os<br>from typing import Any, List<br><br>from sqlalchemy import create_engine<br>from sqlalchemy.orm import Session<br><br><br>def add_entry_to_database(objects: List[Any]):<br>    engine = create_engine(str(os.getenv(&quot;DB_URL&quot;)))<br><br>    with Session(engine) as session:<br>        session.add_all(objects)<br>        session.commit()</pre><p>Running the script to populate should be simple. Now to populate our database (after setting up the DB_URL env variable), we can simple use the command</p><pre>$ python -m scripts.populate_database</pre><h3>Repository Structure</h3><p>As we are putting in more and more scripts it is also important for us to have a look at the current state of the repository structure. The files and folder structure can be summarized below</p><pre>parent_folder<br>│  docker-compose.yml         <br>├─scripts<br>│  │  create_database.py<br>│  │  populate_database.py<br>│  │  stock_downloader.py<br>│  └─ __init__.py<br>└─tables<br>    │  benchmark_dataset.py<br>    │  distribution_table.py<br>    │  stock_price.py<br>    │  training_dataset.py<br>    └─ __init__.py</pre><p>Let’s take a pit stop here. Update on the next post will be here soon…</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c72d6d1566d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Quantitative Finance Project II— Got Bored.]]></title>
            <link>https://medium.com/@msadiq074/building-a-quantitative-finance-project-ii-got-bored-7e5433a1c853?source=rss-f802ef7375fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e5433a1c853</guid>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[fintech]]></category>
            <dc:creator><![CDATA[Mohd Sadiq]]></dc:creator>
            <pubDate>Sun, 25 Feb 2024 01:07:40 GMT</pubDate>
            <atom:updated>2024-02-25T01:25:41.966Z</atom:updated>
            <content:encoded><![CDATA[<h3>Retrieving the Data</h3><p>Well the next steps seem simple enough. We need a way to download data from a third party API and save the onto our custom database. Currently I’m using yfinance python library to download the stock information. I’m planning to retrieve all the history of the stocks present in the S&amp;P 500 at the date 24th February 2024. Downloading the data is simple enough. The code comprises of two lines.</p><pre>import yfinance as yf<br><br>tickerObj = yf.Ticker(ticker)<br>tickerObjHist = tickerObj.history(period=&quot;max&quot;)</pre><p>This should return us a pandas dataframe with the columns of [&#39;Date&#39;, &#39;Open&#39;, &#39;High&#39;, &#39;Low&#39;, &#39;Close&#39;, &#39;Volume&#39;, &#39;Dividends&#39;, &#39;Stock Splits&#39;] . This seems like a fairly detailed description of the history of the stock. We can now alter the tables that were defined in the previous post and have a better schema for us.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wfo00zolxb5zTT9ytmLj3g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SG6DmhsVjU-pDeD4V2VNQg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q7QFI5aFrJnb0HfHUBNewA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1p_fEfjItDJRpTY_IPGfjw.png" /><figcaption>Updated scheme of tables present in the database</figcaption></figure><h3>Setting up the Database</h3><p>I could set up the database within my own system through installation methods described in the original SQL database documentations. Instead I would like to avoid a lot of the setups etc by using a prebuilt container of PostgreSQL. It is a quick and efficient way to setup instances without worrying too much about integrating it within my laptop’s environment. Since it would be in a docker container of its own, it should ideally work as intended. Alongside the postgres docker I would also be setting up an adminer docker. This provides me a WebGUI to access the database. The simplest way to setup the database is by defining their docker configuraitons in a docker-compose.ymlfile.</p><pre>services:<br><br><br>  database:<br>    image: postgres:latest<br>    ports:<br>      - 5432:5432<br>    environment:<br>      - POSTGRES_PASSWORD=devpassword<br>      - POSTGRES_USER=devdocker<br>    restart: always<br><br><br>  adminer:<br>    image: adminer:latest<br>    restart: always<br>    ports:<br>      - 8080:8080</pre><p>Comprehending the docker-compose.yml file is a fairly straightforward process. Under our services section we define the services or in this case our dockers to be setup. I defined by database docker to use the postgres:latestimage and map the ports of my machine to the docker’s ports (5432 to 5432). We also setup our environment variables that include the password and username to gain access to the database. Restart is also set to true so that our docker container loads everytime if docker shuts down. One more optimization that can be done is to mount a location onto the location where the PostgreSQL saves its database. This way one can easily maintain a local copy of the database onto their system without worrying too much about the health of the docker. For the purposes of this project, it is meant to serve as a development database server that we can connect to. Ideally this database can be hosted on a remote server (like an EC2 instance) or on a cloud service like AWS RDS. This way (in prod) we won’t have to setup our docker over and over again. The adminer docker setup pretty much also follows the same blueprint as the database docker.</p><p>To start up the services we can use the following command</p><pre>$ docker compose up -d database<br>$ docker compose up -d adminer</pre><p>This should set up everything for us and we can have a look at our database at the URL https://localhost:8080 . The following credentials can be used to login to access the database.</p><pre>&#39;System&#39;: &#39;PostgreSQL&#39;<br>&#39;Server&#39;: &#39;database&#39;<br>&#39;Username&#39;: &#39;devdocker&#39;<br>&#39;Password&#39;: &#39;devpassword&#39;</pre><p>After logging in, we can create a database called qffun . This database would host all our tables and all our data.</p><h3>Defining the Tables</h3><p>To avoid using SQL commands I would prefer to use a more interesting way to code out my creation and insertion commands for SQL. Python has an interesting library called sqlalchemy . This library contains functionalities to connect to different types of database and abstract the developer from using traditional SQL commands. To define my tables I can define them using ORM of sqlalchemy. For defining our tables I would also like to setup a few prerequisite functions and classes. All the prerequisite tables can be put in a tables folder with a __init__.py for the common functions and classes.</p><pre># tables/__init__.py<br><br>from typing import Any<br>from sqlalchemy.orm import DeclarativeBase<br><br><br>class Base(DeclarativeBase):<br>    &quot;&quot;&quot;Base Class for SQLAlchemy&quot;&quot;&quot;<br><br>    def __str__(self) -&gt; str:<br>        return get_attributes(self)<br><br>    def __repr__(self) -&gt; str:<br>        return get_attributes(self)<br><br><br>def get_attributes(obj: Any) -&gt; str:<br>    &quot;&quot;&quot;Returns the class in a readable format&quot;&quot;&quot;<br><br>    frmt = &quot;{}({})&quot;<br>    class_ = obj.__class__.__name__<br>    attrs = sorted((k, getattr(obj, k)) for k in obj.__mapper__.column.keys())<br>    sattrs = &quot;, &quot;.join(&quot;{}={!r}&quot;.format(*x) for x in attrs)<br>    return frmt.format(class_, sattrs)</pre><p>Once this is setup we can actually define our tables a bit better.</p><pre># tables/distribution_table.py<br><br>import uuid<br>from datetime import datetime<br>from typing import List<br>from sqlalchemy import ARRAY, DateTime, Float, String<br>from sqlalchemy.dialects.postgresql import UUID<br>from sqlalchemy.orm import Mapped, mapped_column<br>from tables import Base, get_attributes<br><br><br>class DistributionDataset(Base):<br>    __tablename__ = &quot;distribution_table&quot;<br><br>    unique_id: Mapped[uuid.UUID] = mapped_column(UUID, primary_key=True)<br>    date: Mapped[datetime] = mapped_column(DateTime)<br>    assets: Mapped[List[str]] = mapped_column(ARRAY(String))<br>    distribution: Mapped[List[float]] = mapped_column(ARRAY(Float))<br><br>    def __repr__(self) -&gt; str:<br>        return get_attributes(self)<br><br>    def __str__(self) -&gt; str:<br>        return get_attributes(self)</pre><pre># tables/benchmark_dataset.py<br><br>from datetime import datetime<br>from typing import List<br>from sqlalchemy import ARRAY, DateTime, String<br>from sqlalchemy.orm import Mapped, mapped_column<br>from tables import Base, get_attributes<br><br><br>class BenchmarkDataset(Base):<br>    __tablename__ = &quot;benchmark_dataset&quot;<br><br>    date: Mapped[datetime] = mapped_column(DateTime, primary_key=True)<br>    assets: Mapped[List[str]] = mapped_column(ARRAY(String), primary_key=True)<br><br>    def __repr__(self) -&gt; str:<br>        return get_attributes(self)<br><br>    def __str__(self) -&gt; str:<br>        return get_attributes(self)</pre><pre># tables/stock_price.py<br><br>from datetime import datetime<br>from sqlalchemy import DateTime, Float, Integer, String<br>from sqlalchemy.orm import Mapped, mapped_column<br>from tables import Base, get_attributes<br><br><br>class StockPriceDataset(Base):<br>    __tablename__ = &quot;stock_price_dataset&quot;<br><br>    date: Mapped[datetime] = mapped_column(DateTime, primary_key=True)<br>    asset: Mapped[str] = mapped_column(String, primary_key=True)<br>    open: Mapped[float] = mapped_column(Float)<br>    high: Mapped[float] = mapped_column(Float)<br>    low: Mapped[float] = mapped_column(Float)<br>    close: Mapped[float] = mapped_column(Float)<br>    volume: Mapped[int] = mapped_column(Integer)<br>    dividends: Mapped[float] = mapped_column(Float)<br>    stock_split: Mapped[float] = mapped_column(Float)<br><br>    def __repr__(self) -&gt; str:<br>        return get_attributes(self)<br><br>    def __str__(self) -&gt; str:<br>        return get_attributes(self)</pre><pre># tables/training_dataset.py<br><br>from datetime import datetime<br>from typing import List<br>from sqlalchemy import ARRAY, DateTime, String<br>from sqlalchemy.orm import Mapped, mapped_column<br>from tables import Base, get_attributes<br><br><br>class TrainingDataset(Base):<br>    __tablename__ = &quot;training_dataset&quot;<br><br>    date: Mapped[datetime] = mapped_column(DateTime, primary_key=True)<br>    assets: Mapped[List[str]] = mapped_column(ARRAY(String), primary_key=True)<br><br>    def __repr__(self) -&gt; str:<br>        return get_attributes(self)<br><br>    def __str__(self) -&gt; str:<br>        return get_attributes(self)</pre><p>Defining the tables generally follow a simple template</p><pre>class TableObj(Base):<br>  ___tablename__ = &quot;{table_name}&quot;  # This attribute defines the name of the table<br>  <br>  field_n: Mapped[_python_datatype_] = mapped_column([sqlalchemy_datatype], primary_key=[optional], foreign_key=[optiona])</pre><p>It is a fairly intuitive structure where the __tablename__ attribute describes the name of the table that will be setup on the database and the rest of the attributes will define the field/column of the table. The __repr__ and __str__methods are optional. I included them just to have a simplified print/logging information be displayed. This helps in printing it in a manner like</p><pre># Example TrainingDataset class<br>&gt;&gt;&gt; td = TrainingDataset(**kwargs)<br>&gt;&gt;&gt; print(td)<br>TrainingDataset(date=&#39;1962-01-02 05:00:00&#39;,assets=[&#39;MMM&#39;])</pre><p>This helps in easy debugging of code and logging information.</p><p>In the next post we will look into creating scripts to help us populate the database as well.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e5433a1c853" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Quantitative Finance Project I — Got Bored.]]></title>
            <link>https://medium.com/@msadiq074/building-a-quantitative-finance-project-i-got-bored-a6a57688fa22?source=rss-f802ef7375fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6a57688fa22</guid>
            <category><![CDATA[fintech-app-development]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mohd Sadiq]]></dc:creator>
            <pubDate>Wed, 21 Feb 2024 05:23:04 GMT</pubDate>
            <atom:updated>2024-02-22T23:27:43.717Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Quantitative Finance Project I — Got Bored.</h3><p>Welcome to Got Bored. A series where I work on side projects when I got bored. In this chapter I’ll try to implement a Quantitative Finance Project. Why you may ask? Well simply because I wanted to familiarize myself with more FinTech + ML related projects. I wanted to know more about the ins and outs of a financial project and how I could use my software engineering skills to be able to make it usable.</p><p>So here I was looking into the broad problem statement of portfolio optimizing. I would like to tackle the sub-problem of optimizing the distribution of investment through the metric of Sharpe ratio.</p><p>The metric I would be considering is to maximize the Sharpe ratio of an investor’s portfolio. The first intuition that may come to my mind is to implement a Reinforcement Learning based algorithm off of the stock price history as an input and the Sharpe Ratio as the reward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FzkcObotX4uIC0TDxO3r7w.png" /><figcaption>Basic outline of the training of the ML Model</figcaption></figure><p>I’m not too worried about the details of the model etc. at this stage but defining how I visualize this project is my main focus. The underlying goal for me is to make this project as flexible as possible. The asset taken into consideration in this project can be swapped out for any kind of asset that the user might be interested in. For having controlled input definitions, I would be looking into the S&amp;P 500 stocks. The next step is to define the kind of inputs and outputs I will be delving into. Assuming the current time this project is run is at time T, I would like to take in T-6 months of asset price and be able to come up with an optimal distribution of investment amongst those assets. This distribution should be “optimized” to give me the maximum Sharpe ratio based off of returns from a period ofT+3 months .</p><p>So for an investor, I’m not recommending them any new assets, but from the pool of investments they have already made, optimize the distribution of their investment within that pool of investment.</p><p>To make it even simpler for me, I’m gonna assume that the investor only has 20 invested assets. But wait… you may be wondering what if the investor has more than 20, like 50 assets under their belt or maybe they have less like 5? Then how will I account for that?</p><p>Well we will look into it further down the line. As I said we are trying to make it as flexible as possible hence if we modularized components in our pipeline, we should be able to ideally just swap one component and we would be able to handle the variable portfolio size as well.</p><p>So to summarize how I am visualizing the flow of the end implementation of the project is shown as below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a7f3GPRbMQr7g7US7ewhCA.png" /><figcaption>Workflow of the Quant Finance Project</figcaption></figure><p>In a nutshell, I want the model to function as follows:</p><ol><li>The user sends the portfolio that they are wishing to optimize. In this scenario the user will send a list/array of assets to our system.</li><li>Once we get the list of assets, we would have the assets prices be fetched via a third party API like yfinance etc. This can be upgraded an in-house database that always has the previous 6 month data of the assets. Currently, to keep it simple I am going the third party API route.</li><li>Once I have the asset prices, I can send those asset prices to a data preprocessor that will modify the pricing data into the respective format the ML Model might need. This includes extracting features or changing the representation of the data</li><li>Once the ML model processes the input and gives us a distribution of the portfolio, we save the distribution in a database alongside the timestamp. This step is done to help in the Phase 2 of the Quantitative Finance Project that we will discuss at a later date</li><li>As simple as sending the distribution output back to the user.</li></ol><h3>Data, Data, Data…</h3><p>Well if I want to train my own model, the first thing I need to consider is how I want my training data to be saved. For the purposes of this project, I’m considering saving the data in a PostgreSQL database. The main tables that are useful for me are defined as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*uHbE4pwKC6yUWJg4TwCymA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*iQLmh8IgFRU7rIweaOdiiQ.png" /><figcaption>Main tables containing the information related to training the ML Model</figcaption></figure><p>Now comes a dilemma on how to save the asset price information. We could have a big table with all the Asset ID, Date and Price, but the consequences for it may lead to an extremely large table with millions of records. This will lead to delays in updating, deleting or reading from the table. But it will be easier to implement. We can look into optimizing this database at a later time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ySYrrLLP9TI-rSVUo1gK3Q.png" /></figure><p>Why not just make API calls to a third party API and get the data while running the code? One of the main reasons why I didn’t want to go down that approach was to avoid any multiple requests error. If I have the data on my own storage device, then I don’t need to rely as much on a third party provider to train my models.</p><p>Having it in an SQL database has its own benefits. I can host this in a docker container and have another adminer docker on top of it to provide me a web-GUI to manage my data. Having it in a docker helps me create a container of it and also gives me the flexibility to host it on a plethora of systems which can include a dedicated custom machine or a cloud based instance like AWS EC2.</p><p>You may have noticed that the ML model is also writing to a database in the workflow. How would I want to save this data?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nFmuXYG1uIyWsysOZZ989g.png" /></figure><p>The main reason to save the distribution is to have a way to measure the model performance as it is churning through user inputs. This distribution data can also give us a few more insights as to which Assets are favored by the masses.</p><h3>Next Steps</h3><p>Well as the foundations are laid down for the project, ideas emerge and eventually you find a way to address them. In this blog I just went over a general idea on how I would go about address such a Quant Finance based project and how I would see it through.</p><p>Will keep you updated for Part II</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6a57688fa22" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I used AI to make my portfolio website — Part 1]]></title>
            <link>https://medium.com/@msadiq074/how-i-used-ai-to-make-my-portfolio-website-part-1-7be312b2f0e7?source=rss-f802ef7375fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/7be312b2f0e7</guid>
            <category><![CDATA[web-design]]></category>
            <category><![CDATA[website]]></category>
            <category><![CDATA[portfolio]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[react]]></category>
            <dc:creator><![CDATA[Mohd Sadiq]]></dc:creator>
            <pubDate>Fri, 26 Jan 2024 02:49:59 GMT</pubDate>
            <atom:updated>2024-01-26T02:49:59.743Z</atom:updated>
            <content:encoded><![CDATA[<h3>How I used AI to make my portfolio website — Part 1</h3><p>Well hello there!! This is part 1 of the series of how I tried to use AI to make my portfolio website. I must admit this is not my original idea but rather something I came across a Youtube shorts that referenced another blog. At this point we’re just copying each other, aren’t we 👀?</p><p>I would have the decency to link the original blog here.</p><h3>Original Portfolio Website</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/929/1*BeF4K3YkE2t4MLNj1W-0aw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/1*qgOJBQ0NU4xrC-PQfxR4Bw.png" /><figcaption>Old Portfolio Website (Dark Theme)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/1*u3pH2kcZzVoWeDmaXKmEPw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*kv4S2H_5P6ofNhy2ClNk1A.png" /><figcaption>Old Portfolio Website (Light Theme)</figcaption></figure><p>The original portfolio website isn’t something terrible. I designed it entirely from scratch and I am pretty proud of the design. Minimalistic and Simple. Uses only 2 (at max 3) colors in the entire website and changes theme based off of user’s preference (Dark or Light).</p><p>BUT. I can’t get over the fact that my friend’s mother said I looked like the guy from KFC in the dark theme. I LOOK LIKE COLONEL SANDERS.</p><p>Ever since then I cannot ‘unsee’ it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/0*RF4f2zOrB_gxKI9H.gif" /></figure><p>So it is inevitable that this portfolio website needs an overhaul. And while I’m at it, I can jot down the things I have done to reach here.</p><p>You can visit the original website <a href="https://ms2892.github.io/">here</a></p><h3>Getting the Inspiration</h3><p>AI certainly isn’t at a stage where I could blindly let it generate a website for me. For some people that might be the case with websites like Wix and Squarespace giving you a WYSIWYG editors alongside some decent customization (if you wish to code your elements etc). For me I find it easier if I work on something from scratch. That way I can define my folder structure, my project, my components. I would know my project inside out. I’m not someone with sound creative skills through which I can draw or make really elegant designs (thanks Indian education system). So how would I accomplish the task of having a basic template or idea?</p><p>Well AI can help me with exactly that!!!</p><p>The first thing to do is to ask AI how would it design a minimalist portfolio website for a machine learning engineer. Since I am stingy I do not want to spend much money on AI generated inspiration. I found this website called Muse AI that let me generate a few images for free. So let’s have a look…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*CZ5gb3BrVkxnAmjuHhDYIg.jpeg" /></figure><p>Ooooohhhhhh Yikes!!!</p><p>I am reminded of the time where I had to convince a friend of mine that me being a Machine Learning Engineer had nothing to do with machines. I haven’t seen a robot let alone touched one. Well let’s see if I asked it to generate a portfolio website for a tech savvy graphic designers. What would it show then?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*zbdiJ3C22lcVLXtdAUL5Iw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*UbiIQPwwd148LtwNv0XUwg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*bUO5L4_pvRS9ON8czR6dvQ.jpeg" /></figure><p>Seems like graphic designers do love the color beige. It does seem to grow on me but we seem to get a basic idea of how the portfolio website would look like. Not bad if I do say so myself. They do look pretty professional and pretty much what I would expect from a professional portfolio website of a graphic designer.</p><p>There was DALL-E as well that could generate images right? What does DALL-E give me if I run the same commands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OCBq2QEQhn3Cn5OsM_TMfQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rQveJR4xbmSiq1Jk3nc97Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OSDviwSigp024YtHrYKvIA.png" /></figure><p>It somewhat knows what a website is but none of them speak out to me much. They seem like they are geared towards a bit more general looking website. Some outright looking a lot like commercial ones. A portfolio website is meant to look personal and unique. (Even though I’m using AI to come up with mine 👀)</p><p>Well the last option I had in my mind was Midjourney. I’ve seen and heard amazing results from Midjourney and I would …… and there is no free option of Midjourney. YIKES!</p><p>How am I going to get inspiration from Midjourney? Well luckily I’m not the only one who wanted to make my own portfolio website. Multiple people before me wanted to make it. So I just searched the word ‘portfolio website’ on the discord server of Midjourney and I came up with a few inspirations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DUKp3JEc2hqEZkz3qzBsqQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j0BG32B2PRiNv7zSQ5MCMg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fDs60XZwTf2IL46d3ZYjhA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l7Gou16Liz8ARXu4LoHyXQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y_LwDdzXKiKNvKRG4aTfHA.jpeg" /></figure><p>I don’t know about you but I really love the results coming out of Midjourney. There is a recurring theme of having a good looking headshot as part of the portfolio. The ones that stand out to me the most, have the photograph with a transparent background. Well seems like I have my inspiration a little bit, now how am I gonna get my photograph to look that aesthetically pleasing?</p><p>Well AI :P …</p><h3>Improving my face</h3><p>In desperate times I turn to youtube shorts that can guide me to a few websites or apps that could help me generate a few AI-based headshots with my face on it. I came across this app called Remini on the App Store and thought of giving it a try. Here are a few results from it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1PYpX7fCArBuMcDoftCANw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IjO2p08lJ15Cq1O00XXVoQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Si5iH8HjdLJ0V3f42UfasA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ET7jnsfAYf_SLbDO1y1EfA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0C45lbPAZUigN0AX44WanA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8uT_eMg0eotVZTctAZBYEw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I-LBKwsXpyilEz_aB0LJMg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nvwnC8_1Cyvh9vHer2AOng.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MUv36i_PR0NUCOsYgypQmQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aDCZhAufioJLHhpCk8FJCA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SxMZVRXipWtcQ40G13oCJw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WhOYLl65aEBVe4M-zajHQw.jpeg" /></figure><p>Oh….. Oh god….. Oh Damnnnn. Didn’t expect it to work so well. I love how to protect some of my features it at times randomly crop those features out and pastes it on top of the generated image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/1*__01Ixm3s_Qfc7S81SaTWw.jpeg" /></figure><h3>Wait for Part 2</h3><p>Well seems like I have a decent amount of inspiration to begin with. Let’s see what I come up with in Part 2.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7be312b2f0e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>