<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mohammed AbuAisha on Medium]]></title>
        <description><![CDATA[Stories by Mohammed AbuAisha on Medium]]></description>
        <link>https://medium.com/@mabuaisha?source=rss-c79e314439f5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*wZ4_ONzqJPItdRAM821NyA.jpeg</url>
            <title>Stories by Mohammed AbuAisha on Medium</title>
            <link>https://medium.com/@mabuaisha?source=rss-c79e314439f5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 04:31:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mabuaisha/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Python Dependencies Management]]></title>
            <link>https://blog.stackademic.com/python-dependencies-management-f22554103354?source=rss-c79e314439f5------2</link>
            <guid isPermaLink="false">https://medium.com/p/f22554103354</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[python-tools]]></category>
            <category><![CDATA[package-management]]></category>
            <category><![CDATA[virtual-environment]]></category>
            <dc:creator><![CDATA[Mohammed AbuAisha]]></dc:creator>
            <pubDate>Tue, 21 Nov 2023 11:59:23 GMT</pubDate>
            <atom:updated>2023-11-23T11:59:34.652Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VyTqznbz6f-mj8nnZwHFww.png" /></figure><p>I have been working as a software engineer since <strong>2015</strong> and have found Python to be my preferred programming language. This is because it has a short learning curve, and a huge community of support, and can be used in various domains such as DevOps, machine learning, data science, web development, IoT, and even mobile development.</p><p>No matter which domain you are working on, your <strong>Python</strong> projects for sure are likely to use dependencies (<strong>SDK</strong>, <strong>API clients</strong>, <strong>Wrappers</strong>, … etc.) developed by the community or internally by your team. Managing these dependencies could be a nightmare especially when it comes to running or testing your application using different dependencies versions or even under different <strong>Python</strong> versions. Fortunately, the <strong>Python</strong> ecosystem provides a lot of tools that can help manage your application’s complex requirements</p><p>In this blog, I will focus on a list of top Python tools that can make developers’ lives easier and save a lot of time and effort during Python project development. Before diving into these tools, I will first explain some basic concepts about <strong>Python</strong> package management &amp; virtual environments.</p><h4><strong>Package Manager</strong></h4><p>This is a generic term for a tool that helps to install packages from the official package registry. <strong>Python</strong> uses <strong>pip</strong> (default), <strong>PHP</strong> uses <strong>Composer</strong> and Ruby uses <strong>RubyGems.</strong></p><h4><strong>Index Registry</strong></h4><p>This is the location where the developers publish their packages so that others can install and use them. The default index registry used by <strong>Python</strong> is <a href="https://pypi.org/">PyPI</a>, <a href="https://www.npmjs.com/">npmjs</a> is used by <strong>NodeJS, </strong>and<strong> </strong><a href="https://packagist.org/">packagist</a><strong> </strong>by<strong> PHP.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bIPaBDu879YFYRrtm3QAQA.png" /><figcaption>PyPI Interface</figcaption></figure><h4><strong>Virtual Environment</strong></h4><p>This term gives the power of package isolation in order to avoid the interference between multiple packages with different versions used by the application and most importantly to avoid clashes with the packages installed on the global Python environment.</p><p>Let us then deep dive into how Python handles packages using different tools. We are going to divide this blog into 3 main categories across 3 blogs:</p><ol><li><a href="https://mabuaisha.medium.com/basic-python-dependencies-management-87e260ff918f">Basic Python Dependencies Management </a>(<a href="https://pip.pypa.io/en/stable/">pip</a>, v<a href="https://virtualenv.pypa.io/en/latest/">irtualenv</a> and v<a href="https://docs.python.org/3/library/venv.html">env</a>)</li><li><a href="https://mabuaisha.medium.com/advanced-python-dependencies-management-e1a7de6ebad9">Advanced Python Dependencies Management</a> (<a href="https://github.com/jazzband/pip-tools">pip-tools</a>, <a href="https://pipenv.pypa.io/en/latest/">pipenv</a>, and <a href="https://python-poetry.org/">poetry</a>)</li><li><a href="https://mabuaisha.medium.com/python-applications-versions-management-c9d61b20465b">Python Applications &amp; Versions Mangement</a>(<a href="https://pypa.github.io/pipx/">pipx</a>, <a href="https://github.com/pyenv/pyenv">pyenv</a>)</li></ol><h3>Summary</h3><p>This blog serves as an introduction to my upcoming posts on dependency management using Python. Check the first blog on <a href="https://mabuaisha.medium.com/87e260ff918f">Basic Python Dependencies Management</a>.</p><h3>Stackademic</h3><p><em>Thank you for reading until the end. Before you go:</em></p><ul><li><em>Please consider </em><strong><em>clapping</em></strong><em> and </em><strong><em>following</em></strong><em> the writer! 👏</em></li><li><em>Follow us on </em><a href="https://twitter.com/stackademichq"><strong><em>Twitter(X)</em></strong></a><em>, </em><a href="https://www.linkedin.com/company/stackademic"><strong><em>LinkedIn</em></strong></a><em>, and </em><a href="https://www.youtube.com/c/stackademic"><strong><em>YouTube</em></strong></a><strong><em>.</em></strong></li><li><em>Visit </em><a href="http://stackademic.com/"><strong><em>Stackademic.com</em></strong></a><em> to find out more about how we are democratizing free programming education around the world.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f22554103354" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/python-dependencies-management-f22554103354">Python Dependencies Management</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Basic Python Dependencies Management]]></title>
            <link>https://blog.stackademic.com/basic-python-dependencies-management-87e260ff918f?source=rss-c79e314439f5------2</link>
            <guid isPermaLink="false">https://medium.com/p/87e260ff918f</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[package-management]]></category>
            <category><![CDATA[virtualenv]]></category>
            <dc:creator><![CDATA[Mohammed AbuAisha]]></dc:creator>
            <pubDate>Tue, 21 Nov 2023 11:58:57 GMT</pubDate>
            <atom:updated>2023-11-23T11:59:40.963Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tUp9p9S3NQdKugNkFfOgjg.jpeg" /></figure><p>In my previous blog <a href="https://mabuaisha.medium.com/python-dependencies-management-f22554103354">post</a>, I introduced some basic glossaries and provided an overview of our series on Python dependencies management. In this post, I will delve deeper into basic Python dependencies management, which is essential knowledge for any Python developer.</p><h3>Pip</h3><p>Pip is the entry point and the basic thing in order to start managing <strong>Python</strong> packages. Pip is simply a package manager tool that helps in managing and installing Python packages from different sources. By default, it will try to install packages from the <strong>Python</strong> Package Index (<a href="https://pypi.org/">PyPI</a>) unless otherwise specified. Starting from <strong>Python 3.4</strong>, the pip is included as part of the Python installation.</p><p>Pip can be installed in different ways:</p><ol><li>Part of <strong>Python</strong> installation (starting from <strong>Python 3.4</strong>)</li><li>Part of a virtual environment</li><li>Using <a href="https://pip.pypa.io/en/stable/installation/#ensurepip">ensurepip</a> module</li><li>Using <a href="http://get-pip.py/">get-pip.py</a> script</li></ol><pre>~ pip --help<br>Usage:<br>  pip &lt;command&gt; [options]<br><br>Commands:<br>  install                     Install packages.<br>  download                    Download packages.<br>  uninstall                   Uninstall packages.<br>  freeze                      Output installed packages in requirements format.<br>  inspect                     Inspect the python environment.<br>  list                        List installed packages.<br>  show                        Show information about installed packages.<br>  check                       Verify installed packages have compatible dependencies.<br>  config                      Manage local and global configuration.<br>  search                      Search PyPI for packages.<br>  cache                       Inspect and manage pip&#39;s wheel cache.<br>  index                       Inspect information available from package indexes.<br>  wheel                       Build wheels from your requirements.<br>  hash                        Compute hashes of package archives.<br>  completion                  A helper command used for command completion.<br>  debug                       Show information useful for debugging.<br>  help                        Show help for commands.<br><br>General Options:<br>  -h, --help                  Show help.<br>  --debug                     Let unhandled exceptions propagate outside the main subroutine, instead of logging them to stderr.<br>  --isolated                  Run pip in an isolated mode, ignoring environment variables and user configuration.<br>  --require-virtualenv        Allow pip to only run in a virtual environment; exit with an error otherwise.<br>  --python &lt;python&gt;           Run pip with the specified Python interpreter.<br>  -v, --verbose               Give more output. Option is additive, and can be used up to 3 times.<br>  -V, --version               Show version and exit.<br>  -q, --quiet                 Give less output. Option is additive, and can be used up to 3 times (corresponding to WARNING, ERROR, and<br>                              CRITICAL logging levels).<br>  --log &lt;path&gt;                Path to a verbose appending log.<br>  --no-input                  Disable prompting for input.<br>  --proxy &lt;proxy&gt;             Specify a proxy in the form scheme://[user:passwd@]proxy.server:port.<br>  --retries &lt;retries&gt;         Maximum number of retries each connection should attempt (default 5 times).<br>  --timeout &lt;sec&gt;             Set the socket timeout (default 15 seconds).<br>  --exists-action &lt;action&gt;    Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.<br>  --trusted-host &lt;hostname&gt;   Mark this host or host:port pair as trusted, even though it does not have valid or any HTTPS.<br>  --cert &lt;path&gt;               Path to PEM-encoded CA certificate bundle. If provided, overrides the default. See &#39;SSL Certificate<br>                              Verification&#39; in pip documentation for more information.<br>  --client-cert &lt;path&gt;        Path to SSL client certificate, a single file containing the private key and the certificate in PEM format.<br>  --cache-dir &lt;dir&gt;           Store the cache data in &lt;dir&gt;.<br>  --no-cache-dir              Disable the cache.<br>  --disable-pip-version-check<br>                              Don&#39;t periodically check PyPI to determine whether a new version of pip is available for download. Implied with<br>                              --no-index.<br>  --no-color                  Suppress colored output.<br>  --no-python-version-warning<br>                              Silence deprecation warnings for upcoming unsupported Pythons.<br>  --use-feature &lt;feature&gt;     Enable new functionality, that may be backward incompatible.<br>  --use-deprecated &lt;feature&gt;  Enable deprecated functionality, that will be removed in the future.</pre><p>The output above shows that <strong>pip</strong> is already installed as part of the Python installation. If <strong>pip</strong> is not installed for some reason, developers can still install pip using ensurepip and this will be useful, especially for offline environments that use at least<strong> Python 3.4.</strong></p><pre>python -m ensurepip --upgrade</pre><p>Moreover, there is still another option to download the script<strong> </strong>get-pip.py and install <strong>pip</strong> locally.</p><pre>~ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py<br>  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current<br>                                 Dload  Upload   Total   Spent    Left  Speed<br>100 2544k  100 2544k    0     0  6161k      0 --:--:-- --:--:-- --:--:-- 6251k</pre><p><strong>pip</strong> supports a lot of commands and options. However, we are not going to cover all of them in this blog as we mainly going to focus on the most important ones.</p><p><strong>Pip install</strong></p><p>This command is used to install packages from the index registry or from any sources that store Python packages.</p><pre>~ pip install requests==2.31.0<br>Collecting requests==2.31.0<br>  Obtaining dependency information for requests==2.31.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata<br>  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)<br>Requirement already satisfied: charset-normalizer&lt;4,&gt;=2 in ./.pyenv/versions/3.11.2/lib/python3.11/site-packages (from requests==2.31.0) (3.2.0)<br>Requirement already satisfied: idna&lt;4,&gt;=2.5 in ./.pyenv/versions/3.11.2/lib/python3.11/site-packages (from requests==2.31.0) (3.4)<br>Requirement already satisfied: urllib3&lt;3,&gt;=1.21.1 in ./.pyenv/versions/3.11.2/lib/python3.11/site-packages (from requests==2.31.0) (2.0.4)<br>Requirement already satisfied: certifi&gt;=2017.4.17 in ./.pyenv/versions/3.11.2/lib/python3.11/site-packages (from requests==2.31.0) (2023.7.22)<br>Using cached requests-2.31.0-py3-none-any.whl (62 kB)<br>Installing collected packages: requests<br>Successfully installed requests-2.31.0</pre><p>If we do not tell the <strong>pip</strong> where to install Python packages, it will install them by default from <a href="https://pypi.org/">PyPI</a>. In some cases, <a href="https://pypi.org/">PyPI</a> may not be a good option — e.g. in offline environments or environments that depend on their own index registry for security reasons.</p><p>--index-url option can be overridden as part of the pip install command as the following:</p><pre>pip install requests==2.31.0 --index-url &lt;YOUR_OWN_URL&gt;</pre><p>or even, the --index-url can be added to the top of the requirements.txt to avoid typing the --index-url every time.</p><pre>--index-url &lt;YOUR_OWN_URL&gt;<br>alembic==1.12.0</pre><p>As we mentioned, there could be a case where we want to install <strong>Python</strong> packages in an offline environment where a request to the internet is not permitted to download your packages. <strong>pip</strong> already supports downloading packaging from local directories.</p><pre>pip install requests --no-index --find-links file:///your-path/to/packages</pre><p>pip install also provided a lot of options that we are not going to cover in this blog as our goal is to focus on the basic options needed to install <strong>Python</strong> packages.</p><p><strong>Pip uninstall</strong></p><p>This command just reverses the operation of the pip install and uninstall packages that were installed before.</p><pre>➜ pip uninstall --help</pre><pre>Usage:<br>  pip uninstall [options] &lt;package&gt; ...<br>  pip uninstall [options] -r &lt;requirements file&gt; ...</pre><p>Uninstalling packages can be done by either specifying the list of packages or even a list of requirements files</p><pre>➜ pip uninstall -y requests<br>Found existing installation: requests 2.31.0<br>Uninstalling requests-2.31.0:<br>  Successfully uninstalled requests-2.31.0</pre><pre>➜ pip uninstall -y -r requirements.txt<br>Found existing installation: requests 2.31.0<br>Uninstalling requests-2.31.0:<br>  Successfully uninstalled requests-2.31.0</pre><p><strong>Pip freeze</strong></p><p>This command helps to output installed packages in the requirements format.</p><p>Note --all include other packages like <em>setuptools, distribute, pip, wheel</em></p><pre>➜  ~ pip freeze --all<br>black=cs=23.7.0<br>certifi==2023.7.22<br>charset-normalizer==3.2.0<br>click==8.1.7<br>confluent-kafka==2.2.0<br>hj-domain-schemas==1.5.1<br>idna==3.4<br>isort==5.12.0<br>mypy-extensions==1.0.0<br>packaging==23.1<br>pathspec==0.11.2<br>pip==23.2.1<br>platformdirs==3.10.0<br>protobuf==4.24.0<br>requests==2.31.0<br>setuptools==65.5.0<br>urllib3==2.0.4</pre><p><strong>Pip list</strong></p><p>This command, lists installed packages, including <strong>editables</strong> using different formats and various options. The output format is the <em>columns</em> (default)</p><pre>➜  ~ pip list<br>Package            Version<br>------------------ ---------<br>black              23.7.0<br>certifi            2023.7.22<br>charset-normalizer 3.2.0<br>click              8.1.7<br>confluent-kafka    2.2.0<br>hj-domain-schemas  1.5.1<br>idna               3.4<br>isort              5.12.0<br>mypy-extensions    1.0.0<br>packaging          23.1<br>pathspec           0.11.2<br>pip                23.2.1<br>platformdirs       3.10.0<br>protobuf           4.24.0<br>requests           2.31.0<br>setuptools         65.5.0<br>urllib3            2.0.4</pre><p><strong>Pip show</strong></p><p>This command helps to display information about one or more installed packages</p><pre>➜  pip show black isort<br>Name: black<br>Version: 23.7.0<br>Summary: The uncompromising code formatter.<br>Home-page:<br>Author:<br>Author-email: Łukasz Langa &lt;lukasz@langa.pl&gt;<br>License: MIT<br>Location: /Users/mabuaisha/.pyenv/versions/3.11.2/lib/python3.11/site-packages<br>Requires: click, mypy-extensions, packaging, pathspec, platformdirs<br>Required-by:<br>---<br>Name: isort<br>Version: 5.12.0<br>Summary: A Python utility / library to sort Python imports.<br>Home-page: https://pycqa.github.io/isort/<br>Author: Timothy Crosley<br>Author-email: timothy.crosley@gmail.com<br>License: MIT<br>Location: /Users/mabuaisha/.pyenv/versions/3.11.2/lib/python3.11/site-packages<br>Requires:<br>Required-by:</pre><h3>Virtualenv</h3><p>As we mentioned before the concept of a virtual environment helps the installation of the packages in an isolated environment instead of using the global Python environment.</p><p>virtualenv is a tool that creates isolated Python environments that help in the following:</p><ol><li>Make development easier especially when we work with multiple projects using different package versions</li><li>Isolation gives the power to have the same package installed on different environment with different packages</li><li>Avoid using a single environment (global Python environment) for multiple projects which adds overhead when switching between projects</li><li>Avoid the conflicting requirements between packages.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/452/1*gnVqhlJiVDKgk2NaZOTm7A.png" /><figcaption>Virtual environments</figcaption></figure><p>virtualenv does not support out-of-the-box switching and managing different Python versions. It only focuses on creating isolated environments. It can be installed simply using <strong>pip</strong> and start using it without any problem to create virtual environments.</p><p>Creating virtual environments is simple as all you need to do is:</p><ol><li>Make sure that you have virtualenv installed</li><li>Create a virtual environment by specifying the name and the location</li><li>Activate the virtual environment in order to start managing the packages you want to modify in your environment</li></ol><pre>➜  pip install virtualenv<br>Collecting virtualenv<br>  Obtaining dependency information for virtualenv from https://files.pythonhosted.org/packages/4e/8b/f0d3a468c0186c603217a6656ea4f49259630e8ed99558501d92f6ff7dc3/virtualenv-20.24.5-py3-none-any.whl.metadata<br>  Downloading virtualenv-20.24.5-py3-none-any.whl.metadata (4.5 kB)<br>Collecting distlib&lt;1,&gt;=0.3.7 (from virtualenv)<br>  Obtaining dependency information for distlib&lt;1,&gt;=0.3.7 from https://files.pythonhosted.org/packages/43/a0/9ba967fdbd55293bacfc1507f58e316f740a3b231fc00e3d86dc39bc185a/distlib-0.3.7-py2.py3-none-any.whl.metadata<br>  Downloading distlib-0.3.7-py2.py3-none-any.whl.metadata (5.1 kB)<br>Collecting filelock&lt;4,&gt;=3.12.2 (from virtualenv)<br>  Obtaining dependency information for filelock&lt;4,&gt;=3.12.2 from https://files.pythonhosted.org/packages/5e/5d/97afbafd9d584ff1b45fcb354a479a3609bd97f912f8f1f6c563cb1fae21/filelock-3.12.4-py3-none-any.whl.metadata<br>  Downloading filelock-3.12.4-py3-none-any.whl.metadata (2.8 kB)<br>Requirement already satisfied: platformdirs&lt;4,&gt;=3.9.1 in /Users/mabuaisha/.pyenv/versions/3.11.2/lib/python3.11/site-packages (from virtualenv) (3.10.0)<br>Downloading virtualenv-20.24.5-py3-none-any.whl (3.7 MB)<br>   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 4.2 MB/s eta 0:00:00<br>Downloading distlib-0.3.7-py2.py3-none-any.whl (468 kB)<br>   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 kB 324.1 kB/s eta 0:00:00<br>Downloading filelock-3.12.4-py3-none-any.whl (11 kB)<br>Installing collected packages: distlib, filelock, virtualenv<br>Successfully installed distlib-0.3.7 filelock-3.12.4 virtualenv-20.24.5<br></pre><pre>➜  ~ virtualenv test_env<br>created virtual environment CPython3.11.2.final.0-64 in 1696ms<br>  creator CPython3Posix(dest=/Users/mabuaisha/test_env, clear=False, no_vcs_ignore=False, global=False)<br>  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/mabuaisha/Library/Application Support/virtualenv)<br>    added seed packages: pip==23.2.1, setuptools==68.2.0, wheel==0.41.2<br>  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator</pre><pre>➜  ~ source test_env/bin/activate</pre><pre>(test_env) ➜  ~</pre><p>virtualenv has two main phases:</p><ol><li>Discover the Python interpreter to create the desired virtual environment. By default, it will be the Python interpreter where virtualenv running from. The option-p/--python can be specified in order to choose which interpreter to create an environment from. So if we have multiple Python versions installed in our system, virtualenv can choose from which Python version we need to create our virtualenv (There are other tools like pyenv that helps to achieve this goal as well)</li><li>Create the virtual environment at the desired location. By default, it will be created in the current directory where you run the command.</li></ol><p>Running the above command will create a directory with some bin and lib sub-directories.</p><pre>(test_env) ➜  test_env ls -lhtra<br>drwxr-xr-x@  3 mabuaisha  staff    96B Oct 15 12:18 lib<br>drwxr-xr-x@ 19 mabuaisha  staff   608B Oct 15 12:18 bin</pre><p>The bin directory contains activate, pip, and links to the Python interpreter.</p><pre>(test_env) ➜  bin ls -lh<br>-rw-r--r--@ 1 mabuaisha  staff   2.2K Oct 15 12:18 activate<br>-rw-r--r--@ 1 mabuaisha  staff   1.5K Oct 15 12:18 activate.csh<br>-rw-r--r--@ 1 mabuaisha  staff   3.0K Oct 15 12:18 activate.fish<br>-rw-r--r--@ 1 mabuaisha  staff   2.7K Oct 15 12:18 activate.nu<br>-rw-r--r--@ 1 mabuaisha  staff   1.6K Oct 15 12:18 activate.ps1<br>-rw-r--r--@ 1 mabuaisha  staff   1.3K Oct 15 12:18 activate_this.py<br>-rwxr-xr-x@ 1 mabuaisha  staff   241B Oct 15 12:18 pip<br>-rwxr-xr-x@ 1 mabuaisha  staff   241B Oct 15 12:18 pip-3.11<br>-rwxr-xr-x@ 1 mabuaisha  staff   241B Oct 15 12:18 pip3<br>-rwxr-xr-x@ 1 mabuaisha  staff   241B Oct 15 12:18 pip3.11<br>lrwxr-xr-x@ 1 mabuaisha  staff    54B Oct 15 12:18 python -&gt; /Users/mabuaisha/.pyenv/versions/3.11.2/bin/python3.11<br>lrwxr-xr-x@ 1 mabuaisha  staff     6B Oct 15 12:18 python3 -&gt; python<br>lrwxr-xr-x@ 1 mabuaisha  staff     6B Oct 15 12:18 python3.11 -&gt; python<br>-rwxr-xr-x@ 1 mabuaisha  staff   228B Oct 15 12:18 wheel<br>-rwxr-xr-x@ 1 mabuaisha  staff   228B Oct 15 12:18 wheel-3.11<br>-rwxr-xr-x@ 1 mabuaisha  staff   228B Oct 15 12:18 wheel3<br>-rwxr-xr-x@ 1 mabuaisha  staff   228B Oct 15 12:18 wheel3.11</pre><p>On, the other hand, the lib, contains the site-packages which represents the installed packages</p><pre>(test_env) ➜  site-packages pwd<br>/Users/mabuaisha/test_env/lib/python3.11/site-packages<br>(test_env) ➜  site-packages ls -lhtra<br>drwxr-xr-x@  3 mabuaisha  staff    96B Oct 15 12:18 ..<br>-rw-r--r--@  1 mabuaisha  staff    18B Oct 15 12:18 _virtualenv.pth<br>-rw-r--r--@  1 mabuaisha  staff   4.2K Oct 15 12:18 _virtualenv.py<br>-rw-r--r--@  1 mabuaisha  staff     0B Oct 15 12:18 setuptools-68.2.0.virtualenv<br>-rw-r--r--@  1 mabuaisha  staff   151B Oct 15 12:18 distutils-precedence.pth<br>-rw-r--r--@  1 mabuaisha  staff     0B Oct 15 12:18 pip-23.2.1.virtualenv<br>drwxr-xr-x@  9 mabuaisha  staff   288B Oct 15 12:18 setuptools-68.2.0.dist-info<br>drwxr-xr-x@ 12 mabuaisha  staff   384B Oct 15 12:18 wheel<br>-rw-r--r--@  1 mabuaisha  staff     0B Oct 15 12:18 wheel-0.41.2.virtualenv<br>drwxr-xr-x@  8 mabuaisha  staff   256B Oct 15 12:18 wheel-0.41.2.dist-info<br>drwxr-xr-x@ 49 mabuaisha  staff   1.5K Oct 15 12:18 setuptools<br>drwxr-xr-x@  5 mabuaisha  staff   160B Oct 15 12:18 pkg_resources<br>drwxr-xr-x@ 10 mabuaisha  staff   320B Oct 15 12:18 pip-23.2.1.dist-info<br>drwxr-xr-x@  5 mabuaisha  staff   160B Oct 15 12:35 _distutils_hack<br>drwxr-xr-x@  9 mabuaisha  staff   288B Oct 15 12:35 pip<br>drwxr-xr-x@ 12 mabuaisha  staff   384B Oct 15 12:36 platformdirs<br>drwxr-xr-x@  7 mabuaisha  staff   224B Oct 15 12:36 platformdirs-3.11.0.dist-info<br>drwxr-xr-x@ 11 mabuaisha  staff   352B Oct 15 12:36 pathspec<br>drwxr-xr-x@  7 mabuaisha  staff   224B Oct 15 12:36 pathspec-0.11.2.dist-info<br>drwxr-xr-x@ 18 mabuaisha  staff   576B Oct 15 12:36 packaging<br>drwxr-xr-x@  9 mabuaisha  staff   288B Oct 15 12:36 packaging-23.2.dist-info<br>-rw-r--r--@  1 mabuaisha  staff   6.1K Oct 15 12:36 mypy_extensions.py<br>drwxr-xr-x@  8 mabuaisha  staff   256B Oct 15 12:36 mypy_extensions-1.0.0.dist-info<br>drwxr-xr-x@ 20 mabuaisha  staff   640B Oct 15 12:36 click<br>drwxr-xr-x@  8 mabuaisha  staff   256B Oct 15 12:36 click-8.1.7.dist-info<br>-rwxr-xr-x@  1 mabuaisha  staff   6.7M Oct 15 12:36 2ec0e72aa72355e6eccf__mypyc.cpython-311-darwin.so<br>-rw-r--r--@  1 mabuaisha  staff    19B Oct 15 12:36 _black_version.py<br>drwxr-xr-x@ 33 mabuaisha  staff   1.0K Oct 15 12:36 .<br>drwxr-xr-x@  5 mabuaisha  staff   160B Oct 15 12:36 __pycache__<br>drwxr-xr-x@ 42 mabuaisha  staff   1.3K Oct 15 12:36 black<br>drwxr-xr-x@  6 mabuaisha  staff   192B Oct 15 12:36 blackd<br>drwxr-xr-x@ 13 mabuaisha  staff   416B Oct 15 12:36 blib2to3<br>drwxr-xr-x@  9 mabuaisha  staff   288B Oct 15 12:36 black-23.9.1.dist-info</pre><p>The exit from the active virtual enviroment can be simply done by running deactivate command.</p><h3>Venv</h3><p>Starting from <strong>Python 3.3</strong>, a subset of the virtualenv has been integrated into the standard library under a new module called <a href="https://docs.python.org/3/library/venv.html">venv</a> module. venv supports creating lightweight “virtual environments” without having the full features that virtualenv support.</p><pre>➜  ~ python -m venv --help<br>usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear] [--upgrade] [--without-pip] [--prompt PROMPT] [--upgrade-deps]<br>            ENV_DIR [ENV_DIR ...]</pre><pre>Creates virtual Python environments in one or more target directories.</pre><pre>positional arguments:<br>  ENV_DIR               A directory to create the environment in.</pre><pre>options:<br>  -h, --help            show this help message and exit<br>  --system-site-packages<br>                        Give the virtual environment access to the system site-packages dir.<br>  --symlinks            Try to use symlinks rather than copies, when symlinks are not the default for the platform.<br>  --copies              Try to use copies rather than symlinks, even when symlinks are the default for the platform.<br>  --clear               Delete the contents of the environment directory if it already exists, before environment creation.<br>  --upgrade             Upgrade the environment directory to use this version of Python, assuming Python has been upgraded in-place.<br>  --without-pip         Skips installing or upgrading pip in the virtual environment (pip is bootstrapped by default)<br>  --prompt PROMPT       Provides an alternative prompt prefix for this environment.<br>  --upgrade-deps        Upgrade core dependencies: pip setuptools to the latest version in PyPI</pre><p>Simply, a virtual environment can be created natively using Python if you have <strong>Python ≥=3.3</strong></p><pre>➜  ~ python -m venv venv<br>➜  ~ source venv/bin/activate<br>(venv) ➜  ~</pre><h3>Summary</h3><p>In this blog post, we have discussed various tools for managing Python dependencies. These tools are crucial for developing Python applications. However, they have their limitations and cannot solve problems like deterministic builds. We will discuss that in more detail in our next <a href="https://mabuaisha.medium.com/advanced-python-dependencies-management-e1a7de6ebad9">blog post</a>.</p><h3>Stackademic</h3><p><em>Thank you for reading until the end. Before you go:</em></p><ul><li><em>Please consider </em><strong><em>clapping</em></strong><em> and </em><strong><em>following</em></strong><em> the writer! 👏</em></li><li><em>Follow us on </em><a href="https://twitter.com/stackademichq"><strong><em>Twitter(X)</em></strong></a><em>, </em><a href="https://www.linkedin.com/company/stackademic"><strong><em>LinkedIn</em></strong></a><em>, and </em><a href="https://www.youtube.com/c/stackademic"><strong><em>YouTube</em></strong></a><strong><em>.</em></strong></li><li><em>Visit </em><a href="http://stackademic.com/"><strong><em>Stackademic.com</em></strong></a><em> to find out more about how we are democratizing free programming education around the world.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87e260ff918f" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/basic-python-dependencies-management-87e260ff918f">Basic Python Dependencies Management</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Advanced Python Dependencies Management]]></title>
            <link>https://blog.stackademic.com/advanced-python-dependencies-management-e1a7de6ebad9?source=rss-c79e314439f5------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1a7de6ebad9</guid>
            <category><![CDATA[package-management]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[python-tools]]></category>
            <dc:creator><![CDATA[Mohammed AbuAisha]]></dc:creator>
            <pubDate>Tue, 21 Nov 2023 11:57:59 GMT</pubDate>
            <atom:updated>2023-11-23T11:59:50.598Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Vsu_KluXv4uL2c_gkl-6g.png" /></figure><p>In my previous blog <a href="https://mabuaisha.medium.com/basic-python-dependencies-management-87e260ff918f">post</a>, I discussed the fundamental tools that can be used for managing <strong>Python</strong> dependencies. However, tools such as <strong>pip</strong>, <strong>virtualenv</strong>, or <strong>venv</strong> have certain limitations, such as ensuring deterministic builds. In this post, I will delve into more detail about some advanced tools that are considered important in order to overcome issues like this.</p><h3>Pip Tools</h3><p>Making your Python environment’s build deterministic is important. You can easily achieve this by pinning your dependencies to the stable versions in <strong>requirements.txt</strong>.</p><p>So instead of letting pip install the latest version always, we need to make sure to specify the desired version to avoid any conflicts with other dependencies, unstable behavior of the latest version,…etc. For example, if we have the Django framework pinned to 4.2.6</p><pre>Django==4.2.6</pre><p>Django depends on other packages like asgiref &gt;= 3.7.0 &amp; sqlparse &gt;= 0.3.1 . By default, <strong>pip</strong> installs the latest version of both packages, which may not be desirable.</p><p>Manually adding all the sub-dependencies to the <strong>requirements.txt</strong> file can be a solution to the problem. However, it can become particularly challenging when your project has multiple dependencies, and each of these dependencies has further sub-dependencies. Updating the files manually under such circumstances would require a significant amount of effort and time..</p><pre>(venv) ➜  venv pip freeze<br>asgiref==3.7.2<br>Django==4.2.6<br>sqlparse==0.4.4</pre><p>In addition, <strong>pip</strong> alone is unable to handle dependency resolution. Dependency resolution comes into play when your main dependencies share sub-dependencies with varying versions. Using pip is insufficient to solve this problem, and will result in failure while attempting to install the dependencies. To illustrate, consider the following scenario:</p><pre>package_x<br>package_y</pre><p>package_x depends on the sub-package called package_z &gt;= 2.0.0 However, package_y also depends on the same sub-package but with a different version package_z &lt;= 3.0.0 . Using <strong>pip</strong> to install the above packages will fail as <strong>pip</strong> is not smart enough to resolve these dependencies</p><ol><li><strong>pip</strong> will try to install package_x and fulfill its requirements and install the latest version of package_z which could be 4.0.0</li><li>When it comes to install the package_y which has package_z &lt;= 3.0.0 it cannot install that because package_z already installed with the latest version 4.0.0</li></ol><p>I haven’t talked about <strong>pip-tools</strong> yet, but I need to explain why we may need to use them to address the previous issues.</p><p>pip-tools is a set of command line tools that helps to build predictable and deterministic dependencies for your Python application and synchronize your changes to your virtual environment. pip-tools ships with two tools which are pip-compile &amp; pip-sync .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*yI0gwTdEPG5vbWyZCgzOgQ.png" /><figcaption>pip tools diagram</figcaption></figure><h4>pip-compile</h4><p>This tool helps to generate and compile your <strong>requirements.txt </strong>files from different sources. Mainly, <strong>pip-compile</strong> helps in solving the issues we mentioned before including the dependency resolution. <strong>pip-compile</strong> supports compiling your dependencies from the following sources:</p><ol><li><strong>pyproject.toml</strong></li><li><strong>setup.cfg</strong></li><li><strong>setup.py</strong></li><li><strong>requirements.in</strong></li></ol><p>We are going to focus on requirements.insince it is the most common one. pip-tools can be simply installed using <strong>pip</strong> as the following:</p><pre>(env) ➜  ~ pip install pip-tools<br>Collecting pip-tools<br>  Downloading pip_tools-7.3.0-py3-none-any.whl (57 kB)<br>     ━━━━━━━━━━━━━━━━━━━━━━━━ 57.4/57.4 kB 1.3 MB/s eta 0:00:00<br>Collecting build<br>  Downloading build-1.0.3-py3-none-any.whl (18 kB)<br>Collecting click&gt;=8<br>  Using cached click-8.1.7-py3-none-any.whl (97 kB)<br>Requirement already satisfied: pip&gt;=22.2 in ./env/lib/python3.11/site-packages (from pip-tools) (22.3.1)<br>Requirement already satisfied: setuptools in ./env/lib/python3.11/site-packages (from pip-tools) (65.5.0)<br>Collecting wheel<br>  Using cached wheel-0.41.2-py3-none-any.whl (64 kB)<br>Collecting packaging&gt;=19.0<br>  Using cached packaging-23.2-py3-none-any.whl (53 kB)<br>Collecting pyproject_hooks<br>  Downloading pyproject_hooks-1.0.0-py3-none-any.whl (9.3 kB)<br>Installing collected packages: wheel, pyproject_hooks, packaging, click, build, pip-tools<br>Successfully installed build-1.0.3 click-8.1.7 packaging-23.2 pip-tools-7.3.0 pyproject_hooks-1.0.0 wheel-0.41.2</pre><p>All you need to do is to define the source file that needs to be compiled. We can compile our source easily into deterministic dependencies using pip-compile</p><pre>(env) ➜  pip-tools pip-compile -o requirements.txt requirements.in<br>#<br># This file is autogenerated by pip-compile with Python 3.11<br># by the following command:<br>#<br>#    pip-compile --output-file=requirements.txt requirements.in<br>#<br>asgiref==3.7.2<br>    # via django<br>django==4.2.6<br>    # via -r requirements.in<br>sqlparse==0.4.4<br>    # via django</pre><p>The above command compiles the requiremets.in into dependencies that also include the sub-dependencies for Django</p><p>Some notes to consider:</p><ol><li>By default pip-compile will try to read from requirements.in if not specified in the current directory</li><li>By default pip-compile will compile the results into requirements.txt in the current directory if not specified</li><li>You can use -q/--quiet to avoid less output</li><li>You can use -U/--upgrade option in order to force upgrade all dependencies to their latest versions.</li></ol><h4>pip-sync</h4><p>It helps to sync all of your dependencies into the active virtual environment as it detects the changes and tries to always make the compiled requirements.txt match the virtual environment. This means that pip-sync can install/uninstall dependencies based on the state of the compiled dependencies files.</p><pre>(env) ➜  pip-tools pip-sync<br>Collecting asgiref==3.7.2<br>  Using cached asgiref-3.7.2-py3-none-any.whl (24 kB)<br>Collecting django==4.2.6<br>  Using cached Django-4.2.6-py3-none-any.whl (8.0 MB)<br>Collecting sqlparse==0.4.4<br>  Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)<br>Installing collected packages: sqlparse, asgiref, django<br>Successfully installed asgiref-3.7.2 django-4.2.6 sqlparse-0.4.4</pre><pre>(env) ➜  pip-tools pip freeze<br>asgiref==3.7.2<br>Django==4.2.6<br>sqlparse==0.4.4</pre><h3>Pipenv</h3><p>Pipenv is a Python management tool that simplifies the process of managing dependencies by using only one tool instead of using multiple tools separately. In addition to that, pipenv helps in solving the common issues that we explained before and even provides some other cool features that we are going to explain in a bit.</p><p><strong>Installation</strong></p><p>Simply, pipenv can be installed using different options and among these options is <strong>pip</strong></p><pre>pip install --user pipenv</pre><pre>➜ pipenv --help<br>Usage: pipenv [OPTIONS] COMMAND [ARGS]...</pre><pre>Options:<br>  --where                         Output project home information.<br>  --venv                          Output virtualenv information.<br>  --py                            Output Python interpreter information.<br>  --envs                          Output Environment Variable options.<br>  --rm                            Remove the virtualenv.<br>  --bare                          Minimal output.<br>  --man                           Display manpage.<br>  --support                       Output diagnostic information for use in<br>                                  GitHub issues.<br>  --site-packages / --no-site-packages<br>                                  Enable site-packages for the virtualenv.<br>                                  [env var: PIPENV_SITE_PACKAGES]<br>  --python TEXT                   Specify which version of Python virtualenv<br>                                  should use.<br>  --clear                         Clears caches (pipenv, pip).  [env var:<br>                                  PIPENV_CLEAR]<br>  -q, --quiet                     Quiet mode.<br>  -v, --verbose                   Verbose mode.<br>  --pypi-mirror TEXT              Specify a PyPI mirror.<br>  --version                       Show the version and exit.<br>  -h, --help                      Show this message and exit.<br></pre><pre>Usage Examples:<br>   Create a new project using Python 3.7, specifically:<br>   $ pipenv --python 3.7</pre><pre>   Remove project virtualenv (inferred from current directory):<br>   $ pipenv --rm</pre><pre>   Install all dependencies for a project (including dev):<br>   $ pipenv install --dev</pre><pre>   Create a lockfile containing pre-releases:<br>   $ pipenv lock --pre</pre><pre>   Show a graph of your installed dependencies:<br>   $ pipenv graph</pre><pre>   Check your installed dependencies for security vulnerabilities:<br>   $ pipenv check</pre><pre>   Install a local setup.py into your virtual environment/Pipfile:<br>   $ pipenv install -e .</pre><pre>   Use a lower-level pip command:<br>   $ pipenv run pip freeze</pre><pre>Commands:<br>  check         Checks for PyUp Safety security vulnerabilities and against<br>                PEP 508 markers provided in Pipfile.<br>  clean         Uninstalls all packages not specified in Pipfile.lock.<br>  graph         Displays currently-installed dependency graph information.<br>  install       Installs provided packages and adds them to Pipfile, or (if no<br>                packages are given), installs all packages from Pipfile.<br>  lock          Generates Pipfile.lock.<br>  open          View a given module in your editor.<br>  requirements  Generate a requirements.txt from Pipfile.lock.<br>  run           Spawns a command installed into the virtualenv.<br>  scripts       Lists scripts in current environment config.<br>  shell         Spawns a shell within the virtualenv.<br>  sync          Installs all packages specified in Pipfile.lock.<br>  uninstall     Uninstalls a provided package and removes it from Pipfile.<br>  update        Runs lock, then sync.<br>  upgrade       Resolves provided packages and adds them to Pipfile, or (if no<br>                packages are given), merges results to Pipfile.lock<br>  verify        Verify the hash in Pipfile.lock is up-to-date.</pre><p>pipenv provides a lot of options when it comes to manage dependencies. These are the top features that pipenv provide:</p><ol><li>Predictable and deterministic build.</li><li>Create a virtual environment automatically in the standard location.</li><li>Create Pipfile &amp; Pipfile.lock automatically.</li><li>Update Pipfile when adding/removing packages.</li><li>Loads the .env file if it exists (This is useful for testing purposes)</li><li>Install the required Python version automatically. (When pyenv is installed)</li></ol><p><strong>Pipfile</strong></p><p>The Pipfile is the format the pipenv used to manage the dependencies. Mainly, two files exist in order to handle the dependencies: Pipfile &amp; Pipfile.lock . The Pipfile and its sister is a new standard that comes as a replacement for the requirement.txt . The <a href="https://www.pypy.org/">pypy</a> is the one that introduced this format.</p><p>Pipfile is the main file that contains the primary dependencies and is used as a source to generate the Pipfile.lock which represents all dependencies including the sub-dependencies to build a predictable and deterministic environment.</p><p><strong>Pipenv Install</strong></p><p>Let us imagine that we want to install requests==2.30.0 using pipenv</p><pre>➜  pipenv install requests==2.30.0<br>Warning: the environment variable LANG is not set!<br>We recommend setting this in ~/.profile (or equivalent) for proper expected behavior.<br>Creating a virtualenv for this project...<br>Pipfile: /Users/mabuaisha/test/Pipfile<br>Using default python from /Users/mabuaisha/.local/pipx/venvs/pipenv/bin/python (3.11.2) to create virtualenv...<br>⠼ Creating virtual environment...created virtual environment CPython3.11.2.final.0-64 in 313ms<br>  creator CPython3Posix(dest=/Users/mabuaisha/.local/share/virtualenvs/test-CfuChXcD, clear=False, no_vcs_ignore=False, global=False)<br>  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/mabuaisha/Library/Application Support/virtualenv)<br>    added seed packages: pip==23.2.1, setuptools==68.2.2, wheel==0.41.2<br>  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator<br><br>✔ Successfully created virtual environment!<br>Virtualenv location: /Users/mabuaisha/.local/share/virtualenvs/test-CfuChXcD<br>Creating a Pipfile for this project...<br>Installing requests==2.30.0...<br>Resolving requests==2.30.0...<br>Added requests to Pipfile&#39;s [packages] ...<br>✔ Installation Succeeded<br>Pipfile.lock not found, creating...<br>Locking [packages] dependencies...<br>Building requirements...<br>Resolving dependencies...<br>✔ Success!<br>Locking [dev-packages] dependencies...<br>Updated Pipfile.lock (d49ddfed48295f4e1ec62cfa5bdd6c43789b6415d94931a84c88d7ecff073c99)!<br>Installing dependencies from Pipfile.lock (073c99)...<br>To activate this project&#39;s virtualenv, run pipenv shell.<br>Alternatively, run a command inside the virtualenv with pipenv run.s</pre><pre>✔ Successfully created virtual environment!<br>Virtualenv location: /Users/mabuaisha/.local/share/virtualenvs/test-CfuChXcD<br>Creating a Pipfile for this project...<br>Installing requests==2.30.0...<br>Resolving requests==2.30.0...<br>Added requests to Pipfile&#39;s [packages] ...<br>✔ Installation Succeeded<br>Pipfile.lock not found, creating...<br>Locking [packages] dependencies...<br>Building requirements...<br>Resolving dependencies...<br>✔ Success!<br>Locking [dev-packages] dependencies...<br>Updated Pipfile.lock (d49ddfed48295f4e1ec62cfa5bdd6c43789b6415d94931a84c88d7ecff073c99)!<br>Installing dependencies from Pipfile.lock (073c99)...<br>To activate this project&#39;s virtualenv, run pipenv shell.<br>Alternatively, run a command inside the virtualenv with pipenv run.s</pre><ol><li>Automatically create virtualenv under ~/.local/share/virtualenvs/test-CfuChXcD</li><li>Automatically create Pipfile if it does not exist</li><li>Automatically create Pipfile.lock if it does not exist</li><li>Automatically add the requests to the Pipfile and resolve all the sub-dependencies and dump them into Pipfile.lock</li></ol><pre>[[source]]<br>url = &quot;https://pypi.org/simple&quot;<br>verify_ssl = true<br>name = &quot;pypi&quot;<br><br>[packages]<br>requests = &quot;==2.30.0&quot;<br><br>[dev-packages]<br><br>[requires]<br>python_version = &quot;3.11&quot;</pre><p>The above file represents the content of the Pipfile . The Pipfile.lock contains detailed information about the packages and sub-packages. Moreover, it contains the <strong>sha256</strong> hashes of each downloaded package to make sure to get the same packages on any network as specified in the lock file, even on untrusted networks. An example of how the Pipfile.lock looks like can be found below.</p><pre>{<br>    &quot;_meta&quot;: {<br>        &quot;hash&quot;: {<br>            &quot;sha256&quot;: &quot;d49ddfed48295f4e1ec62cfa5bdd6c43789b6415d94931a84c88d7ecff073c99&quot;<br>        },<br>        &quot;pipfile-spec&quot;: 6,<br>        &quot;requires&quot;: {<br>            &quot;python_version&quot;: &quot;3.11&quot;<br>        },<br>        &quot;sources&quot;: [<br>            {<br>                &quot;name&quot;: &quot;pypi&quot;,<br>                &quot;url&quot;: &quot;https://pypi.org/simple&quot;,<br>                &quot;verify_ssl&quot;: true<br>            }<br>        ]<br>    },<br>    &quot;default&quot;: {<br>        &quot;certifi&quot;: {<br>            &quot;hashes&quot;: [<br>                &quot;sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082&quot;,<br>                &quot;sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9&quot;<br>            ],<br>            &quot;markers&quot;: &quot;python_version &gt;= &#39;3.6&#39;&quot;,<br>            &quot;version&quot;: &quot;==2023.7.22&quot;<br>        },<br>        &quot;charset-normalizer&quot;: {<br>            &quot;hashes&quot;: [<br>                &quot;sha256:02673e456dc5ab13659f85196c534dc596d4ef260e4d86e856c3b2773ce09843&quot;,<br>                &quot;sha256:02af06682e3590ab952599fbadac535ede5d60d78848e555aa58d0c0abbde786&quot;,<br>                &quot;sha256:03680bb39035fbcffe828eae9c3f8afc0428c91d38e7d61aa992ef7a59fb120e&quot;,<br>                &quot;sha256:0570d21da019941634a531444364f2482e8db0b3425fcd5ac0c36565a64142c8&quot;,<br>                &quot;sha256:09c77f964f351a7369cc343911e0df63e762e42bac24cd7d18525961c81754f4&quot;,<br>                &quot;sha256:0d3d5b7db9ed8a2b11a774db2bbea7ba1884430a205dbd54a32d61d7c2a190fa&quot;,<br>                &quot;sha256:1063da2c85b95f2d1a430f1c33b55c9c17ffaf5e612e10aeaad641c55a9e2b9d&quot;,<br>                &quot;sha256:12ebea541c44fdc88ccb794a13fe861cc5e35d64ed689513a5c03d05b53b7c82&quot;,<br>                &quot;sha256:153e7b6e724761741e0974fc4dcd406d35ba70b92bfe3fedcb497226c93b9da7&quot;,<br>                &quot;sha256:15b26ddf78d57f1d143bdf32e820fd8935d36abe8a25eb9ec0b5a71c82eb3895&quot;,<br>                &quot;sha256:1872d01ac8c618a8da634e232f24793883d6e456a66593135aeafe3784b0848d&quot;,<br>                &quot;sha256:187d18082694a29005ba2944c882344b6748d5be69e3a89bf3cc9d878e548d5a&quot;,<br>                &quot;sha256:1b2919306936ac6efb3aed1fbf81039f7087ddadb3160882a57ee2ff74fd2382&quot;,<br>                &quot;sha256:232ac332403e37e4a03d209a3f92ed9071f7d3dbda70e2a5e9cff1c4ba9f0678&quot;,<br>                &quot;sha256:23e8565ab7ff33218530bc817922fae827420f143479b753104ab801145b1d5b&quot;,<br>                &quot;sha256:24817cb02cbef7cd499f7c9a2735286b4782bd47a5b3516a0e84c50eab44b98e&quot;,<br>                &quot;sha256:249c6470a2b60935bafd1d1d13cd613f8cd8388d53461c67397ee6a0f5dce741&quot;,<br>                &quot;sha256:24a91a981f185721542a0b7c92e9054b7ab4fea0508a795846bc5b0abf8118d4&quot;,<br>                &quot;sha256:2502dd2a736c879c0f0d3e2161e74d9907231e25d35794584b1ca5284e43f596&quot;,<br>                &quot;sha256:250c9eb0f4600361dd80d46112213dff2286231d92d3e52af1e5a6083d10cad9&quot;,<br>                &quot;sha256:278c296c6f96fa686d74eb449ea1697f3c03dc28b75f873b65b5201806346a69&quot;,<br>                &quot;sha256:2935ffc78db9645cb2086c2f8f4cfd23d9b73cc0dc80334bc30aac6f03f68f8c&quot;,<br>                &quot;sha256:2f4a0033ce9a76e391542c182f0d48d084855b5fcba5010f707c8e8c34663d77&quot;,<br>                &quot;sha256:30a85aed0b864ac88309b7d94be09f6046c834ef60762a8833b660139cfbad13&quot;,<br>                &quot;sha256:380c4bde80bce25c6e4f77b19386f5ec9db230df9f2f2ac1e5ad7af2caa70459&quot;,<br>                &quot;sha256:3ae38d325b512f63f8da31f826e6cb6c367336f95e418137286ba362925c877e&quot;,<br>                &quot;sha256:3b447982ad46348c02cb90d230b75ac34e9886273df3a93eec0539308a6296d7&quot;,<br>                &quot;sha256:3debd1150027933210c2fc321527c2299118aa929c2f5a0a80ab6953e3bd1908&quot;,<br>                &quot;sha256:4162918ef3098851fcd8a628bf9b6a98d10c380725df9e04caf5ca6dd48c847a&quot;,<br>                &quot;sha256:468d2a840567b13a590e67dd276c570f8de00ed767ecc611994c301d0f8c014f&quot;,<br>                &quot;sha256:4cc152c5dd831641e995764f9f0b6589519f6f5123258ccaca8c6d34572fefa8&quot;,<br>                &quot;sha256:542da1178c1c6af8873e143910e2269add130a299c9106eef2594e15dae5e482&quot;,<br>                &quot;sha256:557b21a44ceac6c6b9773bc65aa1b4cc3e248a5ad2f5b914b91579a32e22204d&quot;,<br>                &quot;sha256:5707a746c6083a3a74b46b3a631d78d129edab06195a92a8ece755aac25a3f3d&quot;,<br>                &quot;sha256:588245972aca710b5b68802c8cad9edaa98589b1b42ad2b53accd6910dad3545&quot;,<br>                &quot;sha256:5adf257bd58c1b8632046bbe43ee38c04e1038e9d37de9c57a94d6bd6ce5da34&quot;,<br>                &quot;sha256:619d1c96099be5823db34fe89e2582b336b5b074a7f47f819d6b3a57ff7bdb86&quot;,<br>                &quot;sha256:63563193aec44bce707e0c5ca64ff69fa72ed7cf34ce6e11d5127555756fd2f6&quot;,<br>                &quot;sha256:67b8cc9574bb518ec76dc8e705d4c39ae78bb96237cb533edac149352c1f39fe&quot;,<br>                &quot;sha256:6a685067d05e46641d5d1623d7c7fdf15a357546cbb2f71b0ebde91b175ffc3e&quot;,<br>                &quot;sha256:70f1d09c0d7748b73290b29219e854b3207aea922f839437870d8cc2168e31cc&quot;,<br>                &quot;sha256:750b446b2ffce1739e8578576092179160f6d26bd5e23eb1789c4d64d5af7dc7&quot;,<br>                &quot;sha256:7966951325782121e67c81299a031f4c115615e68046f79b85856b86ebffc4cd&quot;,<br>                &quot;sha256:7b8b8bf1189b3ba9b8de5c8db4d541b406611a71a955bbbd7385bbc45fcb786c&quot;,<br>                &quot;sha256:7f5d10bae5d78e4551b7be7a9b29643a95aded9d0f602aa2ba584f0388e7a557&quot;,<br>                &quot;sha256:805dfea4ca10411a5296bcc75638017215a93ffb584c9e344731eef0dcfb026a&quot;,<br>                &quot;sha256:81bf654678e575403736b85ba3a7867e31c2c30a69bc57fe88e3ace52fb17b89&quot;,<br>                &quot;sha256:82eb849f085624f6a607538ee7b83a6d8126df6d2f7d3b319cb837b289123078&quot;,<br>                &quot;sha256:85a32721ddde63c9df9ebb0d2045b9691d9750cb139c161c80e500d210f5e26e&quot;,<br>                &quot;sha256:86d1f65ac145e2c9ed71d8ffb1905e9bba3a91ae29ba55b4c46ae6fc31d7c0d4&quot;,<br>                &quot;sha256:86f63face3a527284f7bb8a9d4f78988e3c06823f7bea2bd6f0e0e9298ca0403&quot;,<br>                &quot;sha256:8eaf82f0eccd1505cf39a45a6bd0a8cf1c70dcfc30dba338207a969d91b965c0&quot;,<br>                &quot;sha256:93aa7eef6ee71c629b51ef873991d6911b906d7312c6e8e99790c0f33c576f89&quot;,<br>                &quot;sha256:96c2b49eb6a72c0e4991d62406e365d87067ca14c1a729a870d22354e6f68115&quot;,<br>                &quot;sha256:9cf3126b85822c4e53aa28c7ec9869b924d6fcfb76e77a45c44b83d91afd74f9&quot;,<br>                &quot;sha256:9fe359b2e3a7729010060fbca442ca225280c16e923b37db0e955ac2a2b72a05&quot;,<br>                &quot;sha256:a0ac5e7015a5920cfce654c06618ec40c33e12801711da6b4258af59a8eff00a&quot;,<br>                &quot;sha256:a3f93dab657839dfa61025056606600a11d0b696d79386f974e459a3fbc568ec&quot;,<br>                &quot;sha256:a4b71f4d1765639372a3b32d2638197f5cd5221b19531f9245fcc9ee62d38f56&quot;,<br>                &quot;sha256:aae32c93e0f64469f74ccc730a7cb21c7610af3a775157e50bbd38f816536b38&quot;,<br>                &quot;sha256:aaf7b34c5bc56b38c931a54f7952f1ff0ae77a2e82496583b247f7c969eb1479&quot;,<br>                &quot;sha256:abecce40dfebbfa6abf8e324e1860092eeca6f7375c8c4e655a8afb61af58f2c&quot;,<br>                &quot;sha256:abf0d9f45ea5fb95051c8bfe43cb40cda383772f7e5023a83cc481ca2604d74e&quot;,<br>                &quot;sha256:ac71b2977fb90c35d41c9453116e283fac47bb9096ad917b8819ca8b943abecd&quot;,<br>                &quot;sha256:ada214c6fa40f8d800e575de6b91a40d0548139e5dc457d2ebb61470abf50186&quot;,<br>                &quot;sha256:b09719a17a2301178fac4470d54b1680b18a5048b481cb8890e1ef820cb80455&quot;,<br>                &quot;sha256:b1121de0e9d6e6ca08289583d7491e7fcb18a439305b34a30b20d8215922d43c&quot;,<br>                &quot;sha256:b3b2316b25644b23b54a6f6401074cebcecd1244c0b8e80111c9a3f1c8e83d65&quot;,<br>                &quot;sha256:b3d9b48ee6e3967b7901c052b670c7dda6deb812c309439adaffdec55c6d7b78&quot;,<br>                &quot;sha256:b5bcf60a228acae568e9911f410f9d9e0d43197d030ae5799e20dca8df588287&quot;,<br>                &quot;sha256:b8f3307af845803fb0b060ab76cf6dd3a13adc15b6b451f54281d25911eb92df&quot;,<br>                &quot;sha256:c2af80fb58f0f24b3f3adcb9148e6203fa67dd3f61c4af146ecad033024dde43&quot;,<br>                &quot;sha256:c350354efb159b8767a6244c166f66e67506e06c8924ed74669b2c70bc8735b1&quot;,<br>                &quot;sha256:c5a74c359b2d47d26cdbbc7845e9662d6b08a1e915eb015d044729e92e7050b7&quot;,<br>                &quot;sha256:c71f16da1ed8949774ef79f4a0260d28b83b3a50c6576f8f4f0288d109777989&quot;,<br>                &quot;sha256:d47ecf253780c90ee181d4d871cd655a789da937454045b17b5798da9393901a&quot;,<br>                &quot;sha256:d7eff0f27edc5afa9e405f7165f85a6d782d308f3b6b9d96016c010597958e63&quot;,<br>                &quot;sha256:d97d85fa63f315a8bdaba2af9a6a686e0eceab77b3089af45133252618e70884&quot;,<br>                &quot;sha256:db756e48f9c5c607b5e33dd36b1d5872d0422e960145b08ab0ec7fd420e9d649&quot;,<br>                &quot;sha256:dc45229747b67ffc441b3de2f3ae5e62877a282ea828a5bdb67883c4ee4a8810&quot;,<br>                &quot;sha256:e0fc42822278451bc13a2e8626cf2218ba570f27856b536e00cfa53099724828&quot;,<br>                &quot;sha256:e39c7eb31e3f5b1f88caff88bcff1b7f8334975b46f6ac6e9fc725d829bc35d4&quot;,<br>                &quot;sha256:e46cd37076971c1040fc8c41273a8b3e2c624ce4f2be3f5dfcb7a430c1d3acc2&quot;,<br>                &quot;sha256:e5c1502d4ace69a179305abb3f0bb6141cbe4714bc9b31d427329a95acfc8bdd&quot;,<br>                &quot;sha256:edfe077ab09442d4ef3c52cb1f9dab89bff02f4524afc0acf2d46be17dc479f5&quot;,<br>                &quot;sha256:effe5406c9bd748a871dbcaf3ac69167c38d72db8c9baf3ff954c344f31c4cbe&quot;,<br>                &quot;sha256:f0d1e3732768fecb052d90d62b220af62ead5748ac51ef61e7b32c266cac9293&quot;,<br>                &quot;sha256:f5969baeaea61c97efa706b9b107dcba02784b1601c74ac84f2a532ea079403e&quot;,<br>                &quot;sha256:f8888e31e3a85943743f8fc15e71536bda1c81d5aa36d014a3c0c44481d7db6e&quot;,<br>                &quot;sha256:fc52b79d83a3fe3a360902d3f5d79073a993597d48114c29485e9431092905d8&quot;<br>            ],<br>            &quot;markers&quot;: &quot;python_full_version &gt;= &#39;3.7.0&#39;&quot;,<br>            &quot;version&quot;: &quot;==3.3.0&quot;<br>        },<br>        &quot;idna&quot;: {<br>            &quot;hashes&quot;: [<br>                &quot;sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4&quot;,<br>                &quot;sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2&quot;<br>            ],<br>            &quot;markers&quot;: &quot;python_version &gt;= &#39;3.5&#39;&quot;,<br>            &quot;version&quot;: &quot;==3.4&quot;<br>        },<br>        &quot;requests&quot;: {<br>            &quot;hashes&quot;: [<br>                &quot;sha256:10e94cc4f3121ee6da529d358cdaeaff2f1c409cd377dbc72b825852f2f7e294&quot;,<br>                &quot;sha256:239d7d4458afcb28a692cdd298d87542235f4ca8d36d03a15bfc128a6559a2f4&quot;<br>            ],<br>            &quot;index&quot;: &quot;pypi&quot;,<br>            &quot;markers&quot;: &quot;python_version &gt;= &#39;3.7&#39;&quot;,<br>            &quot;version&quot;: &quot;==2.30.0&quot;<br>        },<br>        &quot;urllib3&quot;: {<br>            &quot;hashes&quot;: [<br>                &quot;sha256:c97dfde1f7bd43a71c8d2a58e369e9b2bf692d1334ea9f9cae55add7d0dd0f84&quot;,<br>                &quot;sha256:fdb6d215c776278489906c2f8916e6e7d4f5a9b602ccbcfdf7f016fc8da0596e&quot;<br>            ],<br>            &quot;markers&quot;: &quot;python_version &gt;= &#39;3.7&#39;&quot;,<br>            &quot;version&quot;: &quot;==2.0.7&quot;<br>        }<br>    },<br>    &quot;develop&quot;: {}<br>}</pre><p><strong>Pipenv Graph</strong></p><p>Pipenv provides the option to display the dependencies using the graph command.</p><pre>➜  pipenv graph<br>requests==2.30.0<br>├── certifi [required: &gt;=2017.4.17, installed: 2023.7.22]<br>├── charset-normalizer [required: &gt;=2,&lt;4, installed: 3.3.0]<br>├── idna [required: &gt;=2.5,&lt;4, installed: 3.4]<br>└── urllib3 [required: &gt;=1.21.1,&lt;3, installed: 2.0.7]</pre><p><strong>Pipenv Shell</strong></p><p>This command will activate the virtual environment if it exists and create a new one if it does not exist.</p><pre>➜  pipenv shell<br>Launching subshell in virtual environment...<br> . /Users/mabuaisha/.local/share/virtualenvs/test-CfuChXcD/bin/activate<br>➜  test  . /Users/mabuaisha/.local/share/virtualenvs/test-CfuChXcD/bin/activate<br>(test) ➜</pre><p>Deactivate the virtual environment can be done by typing exit .</p><p><strong>Pipenv Upgrade</strong></p><p>In case we want to upgrade one of our primary packages in Pipfile we can do that easily without manually updating the Pipfile or touch the Pipfile.lock</p><pre>(test) ➜  pipenv upgrade requests==2.31.0<br>Building requirements...<br>Resolving dependencies...<br>✔ Success!<br>Building requirements...<br>Resolving dependencies...<br>✔ Success!</pre><p>The above command will update both Pipfile &amp; Pipfile.lock</p><p><strong>Pipenv Lock</strong></p><p>This command will generate a new Pipfile.lock if it does not exist. If the file already exists, then pipenv try to sync it Pipfile to make both of them compatible and up-to-date.</p><p>Some Notes about the locking:</p><ol><li>Never let your <strong>CI/CD</strong> run any lock files. The locked file should be committed as part of the <strong>PR</strong></li><li>Whenever we install/uninstall any packages, the lock file will be updated automatically</li><li>The Pipfile.lock should be the file used in order to install packages</li></ol><p><strong>Pipenv Sync</strong></p><p>In case you want to install the packages from the Pipfile.lock which I think in my opinion is the option that you should use always, then pipenv sync is the one to go with.</p><p>Let us imagine you join a new team and would like to test the project and install all the packages from Pipfile.lock .</p><pre>(test) ➜  pipenv sync<br>Installing dependencies from Pipfile.lock (121ad4)...<br>All dependencies are now up-to-date!</pre><p><strong>Pipenv Requirements</strong></p><p>Pipenv supports also generating requirements.txt fromPipfile.lock in case you want still to use the old format.</p><pre>pipenv requirements &gt; requirements.txt<br><br>-- requirements.txt<br>-i https://pypi.org/simple<br>blinker==1.6.3; python_version &gt;= &#39;3.7&#39;<br>certifi==2023.7.22; python_version &gt;= &#39;3.6&#39;<br>charset-normalizer==3.3.0; python_full_version &gt;= &#39;3.7.0&#39;<br>click==8.1.7; python_version &gt;= &#39;3.7&#39;<br>flask==3.0.0; python_version &gt;= &#39;3.8&#39;<br>idna==3.4; python_version &gt;= &#39;3.5&#39;<br>itsdangerous==2.1.2; python_version &gt;= &#39;3.7&#39;<br>jinja2==3.1.2; python_version &gt;= &#39;3.7&#39;<br>markupsafe==2.1.3; python_version &gt;= &#39;3.7&#39;<br>requests==2.31.0; python_version &gt;= &#39;3.7&#39;<br>urllib3==2.0.7; python_version &gt;= &#39;3.7&#39;<br>werkzeug==3.0.0; python_version &gt;= &#39;3.8&#39;</pre><h3>Poetry</h3><p>Poetry is another fancy tool similar to pipenv that help to simplify the development process and solve the common issues related to the dependencies resolutions. Poetry’s capabilities include building wheels packages and publishing them and also supports plugins in case you want to extend Poetry’s functionality.</p><p><strong>Installation</strong></p><p>Poetry can be installed using different ways:</p><ol><li>Using <a href="https://python-poetry.org/docs/#installing-with-the-official-installer">Official Installer</a></li><li>Using <a href="https://python-poetry.org/docs/#installing-with-pipx">pipx</a>.</li><li>Manually using <a href="https://python-poetry.org/docs/#installing-manually">pip</a>.</li></ol><p>Using an installer is the simplest approach by downloading the binaries and installing them using Python</p><pre>curl -sSL https://install.python-poetry.org | python3 -<br><br><br>Retrieving Poetry metadata<br><br># Welcome to Poetry!<br><br>This will download and install the latest version of Poetry,<br>a dependency and package manager for Python.<br><br>It will add the `poetry` command to Poetry&#39;s bin directory, located at:<br><br>/Users/mabuaisha/.local/bin<br><br>You can uninstall at any time by executing this script with the --uninstall option,<br>and these changes will be reverted.<br><br>Installing Poetry (1.6.1): Installing Poetry<br>Installing Poetry (1.6.1): Done<br><br>Poetry (1.6.1) is installed now. Great!<br><br>You can test that everything is set up by executing:<br><br>`poetry --version`</pre><p>Make sure that the Poetry command is available, run the following command</p><pre>➜  ~ poetry --version<br>Poetry (version 1.6.1)</pre><p>Poetry supports more options than the pipenv</p><pre>➜  ~ poetry list<br>Poetry (version 1.6.1)<br><br>Usage:<br>  command [options] [arguments]<br><br>Options:<br>  -h, --help                 Display help for the given command. When no command is given display help for the list command.<br>  -q, --quiet                Do not output any message.<br>  -V, --version              Display this application version.<br>      --ansi                 Force ANSI output.<br>      --no-ansi              Disable ANSI output.<br>  -n, --no-interaction       Do not ask any interactive question.<br>      --no-plugins           Disables plugins.<br>      --no-cache             Disables Poetry source caches.<br>  -C, --directory=DIRECTORY  The working directory for the Poetry command (defaults to the current working directory).<br>  -v|vv|vvv, --verbose       Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug.<br><br>Available commands:<br>  about              Shows information about Poetry.<br>  add                Adds a new dependency to pyproject.toml.<br>  build              Builds a package, as a tarball and a wheel by default.<br>  check              Validates the content of the pyproject.toml file and its consistency with the poetry.lock file.<br>  config             Manages configuration settings.<br>  export             Exports the lock file to alternative formats.<br>  help               Displays help for a command.<br>  init               Creates a basic pyproject.toml file in the current directory.<br>  install            Installs the project dependencies.<br>  list               Lists commands.<br>  lock               Locks the project dependencies.<br>  new                Creates a new Python project at &lt;path&gt;.<br>  publish            Publishes a package to a remote repository.<br>  remove             Removes a package from the project dependencies.<br>  run                Runs a command in the appropriate environment.<br>  search             Searches for packages on remote repositories.<br>  shell              Spawns a shell within the virtual environment.<br>  show               Shows information about packages.<br>  update             Update the dependencies as according to the pyproject.toml file.<br>  version            Shows the version of the project or bumps it when a valid bump rule is provided.<br><br> cache<br>  cache clear        Clears a Poetry cache by name.<br>  cache list         List Poetry&#39;s caches.<br><br> debug<br>  debug info         Shows debug information.<br>  debug resolve      Debugs dependency resolution.<br><br> env<br>  env info           Displays information about the current environment.<br>  env list           Lists all virtualenvs associated with the current project.<br>  env remove         Remove virtual environments associated with the project.<br>  env use            Activates or creates a new virtualenv for the current project.<br><br> self<br>  self add           Add additional packages to Poetry&#39;s runtime environment.<br>  self install       Install locked packages (incl. addons) required by this Poetry installation.<br>  self lock          Lock the Poetry installation&#39;s system requirements.<br>  self remove        Remove additional packages from Poetry&#39;s runtime environment.<br>  self show          Show packages from Poetry&#39;s runtime environment.<br>  self show plugins  Shows information about the currently installed plugins.<br>  self update        Updates Poetry to the latest version.<br><br> source<br>  source add         Add source configuration for project.<br>  source remove      Remove source configured for the project.<br>  source show        Show information about sources configured for the project.</pre><p><strong>Project Setup</strong></p><p><strong>Poetry</strong> supports a concept called <strong>project</strong> acting as a boilerplate for your Python application and creating all the necessary files for managing dependencies.</p><p>Poetry supports creating a new project or initialising from the existing directory.</p><pre>➜  ~ poetry new poetry-app<br>Created package poetry_app in poetry-app</pre><p>The above command sets a new project by creating a directory that contains all the files and directories for <strong>Poetry.</strong></p><pre>poetry-app<br>├── pyproject.toml<br>├── README.md<br>├── poetry_app<br>│   └── __init__.py<br>└── tests<br>    └── __init__.py</pre><p>The pyproject.toml is the main file that is responsible for managing the project and its dependencies (Similar to the Pipfile but with extra capabilities).</p><pre>[tool.poetry]<br>name = &quot;poetry-app&quot;<br>version = &quot;0.1.0&quot;<br>description = &quot;&quot;<br>authors = [&quot;Mohammed AbuAisha &lt;mabuaisha@outlook.com&gt;&quot;]<br>readme = &quot;README.md&quot;<br><br>[tool.poetry.dependencies]<br>python = &quot;^3.11&quot;<br><br>[build-system]<br>requires = [&quot;poetry-core&quot;]<br>build-backend = &quot;poetry.core.masonry.api&quot;</pre><p>By default, poetry automatically creates a package name that matches the name of the project but it normalizes the package names by transforming the dash (-) of your project name into an underscore (_) as specified in the example above (project name poetry-app , package name poetry_app ).</p><p>poetry gives the ability to change this behavior by one of the two options:</p><ol><li>Using --name option when creating a new project</li><li>Using the srclayout for the project.</li></ol><p>Choosing the src layout will create sub-directories under src directory</p><pre>➜  ~ poetry new --src poetry-app<br>Created package poetry_app in poetry-app<br><br>poetry-app/<br>│<br>├── src/<br>│   │<br>│   └── poetry_app/<br>│       └── __init__.py<br>│<br>├── tests/<br>│   ├── __init__.py<br>│   <br>│<br>├── README.rst<br>└── pyproject.toml</pre><pre>[tool.poetry]<br>name = &quot;poetry-app&quot;<br>version = &quot;0.1.0&quot;<br>description = &quot;&quot;<br>authors = [&quot;Mohammed AbuAisha &lt;mohammed.abuaisha@hotjar.com&gt;&quot;]<br>readme = &quot;README.md&quot;<br>packages = [{include = &quot;poetry_app&quot;, from = &quot;src&quot;}]<br><br>[tool.poetry.dependencies]<br>python = &quot;^3.11&quot;<br><br><br>[build-system]<br>requires = [&quot;poetry-core&quot;]<br>build-backend = &quot;poetry.core.masonry.api&quot;</pre><p>The configuration for the pyproject.toml was updated, where we can see that packages now refers to the poetry_app under src .</p><p>Note: Mainly the packages configuration is important for the packages and modules to be included in the final distribution</p><p>Someone could ask, what about the pre-existing project? how to configure it using poetry ?. Simply, poetry provide an option for that using init command.</p><pre>cd pre-existing-project<br>poetry init</pre><p>poetry will interactively create and configure the pyproject.toml</p><p><strong>Poetry Dependencies</strong></p><p>New dependencies can be added easily from the poetry command line using add the command.</p><pre>➜  poetry-app poetry add requests<br>Using version ^2.31.0 for requests<br><br>Updating dependencies<br>Resolving dependencies... (0.3s)<br><br>Package operations: 5 installs, 0 updates, 0 removals<br><br>  • Installing certifi (2023.7.22)<br>  • Installing charset-normalizer (3.3.0)<br>  • Installing idna (3.4)<br>  • Installing urllib3 (2.0.7)<br>  • Installing requests (2.31.0)<br><br>Writing lock file</pre><p>The above command will do the following:</p><ol><li>Update pyproject.toml by adding the dependencies</li><li>Update the lock file poetry.lock</li><li>Create a virtual environment if it does not exist</li><li>Install all dependencies inside the created/existing virtual environment associated with the current project</li></ol><pre>[[package]]<br>name = &quot;certifi&quot;<br>version = &quot;2023.7.22&quot;<br>description = &quot;Python package for providing Mozilla&#39;s CA Bundle.&quot;<br>optional = false<br>python-versions = &quot;&gt;=3.6&quot;<br>files = [<br>    {file = &quot;certifi-2023.7.22-py3-none-any.whl&quot;, hash = &quot;sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9&quot;},<br>    {file = &quot;certifi-2023.7.22.tar.gz&quot;, hash = &quot;sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082&quot;},<br>]<br><br>[[package]]<br>name = &quot;charset-normalizer&quot;<br>version = &quot;3.3.0&quot;<br>description = &quot;The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet.&quot;<br>optional = false<br>python-versions = &quot;&gt;=3.7.0&quot;<br>files = [<br>    {file = &quot;charset-normalizer-3.3.0.tar.gz&quot;, hash = &quot;sha256:63563193aec44bce707e0c5ca64ff69fa72ed7cf34ce6e11d5127555756fd2f6&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-macosx_10_9_universal2.whl&quot;, hash = &quot;sha256:effe5406c9bd748a871dbcaf3ac69167c38d72db8c9baf3ff954c344f31c4cbe&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-macosx_10_9_x86_64.whl&quot;, hash = &quot;sha256:4162918ef3098851fcd8a628bf9b6a98d10c380725df9e04caf5ca6dd48c847a&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-macosx_11_0_arm64.whl&quot;, hash = &quot;sha256:0570d21da019941634a531444364f2482e8db0b3425fcd5ac0c36565a64142c8&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl&quot;, hash = &quot;sha256:5707a746c6083a3a74b46b3a631d78d129edab06195a92a8ece755aac25a3f3d&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl&quot;, hash = &quot;sha256:278c296c6f96fa686d74eb449ea1697f3c03dc28b75f873b65b5201806346a69&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl&quot;, hash = &quot;sha256:a4b71f4d1765639372a3b32d2638197f5cd5221b19531f9245fcc9ee62d38f56&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl&quot;, hash = &quot;sha256:f5969baeaea61c97efa706b9b107dcba02784b1601c74ac84f2a532ea079403e&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl&quot;, hash = &quot;sha256:a3f93dab657839dfa61025056606600a11d0b696d79386f974e459a3fbc568ec&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-musllinux_1_1_aarch64.whl&quot;, hash = &quot;sha256:db756e48f9c5c607b5e33dd36b1d5872d0422e960145b08ab0ec7fd420e9d649&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-musllinux_1_1_i686.whl&quot;, hash = &quot;sha256:232ac332403e37e4a03d209a3f92ed9071f7d3dbda70e2a5e9cff1c4ba9f0678&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-musllinux_1_1_ppc64le.whl&quot;, hash = &quot;sha256:e5c1502d4ace69a179305abb3f0bb6141cbe4714bc9b31d427329a95acfc8bdd&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-musllinux_1_1_s390x.whl&quot;, hash = &quot;sha256:2502dd2a736c879c0f0d3e2161e74d9907231e25d35794584b1ca5284e43f596&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-musllinux_1_1_x86_64.whl&quot;, hash = &quot;sha256:23e8565ab7ff33218530bc817922fae827420f143479b753104ab801145b1d5b&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-win32.whl&quot;, hash = &quot;sha256:1872d01ac8c618a8da634e232f24793883d6e456a66593135aeafe3784b0848d&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp310-cp310-win_amd64.whl&quot;, hash = &quot;sha256:557b21a44ceac6c6b9773bc65aa1b4cc3e248a5ad2f5b914b91579a32e22204d&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-macosx_10_9_universal2.whl&quot;, hash = &quot;sha256:d7eff0f27edc5afa9e405f7165f85a6d782d308f3b6b9d96016c010597958e63&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-macosx_10_9_x86_64.whl&quot;, hash = &quot;sha256:6a685067d05e46641d5d1623d7c7fdf15a357546cbb2f71b0ebde91b175ffc3e&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-macosx_11_0_arm64.whl&quot;, hash = &quot;sha256:0d3d5b7db9ed8a2b11a774db2bbea7ba1884430a205dbd54a32d61d7c2a190fa&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl&quot;, hash = &quot;sha256:2935ffc78db9645cb2086c2f8f4cfd23d9b73cc0dc80334bc30aac6f03f68f8c&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl&quot;, hash = &quot;sha256:9fe359b2e3a7729010060fbca442ca225280c16e923b37db0e955ac2a2b72a05&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl&quot;, hash = &quot;sha256:380c4bde80bce25c6e4f77b19386f5ec9db230df9f2f2ac1e5ad7af2caa70459&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl&quot;, hash = &quot;sha256:f0d1e3732768fecb052d90d62b220af62ead5748ac51ef61e7b32c266cac9293&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl&quot;, hash = &quot;sha256:1b2919306936ac6efb3aed1fbf81039f7087ddadb3160882a57ee2ff74fd2382&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-musllinux_1_1_aarch64.whl&quot;, hash = &quot;sha256:f8888e31e3a85943743f8fc15e71536bda1c81d5aa36d014a3c0c44481d7db6e&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-musllinux_1_1_i686.whl&quot;, hash = &quot;sha256:82eb849f085624f6a607538ee7b83a6d8126df6d2f7d3b319cb837b289123078&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-musllinux_1_1_ppc64le.whl&quot;, hash = &quot;sha256:7b8b8bf1189b3ba9b8de5c8db4d541b406611a71a955bbbd7385bbc45fcb786c&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-musllinux_1_1_s390x.whl&quot;, hash = &quot;sha256:5adf257bd58c1b8632046bbe43ee38c04e1038e9d37de9c57a94d6bd6ce5da34&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-musllinux_1_1_x86_64.whl&quot;, hash = &quot;sha256:c350354efb159b8767a6244c166f66e67506e06c8924ed74669b2c70bc8735b1&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-win32.whl&quot;, hash = &quot;sha256:02af06682e3590ab952599fbadac535ede5d60d78848e555aa58d0c0abbde786&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp311-cp311-win_amd64.whl&quot;, hash = &quot;sha256:86d1f65ac145e2c9ed71d8ffb1905e9bba3a91ae29ba55b4c46ae6fc31d7c0d4&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-macosx_10_9_universal2.whl&quot;, hash = &quot;sha256:3b447982ad46348c02cb90d230b75ac34e9886273df3a93eec0539308a6296d7&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-macosx_10_9_x86_64.whl&quot;, hash = &quot;sha256:abf0d9f45ea5fb95051c8bfe43cb40cda383772f7e5023a83cc481ca2604d74e&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-macosx_11_0_arm64.whl&quot;, hash = &quot;sha256:b09719a17a2301178fac4470d54b1680b18a5048b481cb8890e1ef820cb80455&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl&quot;, hash = &quot;sha256:b3d9b48ee6e3967b7901c052b670c7dda6deb812c309439adaffdec55c6d7b78&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl&quot;, hash = &quot;sha256:edfe077ab09442d4ef3c52cb1f9dab89bff02f4524afc0acf2d46be17dc479f5&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl&quot;, hash = &quot;sha256:3debd1150027933210c2fc321527c2299118aa929c2f5a0a80ab6953e3bd1908&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl&quot;, hash = &quot;sha256:86f63face3a527284f7bb8a9d4f78988e3c06823f7bea2bd6f0e0e9298ca0403&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl&quot;, hash = &quot;sha256:24817cb02cbef7cd499f7c9a2735286b4782bd47a5b3516a0e84c50eab44b98e&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-musllinux_1_1_aarch64.whl&quot;, hash = &quot;sha256:c71f16da1ed8949774ef79f4a0260d28b83b3a50c6576f8f4f0288d109777989&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-musllinux_1_1_i686.whl&quot;, hash = &quot;sha256:9cf3126b85822c4e53aa28c7ec9869b924d6fcfb76e77a45c44b83d91afd74f9&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-musllinux_1_1_ppc64le.whl&quot;, hash = &quot;sha256:b3b2316b25644b23b54a6f6401074cebcecd1244c0b8e80111c9a3f1c8e83d65&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-musllinux_1_1_s390x.whl&quot;, hash = &quot;sha256:03680bb39035fbcffe828eae9c3f8afc0428c91d38e7d61aa992ef7a59fb120e&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-musllinux_1_1_x86_64.whl&quot;, hash = &quot;sha256:4cc152c5dd831641e995764f9f0b6589519f6f5123258ccaca8c6d34572fefa8&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-win32.whl&quot;, hash = &quot;sha256:b8f3307af845803fb0b060ab76cf6dd3a13adc15b6b451f54281d25911eb92df&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp312-cp312-win_amd64.whl&quot;, hash = &quot;sha256:8eaf82f0eccd1505cf39a45a6bd0a8cf1c70dcfc30dba338207a969d91b965c0&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-macosx_10_9_x86_64.whl&quot;, hash = &quot;sha256:dc45229747b67ffc441b3de2f3ae5e62877a282ea828a5bdb67883c4ee4a8810&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl&quot;, hash = &quot;sha256:2f4a0033ce9a76e391542c182f0d48d084855b5fcba5010f707c8e8c34663d77&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl&quot;, hash = &quot;sha256:ada214c6fa40f8d800e575de6b91a40d0548139e5dc457d2ebb61470abf50186&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl&quot;, hash = &quot;sha256:b1121de0e9d6e6ca08289583d7491e7fcb18a439305b34a30b20d8215922d43c&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl&quot;, hash = &quot;sha256:1063da2c85b95f2d1a430f1c33b55c9c17ffaf5e612e10aeaad641c55a9e2b9d&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl&quot;, hash = &quot;sha256:70f1d09c0d7748b73290b29219e854b3207aea922f839437870d8cc2168e31cc&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-musllinux_1_1_aarch64.whl&quot;, hash = &quot;sha256:250c9eb0f4600361dd80d46112213dff2286231d92d3e52af1e5a6083d10cad9&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-musllinux_1_1_i686.whl&quot;, hash = &quot;sha256:750b446b2ffce1739e8578576092179160f6d26bd5e23eb1789c4d64d5af7dc7&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-musllinux_1_1_ppc64le.whl&quot;, hash = &quot;sha256:fc52b79d83a3fe3a360902d3f5d79073a993597d48114c29485e9431092905d8&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-musllinux_1_1_s390x.whl&quot;, hash = &quot;sha256:588245972aca710b5b68802c8cad9edaa98589b1b42ad2b53accd6910dad3545&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-musllinux_1_1_x86_64.whl&quot;, hash = &quot;sha256:e39c7eb31e3f5b1f88caff88bcff1b7f8334975b46f6ac6e9fc725d829bc35d4&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-win32.whl&quot;, hash = &quot;sha256:abecce40dfebbfa6abf8e324e1860092eeca6f7375c8c4e655a8afb61af58f2c&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp37-cp37m-win_amd64.whl&quot;, hash = &quot;sha256:24a91a981f185721542a0b7c92e9054b7ab4fea0508a795846bc5b0abf8118d4&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-macosx_10_9_universal2.whl&quot;, hash = &quot;sha256:67b8cc9574bb518ec76dc8e705d4c39ae78bb96237cb533edac149352c1f39fe&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-macosx_10_9_x86_64.whl&quot;, hash = &quot;sha256:ac71b2977fb90c35d41c9453116e283fac47bb9096ad917b8819ca8b943abecd&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-macosx_11_0_arm64.whl&quot;, hash = &quot;sha256:3ae38d325b512f63f8da31f826e6cb6c367336f95e418137286ba362925c877e&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl&quot;, hash = &quot;sha256:542da1178c1c6af8873e143910e2269add130a299c9106eef2594e15dae5e482&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl&quot;, hash = &quot;sha256:30a85aed0b864ac88309b7d94be09f6046c834ef60762a8833b660139cfbad13&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl&quot;, hash = &quot;sha256:aae32c93e0f64469f74ccc730a7cb21c7610af3a775157e50bbd38f816536b38&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl&quot;, hash = &quot;sha256:15b26ddf78d57f1d143bdf32e820fd8935d36abe8a25eb9ec0b5a71c82eb3895&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl&quot;, hash = &quot;sha256:7f5d10bae5d78e4551b7be7a9b29643a95aded9d0f602aa2ba584f0388e7a557&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-musllinux_1_1_aarch64.whl&quot;, hash = &quot;sha256:249c6470a2b60935bafd1d1d13cd613f8cd8388d53461c67397ee6a0f5dce741&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-musllinux_1_1_i686.whl&quot;, hash = &quot;sha256:c5a74c359b2d47d26cdbbc7845e9662d6b08a1e915eb015d044729e92e7050b7&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-musllinux_1_1_ppc64le.whl&quot;, hash = &quot;sha256:b5bcf60a228acae568e9911f410f9d9e0d43197d030ae5799e20dca8df588287&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-musllinux_1_1_s390x.whl&quot;, hash = &quot;sha256:187d18082694a29005ba2944c882344b6748d5be69e3a89bf3cc9d878e548d5a&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-musllinux_1_1_x86_64.whl&quot;, hash = &quot;sha256:81bf654678e575403736b85ba3a7867e31c2c30a69bc57fe88e3ace52fb17b89&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-win32.whl&quot;, hash = &quot;sha256:85a32721ddde63c9df9ebb0d2045b9691d9750cb139c161c80e500d210f5e26e&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp38-cp38-win_amd64.whl&quot;, hash = &quot;sha256:468d2a840567b13a590e67dd276c570f8de00ed767ecc611994c301d0f8c014f&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-macosx_10_9_universal2.whl&quot;, hash = &quot;sha256:e0fc42822278451bc13a2e8626cf2218ba570f27856b536e00cfa53099724828&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-macosx_10_9_x86_64.whl&quot;, hash = &quot;sha256:09c77f964f351a7369cc343911e0df63e762e42bac24cd7d18525961c81754f4&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-macosx_11_0_arm64.whl&quot;, hash = &quot;sha256:12ebea541c44fdc88ccb794a13fe861cc5e35d64ed689513a5c03d05b53b7c82&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl&quot;, hash = &quot;sha256:805dfea4ca10411a5296bcc75638017215a93ffb584c9e344731eef0dcfb026a&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl&quot;, hash = &quot;sha256:96c2b49eb6a72c0e4991d62406e365d87067ca14c1a729a870d22354e6f68115&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl&quot;, hash = &quot;sha256:aaf7b34c5bc56b38c931a54f7952f1ff0ae77a2e82496583b247f7c969eb1479&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl&quot;, hash = &quot;sha256:619d1c96099be5823db34fe89e2582b336b5b074a7f47f819d6b3a57ff7bdb86&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl&quot;, hash = &quot;sha256:a0ac5e7015a5920cfce654c06618ec40c33e12801711da6b4258af59a8eff00a&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-musllinux_1_1_aarch64.whl&quot;, hash = &quot;sha256:93aa7eef6ee71c629b51ef873991d6911b906d7312c6e8e99790c0f33c576f89&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-musllinux_1_1_i686.whl&quot;, hash = &quot;sha256:7966951325782121e67c81299a031f4c115615e68046f79b85856b86ebffc4cd&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-musllinux_1_1_ppc64le.whl&quot;, hash = &quot;sha256:02673e456dc5ab13659f85196c534dc596d4ef260e4d86e856c3b2773ce09843&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-musllinux_1_1_s390x.whl&quot;, hash = &quot;sha256:c2af80fb58f0f24b3f3adcb9148e6203fa67dd3f61c4af146ecad033024dde43&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-musllinux_1_1_x86_64.whl&quot;, hash = &quot;sha256:153e7b6e724761741e0974fc4dcd406d35ba70b92bfe3fedcb497226c93b9da7&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-win32.whl&quot;, hash = &quot;sha256:d47ecf253780c90ee181d4d871cd655a789da937454045b17b5798da9393901a&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-cp39-cp39-win_amd64.whl&quot;, hash = &quot;sha256:d97d85fa63f315a8bdaba2af9a6a686e0eceab77b3089af45133252618e70884&quot;},<br>    {file = &quot;charset_normalizer-3.3.0-py3-none-any.whl&quot;, hash = &quot;sha256:e46cd37076971c1040fc8c41273a8b3e2c624ce4f2be3f5dfcb7a430c1d3acc2&quot;},<br>]<br><br>[[package]]<br>name = &quot;idna&quot;<br>version = &quot;3.4&quot;<br>description = &quot;Internationalized Domain Names in Applications (IDNA)&quot;<br>optional = false<br>python-versions = &quot;&gt;=3.5&quot;<br>files = [<br>    {file = &quot;idna-3.4-py3-none-any.whl&quot;, hash = &quot;sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2&quot;},<br>    {file = &quot;idna-3.4.tar.gz&quot;, hash = &quot;sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4&quot;},<br>]<br><br>[[package]]<br>name = &quot;requests&quot;<br>version = &quot;2.31.0&quot;<br>description = &quot;Python HTTP for Humans.&quot;<br>optional = false<br>python-versions = &quot;&gt;=3.7&quot;<br>files = [<br>    {file = &quot;requests-2.31.0-py3-none-any.whl&quot;, hash = &quot;sha256:58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f&quot;},<br>    {file = &quot;requests-2.31.0.tar.gz&quot;, hash = &quot;sha256:942c5a758f98d790eaed1a29cb6eefc7ffb0d1cf7af05c3d2791656dbd6ad1e1&quot;},<br>]<br><br>[package.dependencies]<br>certifi = &quot;&gt;=2017.4.17&quot;<br>charset-normalizer = &quot;&gt;=2,&lt;4&quot;<br>idna = &quot;&gt;=2.5,&lt;4&quot;<br>urllib3 = &quot;&gt;=1.21.1,&lt;3&quot;<br><br>[package.extras]<br>socks = [&quot;PySocks (&gt;=1.5.6,!=1.5.7)&quot;]<br>use-chardet-on-py3 = [&quot;chardet (&gt;=3.0.2,&lt;6)&quot;]<br><br>[[package]]<br>name = &quot;urllib3&quot;<br>version = &quot;2.0.7&quot;<br>description = &quot;HTTP library with thread-safe connection pooling, file post, and more.&quot;<br>optional = false<br>python-versions = &quot;&gt;=3.7&quot;<br>files = [<br>    {file = &quot;urllib3-2.0.7-py3-none-any.whl&quot;, hash = &quot;sha256:fdb6d215c776278489906c2f8916e6e7d4f5a9b602ccbcfdf7f016fc8da0596e&quot;},<br>    {file = &quot;urllib3-2.0.7.tar.gz&quot;, hash = &quot;sha256:c97dfde1f7bd43a71c8d2a58e369e9b2bf692d1334ea9f9cae55add7d0dd0f84&quot;},<br>]<br><br>[package.extras]<br>brotli = [&quot;brotli (&gt;=1.0.9)&quot;, &quot;brotlicffi (&gt;=0.8.0)&quot;]<br>secure = [&quot;certifi&quot;, &quot;cryptography (&gt;=1.9)&quot;, &quot;idna (&gt;=2.0.0)&quot;, &quot;pyopenssl (&gt;=17.1.0)&quot;, &quot;urllib3-secure-extra&quot;]<br>socks = [&quot;pysocks (&gt;=1.5.6,!=1.5.7,&lt;2.0)&quot;]<br>zstd = [&quot;zstandard (&gt;=0.18.0)&quot;]<br><br>[metadata]<br>lock-version = &quot;2.0&quot;<br>python-versions = &quot;^3.11&quot;<br>content-hash = &quot;a5780ef8e06df616beb6eb67292099db49b8fe658fcbf22940e5e1af96a7c14e&quot;</pre><p>Moreover, removing any package will also remove it from pyproject.toml , poetry.lock and virtual environment</p><p><strong>Poetry</strong> <strong>Lock</strong></p><p>We mentioned before that poetry.lockwill be updated automatically when you add a package from the command line. However, if someone updates the pyproject.toml manually by editing the file and adding a new package under tool-poetry-dependencies the following things must be taken into consideration:</p><ol><li>Avoid using poetry install in this case because poetry.lock will be out of sync with the pyproject.toml</li><li>Make sure to run poetry lock in order to resolve all the dependencies and make poetry.lock compatible with pyproject.toml</li></ol><p>For example, if we add flask == &quot;^2.2.3&quot; to the pyproject.toml and run poetry install you will get a warning</p><pre>➜  poetry-app poetry install<br>Installing dependencies from lock file<br>Warning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run `poetry lock [--no-update]` to fix it.<br><br>Because poetry-app depends on flask (^2.3.3) which doesn&#39;t match any versions, version solving failed.</pre><p>In order to solve the above, you must run poetry lock</p><pre>➜  poetry-app poetry lock<br>Updating dependencies<br>Resolving dependencies... (2.5s)<br><br>Writing lock file</pre><p>So after locking the dependencies, you can easily install them without any problem. Moreover, --no-update is another option that can be used with poetry lock in case you do not want to update the dependencies that are already in the poetry.lock file</p><pre>➜  poetry-app poetry install<br>Installing dependencies from lock file<br><br>Package operations: 7 installs, 1 update, 0 removals<br><br>  • Installing markupsafe (2.1.3)<br>  • Installing blinker (1.6.3)<br>  • Updating charset-normalizer (3.3.0 -&gt; 3.3.1)<br>  • Installing click (8.1.7)<br>  • Installing itsdangerous (2.1.2)<br>  • Installing jinja2 (3.1.2)<br>  • Installing werkzeug (3.0.1)<br>  • Installing flask (2.3.3)<br><br>Installing the current project: poetry-app (0.1.0)</pre><p><strong>Poetry Update</strong></p><p>Packages also can be updated using poetry update &lt;PACKAGE_1&gt; &lt;PACKAGE_2&gt; ... This command will update the specified packages by updating both pyproject.toml + poetry.lock</p><pre>➜  poetry-app poetry update flask<br>Updating dependencies<br>Resolving dependencies... (0.3s)</pre><p><strong>Poetry Requirements</strong></p><p>Someone could argue, what about if I want to generate or use requirements.txt ?. Poetry supports both reading and writing to requirements.txt</p><pre>// This will read from poetry.lock and dump to requirements.txt<br>➜  poetry-app poetry export -o requirements.txt</pre><p>The other option is that if you already have a project that uses requirements.txt , you can still add these packages using poetry add</p><p>For example, if you have the following requirements.txt and you want to add it to the poetry ecosystem, then this can be done easily</p><pre>ipython==8.12.2<br>sphinx==4.1.2<br>diagrams==0.23.3</pre><pre>➜  poetry-app poetry add `cat requirements.txt`<br><br>Updating dependencies<br>Resolving dependencies... (2.5s)<br><br>Package operations: 19 installs, 0 updates, 0 removals<br><br>  • Installing six (1.16.0)<br>  • Installing asttokens (2.4.1)<br>  • Installing executing (2.0.0)<br>  • Installing parso (0.8.3)<br>  • Installing ptyprocess (0.7.0)<br>  • Installing pure-eval (0.2.2)<br>  • Installing traitlets (5.12.0)<br>  • Installing wcwidth (0.2.8)<br>  • Installing appnope (0.1.3)<br>  • Installing backcall (0.2.0)<br>  • Installing decorator (5.1.1)<br>  • Installing jedi (0.19.1)<br>  • Installing matplotlib-inline (0.1.6)<br>  • Installing pexpect (4.8.0)<br>  • Installing pickleshare (0.7.5)<br>  • Installing prompt-toolkit (3.0.39)<br>  • Installing pygments (2.16.1)<br>  • Installing stack-data (0.6.3)<br>  • Installing ipython (8.12.2)<br><br>Writing lock file</pre><p><strong>Poetry Shell</strong></p><p>Poetry supports activating the virtual environment and provides the ability to run any command from the shell directly without the need to specify the binaries.</p><pre>➜  poetry-app poetry shell<br>Spawning shell within /Users/mabuaisha/Library/Caches/pypoetry/virtualenvs/poetry-app-7KqqTxz1-py3.11<br>➜  poetry-app emulate bash -c &#39;. /Users/mabuaisha/Library/Caches/pypoetry/virtualenvs/poetry-app-7KqqTxz1-py3.11/bin/activate<br>&#39;<br>(poetry-app-py3.11) ➜  poetry-app</pre><p>Exiting from this shell can also be done using exit command</p><p><strong>Poetry Config</strong></p><p>Poetry also supports listing some configuration values like cache-dir and other settings.</p><pre>➜  poetry-app poetry config --list<br>cache-dir = &quot;/Users/mabuaisha/Library/Caches/pypoetry&quot;<br>experimental.system-git-client = false<br>installer.max-workers = null<br>installer.modern-installation = true<br>installer.no-binary = null<br>installer.parallel = true<br>virtualenvs.create = true<br>virtualenvs.in-project = null<br>virtualenvs.options.always-copy = false<br>virtualenvs.options.no-pip = false<br>virtualenvs.options.no-setuptools = false<br>virtualenvs.options.system-site-packages = false<br>virtualenvs.path = &quot;{cache-dir}/virtualenvs&quot;  # /Users/mabuaisha/Library/Caches/pypoetry/virtualenvs<br>virtualenvs.prefer-active-python = false<br>virtualenvs.prompt = &quot;{project_name}-py{python_version}&quot;<br>➜  poetry-app</pre><p><strong>Poetry Env</strong></p><p>You can get information about virtual environments and manage them using env commands support by poetry.</p><pre>➜  poetry-app poetry env<br><br>The command &quot;env&quot; does not exist.<br><br>Did you mean one of these?<br>    env use<br>    env info<br>    env list<br>    env remove</pre><pre>➜  poetry-app poetry env  info<br><br>Virtualenv<br>Python:         3.11.2<br>Implementation: CPython<br>Path:           /Users/mabuaisha/Library/Caches/pypoetry/virtualenvs/poetry-app-7KqqTxz1-py3.11<br>Executable:     /Users/mabuaisha/Library/Caches/pypoetry/virtualenvs/poetry-app-7KqqTxz1-py3.11/bin/python<br>Valid:          True<br><br>System<br>Platform:   darwin<br>OS:         posix<br>Python:     3.11.2<br>Path:       /Users/mabuaisha/.pyenv/versions/3.11.2<br>Executable: /Users/mabuaisha/.pyenv/versions/3.11.2/bin/python3.11</pre><pre>➜  poetry-app poetry env list<br>poetry-app-7KqqTxz1-py3.11 (Activated)</pre><h3>Summary</h3><p>In this blog, We discussed different tools for managing Python dependencies and versions. Choosing between these tools depends on your needs. However, I would recommend that when developing any Python application consider having a deterministic build and dependencies to avoid any unexpected behavior and these can be achieved using either pip-compile/pipenv/poetry .</p><p>Choosing between these tools depends on your needs and preferences but I found that pip-compile could be enough to solve this major issue.</p><p>Check the next <a href="https://mabuaisha.medium.com/python-applications-versions-management-c9d61b20465b">blog</a> if you want to know more about how to manage different Python versions and running Python-based applications.</p><h3>Stackademic</h3><p><em>Thank you for reading until the end. Before you go:</em></p><ul><li><em>Please consider </em><strong><em>clapping</em></strong><em> and </em><strong><em>following</em></strong><em> the writer! 👏</em></li><li><em>Follow us on </em><a href="https://twitter.com/stackademichq"><strong><em>Twitter(X)</em></strong></a><em>, </em><a href="https://www.linkedin.com/company/stackademic"><strong><em>LinkedIn</em></strong></a><em>, and </em><a href="https://www.youtube.com/c/stackademic"><strong><em>YouTube</em></strong></a><strong><em>.</em></strong></li><li><em>Visit </em><a href="http://stackademic.com/"><strong><em>Stackademic.com</em></strong></a><em> to find out more about how we are democratizing free programming education around the world.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1a7de6ebad9" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/advanced-python-dependencies-management-e1a7de6ebad9">Advanced Python Dependencies Management</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Applications & Versions Management]]></title>
            <link>https://blog.stackademic.com/python-applications-versions-management-c9d61b20465b?source=rss-c79e314439f5------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9d61b20465b</guid>
            <category><![CDATA[python-tools]]></category>
            <category><![CDATA[dependency-management]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Mohammed AbuAisha]]></dc:creator>
            <pubDate>Tue, 21 Nov 2023 11:56:23 GMT</pubDate>
            <atom:updated>2023-11-23T11:59:59.903Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*E6sFdNQ9fR4O0bsnTKSeyQ.jpeg" /></figure><p>In my previous blog <a href="https://medium.com/p/e1a7de6ebad9">post</a>, I discussed tools for managing Python dependencies. However, I didn’t cover tools for managing Python versions or applications. In this post, which is the last in the series, I will cover two main tools: <strong>pipx</strong> and <strong>pyenv</strong>.</p><h3>Pipx</h3><p>pipx is a tool that helps to install and run user-based applications written in Python. You can think of pipx similar to macOS’s brew, JavaScript&#39;s <a href="https://medium.com/@maybekatz/introducing-npx-an-npm-package-runner-55f7d4bd282b">npx</a>, and Linux&#39;s apt .</p><p>The main difference between pipx and pip is that pip is a generic tool for installing both libraries and apps with no environment isolation. pipx mainly focuses on installing applications by creating an isolated environment for each application and its associated packages.</p><p><strong>Pipx Installation</strong></p><p>The minimal requirement for installing pipx is Python 3.7+. If you are on Mac you can install it easily using brew</p><pre>brew install pipx<br>pipx ensurepath</pre><pre>➜  ~ pipx --help<br>usage: pipx [-h] [--version] {install,uninject,inject,upgrade,upgrade-all,uninstall,uninstall-all,reinstall,reinstall-all,list,run,runpip,ensurepath,environment,completions} ...<br><br>Install and execute apps from Python packages.<br><br>Binaries can either be installed globally into isolated Virtual Environments<br>or run directly in a temporary Virtual Environment.<br><br>Virtual Environment location is /Users/mabuaisha/.local/pipx/venvs.<br>Symlinks to apps are placed in /Users/mabuaisha/.local/bin.<br><br>optional environment variables:<br>  PIPX_HOME             Overrides default pipx location. Virtual Environments will be installed to $PIPX_HOME/venvs.<br>  PIPX_BIN_DIR          Overrides location of app installations. Apps are symlinked or copied here.<br>  PIPX_DEFAULT_PYTHON   Overrides default python used for commands.<br>  USE_EMOJI             Overrides emoji behavior. Default value varies based on platform.<br><br>options:<br>  -h, --help            show this help message and exit<br>  --version             Print version and exit<br><br>subcommands:<br>  Get help for commands with pipx COMMAND --help<br><br>  {install,uninject,inject,upgrade,upgrade-all,uninstall,uninstall-all,reinstall,reinstall-all,list,run,runpip,ensurepath,environment,completions}<br>    install             Install a package<br>    uninject            Uninstall injected packages from an existing Virtual Environment<br>    inject              Install packages into an existing Virtual Environment<br>    upgrade             Upgrade a package<br>    upgrade-all         Upgrade all packages. Runs `pip install -U &lt;pkgname&gt;` for each package.<br>    uninstall           Uninstall a package<br>    uninstall-all       Uninstall all packages<br>    reinstall           Reinstall a package<br>    reinstall-all       Reinstall all packages<br>    list                List installed packages<br>    run                 Download the latest version of a package to a temporary virtual environment, then run an app from it. Also compatible with local `__pypackages__` directory (experimental).<br>    runpip              Run pip in an existing pipx-managed Virtual Environment<br>    ensurepath          Ensure directories necessary for pipx operation are in your PATH environment variable.<br>    environment         Print a list of variables used in pipx.constants.<br>    completions         Print instructions on enabling shell completions for pipx</pre><p>We are going to focus mainly on how pipx can install applications and run them.</p><p><strong>Pipx Usage</strong></p><p>We can simply install our application using pipx install &lt;PACKAGE_NAME&gt; . Pipx will also make the installed application available through your shell so that you can access it.</p><pre>➜  ~ pipx install black<br>  installed package black 23.10.1, installed using Python 3.11.2<br>  These apps are now globally available<br>    - black<br>    - blackd<br>done! ✨ 🌟 ✨</pre><pre>➜  ~ pipx list<br>venvs are in /Users/mabuaisha/.local/pipx/venvs<br>apps are exposed on your $PATH at /Users/mabuaisha/.local/bin<br>   package black 23.10.1, installed using Python 3.11.2<br>    - black<br>    - blackd<br>   package pipenv 2023.10.3, installed using Python 3.11.2<br>    - pipenv<br>    - pipenv-resolver</pre><p>behind the scenes, pipx creates a virtual environment for each installed application under the path specified by this environment variable PIPX_LOCAL_VENVS</p><pre>➜  ~ pipx environment<br>PIPX_HOME=/Users/mabuaisha/.local/pipx<br>PIPX_BIN_DIR=/Users/mabuaisha/.local/bin<br>PIPX_SHARED_LIBS=/Users/mabuaisha/.local/pipx/shared<br>PIPX_LOCAL_VENVS=/Users/mabuaisha/.local/pipx/venvs<br>PIPX_LOG_DIR=/Users/mabuaisha/.local/pipx/logs<br>PIPX_TRASH_DIR=/Users/mabuaisha/.local/pipx/.trash<br>PIPX_VENV_CACHEDIR=/Users/mabuaisha/.local/pipx/.cache</pre><p>More examples can be found on pipx<a href="https://pypa.github.io/pipx/examples/">documentation</a></p><h3>Pyenv</h3><p>All the tools that we mentioned before are mainly used in order to manage Python dependencies. Someone may ask how could we then manage different Python versions. Testing and trying new features for the latest Python versions? Testing and developing a project that can run under different Python versions? or installing multiple Python versions on the same machine without messing up the development environment. The answer to these questions is to usepyenv .</p><p>In nutshell, pyenv is a tool that helps in managing multiple Python versions. There are a lot of use cases that we can think of where pyenv could be helpful:</p><ol><li>Mainly running and installing multiple Python versions on the same machine</li><li>Testing new features for the latest Python versions</li><li>Switching between different Python versions easily</li><li>Testing the same project under different Python versions</li></ol><p><strong>Pyenv Installation</strong></p><p>Before installing the pyenv , your system requires some OS dependencies which are different based on the OS. Since I’m a macOS user, I will mention only the macOS dependencies that can installed via brew</p><pre>$ brew install openssl readline sqlite3 xz zlib</pre><p>Once you install these dependencies, it is time to install the pyenv . There are different ways to install pyenv but personally, I prefer using a <a href="https://github.com/pyenv/pyenv-installer">pyenv-installer</a> which helps to install pyenv with other plugins in one shot. This tool is also a project maintained by the owner of the pyenv project that helps to install the following:</p><ol><li>The basic installation of pyenv</li><li>The Installation of pyenv-virtualenv which is a plugin that works with pyenv and virtual environnts</li><li>The installation of pyenv-which-ext which is a plugin to lookup the system command automatically</li><li>The installation of pyenv-docotr which is a plugin to validate that all the dependencies are installed</li><li>The installation of pyenv-update which is a plugin to update the pyenv</li></ol><p>Installing the pyenv can be done from that installer easily using the pyenv.run which is the pyenv-installer</p><pre>$ curl https://pyenv.run | bash<br><br><br>WARNING: seems you still have not added &#39;pyenv&#39; to the load path.<br><br>Load pyenv automatically by adding<br>the following to ~/.bashrc:<br><br>export PATH=&quot;$HOME/.pyenv/bin:$PATH&quot;<br>eval &quot;$(pyenv init -)&quot;<br>eval &quot;$(pyenv virtualenv-init -)&quot;<br><br># This is helpful to reload your configuration so that you can use the pyenv<br>exec &quot;$SHELL&quot;</pre><p>You need to make sure to update your shell configuration in order to make pyenv command available. If you are using .zshrc like me, you need to update the .zshrc instead.</p><pre>➜  ~ pyenv<br>pyenv 2.3.23<br>Usage: pyenv &lt;command&gt; [&lt;args&gt;]<br><br>Some useful pyenv commands are:<br>   --version   Display the version of pyenv<br>   activate    Activate virtual environment<br>   commands    List all available pyenv commands<br>   deactivate   Deactivate virtual environment<br>   doctor      Verify pyenv installation and development tools to build pythons.<br>   exec        Run an executable with the selected Python version<br>   global      Set or show the global Python version(s)<br>   help        Display help for a command<br>   hooks       List hook scripts for a given pyenv command<br>   init        Configure the shell environment for pyenv<br>   install     Install a Python version using python-build<br>   latest      Print the latest installed or known version with the given prefix<br>   local       Set or show the local application-specific Python version(s)<br>   prefix      Display prefixes for Python versions<br>   rehash      Rehash pyenv shims (run this after installing executables)<br>   root        Display the root directory where versions and shims are kept<br>   shell       Set or show the shell-specific Python version<br>   shims       List existing pyenv shims<br>   uninstall   Uninstall Python versions<br>   update      Update pyenv, its plugins including the list of available versions<br>   version     Show the current Python version(s) and its origin<br>   version-file   Detect the file that sets the current pyenv version<br>   version-name   Show the current Python version<br>   version-origin   Explain how the current Python version is set<br>   versions    List all Python versions available to pyenv<br>   virtualenv   Create a Python virtualenv using the pyenv-virtualenv plugin<br>   virtualenv-delete   Uninstall a specific Python virtualenv<br>   virtualenv-init   Configure the shell environment for pyenv-virtualenv<br>   virtualenv-prefix   Display real_prefix for a Python virtualenv version<br>   virtualenvs   List all Python virtualenvs found in `$PYENV_ROOT/versions/*&#39;.<br>   whence      List all Python versions that contain the given executable<br>   which       Display the full path to an executable<br><br>See `pyenv help &lt;command&gt;&#39; for information on a specific command.<br>For full documentation, see: https://github.com/pyenv/pyenv#readme</pre><p><strong>Install Python</strong></p><p>In order to check the Python versions that can be installed using pyenv , you can use pyenv install --list to figure out the desired version to install.</p><pre>➜  ~ pyenv install --list  | grep &#39;3.11&#39;<br>  3.11.0<br>  3.11-dev<br>  3.11.1<br>  3.11.2<br>  3.11.3<br>  3.11.4<br>  miniconda3-3.11-23.5.0-3<br>  miniconda3-4.3.11<br>  pypy2.7-7.3.11-src<br>  pypy2.7-7.3.11<br>  pypy3.8-7.3.11-src<br>  pypy3.8-7.3.11<br>  pypy3.9-7.3.11-src<br>  pypy3.9-7.3.11</pre><p>For example, if you want to install 3.12.0 that can be done using pyenv install as well</p><pre>➜  ~ pyenv install 3.12.0<br>python-build: use openssl@3 from homebrew<br>python-build: use readline from homebrew<br>Installing Python-3.12.0...<br>python-build: use readline from homebrew<br>python-build: use zlib from xcode sdk<br>Installed Python-3.12.0 to /Users/mabuaisha/.pyenv/versions/3.12.0</pre><p>You can also view all the installed Python versions as well.</p><pre>➜  ~ pyenv versions<br>  system<br>  3.9.7<br>  3.10.0<br>* 3.11.2 (set by /Users/mabuaisha/.pyenv/version)<br>  3.12.0</pre><p>Uninstalling Python is also can be done using pyenv uninstall &lt;PY_VERSION&gt;</p><p><strong>Choosing Python</strong></p><p>pyenv supports 3 different modes to select and choose which Python to use.</p><ol><li><strong>Shell</strong>: The Python version will be on the shell level. As long you on the current shell, then pyenv will look and use the same Python version. <strong>pyenv</strong> will depend on the value of $PYENV_VERSION to detect which version to use. Setting which Python to use on the shell level can be configured via pyenv shell &lt;PY_VERSION&gt; which will then set the value of $PYENV_VERSION</li><li><strong>Local</strong>: The Python version will be on the directory level. <strong>pyenv</strong> will detect which version to use depending on the .python-version file. Setting which Python to use on locally can be configured via pyenv local &lt;PY_VERSION&gt; which will then create .python-version that contains the correct Python version.</li><li><strong>Global: </strong>The Python version will be globally configured and used as the default Python version. <strong>pyenv </strong>will depend on<strong> </strong>~/.pyenv/version file in order to determine which Python to use. Setting which Pythong to use globally can be configured via pyenv gloabl &lt;PY_VERSION&gt;</li></ol><p><strong>Pyenv Shell</strong></p><pre>➜  ~ pyenv versions<br>  system<br>* 3.9.7 (set by /Users/mabuaisha/.pyenv/version)<br>  3.10.0<br>  3.11.2<br>  3.12.0<br><br>➜  ~ python<br>Python 3.9.7 (default, Aug  4 2023, 16:57:07)<br>[Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin<br>Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.<br>&gt;&gt;&gt;<br><br>➜  ~ pyenv shell 3.12.0<br>➜  ~ echo $PYENV_VERSION<br>3.12.0<br><br>➜  ~ python<br>Python 3.12.0 (main, Oct 29 2023, 11:15:12) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin<br>Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.<br>&gt;&gt;&gt;<br><br><br># The gloabl version is still set to the 3.9.7, since pyenv shell take precedence <br>➜  ~ cat .pyenv/version<br>3.9.7</pre><p><strong>Pyenv Local</strong></p><pre>➜  project-x pyenv local 3.11.2<br><br>➜  project-x ls -ltrha<br>total 8<br>-rw-r--r--@  1 mabuaisha  staff     7B Oct 29 12:12 .python-version<br><br>➜  project-x cat .python-version<br>3.11.2<br><br>➜  project-x python<br>Python 3.11.2 (main, Aug 16 2023, 11:17:37) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin<br>Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.<br>&gt;&gt;&gt;<br><br># The gloabl version is still set to the 3.9.7, since pyenv local take precedence <br>➜  ~ cat .pyenv/version<br>3.9.7</pre><p><strong>Pyenv Global</strong></p><pre>➜  ~ pyenv global 3.12.0<br><br>➜  ~ cat .pyenv/version<br>3.12.0<br><br>➜  ~ Python<br>Python 3.12.0 (main, Oct 29 2023, 11:15:12) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin<br>Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.<br>&gt;&gt;&gt;</pre><p><strong>Pyenv Virtualenv</strong></p><p>Since <strong>pyenv</strong> has support for plugins, pyenv-installer already installs some plugins and among these plugins, there is pyenv-virtualenv a plugin that helps to manage <strong>virtualenv</strong> and specify the Python version associated with it. <strong>Pyenv</strong> support creates <strong>virtualenv</strong> and also activates it.</p><pre>➜  ~ pyenv virtualenv 3.12.0 venv-3-12<br>➜  ~ pyenv activate venv-3-12<br>(venv-3-12) ➜  ~ python<br>Python 3.12.0 (main, Oct 29 2023, 11:15:12) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin<br>Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.<br>&gt;&gt;&gt;<br><br>(venv-3-12) ➜  ~ pyenv versions<br>  system<br>  3.9.7<br>  3.10.0<br>  3.11.2<br>  3.12.0<br>  3.12.0/envs/venv-3-12<br>* venv-3-12 --&gt; /Users/mabuaisha/.pyenv/versions/3.12.0/envs/venv-3-12 (set by PYENV_VERSION environment variable)<br><br><br>(venv-3-12) ➜  ~ pyenv deactivate<br>➜  ~<br><br>➜  ~ pyenv versions<br>  system<br>  3.9.7<br>  3.10.0<br>  3.11.2<br>* 3.12.0 (set by /Users/mabuaisha/.pyenv/version)<br>  3.12.0/envs/venv-3-12<br>  venv-3-12 --&gt; /Users/mabuaisha/.pyenv/versions/3.12.0/envs/venv-3-12<br><br><br>➜  ~ pyenv virtualenv-delete venv-3-12<br>pyenv-virtualenv: remove /Users/mabuaisha/.pyenv/versions/3.12.0/envs/venv-3-12? (y/N) y<br><br>➜  ~<br><br>➜  ~ pyenv versions<br>  system<br>  3.9.7<br>  3.10.0<br>  3.11.2<br>* 3.12.0 (set by /Users/mabuaisha/.pyenv/version)</pre><p>The virtualenv will be also included on the versions detected by pyenv and the activated <strong>virtualenv</strong> will linked to the specified Python version.</p><h3>Summary</h3><p>In this blog post, we discussed Pipx, which is an efficient tool for managing and installing Python-based applications such as <strong>black</strong>, <strong>isort</strong>, or any other executable Python application. Although we didn’t cover it in this series, another useful tool for installing and running Python applications is called <a href="https://github.com/pantsbuild/pex"><strong>pex</strong></a> which helps to run and install executable Python apps.</p><p>It’s worth noting that <strong>pyenv</strong> is an excellent tool for managing multiple Python versions, which can be particularly useful for testing purposes. Other tools such as <strong>conda</strong> and <strong>anaconda</strong> can also be utilized by data scientists and data engineers, and I plan on writing about them in future blog posts.</p><h3>Stackademic</h3><p><em>Thank you for reading until the end. Before you go:</em></p><ul><li><em>Please consider </em><strong><em>clapping</em></strong><em> and </em><strong><em>following</em></strong><em> the writer! 👏</em></li><li><em>Follow us on </em><a href="https://twitter.com/stackademichq"><strong><em>Twitter(X)</em></strong></a><em>, </em><a href="https://www.linkedin.com/company/stackademic"><strong><em>LinkedIn</em></strong></a><em>, and </em><a href="https://www.youtube.com/c/stackademic"><strong><em>YouTube</em></strong></a><strong><em>.</em></strong></li><li><em>Visit </em><a href="http://stackademic.com/"><strong><em>Stackademic.com</em></strong></a><em> to find out more about how we are democratizing free programming education around the world.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9d61b20465b" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/python-applications-versions-management-c9d61b20465b">Python Applications &amp; Versions Management</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Service Management Using Supervisor]]></title>
            <link>https://mabuaisha.medium.com/service-management-using-supervisor-a7b200a0ff80?source=rss-c79e314439f5------2</link>
            <guid isPermaLink="false">https://medium.com/p/a7b200a0ff80</guid>
            <dc:creator><![CDATA[Mohammed AbuAisha]]></dc:creator>
            <pubDate>Thu, 15 Oct 2020 13:50:05 GMT</pubDate>
            <atom:updated>2020-10-15T13:55:40.766Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*ktIl0twJISAqmuRxaX_iyw.jpeg" /><figcaption><strong>Supervisor Process Management</strong></figcaption></figure><p>As R&amp;D Software Engineer at <a href="https://cloudify.co/">Cloudify.co</a> I started to add support for running Cloudify services using <a href="http://supervisord.org/"><strong>Supervisor</strong></a><strong> </strong>in order to run Cloudify using non-privilege containers for better security, providing light weight containers, alternative for SystemD and be more cloud native.</p><p>In this blog I’m going to focus on the following:</p><ol><li>What is Supervisor<strong> </strong>?</li><li>Supervisor Components.</li><li>Supervisor Installation.</li><li>Controlling Program using Supervisor.</li><li>Controlling Supervisor<strong> </strong>using XML-RPC interface.</li></ol><h3>What is Supervisor ?</h3><blockquote><strong>Supervisor</strong> is a client/server system that allows its users to control a number of processes on UNIX-like operating systems.</blockquote><blockquote><a href="http://www.supervisord.org/introduction.html">http://www.supervisord.org/introduction.html</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*N7tVaGCxMcMKi2ljgxAEUQ.png" /><figcaption><strong>Supervisor Logo</strong></figcaption></figure><ol><li>Written entirely in <strong>Python</strong>.</li><li>Simpler than <a href="https://www.freebsd.org/cgi/man.cgi?query=rc&amp;sektion=8"><strong>rc</strong></a> &amp; <a href="https://systemd.io/"><strong>systemd</strong></a>.</li><li>Easy to configure using<strong> INI-style</strong> config format.</li><li>Easy to control processes using command line &amp; web interface.</li><li><strong>Supervisor</strong> invokes subprocesses using fork/exec without demonizing them.</li><li>Easy to extend and control using the exposed <strong>XML-RPC</strong> interface.</li><li>Portable across UNIX-like environments <strong>Linux</strong>, <strong>Mac OS X</strong>, <strong>Solaris</strong>, and <strong>FreeBSD</strong></li></ol><h3>Supervisor Components</h3><p>As we mentioned before, <strong>Supervisor </strong>is a client/server system where mainly has the following components:</p><ol><li><strong>supervisord</strong>: The server side that controls/manages processes</li><li><strong>supervisorctl: </strong>The client side that communicates with <strong>supervisord </strong>demon.</li><li><strong>Web Server: </strong>Simple web UI console<strong> </strong>similar to<strong> supervisorctl </strong>that can be accessed on<strong> </strong><a href="http://localhost:9001/">http://localhost:9001/</a></li><li><strong>XML-RPC Interface: </strong>XMP-RPC interface served by the same web server of the web UI console which allows to integrate <strong>supervisor </strong>with other applications.</li></ol><h3>Supervisor Installation</h3><p>In order to setup and install <strong>Supervisor</strong>, the following requirements must be met:</p><ol><li>UNIX-like Operating System (<strong>Linux</strong>, <strong>Mac OS X</strong>, <strong>Solaris, FreeBSD</strong>)</li><li><strong>Python 3</strong> (3.4 or later) or <strong>Python 2</strong> version 2.7.X</li></ol><p>One of the most common ways to download &amp; install <strong>Supervisor </strong>is to use<strong> </strong><a href="https://pip.pypa.io/en/stable/"><strong>pip</strong></a><strong> (</strong>Python Package Installer<strong>) </strong>that helps to download it from<strong> </strong><a href="https://pypi.org/"><strong>pypi</strong></a><strong> </strong>and install it on the system.</p><p>So let’s start and install <strong>Supervisor </strong>using <a href="https://pip.pypa.io/en/stable/">pip</a>.</p><ol><li>Create python <a href="https://virtualenv.pypa.io/en/latest/"><strong>virtualenv</strong></a> to install <strong>Supervisor </strong>inside it. I’m going to use <strong>venv</strong> command supported by <strong>python3</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*AKspnuT0TLVIJB6brY3IgQ.png" /></figure><p>2. Running the above command, will generate a <a href="https://virtualenv.pypa.io/en/latest/"><strong>virtualenv</strong></a><strong> </strong>on the current directory. Now its time to activate that env “<strong>supervisor-env</strong>”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*6MS41hWcPFijdQcfP8RcRw.png" /></figure><p>3. After the <a href="https://virtualenv.pypa.io/en/latest/"><strong>virtualenv</strong></a> get activated. Let’s install <strong>supervisor </strong>from <a href="https://pypi.org/"><strong>pypi</strong></a><strong> by running “pip install supervisor==4.2.1” </strong>as the latest version when writing this blog was <strong>4.2.1</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r6Eqhd3LltTZmK5nolNucQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ezb5lzftK--t59KcY897zw.png" /></figure><p>4. Validate that both <strong>supervisord &amp; supervisorctl </strong>are installed by running help commands</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*pwAYr4BI6bgiED0B29zCiw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*Qp7xa5NeW6hfZilnuUDBTA.png" /></figure><h3>Controlling Program using Supervisor</h3><p>I created a simple “Hello World” web application using <a href="https://flask.palletsprojects.com/en/1.1.x/"><strong>Flask</strong></a> framework running by <a href="https://gunicorn.org/"><strong>Gunicorn</strong></a><strong> </strong>HTTP server in order to control Web server using Supervisor.</p><p>The <a href="https://github.com/mabuaisha/supervisord-on-action">https://github.com/mabuaisha/supervisord-on-action</a> contains python source code and Supervisor configuration required to run and control the <a href="https://gunicorn.org/"><strong>Gunicorn</strong></a> process</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lZO0-c2jvR7bkhkLDNmUag.png" /></figure><p>The repository contains 4 main items:</p><ol><li><strong>simple_app</strong>: This directory contains the <a href="https://flask.palletsprojects.com/en/1.1.x/"><strong>Flask</strong></a> web app and <a href="https://wsgi.readthedocs.io/en/latest/what.html"><strong>wsgi</strong></a> module.</li><li><strong>app.conf</strong>: Represents the configuration file required to work with Supervisor and to control <a href="https://gunicorn.org/"><strong>Gunicorn</strong></a></li><li><strong>parse.py: </strong>Simple parser that communicate with<strong> </strong>supervisor over XML-RPC</li><li><strong>requirement.txt: </strong>Requirement file contains Python packages to install.</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b8323a08745955f191645d904becf2e6/href">https://medium.com/media/b8323a08745955f191645d904becf2e6/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1a4e59c1bbba5829c9da13bcad0aadc5/href">https://medium.com/media/1a4e59c1bbba5829c9da13bcad0aadc5/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/810fab0c140b26b4cf97add28096f625/href">https://medium.com/media/810fab0c140b26b4cf97add28096f625/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ca332ef9e15e24da35c35900e150c1a2/href">https://medium.com/media/ca332ef9e15e24da35c35900e150c1a2/href</a></iframe><p>The focus here will be on the <strong>Supervisor</strong> configuration file where it is written as <strong>INI-style</strong> format and organized as sections.</p><ol><li><strong>[supervisord] Section: </strong>Global settings related to the <strong>supervisord</strong> process contains configuration of logs, users, environment variables settings and much more. Full list of available settings are explained in details on <strong>s</strong><a href="http://supervisord.org/configuration.html#supervisord-section-settings"><strong>upervisord doc section</strong></a></li><li><strong>[supervisorctl] Section: </strong>The configuration related for the <strong>supervisorctl</strong> interactive shell program where it mainly contains basic configuration like (server url, username, password, ..etc) in order to communicate with <strong>supervisord </strong>process. More details about this section can be found on <a href="http://supervisord.org/configuration.html#supervisorctl-section-settings"><strong>supervisorctl doc section</strong></a></li><li><strong>[unix_http_server] Section: </strong>The<strong> </strong>configuration related for HTTP server that listens on a <a href="https://en.wikipedia.org/wiki/Unix_domain_socket"><strong>UNIX domain socket</strong></a> that contains basic configuration (file socket, username, password, ..etc). More details about this section are available on <a href="http://supervisord.org/configuration.html#unix-http-server-section-settings"><strong>Unix HTTP server doc section</strong></a></li><li><strong>[inet_http_server] Section: </strong>The<strong> </strong>configuration related for HTTP server that listens on a TCP (internet) socket that contains basic configuration like (port, username, password). More details about this section are available on <a href="http://supervisord.org/configuration.html#inet-http-server-section-settings"><strong>INET HTTP server doc section</strong></a></li><li><strong>[program:x] Section</strong>: This is configuration for the program that is going to be managed by <strong>Supervisor</strong> where <strong>X </strong>represents the name of that program where we can have multiple program sections on the configuration file. It mainly contains information related to the command need to be invoked, logs, environment variables and much more. More details about this section are available on <a href="http://supervisord.org/configuration.html#program-x-section-settings"><strong>Program doc section</strong></a><strong>.</strong></li></ol><p>There are other sections available for <strong>Supervisor </strong>which I’m not going to cover on this blog.</p><p>Now back again to our “Hello World” Flask web app. In order to use that application and integrate it with <strong>Supervisor</strong>, do the following:</p><ol><li>Clone the repository git clone <a href="https://https//github.com/mabuaisha">https://github.com/mabuaisha</a>/supervisord-on-action.git</li><li>Move <strong>supervisord-on-action</strong> to /opt</li><li>Make sure that you are using the same <strong>virtualenv</strong> we used before <strong>supervisor-env</strong></li><li>Install required python packages for the app pip install -r requirement.tx</li></ol><p>Everything is ready now to start running the <strong>supervisord </strong>process</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*_6_r_DZYt-HJ3lYtFoHmeA.png" /></figure><p>We run the main process for <strong>supervisord </strong>by specifying the location of <strong>configuration file (app.conf) </strong>and from the logs we can see that the <strong>gunicorn</strong> program is running and start listening on port 5000.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*8kpF32r8YXyi0gY9wy7hKw.png" /></figure><p>On the other hand, we can use the <strong>supervisorctl</strong> command to query about the status of all services managed by <strong>supervisord </strong>where configuration file for supervisor is specified to the command</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*_NolSw3pqdeeGaoIFMvFNQ.png" /></figure><p>We can also control services using other actions supported by the client (restart, stop, start, ..etc)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*hwSu9fXoOizCLRoQIPqBLw.png" /></figure><p>So thats good so far, let’s move to the next part and interact with <strong>supervisor </strong>over<strong> XML-RPC calls.</strong></p><h3>Controlling Supervisor<strong> </strong>using XML-RPC interface</h3><p>As part of the <strong>supervisord-on-action</strong> repository, I implemented a simple command line app which allows to communicate with <strong>supervisor</strong> to run simple actions (start/stop) services.</p><p>The snippet code illustrated below shows the built-in python packages (<strong>socket, xmlrpc, http, argparse</strong>) were used in order to accomplish this task.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/91b8094d92d0af6d55ae26dd80a800e8/href">https://medium.com/media/91b8094d92d0af6d55ae26dd80a800e8/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LKTA_eRfs8F-2xcZuY-VNw.png" /></figure><p>This would be useful in case you have an application that consists of multiple services and you want to check the overall health status of your application and then expose the result over REST API endpoint.</p><p>To Recap, the main goal of this blog is to show the usage of <strong>supervisor</strong> with your application. All the shared code, configuration and the way we installed <strong>supervisor </strong>was for demo purposes and not recommended for production usage. On the next coming blog I’m going to talk about the usage of <strong>supervisor </strong>with<strong> Non-Privilege Container </strong>and<strong> Kubernetes</strong> deployment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a7b200a0ff80" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serverless Orchestration Using Cloudify]]></title>
            <link>https://mabuaisha.medium.com/serverless-orchestration-using-cloudify-50193fdc5aa4?source=rss-c79e314439f5------2</link>
            <guid isPermaLink="false">https://medium.com/p/50193fdc5aa4</guid>
            <category><![CDATA[orchestration]]></category>
            <category><![CDATA[cloudify]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Mohammed AbuAisha]]></dc:creator>
            <pubDate>Thu, 20 Feb 2020 12:01:13 GMT</pubDate>
            <atom:updated>2021-02-22T07:29:54.931Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A_UQBdCeNbH8V0lkbI2zbw.png" /></figure><p>Serverless is a buzzword in cloud computing world where developers do not need to worry about servers management at all as they only need to focus on writing code and deploy it. <strong>Amazon</strong> introduced its Serverless service <a href="https://aws.amazon.com/lambda/"><strong>AWS Lambda</strong></a> in 2014 and then other cloud providers started introducing their Serverless services such as <a href="https://azure.microsoft.com/en-us/services/functions/"><strong>Azure Function</strong></a><strong>, </strong><a href="https://cloud.google.com/functions/"><strong>Google Functions</strong></a><strong>, </strong><a href="https://cloud.ibm.com/functions/"><strong>IBM Cloud Functions</strong></a><strong> </strong>and many others.</p><p>Every cloud provider has its own way for configuring, deploying and managing Serverless. For example AWS Lambda used <a href="https://aws.amazon.com/cloudformation/">AWS CloudFormation</a> or <a href="https://github.com/awslabs/serverless-application-model">SAM</a>, Azure Functions use <a href="https://azure.microsoft.com/en-us/resources/templates/">ARM</a> and Google Functions use <a href="https://cloud.google.com/deployment-manager">Cloud Deployment Manager</a>. However, managing Serverless functions from different providers is not an easy job especially when it comes to use CI/CD on Multi-Cloud environment. There are multiple tools/frameworks used to abstract and hide the complexity of the underlying Serverless providers using one interface and unified yaml syntax.<a href="https://serverless.com/"> <strong>Serverless Framework</strong></a> is one of popular tools which can be used to deploy functions across different cloud providers.</p><h3>Serverless Framework</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*WYRrpVXMYH7xRrRO2PMRng.png" /><figcaption>Serverless Framework</figcaption></figure><p>It’s an open source framework which allows to deploy and configure Serverless functions across multiple cloud providers. I’m going to create a simple function and deploy it to AWS Lambda using framework itself before integrating anything with Cloudify.</p><p><strong>Note:</strong> Serverless Framework is written in <strong>NodeJS</strong> which requires <strong>NodeJS</strong> to be installed on your machine. More information can be found <a href="https://serverless.com/framework/docs/providers/aws/guide/installation/">here</a></p><ol><li>Make sure that Serverless is installed on your machine.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*AlRz9V6vEYRrry7I2dLWqQ.png" /><figcaption>Serverless version</figcaption></figure><p>2. Create Python3 function on AWS called <strong>demo</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z8rM-Htvv6rPBiQSxlbWPw.png" /><figcaption>Serverless boilerplate</figcaption></figure><p>3. The above command will generate two files: <strong>serverless.yml &amp; handler.py</strong></p><p>4. We can use the generated boilerplate and deploy it to AWS.</p><p>5. Now it’s time to deploy the function to AWS but make sure that AWS credentials are set.</p><pre>export AWS_ACCESS_KEY_ID=&lt;your-key-here&gt;<br>export AWS_SECRET_ACCESS_KEY=&lt;your-secret-key-here&gt;</pre><p>6. Run serverless deploy</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PCK1uMzlMYfLHVeyXMzmxA.png" /><figcaption>Serverless Deployment</figcaption></figure><p>7. Once the function get deployed to AWS we can invoke it using serverless invole -f hello</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RwY3evEYX6RETYgol62K-w.png" /><figcaption>Serverless Function Invocation</figcaption></figure><p>8. Serverless Framework also allows us to remove all of the created resources associated with the deployed function by running serverless remove</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/1*9iZpI458UEIEgW1qbMTUHw.png" /><figcaption>Severless Function Removal</figcaption></figure><p>So all the previous steps are required in order to manage the full lifecycle of Serverless function using AWS lambada. But wait !!! someone will ask , the title of this blog is <strong>Serverless Orchestration Using Cloudify, </strong>what is cloudify ? and where is <strong>Cloudify</strong> from this ? be patient will come to this in a moment ;)</p><h3><strong>Cloudify</strong></h3><p><a href="https://docs.cloudify.co/5.0.5/"><strong>Cloudify</strong></a> is an open source cloud orchestration framework which is not limited to infrastructure provisioning but allows to model applications and services and automate their entire life cycle, including deployment on any cloud or data center environment, monitoring all aspects of a deployed application, detecting issues and failure and its based on TOSCA DSL.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*izcDfHxX3q6mO9fb7eJpwg.png" /><figcaption><strong>Cloudify</strong></figcaption></figure><p><strong>Pluggability</strong> is one of the cool features provided by Cloudify where it can be integrated with any configuration management tool, cloud providers, containers, Kubernetes and the list goes on. Cloudify is written in Python and has already a lot of official plugins that can run on top of it. I decided to build a Cloudify plugin for Serverless Framework which enables to write standard TOSCA blueprint (DSL) that describe the functions I need to deploy.</p><p><strong>Cloudify Setup</strong></p><ol><li>Install <strong>Cloudify</strong> from <a href="https://cloudify.co/download/"><strong>here</strong></a><strong>.</strong></li><li>Run Hello World App on AWS using Cloudify from <a href="https://github.com/cloudify-community/blueprint-examples/tree/master/hello-world-example"><strong>here</strong></a></li></ol><p><strong>Steps to create Plugin</strong></p><ol><li>Prepare the DSL file that describe the new node types.</li><li>Add the required Python code which mapped to the DSL node types.</li></ol><p><strong>Note</strong>: In order to create a custom plugin follow this <a href="https://docs.cloudify.co/5.0.5/developer/writing_plugins/creating-your-own-plugin/">doc</a></p><h4><strong>Cloudify Serverless Plugin</strong></h4><p>This plugin is a <strong>POC</strong> that shows how Cloudify can easily be integrated with any third party tool such as <strong>Serverless Framework</strong>.</p><p>The plugin mainly does the following:</p><ol><li>Service management.</li><li>Functions Management.</li><li>Function Invocation.</li></ol><p>This blueprint defines the following</p><ol><li>Service Configuration.</li><li>Serverless provider (AWS Lambda).</li><li>Functions associated with the service to deploy on AWS.</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/736aa0ffd1aca00a7700156077c6321f/href">https://medium.com/media/736aa0ffd1aca00a7700156077c6321f/href</a></iframe><p><strong>Note:</strong> Before deploy the above blueprint and start actual execution we need to make sure to do the following:</p><ol><li>Clone<strong> </strong><a href="https://github.com/cloudify-incubator/cloudify-serverless-plugin"><strong>cloudify-serverless-plugin</strong></a><strong> </strong>plugin (Contains example + code)</li><li>Cloudify manager has the <strong>NodeJS</strong> runtimes and <strong>Serverless Framework</strong> itself .</li><li>Upload plugin to Cloudify manager</li><li>Create <strong>Cloudify</strong> Secrets <strong>aws_access_key_id</strong> &amp; <strong>aws_secret_access_key</strong></li></ol><p><strong>Steps to deploy Serverless functions in Cloudify</strong></p><ol><li>Upload blueprint cfy blueprint upload -b aws.yaml aws-serverless</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9WGbFXYKnY25CEl8ooiCOw.png" /><figcaption>Cloudify Blueprint Uploading</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ps9wNT8H-u-rmZYQqPIPQQ.png" /><figcaption>Blueprint in Cloudify Console</figcaption></figure><p>2. Create Deployment from the uploaded blueprint.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ENTTNwC3XyOXobsIWIUSNw.png" /><figcaption>Cloudify Deployment Creation</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ue96UqykqPk82AbytiFCbQ.png" /><figcaption>Deployment in Cloudify Console</figcaption></figure><p>3. Run Install workflow to start creating resources on AWS using Serverless</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zCNwt-i-z7o-WRzPgbQDxA.png" /><figcaption>Cloudify Install Workflow</figcaption></figure><p>4. The installation result created a boilerplate serverless configuration ready for deployment and then deploy all functions as specified on the blueprint</p><p>5. The plugin also has the ability to invoke functions using <strong>execute_operation </strong>workflow</p><pre><strong>cfy executions start execute_operation -d aws-serverless -p inputs.yaml</strong></pre><pre>allow_kwargs_override: true<br>operation_kwargs:<br>  functions:<br>   - hello_1<br>   - hello_2<br>operation: &#39;serverless.interface.invoke&#39;<br>node_instance_ids:<br>  - example-aws-serverless_uionc3</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sPxAImjfnbEXhJpepSuuVw.png" /><figcaption>Cloudify Functions Invocations</figcaption></figure><p>6. Tear down all resources using cfy uninstall -b aws-serverless</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mWFAirRBJdea0tJhdWbW8Q.png" /><figcaption>Cloudify Uninstall Workflow</figcaption></figure><p>To Recap, the <strong>cloudify-serverless-plugin</strong> is just a POC showing how Cloudify can be easily used to orchestrate frameworks/tools such as Serverless using standard DSL format. The code is available on Github <a href="https://github.com/cloudify-incubator/cloudify-serverless-plugin">https://github.com/cloudify-incubator/cloudify-serverless-plugin</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50193fdc5aa4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Attending First Open Infrastructure Summit 2019]]></title>
            <link>https://mabuaisha.medium.com/attending-first-open-infrastructure-summit-2019-910600b3df1?source=rss-c79e314439f5------2</link>
            <guid isPermaLink="false">https://medium.com/p/910600b3df1</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[openstack]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Mohammed AbuAisha]]></dc:creator>
            <pubDate>Mon, 17 Jun 2019 21:56:58 GMT</pubDate>
            <atom:updated>2019-06-17T21:56:58.428Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*0BJOD_cgFqjuxYEWz-oEEQ.jpeg" /></figure><p>This is my first post at Medium and I’m so happy that finally I have time to start writing. I’m writing about my participation at <a href="https://www.openstack.org/summit/denver-2019/"><strong>Open infrastructure Summit 2019</strong></a><strong> (Formally known as OpenStack Summit) </strong>which was held at <strong>Denver, Colorado, USA.</strong></p><p>So let’s begin<strong> , </strong>I’m Mohammed AbuAisha, a Software Engineer from <strong>Palestine</strong> interested in cloud computing and DevOps with 6 years of experience in software development. In the last 3 years I have become interested more in DevOps &amp; cloud computing. Currently I’m working with a company called <a href="http://radixtechs.com/"><strong>Radix Technologies</strong></a><strong> </strong>as an integration developer building plugins(<strong>extensions</strong>) for cloud orchestration product that can be integrated with many cloud providers including: <a href="https://aws.amazon.com/"><strong>AWS</strong></a>, <strong>Az</strong><a href="https://azure.microsoft.com/en-us/"><strong>ure</strong></a>, <a href="https://cloud.google.com/"><strong>GCP</strong></a><strong> </strong>and any private cloud powered by OpenStack platform(<a href="https://www.rackspace.com/"><strong>Rackspace</strong></a>). As an integration developer, I have the chance to develop plugins for open source products like: <a href="https://kubernetes.io/"><strong>Kubernetes</strong></a><strong>, </strong><a href="https://www.openstack.org/"><strong>Openstack</strong></a><strong>, </strong><a href="https://www.ansible.com/"><strong>Ansible</strong></a><strong> </strong>and many others (AWS, GCP, Azure).</p><p>For the last 4 months I was checking the <a href="https://developer.openstack.org/api-guide/quick-start/"><strong>OpenStack API documentations </strong></a>heavily as I’m working on project that should be integrated with most of the OpenStack services. The API docs has a link directs developers to the <a href="https://www.openstack.org/"><strong>OpenStack</strong></a> foundation website which all events/summits are published there. One of the main events on the OpenStack list was <strong>Open Infrastructure summit 2019 Denver, </strong>I had the curiosity to read more about it, so I found an interesting summit to attend, and I applied for a travel support program to cover the expenses of travel and summit fees. In <strong>19/03/2019</strong> I received an email from OpenStack Foundation that they can only provide me with a free ticket to attend the summit, so I moved forward and completed the necessary procedures. My company offered a finical help to cover the travel expenses since it is related to what we are currently working on and they believed it is going to be an added value.</p><p>The Open Infrastructure summit 2019 took place at Denver from <strong>April 29, May 4 2019 </strong>and divided as the following:</p><ol><li><a href="https://www.openstack.org/summit/denver-2019/summit-schedule/events/22721/openstack-upstream-institute-sponsored-by-lenovo-day-1-rsvp-required"><strong>OpenStack Upstream Institute</strong></a><strong> : </strong>This<strong> </strong>was an intensive program designed to share knowledge about the different ways of contributing to OpenStack, unfortunately I could not make it because my flight landed at Denver at April 28 2019 after 22 hours of travel (all I need is sleep :P)</li><li><a href="https://www.openstack.org/summit/denver-2019/summit-schedule/global-search?t=Keynotes"><strong>Keynotes</strong></a>: This was the keynotes for OpenStack foundation CEO, the summit’s sponsors and the announcement for open source projects like: <a href="https://katacontainers.io/"><strong>Kata Containers</strong></a>, <a href="https://www.starlingx.io/"><strong>Starlingx</strong></a><strong>, </strong><a href="https://zuul-ci.org/docs/zuul/"><strong>CI/CD Zuul</strong></a><strong>. </strong>Moreover<strong>, </strong>keynotes about<strong> </strong>5G built on Open Infrastructure and finally announcing <a href="https://superuser.openstack.org/articles/openstack-contributor-awards-nominations/"><strong>Superuser/Community Contributor Awards</strong></a></li><li><a href="https://www.openstack.org/summit/denver-2019/summit-schedule#day=2019-04-29"><strong>Breakout Sessions</strong></a><strong>: </strong>These were 40 minutes<strong> </strong>sessions about different topics like: <strong>AI</strong>, <strong>CI/CD, Containers, Edge Computing, Hybrid Cloud, NVF (Network Virtual Function) and Private/Public Cloud</strong> which can be related to OpenStack projects<strong> </strong>or<strong> </strong>any other open source projects that can be integrated with OpenStack like <a href="https://ceph.com/"><strong>Ceph</strong></a>, <strong>Kubernetes</strong> and <a href="https://www.ansible.com/"><strong>Ansible</strong></a></li><li><a href="https://wiki.openstack.org/wiki/Forum?_ga=2.195900780.1516498167.1560795076-1218398374.1552597610"><strong>The Forum</strong></a><strong>: </strong>The<strong> </strong>OpenStack community gathered for brainstorming about the requirements for the next release, get feedback on the past version</li><li><a href="https://www.openstack.org/summit/denver-2019/summit-schedule/#day=2019-04-29"><strong>WorkShops &amp; Training</strong></a><strong>: </strong>These were practical sessions and training for how to use some of the OpenStack projects and any other tools</li><li><a href="https://www.openstack.org/summit/denver-2019/summit-schedule/events/23763/open-infrastructure-marketplace-op"><strong>Open Infrastructure Marketplace:</strong></a> The marketplace was the place where all companies in the summit were gathered to offer services they provide, how they are using/integrating OpenStack projects and it was a place to meet developers from all over the world.</li><li><a href="https://www.openstack.org/ptg"><strong>Project Team Gathering (PTG):</strong></a><strong> </strong>It was an event that<strong> </strong>allowed the various technical community groups working on OSF projects to meet in-person, exchange and get work done in a productive setting. (I did not have a chance to attend this event since I had a valid ticket to attend only the 3 days summit <strong>April 29, May 1 2019</strong>).</li></ol><p>A good percentage of the sessions/trainings were related to topics like <strong>CI/CD</strong>, <strong>Kubernetes</strong>, <strong>Ansible</strong>, <strong>Service Mesh &amp; Microservices </strong>which gave an indication how important these topics were and gave me a good reason to attend most of the sessions which were related to these topics. Moreover, one of the sessions mentioned how OpenStack platform can be deployed to production using different methods and tools as the following:</p><ol><li><a href="https://docs.openstack.org/install-guide/"><strong>Manual Installation</strong></a></li><li><a href="https://docs.openstack.org/devstack/latest/"><strong>DevStack</strong></a></li><li><a href="https://docs.openstack.org/openstack-ansible/latest/"><strong>Openstack Ansible</strong></a></li><li><a href="https://docs.openstack.org/kolla/latest/"><strong>Kolla &amp; Kolla-Ansible</strong></a></li><li><a href="https://docs.openstack.org/openstack-helm/latest/readme.html"><strong>OpenStack Helm</strong></a></li><li><a href="https://docs.openstack.org/tripleo-docs/latest/"><strong>Triple-O</strong></a></li><li><a href="https://kayobe.readthedocs.io/en/latest"><strong>Kayobe</strong></a></li><li><a href="https://www.airshipit.org/"><strong>Airship</strong></a></li></ol><p><strong>Ansible </strong>project<strong> </strong>which is used to automate apps and IT infrastructure had a high demands among the developers to attend its sessions/training during the summit and they were all full. <strong>On the other hand</strong>, there were some projects developed/maintained under OpenStack foundation that I had never heard of them before which had sessions and presented in different places and it is worth to spend some time to review, such as:</p><ol><li><a href="https://katacontainers.io/"><strong>Kata Containers</strong></a></li><li><a href="https://www.airshipit.org/"><strong>AirShip</strong></a></li><li><a href="https://www.starlingx.io/"><strong>Starlingx</strong></a></li><li><a href="https://zuul-ci.org/docs/zuul/"><strong>Zuul</strong></a></li><li><a href="https://ceph.com/"><strong>Ceph</strong></a></li><li><a href="https://wiki.openstack.org/wiki/Ironic"><strong>Ironic Openstack</strong></a></li><li><a href="https://docs.openstack.org/tripleo-docs/latest/"><strong>Triple-O</strong></a></li></ol><p>Finally, I can say that attending the summit was a great experience for me and added a-lot by focusing more on the topics which were interesting to the most of the developers and companies from all over the world. If everything went well I’m planning to register and attend the next summit which will take place in <a href="https://www.openstack.org/summit/shanghai-2019/"><strong>Shanghai, China 2019</strong></a> next November.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=910600b3df1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>