This opinionated guide exists to provide both novice and expert Python developers a best practice handbook to the installation, configuration, and usage of Python on a daily basis.
-
-
-
Stay Informed
-
Receive updates on new releases and upcoming projects.
This opinionated guide exists to provide both novice and expert Python developers a best practice handbook to the installation, configuration, and usage of Python on a daily basis.
-
-
Get Updates
-
Receive updates on new releases and upcoming projects.
+
diff --git a/docs/conf.py b/docs/conf.py
index b9e1c63a7..d732ac29f 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -11,7 +11,9 @@
# All configuration values have a default; values that are commented out
# serve to show the default.
-import sys, os
+import datetime
+import os
+import sys
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
@@ -45,8 +47,11 @@
master_doc = 'index'
# General information about the project.
+current_year = datetime.datetime.now().year
project = u'pythonguide'
-copyright = u'2016. A Kenneth Reitz Project. CC BY-NC-SA 3.0'
+copyright = (u'2011-{} Kenneth Reitz'
+ ' & Real Python.'
+ ' CC BY-NC-SA 3.0').format(current_year)
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
@@ -106,10 +111,11 @@
# documentation.
html_theme_options = {
'show_powered_by': False,
- 'github_user': 'kennethreitz',
+ 'github_user': 'realpython',
'github_repo': 'python-guide',
'github_banner': True,
- 'show_related': False
+ 'show_related': False,
+ 'note_bg': '#FFF59C',
}
# Add any paths that contain custom themes here, relative to this directory.
@@ -240,7 +246,7 @@
epub_title = u'pythonguide'
epub_author = u'Kenneth Reitz'
epub_publisher = u'Kenneth Reitz'
-epub_copyright = u'2016, Kenneth Reitz'
+epub_copyright = u'2011–{}, Kenneth Reitz & Real Python'.format(current_year)
# The language of the text. It defaults to the language option
# or en if the language is not set.
@@ -278,5 +284,5 @@
todo_include_todos = True
intersphinx_mapping = {
- 'python': ('http://docs.python.org/', None),
+ 'python': ('https://docs.python.org/3', None),
}
diff --git a/docs/contents.rst.inc b/docs/contents.rst.inc
index c2c603dec..29b076197 100644
--- a/docs/contents.rst.inc
+++ b/docs/contents.rst.inc
@@ -1,24 +1,50 @@
Getting Started with Python
---------------------------
-New to Python? Let's properly setup up your Python environment.
+New to Python? Let's properly setup up your Python environment:
.. toctree::
:maxdepth: 2
starting/which-python
-- Properly Install Python
+- Properly Install Python on your system:
.. toctree::
:maxdepth: 1
starting/installation
+ starting/install3/osx
+ starting/install3/win
+ starting/install3/linux
starting/install/osx
starting/install/win
starting/install/linux
+- Using Virtualenvs with Pipenv:
+
+ .. toctree::
+ :maxdepth: 2
+
+ dev/virtualenvs
+
+
+Python Development Environments
+-------------------------------
+
+This part of the guide focuses on the Python development environment,
+and the best-practice tools that are available for writing Python code.
+
+.. toctree::
+ :maxdepth: 2
+
+ dev/env
+ dev/virtualenvs
+ dev/pip-virtualenv
+
+
+
Writing Great Python Code
-------------------------
@@ -64,42 +90,28 @@ different scenarios.
scenarios/xml
scenarios/json
scenarios/crypto
+ scenarios/ml
scenarios/clibs
Shipping Great Python Code
--------------------------
-This part of the guide focuses on deploying your Python code.
+This part of the guide focuses on sharing and deploying your Python code.
.. toctree::
:maxdepth: 2
+ shipping/publishing
shipping/packaging
shipping/freezing
-Python Development Environments
--------------------------------
-
-This part of the guide focus on the Python development environment,
-and the best-practice tools that are available for writing Python code.
-
-.. toctree::
- :maxdepth: 2
-
- dev/env
- dev/virtualenvs
- dev/pip-virtualenv
-
-
-
-
Additional Notes
----------------
This part of the guide, which is mostly prose, begins with some
-background information about Python, then focuses on next steps.
+background information about Python, and then focuses on next steps.
.. toctree::
:maxdepth: 2
@@ -126,8 +138,3 @@ Contribution notes and legal information (for those interested).
notes/contribute
notes/license
notes/styleguide
-
-
-
-
-
diff --git a/docs/dev/env.rst b/docs/dev/env.rst
index d5870274a..7da63c0e6 100644
--- a/docs/dev/env.rst
+++ b/docs/dev/env.rst
@@ -1,11 +1,13 @@
Your Development Environment
============================
+.. image:: /_static/photos/33175624924_7febc46cc4_k_d.jpg
+
Text Editors
::::::::::::
-Just about anything that can edit plain text will work for writing Python code,
+Just about anything that can edit plain text will work for writing Python code;
however, using a more powerful editor may make your life a bit easier.
@@ -35,11 +37,11 @@ source files.
There is also a handy syntax plugin called syntax_ featuring some improvements
over the syntax file included in Vim 6.1.
-These plugins supply you with a basic environment for developing in Python.
-To get the most out of Vim, you should continually check your code for syntax
-errors and PEP8 compliance. Luckily PEP8_ and Pyflakes_ will do this for you.
-If your Vim is compiled with :option:`+python` you can also utilize some very
-handy plugins to do these checks from within the editor.
+These plugins supply you with a basic environment for developing in Python. To
+get the most out of Vim, you should continually check your code for syntax
+errors and PEP8 compliance. Luckily pycodestyle_ and Pyflakes_ will do this
+for you. If your Vim is compiled with ``+python`` you can also utilize some
+very handy plugins to do these checks from within the editor.
For PEP8 checking and pyflakes, you can install vim-flake8_. Now you can map the
function ``Flake8`` to any hotkey or action you want in Vim. The plugin will
@@ -68,12 +70,12 @@ Python-mode
Python-mode_ is a complex solution for working with Python code in Vim.
It has:
-- Asynchronous Python code checking (``pylint``, ``pyflakes``, ``pep8``, ``mccabe``) in any combination
+- Asynchronous Python code checking (``pylint``, ``pyflakes``, ``pycodestyle``, ``mccabe``) in any combination
- Code refactoring and autocompletion with Rope
- Fast Python folding
- Virtualenv support
- Search through Python documentation and run Python code
-- Auto PEP8_ error fixes
+- Auto pycodestyle_ error fixes
And more.
@@ -85,29 +87,29 @@ using ```` key or any other customized keys.
.. _indent: http://www.vim.org/scripts/script.php?script_id=974
.. _syntax: http://www.vim.org/scripts/script.php?script_id=790
-.. _Pyflakes: http://pypi.python.org/pypi/pyflakes/
-.. _PEP8: http://pypi.python.org/pypi/pep8/
-.. _syntastic: https://github.com/scrooloose/syntastic
-.. _Python-mode: https://github.com/klen/python-mode
+.. _Pyflakes: http://pypi.org/project/pyflakes/
+.. _pycodestyle: https://pypi.org/project/pycodestyle/
+.. _syntastic: https://github.com/vim-syntastic/syntastic
+.. _Python-mode: https://github.com/python-mode/python-mode
.. _SuperTab: http://www.vim.org/scripts/script.php?script_id=1643
.. _vim-flake8: https://github.com/nvie/vim-flake8
Emacs
-----
-Emacs is another powerful text editor. It is fully programmable (lisp), but
+Emacs is another powerful text editor. It is fully programmable (Lisp), but
it can be some work to wire up correctly. A good start if you're already an
Emacs user is `Python Programming in Emacs`_ at EmacsWiki.
1. Emacs itself comes with a Python mode.
-.. _Python Programming in Emacs: http://emacswiki.org/emacs/PythonProgrammingInEmacs
+.. _Python Programming in Emacs: https://www.emacswiki.org/emacs/PythonProgrammingInEmacs
TextMate
--------
`TextMate `_ brings Apple's approach to operating
- systems into the world of text editors. By bridging UNIX underpinnings and
+ systems into the world of text editors. By bridging Unix underpinnings and
GUI, TextMate cherry-picks the best of both worlds to the benefit of expert
scripters and novice users alike.
@@ -115,8 +117,8 @@ Sublime Text
------------
`Sublime Text `_ is a sophisticated text
- editor for code, markup and prose. You'll love the slick user interface,
- extraordinary features and amazing performance.
+ editor for code, markup, and prose. You'll love the slick user interface,
+ extraordinary features, and amazing performance.
Sublime Text has excellent support for editing Python code and uses Python for
its plugin API. It also has a diverse variety of plugins,
@@ -131,11 +133,19 @@ Atom
editors.
Atom is web native (HTML, CSS, JS), focusing on modular design and easy plugin
-development. It comes with native package control and plethora of packages.
+development. It comes with native package control and a plethora of packages.
Recommended for Python development is
-`Linter `_ combined with
+`Linter `_ combined with
`linter-flake8 `_.
+Python (on Visual Studio Code)
+------------------------------
+
+`Python for Visual Studio `_ is an extension for the `Visual Studio Code `_.
+This is a free, lightweight, open source code editor, with support for Mac, Windows, and Linux.
+Built using open source technologies such as Node.js and Python, with compelling features such as Intellisense (autocompletion), local and remote debugging, linting, and the like.
+
+MIT licensed.
IDEs
::::
@@ -148,34 +158,26 @@ known for IntelliJ IDEA. Both share the same code base and most of PyCharm's
features can be brought to IntelliJ with the free
`Python Plug-In `_. There are two
versions of PyCharm: Professional Edition (Free 30-day trial) and Community
-Edition(Apache 2.0 License) with fewer features.
+Edition (Apache 2.0 License) with fewer features.
-Python (on Visual Studio Code)
------------------------------
-
-`Python for Visual Studio `_ is an extension for the `Visual Studio Code IDE `_.
-This is a free, light weight, open source IDE, with support for Mac, Windows, and Linux.
-Built using open source technologies such as Node.js and Python, with compelling features such as Intellisense (autocompletion), local and remote debugging, linting, and the like.
-
-MIT licensed.
Enthought Canopy
----------------
-`Enthought Canopy `_ is a Python
-IDE which is focused towards Scientists and Engineers as it provides pre
-installed libraries for data analysis.
+`Enthought Canopy `_ is a Python
+IDE which is focused towards Scientists and Engineers as it provides pre
+installed libraries for data analysis.
Eclipse
-------
The most popular Eclipse plugin for Python development is Aptana's
-`PyDev `_.
+`PyDev `_.
Komodo IDE
----------
-`Komodo IDE `_ is developed by
+`Komodo IDE `_ is developed by
ActiveState and is a commercial IDE for Windows, Mac, and Linux.
`KomodoEdit `_ is the open source
alternative.
@@ -186,11 +188,11 @@ Spyder
`Spyder `_ is an IDE specifically geared
toward working with scientific Python libraries (namely
-`Scipy `_). It includes integration with pyflakes_,
-`pylint `_ and
+`SciPy `_). It includes integration with pyflakes_,
+`pylint `_ and
`rope `_.
-Spyder is open-source (free), offers code completion, syntax highlighting,
+Spyder is open source (free), offers code completion, syntax highlighting,
a class and function browser, and object inspection.
@@ -198,7 +200,7 @@ WingIDE
-------
`WingIDE `_ is a Python specific IDE. It runs on Linux,
-Windows and Mac (as an X11 application, which frustrates some Mac users).
+Windows, and Mac (as an X11 application, which frustrates some Mac users).
WingIDE offers code completion, syntax highlighting, source browser, graphical
debugger and support for version control systems.
@@ -209,11 +211,11 @@ NINJA-IDE
`NINJA-IDE `_ (from the recursive acronym: "Ninja-IDE
Is Not Just Another IDE") is a cross-platform IDE, specially designed to build
-Python applications, and runs on Linux/X11, Mac OS X and Windows desktop
+Python applications, and runs on Linux/X11, Mac OS X, and Windows desktop
operating systems. Installers for these platforms can be downloaded from the
website.
-NINJA-IDE is open-source software (GPLv3 licence) and is developed
+NINJA-IDE is open source software (GPLv3 licence) and is developed
in Python and Qt. The source files can be downloaded from
`GitHub `_.
@@ -222,13 +224,25 @@ Eric (The Eric Python IDE)
--------------------------
`Eric `_ is a full featured Python IDE
-offering sourcecode autocompletion, syntax highlighting, support for version
-control systems, python 3 support, integrated web browser, python shell,
-integrated debugger and a flexible plug-in system. Written in python, it is
-based on the Qt gui toolkit, integrating the Scintilla editor control. Eric
-is an open-source software project (GPLv3 licence) with more than ten years of
+offering source code autocompletion, syntax highlighting, support for version
+control systems, Python 3 support, integrated web browser, python shell,
+integrated debugger, and a flexible plug-in system. Written in Python, it is
+based on the Qt GUI toolkit, integrating the Scintilla editor control. Eric
+is an open source software project (GPLv3 licence) with more than ten years of
active development.
+Mu
+--
+
+`Mu `_ is a minimalist Python IDE which can run Python 3 code
+locally and can also deploy code to the BBC micro:bit and to Adafruit boards running
+CircuitPython.
+
+Intended for beginners, mu includes a Python 3 interpreter, and is easy to install
+on Windows, OS/X and Linux. It runs well on the Raspberry Pi.
+
+There's an active support community on gitter.
+
Interpreter Tools
:::::::::::::::::
@@ -240,19 +254,19 @@ Virtual Environments
Virtual Environments provide a powerful way to isolate project package dependencies. This means that you can use packages particular to a Python project without installing them system wide and thus avoiding potential version conflicts.
To start using and see more information:
-`Virtual Environments `_ docs.
+`Virtual Environments `_ docs.
pyenv
-----
-`pyenv `_ is a tool to allow multiple versions
+`pyenv `_ is a tool to allow multiple versions
of the Python interpreter to be installed at the same time. This solves the
problem of having different projects requiring different versions of Python.
For example, it becomes very easy to install Python 2.7 for compatibility in
-one project, whilst still using Python 3.4 as the default interpreter.
-pyenv isn't just limited to the CPython versions - it will also install PyPy,
-anaconda, miniconda, stackless, jython, and ironpython interpreters.
+one project, while still using Python 3.4 as the default interpreter.
+pyenv isn't just limited to the CPython versions – it will also install PyPy,
+Anaconda, miniconda, stackless, Jython, and IronPython interpreters.
pyenv works by filling a ``shims`` directory with fake versions of the Python
interpreter (plus other tools like ``pip`` and ``2to3``). When the system
@@ -262,7 +276,7 @@ pyenv. pyenv then works out which version of Python should be run based on
environment variables, ``.python-version`` files, and the global default.
pyenv isn't a tool for managing virtual environments, but there is the plugin
-`pyenv-virtualenv `_ which automates
+`pyenv-virtualenv `_ which automates
the creation of different environments, and also makes it possible to use the
existing pyenv tools to switch to different environments based on environment
variables or ``.python-version`` files.
@@ -274,7 +288,7 @@ IDLE
----
:ref:`IDLE ` is an integrated development environment that is
-part of Python standard library. It is completely written in Python and uses
+part of the Python standard distribution. It is completely written in Python and uses
the Tkinter GUI toolkit. Though IDLE is not suited for full-blown development
using Python, it is quite helpful to try out small Python snippets and
experiment with different features in Python.
@@ -292,18 +306,18 @@ IPython
`IPython `_ provides a rich toolkit to help you make the
most out of using Python interactively. Its main components are:
-* Powerful Python shells (terminal- and Qt-based).
+* Powerful Python shells (terminal- and Qt-based)
* A web-based notebook with the same core features but support for rich media,
- text, code, mathematical expressions and inline plots.
-* Support for interactive data visualization and use of GUI toolkits.
-* Flexible, embeddable interpreters to load into your own projects.
-* Tools for high level and interactive parallel computing.
+ text, code, mathematical expressions and inline plots
+* Support for interactive data visualization and use of GUI toolkits
+* Flexible, embeddable interpreters to load into your own projects
+* Tools for high level and interactive parallel computing
.. code-block:: console
$ pip install ipython
-To download and install IPython with all it's optional dependencies for the notebook, qtconsole, tests, and other functionalities
+To download and install IPython with all its optional dependencies for the notebook, qtconsole, tests, and other functionalities:
.. code-block:: console
@@ -312,18 +326,18 @@ To download and install IPython with all it's optional dependencies for the note
BPython
-------
-`bpython `_ is an alternative interface to the
+`bpython `_ is an alternative interface to the
Python interpreter for Unix-like operating systems. It has the following
features:
-* In-line syntax highlighting.
-* Readline-like autocomplete with suggestions displayed as you type.
-* Expected parameter list for any Python function.
-* "Rewind" function to pop the last line of code from memory and re-evaluate.
-* Send entered code off to a pastebin.
-* Save entered code to a file.
-* Auto-indentation.
-* Python 3 support.
+* In-line syntax highlighting
+* Readline-like autocomplete with suggestions displayed as you type
+* Expected parameter list for any Python function
+* "Rewind" function to pop the last line of code from memory and re-evaluate
+* Send entered code off to a pastebin
+* Save entered code to a file
+* Auto-indentation
+* Python 3 support
.. code-block:: console
@@ -332,19 +346,19 @@ features:
ptpython
--------
-`ptpython `_ is a REPL build
-on top of the `prompt_toolkit `_
+`ptpython `_ is a REPL build
+on top of the `prompt_toolkit `_
library. It is considered to be an alternative to BPython_. Features include:
* Syntax highlighting
* Autocompletion
* Multiline editing
-* Emacs and VIM Mode
+* Emacs and Vim Modes
* Embedding REPL inside of your code
-* Syntax Validation
+* Syntax validation
* Tab pages
* Support for integrating with IPython_'s shell, by installing IPython
- ``pip install ipython`` and running ``ptipython``.
+ (``pip install ipython``) and running ``ptipython``.
.. code-block:: console
diff --git a/docs/dev/pip-virtualenv.rst b/docs/dev/pip-virtualenv.rst
index 51927e689..667f30c7a 100644
--- a/docs/dev/pip-virtualenv.rst
+++ b/docs/dev/pip-virtualenv.rst
@@ -1,8 +1,10 @@
.. _pip-virtualenv:
-Further Configuration of Pip and Virtualenv
+Further Configuration of pip and Virtualenv
===========================================
+.. image:: /_static/photos/34018732105_f0e6758859_k_d.jpg
+
Requiring an active virtual environment for ``pip``
---------------------------------------------------
@@ -18,8 +20,8 @@ environment of the project. Over time this can result in a messy global package
list.
In order to make sure that you install packages to your active virtual
-environment when you use ``pip install``, consider adding the following two
-lines to your :file:`~/.bashrc` file:
+environment when you use ``pip install``, consider adding the following
+line to your :file:`~/.bashrc` file:
.. code-block:: console
@@ -49,7 +51,7 @@ can be found at:
.. code-block:: console
- %HOME%\pip\pip.ini
+ %USERPROFILE%\pip\pip.ini
If you don't have a :file:`pip.conf` or :file:`pip.ini` file at these locations,
you can create a new file with the correct name for your operating system.
@@ -77,7 +79,7 @@ adding the following to your :file:`~/.bashrc` file:
.. code-block:: console
gpip() {
- PIP_REQUIRE_VIRTUALENV="" pip "$@"
+ PIP_REQUIRE_VIRTUALENV=false pip "$@"
}
After saving the changes and sourcing your :file:`~/.bashrc` file you can now
@@ -94,11 +96,16 @@ that you use. For example, you may be using the ``requests`` library in a lot
of different projects.
It is surely unnecessary to re-download the same packages/libraries each time
-you start working on a new project (and in a new virtual environment as a result).
-Fortunately, you can configure pip in such a way that it tries to reuse already
-installed packages.
+you start working on a new project (and in a new virtual environment as a
+result). Fortunately, starting with version 6.0, pip provides an `on-by-default
+caching mechanism
+`_ that doesn't
+need any configuration.
+
+When using older versions, you can configure pip in such a way that it tries to
+reuse already installed packages, too.
-On UNIX systems, you can add the following line to your :file:`.bashrc` or
+On Unix systems, you can add the following line to your :file:`.bashrc` or
:file:`.bash_profile` file.
.. code-block:: console
@@ -115,9 +122,9 @@ add the following line to your :file:`pip.ini` file under ``[global]`` settings:
.. code-block:: console
- download-cache = %HOME%\pip\cache
+ download-cache = %USERPROFILE%\pip\cache
-Similarly, on UNIX systems you should simply add the following line to your
+Similarly, on Unix systems you should simply add the following line to your
:file:`pip.conf` file under ``[global]`` settings:
.. code-block:: console
diff --git a/docs/dev/virtualenvs.rst b/docs/dev/virtualenvs.rst
index bb3f9487c..fe6bd3f9e 100644
--- a/docs/dev/virtualenvs.rst
+++ b/docs/dev/virtualenvs.rst
@@ -1,22 +1,210 @@
.. _virtualenvironments-ref:
-Virtual Environments
-====================
+Pipenv & Virtual Environments
+=============================
-A Virtual Environment is a tool to keep the dependencies required by different
-projects in separate places, by creating virtual Python environments for them.
-It solves the "Project X depends on version 1.x but, Project Y needs 4.x"
-dilemma, and keeps your global site-packages directory clean and manageable.
+.. image:: /_static/photos/35294660055_42c02b2316_k_d.jpg
-For example, you can work on a project which requires Django 1.3 while also
-maintaining a project which requires Django 1.0.
+This tutorial walks you through installing and using Python packages.
-virtualenv
+It will show you how to install and use the necessary tools and make strong
+recommendations on best practices. Keep in mind that Python is used for a great
+many different purposes, and precisely how you want to manage your dependencies
+may change based on how you decide to publish your software. The guidance
+presented here is most directly applicable to the development and deployment of
+network services (including web applications), but is also very well suited to
+managing development and testing environments for any kind of project.
+
+.. Note:: This guide is written for Python 3, however, these instructions
+ should work fine on Python 2.7—if you are still using it, for some reason.
+
+
+Make sure you've got Python & pip
+---------------------------------
+
+Before you go any further, make sure you have Python and that it's available
+from your command line. You can check this by simply running:
+
+.. code-block:: console
+
+ $ python --version
+
+You should get some output like ``3.6.2``. If you do not have Python, please
+install the latest 3.x version from `python.org`_ or refer to the
+`Installing Python`_ section of this guide.
+
+.. Note:: If you're newcomer and you get an error like this:
+
+ .. code-block:: python
+
+ >>> python
+ Traceback (most recent call last):
+ File "", line 1, in
+ NameError: name 'python' is not defined
+
+ It's because this command is intended to be run in a *shell* (also called
+ a *terminal* or *console*). See the Python for Beginners
+ `getting started tutorial`_ for an introduction to using your operating
+ system's shell and interacting with Python.
+
+Additionally, you'll need to make sure you have `pip`_ available. You can
+check this by running:
+
+.. code-block:: console
+
+ $ pip --version
+
+If you installed Python from source, with an installer from `python.org`_, or
+via `Homebrew`_ you should already have pip. If you're on Linux and installed
+using your OS package manager, you may have to `install pip `_ separately.
+
+.. _getting started tutorial: https://opentechschool.github.io/python-beginners/en/getting_started.html#what-is-python-exactly
+.. _python.org: https://python.org
+.. _pip: https://pypi.org/project/pip/
+.. _Homebrew: https://brew.sh
+.. _Installing Python: https://docs.python-guide.org/starting/installation/
+
+
+Installing Pipenv
+-----------------
+
+`Pipenv`_ is a dependency manager for Python projects. If you're familiar
+with Node.js' `npm`_ or Ruby's `bundler`_, it is similar in spirit to those
+tools. While `pip`_ can install Python packages, Pipenv is recommended as
+it's a higher-level tool that simplifies dependency management for common use
+cases.
+
+Use ``pip`` to install Pipenv:
+
+.. code-block:: console
+
+ $ pip install --user pipenv
+
+
+.. Note:: This does a `user installation`_ to prevent breaking any system-wide
+ packages. If ``pipenv`` isn't available in your shell after installation,
+ you'll need to add the `user base`_'s binary directory to your ``PATH``.
+
+ On Linux and macOS you can find the user base binary directory by running
+ ``python -m site --user-base`` and adding ``bin`` to the end. For example,
+ this will typically print ``~/.local`` (with ``~`` expanded to the
+ absolute path to your home directory) so you'll need to add
+ ``~/.local/bin`` to your ``PATH``. You can set your ``PATH`` permanently by
+ `modifying ~/.profile`_.
+
+ On Windows you can find the user base binary directory by running
+ ``py -m site --user-site`` and replacing ``site-packages`` with
+ ``Scripts``. For example, this could return
+ ``C:\Users\Username\AppData\Roaming\Python36\site-packages`` so you would
+ need to set your ``PATH`` to include
+ ``C:\Users\Username\AppData\Roaming\Python36\Scripts``. You can set your
+ user ``PATH`` permanently in the `Control Panel`_. You may need to log
+ out for the ``PATH`` changes to take effect.
+
+.. _Pipenv: https://pipenv.kennethreitz.org/
+.. _npm: https://www.npmjs.com/
+.. _bundler: http://bundler.io/
+.. _user base: https://docs.python.org/3/library/site.html#site.USER_BASE
+.. _user installation: https://pip.pypa.io/en/stable/user_guide/#user-installs
+.. _modifying ~/.profile: https://stackoverflow.com/a/14638025
+.. _Control Panel: https://msdn.microsoft.com/en-us/library/windows/desktop/bb776899(v=vs.85).aspx
+
+Installing packages for your project
+------------------------------------
+
+Pipenv manages dependencies on a per-project basis. To install packages,
+change into your project's directory (or just an empty directory for this
+tutorial) and run:
+
+.. code-block:: console
+
+ $ cd project_folder
+ $ pipenv install requests
+
+Pipenv will install the excellent `Requests`_ library and create a ``Pipfile``
+for you in your project's directory. The `Pipfile`_ is used to track which
+dependencies your project needs in case you need to re-install them, such as
+when you share your project with others. You should get output similar to this
+(although the exact paths shown will vary):
+
+.. _Pipfile: https://github.com/pypa/pipfile
+
+.. code-block:: text
+
+ Creating a Pipfile for this project...
+ Creating a virtualenv for this project...
+ Using base prefix '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6'
+ New python executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python3.6
+ Also creating executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python
+ Installing setuptools, pip, wheel...done.
+
+ Virtualenv location: ~/.local/share/virtualenvs/tmp-agwWamBd
+ Installing requests...
+ Collecting requests
+ Using cached requests-2.18.4-py2.py3-none-any.whl
+ Collecting idna<2.7,>=2.5 (from requests)
+ Using cached idna-2.6-py2.py3-none-any.whl
+ Collecting urllib3<1.23,>=1.21.1 (from requests)
+ Using cached urllib3-1.22-py2.py3-none-any.whl
+ Collecting chardet<3.1.0,>=3.0.2 (from requests)
+ Using cached chardet-3.0.4-py2.py3-none-any.whl
+ Collecting certifi>=2017.4.17 (from requests)
+ Using cached certifi-2017.7.27.1-py2.py3-none-any.whl
+ Installing collected packages: idna, urllib3, chardet, certifi, requests
+ Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22
+
+ Adding requests to Pipfile's [packages]...
+ P.S. You have excellent taste! ✨ 🍰 ✨
+
+.. _Requests: https://requests.readthedocs.io/en/latest/
+
+
+Using installed packages
+------------------------
+
+Now that Requests is installed you can create a simple ``main.py`` file to
+use it:
+
+.. code-block:: python
+
+ import requests
+
+ response = requests.get('https://httpbin.org/ip')
+
+ print('Your IP is {0}'.format(response.json()['origin']))
+
+Then you can run this script using ``pipenv run``:
+
+.. code-block:: console
+
+ $ pipenv run python main.py
+
+You should get output similar to this:
+
+.. code-block:: text
+
+ Your IP is 8.8.8.8
+
+Using ``$ pipenv run`` ensures that your installed packages are available to
+your script. It's also possible to spawn a new shell that ensures all commands
+have access to your installed packages with ``$ pipenv shell``.
+
+
+Next steps
----------
-`virtualenv `_ is a tool to create
-isolated Python environments. virtualenv creates a folder which contains all the
-necessary executables to use the packages that a Python project would need.
+Congratulations, you now know how to install and use Python packages! ✨ 🍰 ✨
+
+
+
+Lower level: virtualenv
+=======================
+
+`virtualenv `_ is a tool to create
+isolated Python environments. virtualenv creates a folder which contains all the
+necessary executables to use the packages that a Python project would need.
+
+It can be used standalone, in place of Pipenv.
Install virtualenv via pip:
@@ -24,14 +212,20 @@ Install virtualenv via pip:
$ pip install virtualenv
+Test your installation:
+
+.. code-block:: console
+
+ $ virtualenv --version
+
Basic Usage
-~~~~~~~~~~~
+-----------
1. Create a virtual environment for a project:
.. code-block:: console
- $ cd my_project_folder
+ $ cd project_folder
$ virtualenv venv
``virtualenv venv`` will create a folder in the current directory which will
@@ -40,16 +234,24 @@ can use to install other packages. The name of the virtual environment (in this
case, it was ``venv``) can be anything; omitting the name will place the files
in the current directory instead.
+.. note::
+ 'venv' is the general convention used globally. As it is readily available in ignore files (eg: .gitignore')
+
This creates a copy of Python in whichever directory you ran the command in,
placing it in a folder named :file:`venv`.
-You can also use a Python interpreter of your choice.
+You can also use the Python interpreter of your choice (like
+``python2.7``).
.. code-block:: console
$ virtualenv -p /usr/bin/python2.7 venv
-This will use the Python interpreter in :file:`/usr/bin/python2.7`
+or change the interpreter globally with an env variable in ``~/.bashrc``:
+
+.. code-block:: console
+
+ $ export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python2.7
2. To begin using the virtual environment, it needs to be activated:
@@ -57,12 +259,20 @@ This will use the Python interpreter in :file:`/usr/bin/python2.7`
$ source venv/bin/activate
-The name of the current virtual environment will now appear on the left of
-the prompt (e.g. ``(venv)Your-Computer:your_project UserName$)`` to let you know
-that it's active. From now on, any package that you install using pip will be
+The name of the current virtual environment will now appear on the left of
+the prompt (e.g. ``(venv)Your-Computer:project_folder UserName$``) to let you know
+that it's active. From now on, any package that you install using pip will be
placed in the ``venv`` folder, isolated from the global Python installation.
-Install packages as usual, for example:
+For Windows, the same command mentioned in step 1 can be used to create a virtual environment. However, activating the environment requires a slightly different command.
+
+Assuming that you are in your project directory:
+
+.. code-block:: console
+
+ C:\Users\SomeUser\project_folder> venv\Scripts\activate
+
+Install packages using the ``pip`` command:
.. code-block:: console
@@ -73,28 +283,31 @@ Install packages as usual, for example:
.. code-block:: console
- $ deactivate
+ $ deactivate
This puts you back to the system's default Python interpreter with all its
installed libraries.
-To delete a virtual environment, just delete its folder. (In this case,
+To delete a virtual environment, just delete its folder. (In this case,
it would be ``rm -rf venv``.)
After a while, though, you might end up with a lot of virtual environments
-littered across your system, and its possible you'll forget their names or
+littered across your system, and it's possible you'll forget their names or
where they were placed.
+.. note::
+ Python has included venv module from version 3.3. For more details: `venv `_.
+
Other Notes
-~~~~~~~~~~~
+-----------
-Running ``virtualenv`` with the option :option:`--no-site-packages` will not
+Running ``virtualenv`` with the option ``--no-site-packages`` will not
include the packages that are installed globally. This can be useful
for keeping the package list clean in case it needs to be accessed later.
[This is the default behavior for ``virtualenv`` 1.7 and later.]
In order to keep your environment consistent, it's a good idea to "freeze"
-the current state of the environment packages. To do this, run
+the current state of the environment packages. To do this, run:
.. code-block:: console
@@ -102,8 +315,8 @@ the current state of the environment packages. To do this, run
This will create a :file:`requirements.txt` file, which contains a simple
list of all the packages in the current environment, and their respective
-versions. You can see the list of installed packages without the requirements
-format using "pip list". Later it will be easier for a different developer
+versions. You can see the list of installed packages without the requirements
+format using ``pip list``. Later it will be easier for a different developer
(or you, if you need to re-create the environment) to install the same packages
using the same versions:
@@ -115,14 +328,14 @@ This can help ensure consistency across installations, across deployments,
and across developers.
Lastly, remember to exclude the virtual environment folder from source
-control by adding it to the ignore list.
+control by adding it to the ignore list (see :ref:`Version Control Ignores`).
.. _virtualenvwrapper-ref:
virtualenvwrapper
-----------------
-`virtualenvwrapper `_
+`virtualenvwrapper `_
provides a set of commands which makes working with virtual environments much
more pleasant. It also places all your virtual environments in one place.
@@ -134,7 +347,7 @@ To install (make sure **virtualenv** is already installed):
$ export WORKON_HOME=~/Envs
$ source /usr/local/bin/virtualenvwrapper.sh
-(`Full virtualenvwrapper install instructions `_.)
+(`Full virtualenvwrapper install instructions `_.)
For Windows, you can use the `virtualenvwrapper-win `_.
@@ -143,8 +356,8 @@ To install (make sure **virtualenv** is already installed):
.. code-block:: console
$ pip install virtualenvwrapper-win
-
-In Windows, the default path for WORKON_HOME is %USERPROFILE%\Envs
+
+In Windows, the default path for WORKON_HOME is %USERPROFILE%\\Envs
Basic Usage
~~~~~~~~~~~
@@ -153,23 +366,23 @@ Basic Usage
.. code-block:: console
- $ mkvirtualenv venv
+ $ mkvirtualenv project_folder
-This creates the :file:`venv` folder inside :file:`~/Envs`.
+This creates the :file:`project_folder` folder inside :file:`~/Envs`.
2. Work on a virtual environment:
.. code-block:: console
- $ workon venv
+ $ workon project_folder
Alternatively, you can make a project, which creates the virtual environment,
-and also a project directory inside ``$PROJECT_HOME``, which is ``cd`` -ed into
-when you ``workon myproject``.
+and also a project directory inside ``$WORKON_HOME``, which is ``cd``-ed into
+when you ``workon project_folder``.
.. code-block:: console
- $ mkproject myproject
+ $ mkproject project_folder
**virtualenvwrapper** provides tab-completion on environment names. It really
helps when you have a lot of environments and have trouble remembering their
@@ -206,7 +419,7 @@ Other useful commands
``lssitepackages``
Shows contents of :file:`site-packages` directory.
-`Full list of virtualenvwrapper commands `_.
+`Full list of virtualenvwrapper commands `_.
virtualenv-burrito
------------------
@@ -214,20 +427,15 @@ virtualenv-burrito
With `virtualenv-burrito `_, you
can have a working virtualenv + virtualenvwrapper environment in a single command.
-autoenv
+direnv
-------
-When you ``cd`` into a directory containing a :file:`.env`, `autoenv `_
+When you ``cd`` into a directory containing a :file:`.env`, `direnv `_
automagically activates the environment.
Install it on Mac OS X using ``brew``:
.. code-block:: console
- $ brew install autoenv
-
-And on Linux:
-
-.. code-block:: console
+ $ brew install direnv
- $ git clone git://github.com/kennethreitz/autoenv.git ~/.autoenv
- $ echo 'source ~/.autoenv/activate.sh' >> ~/.bashrc
+On Linux follow the instructions at `direnv.net `_
diff --git a/docs/index.rst b/docs/index.rst
index 30b73e759..df7fcb8ed 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -3,16 +3,21 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
+.. meta::
+ :description: An opinionated guide to the Python programming language and a best practice handbook for the installation, configuration, and usage of Python on a daily basis.
+
+
+#################################
The Hitchhiker's Guide to Python!
-=================================
+#################################
Greetings, Earthling! Welcome to The Hitchhiker's Guide to Python.
**This is a living, breathing guide.** If you'd like to contribute,
-`fork us on GitHub `_!
+`fork us on GitHub `_!
This handcrafted guide exists to provide both novice and expert Python
-developers a best practice handbook to the installation, configuration, and
+developers a best practice handbook for the installation, configuration, and
usage of Python on a daily basis.
This guide is **opinionated** in a way that is almost, but not quite, entirely
@@ -20,6 +25,8 @@ This guide is **opinionated** in a way that is almost, but not quite, entirely
available here. Rather, you'll find a nice concise list of highly recommended
options.
+.. note:: The use of **Python 3** is *highly* recommended over Python 2. Consider upgrading your applications and infrastructures if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste.
+ —*Kenneth Reitz*
Let's get started! But first, let's make sure you know where your towel is.
diff --git a/docs/intro/community.rst b/docs/intro/community.rst
index 754d53307..4ce563658 100644
--- a/docs/intro/community.rst
+++ b/docs/intro/community.rst
@@ -1,18 +1,25 @@
.. _the-community:
+
+#############
The Community
-=============
+#############
+
+.. image:: /_static/photos/34689432801_78d97ecec9_k_d.jpg
+
+****
BDFL
-----
+****
Guido van Rossum, the creator of Python, is often referred to as the BDFL — the
Benevolent Dictator For Life.
-
+**************************
Python Software Foundation
---------------------------
+**************************
+
The mission of the Python Software Foundation is to promote, protect, and
advance the Python programming language, and to support and facilitate the
@@ -21,8 +28,9 @@ growth of a diverse and international community of Python programmers.
`Learn More about the PSF `_.
+****
PEPs
-----
+****
PEPs are *Python Enhancement Proposals*. They describe changes to Python itself,
or the standards around it.
@@ -68,8 +76,9 @@ Here's an overview of the PEP acceptance workflow:
.. image:: ../_static/pep-0001-1.png
+******************
Python Conferences
---------------------------
+******************
The major events for the Python community are developer conferences. The two
most notable conferences are PyCon, which is held in the US, and its European
@@ -78,9 +87,29 @@ sibling, EuroPython.
A comprehensive list of conferences is maintained at `pycon.org `_.
+******************
Python User Groups
---------------------------
+******************
User Groups are where a bunch of Python developers meet to present or talk
about Python topics of interest. A list of local user groups is maintained at
the `Python Software Foundation Wiki `_.
+
+
+******************
+Online Communities
+******************
+
+`PythonistaCafe `_ is an invite-only, online community
+of Python and software development enthusiasts helping each other succeed and grow.
+Think of it as a club of mutual improvement for Pythonistas where a broad range of
+programming questions, career advice, and other topics are discussed every day.
+
+
+*****************
+Python Job Boards
+*****************
+
+`Python Jobs HQ `_ is a Python job board, by Python Developers
+for Python Developers. The site aggregates Python job postings from across the web and
+also allows employers to post Python job openings directly on the site.
diff --git a/docs/intro/documentation.rst b/docs/intro/documentation.rst
index 1db5bcedb..887d0982b 100644
--- a/docs/intro/documentation.rst
+++ b/docs/intro/documentation.rst
@@ -1,8 +1,15 @@
+
+
+#############
Documentation
-=============
+#############
+
+.. image:: /_static/photos/33928823133_2f3d32cf32_k_d.jpg
+
+**********************
Official Documentation
-----------------------
+**********************
The official Python Language and Library documentation can be found here:
@@ -10,8 +17,9 @@ The official Python Language and Library documentation can be found here:
- `Python 3.x `_
+*************
Read the Docs
--------------
+*************
Read the Docs is a popular community project that hosts documentation
for open source software. It holds documentation for many Python modules,
@@ -20,20 +28,21 @@ both popular and exotic.
`Read the Docs `_
+*****
pydoc
------
+*****
:program:`pydoc` is a utility that is installed when you install Python.
It allows you to quickly retrieve and search for documentation from your
shell. For example, if you needed a quick refresher on the
-:mod:`time` module, pulling up documentation would be as simple as
+:mod:`time` module, pulling up documentation would be as simple as:
.. code-block:: console
$ pydoc time
The above command is essentially equivalent to opening the Python REPL
-and running
+and running:
.. code-block:: pycon
diff --git a/docs/intro/duction.rst b/docs/intro/duction.rst
index e18d146c2..16baebc0d 100644
--- a/docs/intro/duction.rst
+++ b/docs/intro/duction.rst
@@ -1,5 +1,10 @@
+
+
+############
Introduction
-============
+############
+
+.. image:: /_static/photos/34725946825_0f85497e60_k_d.jpg
From the `official Python website `_:
@@ -23,11 +28,11 @@ include:
object serialization, and much more.
Additionally, the
- `Python Package Index `_ is available
+ `Python Package Index `_ is available
for users to submit their packages for widespread use, similar to
- Perl's `CPAN `_. There is a thriving community
+ Perl's `CPAN `_. There is a thriving community
of very powerful Python frameworks and tools like
- the `Django `_ web framework and the
+ the `Django `_ web framework and the
`NumPy `_ set of math routines.
* **integration with other systems**
@@ -54,8 +59,10 @@ include:
.. _about-ref:
+
+****************
About This Guide
-----------------
+****************
Purpose
~~~~~~~
diff --git a/docs/intro/learning.rst b/docs/intro/learning.rst
index 8ab660867..b0c957398 100644
--- a/docs/intro/learning.rst
+++ b/docs/intro/learning.rst
@@ -1,8 +1,15 @@
+
+
+###############
Learning Python
-===============
+###############
+
+.. image:: /_static/photos/32800783863_11a00db52c_k_d.jpg
+
+********
Beginner
---------
+********
The Python Tutorial
~~~~~~~~~~~~~~~~~~~~
@@ -13,37 +20,60 @@ quick-start guide to the language.
`The Python Tutorial `_
-Python for Beginners
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Real Python
+~~~~~~~~~~~
+
+Real Python is a repository of free and in-depth Python tutorials created by a diverse team of professional Python developers. At Real Python you can learn all things Python from the ground up. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond.
-thepythonguru.com is a tutorial focuses on beginner programmers. It covers many python concepts
-in depth. It also teaches you some advance constructs of python like lambda expression, regular expression.
-At last it finishes off with tutorial "How to access MySQL db using python"
+ `Real Python `_
+Python Basics
+~~~~~~~~~~~~~
- `Python for beginners `_
+pythonbasics.org is an introductory tutorial for beginners. The tutorial includes exercises. It covers the basics and there are also in-depth lessons like object oriented programming and regular expressions.
+
+ `Python basics `_
+
+Python for Beginners
+~~~~~~~~~~~~~~~~~~~~
+
+thepythonguru.com is a tutorial focused on beginner programmers. It covers many Python concepts
+in depth. It also teaches you some advanced constructs of Python like lambda expressions and regular expressions.
+And last it finishes off with the tutorial "How to access MySQL db using Python"
+
+ `Python for Beginners `_
Learn Python Interactive Tutorial
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Learnpython.org is an easy non-intimidating way to get introduced to Python.
The website takes the same approach used on the popular
-`Try Ruby `_ website, it has an interactive Python
+`Try Ruby `_ website. It has an interactive Python
interpreter built into the site that allows you to go through the lessons
without having to install Python locally.
`Learn Python `_
+Python for You and Me
+~~~~~~~~~~~~~~~~~~~~~
If you want a more traditional book, *Python For You and Me* is an excellent
resource for learning all aspects of the language.
- `Python for You and Me `_
+ `Python for You and Me `_
+
+Learn Python Step by Step
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Techbeamers.com provides step-by-step tutorials to teach Python. Each tutorial is supplemented with logically added coding snippets and equips with a follow-up quiz on the subject learned. There is a section for `Python interview questions `_ to help job seekers. You can also read essential `Python tips `_ and learn `best coding practices `_ for writing quality code. Here, you'll get the right platform to learn Python quickly.
+
+`Learn Python Basic to Advanced `_
+
Online Python Tutor
~~~~~~~~~~~~~~~~~~~
-Online Python Tutor gives you a visual step by step
+Online Python Tutor gives you a visual step-by-step
representation of how your program runs. Python Tutor
helps people overcome a fundamental barrier to learning
programming by understanding what happens as the computer
@@ -78,7 +108,7 @@ Learn Python the Hard Way
This is an excellent beginner programmer's guide to Python. It covers "hello
world" from the console to the web.
- `Learn Python the Hard Way `_
+ `Learn Python the Hard Way `_
Crash into Python
@@ -87,7 +117,7 @@ Crash into Python
Also known as *Python for Programmers with 3 Hours*, this guide gives
experienced developers from other languages a crash course on Python.
- `Crash into Python `_
+ `Crash into Python `_
Dive Into Python 3
@@ -97,7 +127,7 @@ Dive Into Python 3 is a good book for those ready to jump in to Python 3. It's
a good read if you are moving from Python 2 to 3 or if you already have some
experience programming in another language.
- `Dive Into Python 3 `_
+ `Dive Into Python 3 `_
Think Python: How to Think Like a Computer Scientist
@@ -105,7 +135,7 @@ Think Python: How to Think Like a Computer Scientist
Think Python attempts to give an introduction to basic concepts in computer
science through the use of the Python language. The focus was to create a book
-with plenty of exercises, minimal jargon and a section in each chapter devoted
+with plenty of exercises, minimal jargon, and a section in each chapter devoted
to the subject of debugging.
While exploring the various features available in the Python language the
@@ -113,7 +143,7 @@ author weaves in various design patterns and best practices.
The book also includes several case studies which have the reader explore the
topics discussed in the book in greater detail by applying those topics to
-real-world examples. Case studies include assignments in GUI and Markov
+real-world examples. Case studies include assignments in GUI programming and Markov
Analysis.
`Think Python `_
@@ -123,7 +153,7 @@ Python Koans
~~~~~~~~~~~~
Python Koans is a port of Edgecase's Ruby Koans. It uses a test-driven
-approach, q.v. TEST DRIVEN DESIGN SECTION to provide an interactive tutorial
+approach to provide an interactive tutorial
teaching basic Python concepts. By fixing assertion statements that fail in a
test script, this provides sequential steps to learning Python.
@@ -131,11 +161,11 @@ For those used to languages and figuring out puzzles on their own, this can be
a fun, attractive option. For those new to Python and programming, having an
additional resource or reference will be helpful.
- `Python Koans `_
+ `Python Koans `_
More information about test driven development can be found at these resources:
- `Test Driven Development `_
+ `Test Driven Development `_
A Byte of Python
@@ -145,34 +175,59 @@ A free introductory book that teaches Python at the beginner level, it assumes
no previous programming experience.
`A Byte of Python for Python 2.x `_
- `A Byte of Python for Python 3.x `_
+ `A Byte of Python for Python 3.x `_
-Learn to Program in Python with Codeacademy
+Computer Science Path on Codecademy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A Codeacademy course for the absolute Python beginner. This free and interactive course provides and teaches the basics (and beyond) of Python programming whilst testing the user's knowledge in between progress.
+A Codecademy course for the absolute Python beginner. This free and interactive
+course provides and teaches the basics (and beyond) of Python programming while
+testing the user's knowledge in between progress.
This course also features a built-in interpreter for receiving instant feedback on your learning.
- `Learn to Program in Python with Codeacademy `_
+ `Computer Science Path on Codecademy `_
+
+
+Code the blocks
+~~~~~~~~~~~~~~~
+
+*Code the blocks* provides free and interactive Python tutorials for
+beginners. It combines Python programming with a 3D environment where
+you "place blocks" and construct structures. The tutorials teach you
+how to use Python to create progressively more elaborate 3D structures,
+making the process of learning Python fun and engaging.
+ `Code the blocks `_
+
+************
Intermediate
-------------
+************
+
+Python Tricks: The Book
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Discover Python's best practices with simple examples and start writing even more beautiful + Pythonic code. *Python Tricks: The Book* shows you exactly how.
+
+You’ll master intermediate and advanced-level features in Python with practical examples and a clear narrative.
+
+ `Python Tricks: The Book `_
Effective Python
~~~~~~~~~~~~~~~~
This book contains 59 specific ways to improve writing Pythonic code. At 227
-pages, it is a very brief overview of some of the most commons adapations
+pages, it is a very brief overview of some of the most common adaptations
programmers need to make to become efficient intermediate level Python
programmers.
- `Effective Python `_
+ `Effective Python `_
+********
Advanced
---------
+********
Pro Python
~~~~~~~~~~
@@ -181,7 +236,7 @@ This book is for intermediate to advanced Python programmers who are looking to
understand how and why Python works the way it does and how they can take their
code to the next level.
- `Pro Python `_
+ `Pro Python `_
Expert Python Programming
@@ -190,15 +245,15 @@ Expert Python Programming deals with best practices in programming Python and
is focused on the more advanced crowd.
It starts with topics like decorators (with caching, proxy, and context manager
-case-studies), method resolution order, using super() and meta-programming, and
+case studies), method resolution order, using super() and meta-programming, and
general :pep:`8` best practices.
It has a detailed, multi-chapter case study on writing and releasing a package
and eventually an application, including a chapter on using zc.buildout. Later
chapters detail best practices such as writing documentation, test-driven
-development, version control, optimization and profiling.
+development, version control, optimization, and profiling.
- `Expert Python Programming `_
+ `Expert Python Programming `_
A Guide to Python's Magic Methods
@@ -210,9 +265,13 @@ and can make classes and objects behave in different and magical ways.
`A Guide to Python's Magic Methods `_
+.. note:: Rafekettler.com is currently down; you can go to their GitHub version directly. Here you can find a PDF version:
+ `A Guide to Python's Magic Methods (repo on GitHub) `_
+
+****************************
For Engineers and Scientists
-----------------------------
+****************************
A Primer on Scientific Programming with Python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -221,7 +280,7 @@ A Primer on Scientific Programming with Python, written by Hans Petter
Langtangen, mainly covers Python's usage in the scientific field. In the book,
examples are chosen from mathematics and the natural sciences.
- `A Primer on Scientific Programming with Python `_
+ `A Primer on Scientific Programming with Python `_
Numerical Methods in Engineering with Python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -229,10 +288,12 @@ Numerical Methods in Engineering with Python
Numerical Methods in Engineering with Python, written by Jaan Kiusalaas,
puts the emphasis on numerical methods and how to implement them in Python.
- `Numerical Methods in Engineering with Python `_
+ `Numerical Methods in Engineering with Python `_
+
-Miscellaneous topics
---------------------
+********************
+Miscellaneous Topics
+********************
Problem Solving with Algorithms and Data Structures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -260,7 +321,7 @@ Transforming Code into Beautiful, Idiomatic Python
Transforming Code into Beautiful, Idiomatic Python is a video by Raymond Hettinger.
Learn to take better advantage of Python's best features and improve existing code
-through a series of code transformations, "When you see this, do that instead."
+through a series of code transformations: "When you see this, do that instead."
`Transforming Code into Beautiful, Idiomatic Python `_
@@ -271,7 +332,7 @@ Fullstack Python
Fullstack Python offers a complete top-to-bottom resource for web development
using Python.
-From setting up the webserver, to designing the front-end, choosing a database,
+From setting up the web server, to designing the front-end, choosing a database,
optimizing/scaling, etc.
As the name suggests, it covers everything you need to build and run a complete
@@ -280,14 +341,23 @@ web app from scratch.
`Fullstack Python `_
+PythonistaCafe
+~~~~~~~~~~~~~~
+
+PythonistaCafe is an invite-only, online community of Python and software development enthusiasts helping each other succeed and grow. Think of it as a club of mutual improvement for Pythonistas where a broad range of programming questions, career advice, and other topics are discussed every day.
+
+ `PythonistaCafe `_
+
+
+**********
References
-----------
+**********
Python in a Nutshell
~~~~~~~~~~~~~~~~~~~~
Python in a Nutshell, written by Alex Martelli, covers most cross-platform
-Python's usage, from its syntax to built-in libraries to advanced topics such
+Python usage, from its syntax to built-in libraries to advanced topics such
as writing C extensions.
`Python in a Nutshell `_
@@ -295,7 +365,7 @@ as writing C extensions.
The Python Language Reference
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-This is Python's reference manual, it covers the syntax and the core semantics
+This is Python's reference manual. It covers the syntax and the core semantics
of the language.
`The Python Language Reference `_
@@ -322,7 +392,7 @@ Python Cookbook
~~~~~~~~~~~~~~~
Python Cookbook, written by David Beazley and Brian K. Jones, is packed with
-practical recipes. This book covers the core python language as well as tasks
+practical recipes. This book covers the core Python language as well as tasks
common to a wide variety of application domains.
`Python Cookbook `_
@@ -330,13 +400,13 @@ common to a wide variety of application domains.
Writing Idiomatic Python
~~~~~~~~~~~~~~~~~~~~~~~~
-"Writing Idiomatic Python", written by Jeff Knupp, contains the most common and
+Writing Idiomatic Python, written by Jeff Knupp, contains the most common and
important Python idioms in a format that maximizes identification and
understanding. Each idiom is presented as a recommendation of a way to write
some commonly used piece of code, followed by an explanation of why the idiom
is important. It also contains two code samples for each idiom: the "Harmful"
way to write it and the "Idiomatic" way.
- `For Python 2.7.3+ `_
+ `For Python 2.7.3+ `_
- `For Python 3.3+ `_
+ `For Python 3.3+ `_
diff --git a/docs/intro/news.rst b/docs/intro/news.rst
index 54f8ff70f..d1cda93be 100644
--- a/docs/intro/news.rst
+++ b/docs/intro/news.rst
@@ -1,56 +1,102 @@
+
+
+####
News
-====
+####
+
+.. image:: /_static/photos/33573767786_eececc5d27_k_d.jpg
+
+
+****************
+PyCoder’s Weekly
+****************
+
+PyCoder’s Weekly is a free weekly Python newsletter for Python developers
+by Python developers (Projects, Articles, News, and Jobs).
+
+ `PyCoder’s Weekly `_
+
+
+***********
+Real Python
+***********
+
+At Real Python you can learn all things Python from the ground up, with weekly free and in-depth tutorials. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond.
+
+ `Real Python `_
+
+*************
Planet Python
-~~~~~~~~~~~~~
+*************
This is an aggregate of Python news from a growing number of developers.
- `Planet Python `_
+ `Planet Python `_
+
+*********
/r/python
-~~~~~~~~~
+*********
/r/python is the Reddit Python community where users contribute and vote on
Python-related news.
- `/r/python `_
+ `/r/python `_
-Pycoder's Weekly
-~~~~~~~~~~~~~~~~
-Pycoder's Weekly is a free weekly Python newsletter for Python developers
-by Python developers (Projects, Articles, News, and Jobs).
+*******************
+Talk Python Podcast
+*******************
+
+The #1 Python-focused podcast covering the people and ideas in Python.
+
+ `Talk Python To Me `_
+
- `Pycoder's Weekly `_
+********************
+Python Bytes Podcast
+********************
+A short-form Python podcast covering recent developer headlines.
+
+ `Python Bytes `_
+
+
+*************
Python Weekly
-~~~~~~~~~~~~~
+*************
Python Weekly is a free weekly newsletter featuring curated news, articles,
new releases, jobs, etc. related to Python.
- `Python Weekly `_
+ `Python Weekly `_
+
+***********
Python News
-~~~~~~~~~~~~~
+***********
Python News is the news section in the official Python web site
(www.python.org). It briefly highlights the news from the Python community.
- `Python News `_
+ `Python News `_
+
+********************
Import Python Weekly
-~~~~~~~~~~~~~~~~~~~~
+********************
-Weekly Python Newsletter containing Python Articles, Projects, Videos, Tweets
+Weekly Python Newsletter containing Python Articles, Projects, Videos, and Tweets
delivered in your inbox. Keep Your Python Programming Skills Updated.
`Import Python Weekly Newsletter `_
+
+*************************
Awesome Python Newsletter
-~~~~~~~~~~~~~~~~~~~~
+*************************
-A weekly overview of the most popular Python news, articles and packages.
+A weekly overview of the most popular Python news, articles, and packages.
`Awesome Python Newsletter `_
diff --git a/docs/notes/contribute.rst b/docs/notes/contribute.rst
index 0d709f29f..b6c72bb4f 100644
--- a/docs/notes/contribute.rst
+++ b/docs/notes/contribute.rst
@@ -1,5 +1,8 @@
+##########
Contribute
-~~~~~~~~~~
+##########
+
+.. image:: /_static/photos/33573769116_49c1ef51e7_k_d.jpg
Python-guide is under active development, and contributors are welcome.
@@ -8,20 +11,24 @@ issue on GitHub_. To submit patches, please send a pull request on GitHub_.
Once your changes get merged back in, you'll automatically be added to the
`Contributors List `_.
+
+***********
Style Guide
------------
+***********
For all contributions, please follow the :ref:`guide-style-guide`.
.. _todo-list-ref:
+
+*********
Todo List
----------
+*********
If you'd like to contribute, there's plenty to do. Here's a short todo_ list.
.. include:: ../../TODO.rst
-.. _GitHub: http://github.com/kennethreitz/python-guide/
+.. _GitHub: https://github.com/kennethreitz/python-guide/
.. _todo: https://github.com/kennethreitz/python-guide/blob/master/TODO.rst
diff --git a/docs/notes/license.rst b/docs/notes/license.rst
index 5a7f54351..1dd3d340a 100644
--- a/docs/notes/license.rst
+++ b/docs/notes/license.rst
@@ -1,5 +1,7 @@
-=======
+#######
License
-=======
+#######
-The Guide is licensed under the `Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license `_.
\ No newline at end of file
+.. image:: /_static/photos/32800805573_568d6b72fd_k_d.jpg
+
+The Guide is licensed under the `Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license `_.
diff --git a/docs/notes/styleguide.rst b/docs/notes/styleguide.rst
index 768240b0e..5a715332e 100644
--- a/docs/notes/styleguide.rst
+++ b/docs/notes/styleguide.rst
@@ -1,8 +1,10 @@
.. _guide-style-guide:
-=====================
+#####################
The Guide Style Guide
-=====================
+#####################
+
+.. image:: /_static/photos/33573755856_7f43d43adf_k_d.jpg
As with all documentation, having a consistent format helps make the
document more understandable. In order to make The Guide easier to digest,
@@ -17,8 +19,10 @@ The Guide is written as :ref:`restructuredtext-ref`.
.. note:: On any page of the rendered HTML you can click "Show Source" to
see how authors have styled the page.
+
+*********
Relevancy
----------
+*********
Strive to keep any contributions relevant to the :ref:`purpose of The Guide
`.
@@ -27,15 +31,17 @@ Strive to keep any contributions relevant to the :ref:`purpose of The Guide
relate to Python development.
* Prefer to link to other sources if the information is already out there.
Be sure to describe what and why you are linking.
-* `Cite `_
+* `Cite `_
references where needed.
* If a subject isn't directly relevant to Python, but useful in conjunction
with Python (e.g., Git, GitHub, Databases), reference by linking to useful
resources, and describe why it's useful to Python.
* When in doubt, ask.
+
+********
Headings
---------
+********
Use the following styles for headings.
@@ -51,31 +57,35 @@ Page title:
.. code-block:: rest
- ===================
+ *******************
Time is an Illusion
- ===================
+ *******************
Section headings:
.. code-block:: rest
Lunchtime Doubly So
- -------------------
+ ===================
Sub section headings:
.. code-block:: rest
Very Deep
- ~~~~~~~~~
+ ---------
+
+*****
Prose
------
+*****
Wrap text lines at 78 characters. Where necessary, lines may exceed 78
characters, especially if wrapping would make the source text more difficult
to read.
+Use Standard American English, not British English.
+
Use of the `serial comma `_
(also known as the Oxford comma) is 100% non-optional. Any attempt to
submit content with a missing serial comma will result in permanent banishment
@@ -83,8 +93,10 @@ from this project, due to complete and total lack of taste.
Banishment? Is this a joke? Hopefully we will never have to find out.
+
+*************
Code Examples
--------------
+*************
Wrap all code examples at 70 characters to avoid horizontal scrollbars.
@@ -97,7 +109,10 @@ Command line examples:
$ run command --help
$ ls ..
-Be sure to include the ``$`` prefix before each line.
+Be sure to include the ``$`` prefix before each line for Unix console examples.
+
+For Windows console examples, use ``doscon`` or ``powershell`` instead of
+``console``, and omit the ``$`` prefix.
Python interpreter examples:
@@ -120,32 +135,36 @@ Python examples:
def get_answer():
return 42
+
+******************
Externally Linking
-------------------
+******************
-* Prefer labels for well known subjects (ex: proper nouns) when linking:
+* Prefer labels for well known subjects (e.g. proper nouns) when linking:
.. code-block:: rest
Sphinx_ is used to document Python.
- .. _Sphinx: http://sphinx.pocoo.org
+ .. _Sphinx: https://www.sphinx-doc.org
* Prefer to use descriptive labels with inline links instead of leaving bare
links:
.. code-block:: rest
- Read the `Sphinx Tutorial `_
+ Read the `Sphinx Tutorial `_
-* Avoid using labels such as "click here", "this", etc. preferring
+* Avoid using labels such as "click here", "this", etc., preferring
descriptive labels (SEO worthy) instead.
+
+********************************
Linking to Sections in The Guide
---------------------------------
+********************************
To cross-reference other parts of this documentation, use the `:ref:
-`_
+`_
keyword and labels.
To make reference labels more clear and unique, always add a ``-ref`` suffix:
@@ -157,11 +176,13 @@ To make reference labels more clear and unique, always add a ``-ref`` suffix:
Some Section
------------
+
+******************
Notes and Warnings
-------------------
+******************
Make use of the appropriate `admonitions directives
-`_ when making notes.
+`_ when making notes.
Notes:
@@ -178,11 +199,13 @@ Warnings:
.. warning:: DON'T PANIC
+
+*****
TODOs
------
+*****
Please mark any incomplete areas of The Guide with a `todo directive
-`_. To
+`_. To
avoid cluttering the :ref:`todo-list-ref`, use a single ``todo`` for stub
documents or large incomplete sections.
@@ -191,4 +214,3 @@ documents or large incomplete sections.
.. todo::
Learn the Ultimate Answer to the Ultimate Question
of Life, The Universe, and Everything
-
diff --git a/docs/scenarios/admin.rst b/docs/scenarios/admin.rst
index 0bd928dd5..f6433c213 100644
--- a/docs/scenarios/admin.rst
+++ b/docs/scenarios/admin.rst
@@ -1,8 +1,14 @@
+
+######################
Systems Administration
-======================
+######################
+
+.. image:: /_static/photos/34435690580_3afec7d4cd_k_d.jpg
+
+******
Fabric
-------
+******
`Fabric `_ is a library for simplifying system
administration tasks. While Chef and Puppet tend to focus on managing servers
@@ -17,7 +23,7 @@ Install Fabric:
The following code will create two tasks that we can use: ``memory_usage`` and
``deploy``. The former will output the memory usage on each machine. The
-latter will ssh into each server, cd to our project directory, activate the
+latter will SSH into each server, cd to our project directory, activate the
virtual environment, pull the newest codebase, and restart the application
server.
@@ -69,8 +75,10 @@ programs, and host grouping.
`Fabric Documentation `_
+
+****
Salt
-----
+****
`Salt `_ is an open source infrastructure management
tool. It supports remote command execution from a central point (master host)
@@ -93,7 +101,7 @@ The following command lists all available minion hosts, using the ping module.
$ salt '*' test.ping
-The host filtering is accomplished by matching the minion id,
+The host filtering is accomplished by matching the minion id
or using the grains system. The
`grains `_
system uses static host information like the operating system version or the
@@ -123,16 +131,17 @@ it will install and start the Apache server:
- require:
- pkg: apache
-State files can be written using YAML, the Jinja2 template system or pure Python.
+State files can be written using YAML, the Jinja2 template system, or pure Python.
`Salt Documentation `_
+******
Psutil
-------
+******
-`Psutil `_ is an interface to different
-system information (e.g. CPU, memory, disks, network, users and processes).
+`Psutil `_ is an interface to different
+system information (e.g. CPU, memory, disks, network, users, and processes).
Here is an example to be aware of some server overload. If any of the
tests (net, CPU) fail, it will send an email.
@@ -146,49 +155,49 @@ tests (net, CPU) fail, it will send an email.
# Package for email services:
import smtplib
import string
- MAX_NET_USAGE = 400000
+ MAX_NET_USAGE = 400000 # bytes per seconds
MAX_ATTACKS = 4
attack = 0
- counter = 0
while attack <= MAX_ATTACKS:
sleep(4)
- counter = counter + 1
- # Check the cpu usage
- if cpu_percent(interval = 1) > 70:
- attack = attack + 1
- # Check the net usage
- neti1 = net_io_counters()[1]
- neto1 = net_io_counters()[0]
+
+ # Check the net usage wit named tuples
+ neti1 = net_io_counters().bytes_recv
+ neto1 = net_io_counters().bytes_sent
sleep(1)
- neti2 = net_io_counters()[1]
- neto2 = net_io_counters()[0]
+ neti2 = net_io_counters().bytes_recv
+ neto2 = net_io_counters().bytes_sent
+
# Calculate the bytes per second
net = ((neti2+neto2) - (neti1+neto1))/2
- if net > MAX_NET_USAGE:
- attack = attack + 1
- if counter > 25:
- attack = 0
- counter = 0
+
+ # Check the net and cpu usage
+ if (net > MAX_NET_USAGE) or (cpu_percent(interval = 1) > 70):
+ attack+=1
+ elif attack > 1:
+ attack-=1
+
# Write a very important email if attack is higher than 4
TO = "you@your_email.com"
FROM = "webmaster@your_domain.com"
SUBJECT = "Your domain is out of system resources!"
text = "Go and fix your server!"
- BODY = string.join(("From: %s" %FROM,"To: %s" %TO,"Subject: %s" %SUBJECT, "",text), "\r\n")
+ string="\r\n"
+ BODY = string.join(("From: %s" %FROM,"To: %s" %TO,
+ "Subject: %s" %SUBJECT, "",text))
server = smtplib.SMTP('127.0.0.1')
server.sendmail(FROM, [TO], BODY)
server.quit()
-A full terminal application like a widely extended top which is based on
-psutil and with the ability of a client-server monitoring is
-`glance `_.
+A full terminal application like a widely extended top is `Glance `_, which is based on psutil and has the ability for client-server monitoring.
+*******
Ansible
--------
+*******
`Ansible `_ is an open source system automation tool.
-The biggest advantage over Puppet or Chef is it does not require an agent on
+Its biggest advantage over Puppet or Chef is that it does not require an agent on
the client machine. Playbooks are Ansible’s configuration, deployment, and
orchestration language and are written in YAML with Jinja2 for templating.
@@ -232,43 +241,47 @@ The Ansible playbook will ping all of the servers in the :file:`hosts.yml` file.
You can also select groups of servers using Ansible. For more information
about Ansible, read the `Ansible Docs `_.
-`An Ansible tutorial `_ is also a
+`An Ansible tutorial `_ is also a
great and detailed introduction to getting started with Ansible.
+****
Chef
-----
-`Chef `_ is a systems and cloud infrastructure automation
-framework that makes it easy to deploy servers and applications to any physical,
-virtual, or cloud location. In case this is your choice for configuration management,
-you will primarily use Ruby to write your infrastructure code.
-
-Chef clients run on every server that is part of your infrastructure and these regularly
-check with your Chef server to ensure your system is always aligned and represents the
-desired state. Since each individual server has its own distinct Chef client, each server
+****
+
+`Chef `_ is a systems and cloud infrastructure automation
+framework that makes it easy to deploy servers and applications to any physical,
+virtual, or cloud location. In case this is your choice for configuration management,
+you will primarily use Ruby to write your infrastructure code.
+
+Chef clients run on every server that is part of your infrastructure and these regularly
+check with your Chef server to ensure your system is always aligned and represents the
+desired state. Since each individual server has its own distinct Chef client, each server
configures itself and this distributed approach makes Chef a scalable automation platform.
-Chef works by using custom recipes (configuration elements), implemented in cookbooks. Cookbooks, which are basically
-packages for infrastructure choices, are usually stored in your Chef server.
-Read the `Digital Ocean tutorial series
-`_
-on chef to learn how to create a simple Chef Server.
+Chef works by using custom recipes (configuration elements), implemented in cookbooks. Cookbooks, which are basically
+packages for infrastructure choices, are usually stored in your Chef server.
+Read the `DigitalOcean tutorial series
+`_
+on Chef to learn how to create a simple Chef Server.
To create a simple cookbook the `knife `_ command is used:
-.. code-block:: console
+.. code-block:: console
knife cookbook create cookbook_name
-`Getting started with Chef `_
-is a good starting point for Chef Beginners and many community maintained cookbooks that can
-serve as a good reference or tweaked to serve your infrastructure configuration needs can be
+`Getting started with Chef `_
+is a good starting point for Chef Beginners and many community maintained cookbooks that can
+serve as a good reference or tweaked to serve your infrastructure configuration needs can be
found on the `Chef Supermarket `_.
- `Chef Documentation `_
+
+******
Puppet
-------
+******
`Puppet `_ is IT Automation and configuration management
software from Puppet Labs that allows System Administrators to define the state
@@ -284,8 +297,8 @@ Puppet Agents are installed on nodes whose state needs to be monitored or
changed. A designated server known as the Puppet Master is responsible for
orchestrating the agent nodes.
-Agent nodes send basic facts about the system such as to the operating system,
-kernel, architecture, ip address, hostname etc. to the Puppet Master.
+Agent nodes send basic facts about the system such as the operating system,
+kernel, architecture, IP address, hostname, etc. to the Puppet Master.
The Puppet Master then compiles a catalog with information provided by the
agents on how each node should be configured and sends it to the agent. The
agent enforces the change as prescribed in the catalog and sends a report back
@@ -302,10 +315,10 @@ your Puppet modules.
.. code-block:: console
$ facter operatingsystem
- Ubuntu
+ Ubuntu
Writing Modules in Puppet is pretty straight forward. Puppet Manifests together
-form Puppet Modules. Puppet manifest end with an extension of ``.pp``.
+form Puppet Modules. Puppet manifests end with an extension of ``.pp``.
Here is an example of 'Hello World' in Puppet.
.. code-block:: puppet
@@ -319,7 +332,7 @@ Here is an example of 'Hello World' in Puppet.
Here is another example with system based logic. Note how the operating system
fact is being used as a variable prepended with the ``$`` sign. Similarly, this
holds true for other facts such as hostname which can be referenced by
-``$hostname``
+``$hostname``.
.. code-block:: puppet
@@ -331,10 +344,10 @@ holds true for other facts such as hostname which can be referenced by
}
There are several resource types for Puppet but the package-file-service
-paradigm is all you need for undertaking majority of the configuration
+paradigm is all you need for undertaking the majority of the configuration
management. The following Puppet code makes sure that the OpenSSH-Server
package is installed in a system and the sshd service is notified to restart
-everytime the sshd configuration file is changed.
+every time the sshd configuration file is changed.
.. code-block:: puppet
@@ -363,31 +376,22 @@ everytime the sshd configuration file is changed.
For more information, refer to the `Puppet Labs Documentation `_
+
+*********
Blueprint
----------
+*********
.. todo:: Write about Blueprint
+
+********
Buildout
---------
+********
`Buildout `_ is an open source software build tool.
-Buildout is created using the Python programming language. It implements a
-principle of separation of configuration from the scripts that do the setting up.
-Buildout is primarily used to download and set up dependencies in Python eggs
-format of the software being developed or deployed. Recipes for build tasks in any
-environment can be created, and many are already available.
-
-Buidout is written in Python.
-
-Shinken
--------
-
-`Shinken `_ is a modern, Nagios compatible
-monitoring framework written in Python. Its main goal is to give users a flexible
-architecture for their monitoring system that is designed to scale to large
-environments.
-
-Shinken is backwards-compatible with the Nagios configuration standard, and
-plugins.It works on any operating system, and architecture that supports Python
-which includes Windows, GNU/Linux, and FreeBSD.
+Buildout is created using the Python programming language. It implements a
+principle of separation of configuration from the scripts that do the setting
+up. Buildout is primarily used to download and set up dependencies in `Python
+eggs `_
+format of the software being developed or deployed. Recipes for build tasks in
+any environment can be created, and many are already available.
diff --git a/docs/scenarios/ci.rst b/docs/scenarios/ci.rst
index aa8d1968f..4296679c7 100644
--- a/docs/scenarios/ci.rst
+++ b/docs/scenarios/ci.rst
@@ -1,12 +1,20 @@
+
+######################
Continuous Integration
-======================
+######################
+
+.. image:: /_static/photos/33907150594_9abba7ad0a_k_d.jpg
+
+.. note::
+ For advice on writing your tests, see :doc:`/writing/tests`.
+****
Why?
-----
+****
Martin Fowler, who first wrote about `Continuous Integration `_
-(short: CI) together with Kent Beck, describes the CI as follows:
+(short: CI) together with Kent Beck, describes CI as follows:
Continuous Integration is a software development practice where members of
a team integrate their work frequently, usually each person integrates at
@@ -16,27 +24,28 @@ Martin Fowler, who first wrote about `Continuous Integration `_ is an extensible continuous integration
-engine. Use it.
+*******
+Jenkins
+*******
+`Jenkins CI `_ is an extensible Continuous Integration engine. Use it.
+********
Buildbot
---------
+********
`Buildbot `_ is a Python system to
automate the compile/test cycle to validate code changes.
-
+***
Tox
----
+***
-`tox `_ is an automation tool providing
-packaging, testing and deployment of Python software right from the console or
+`tox `_ is an automation tool providing
+packaging, testing, and deployment of Python software right from the console or
CI server. It is a generic virtualenv management and test command line tool
which provides the following features:
@@ -45,17 +54,18 @@ which provides the following features:
* Running tests in each of the environments, configuring your test tool of
choice
* Acting as a front-end to Continuous Integration servers, reducing boilerplate
- and merging CI and shell-based testing.
+ and merging CI and shell-based testing
+*********
Travis-CI
----------
+*********
`Travis-CI `_ is a distributed CI server which builds
tests for open source projects for free. It provides multiple workers to run
Python tests on and seamlessly integrates with GitHub. You can even have it
comment on your Pull Requests whether this particular changeset breaks the
-build or not. So if you are hosting your code on GitHub, travis-ci is a great
+build or not. So, if you are hosting your code on GitHub, Travis-CI is a great
and easy way to get started with Continuous Integration.
In order to get started, add a :file:`.travis.yml` file to your repository with
@@ -75,12 +85,12 @@ this example content::
This will get your project tested on all the listed Python versions by
-running the given script, and will only build the master branch. There are a
-lot more options you can enable, like notifications, before and after steps
-and much more. The `travis-ci docs `_
+running the given script, and will only build the ``master`` branch. There are a
+lot more options you can enable, like notifications, before and after steps,
+and much more. The `Travis-CI docs `_
explain all of these options, and are very thorough.
-In order to activate testing for your project, go to `the travis-ci site `_
+In order to activate testing for your project, go to `the Travis-CI site `_
and login with your GitHub account. Then activate your project in your
profile settings and you're ready to go. From now on, your project's tests
will be run on every push to GitHub.
diff --git a/docs/scenarios/cli.rst b/docs/scenarios/cli.rst
index c8cdcec73..59f311e83 100644
--- a/docs/scenarios/cli.rst
+++ b/docs/scenarios/cli.rst
@@ -1,64 +1,91 @@
+
+#########################
Command-line Applications
-=========================
+#########################
+
+.. image:: /_static/photos/34435690330_11930b5987_k_d.jpg
-Command-line applications, also referred to as
-`Console Applications `_,
-are computer programs designed to be used from a text interface, such as a
-`shell `_. Command-line
-applications usually accept various inputs as arguments, often referred to as
-parameters or sub-commands, as well as options, often referred to as flags or
-switches.
+Command-line applications, also referred to as `Console Applications
+`_, are computer programs
+designed to be used from a text interface, such as a `shell
+`_. Command-line applications
+usually accept various inputs as arguments, often referred to as parameters or
+sub-commands, as well as options, often referred to as flags or switches.
Some popular command-line applications include:
-* `Grep `_ - A plain-text data search utility
+* `grep `_ - A plain-text data search utility
* `curl `_ - A tool for data transfer with URL syntax
-* `httpie `_ - A command line HTTP
+* `httpie `_ - A command-line HTTP
client, a user-friendly cURL replacement
-* `git `_ - A distributed version control system
-* `mercurial `_ - A distributed version control
+* `Git `_ - A distributed version control system
+* `Mercurial `_ - A distributed version control
system primarily written in Python
-Clint
------
-
-`clint `_ is a Python module which is
-filled with very useful tools for developing command-line applications.
-It supports features such as; CLI colors and indents, simple and powerful
-column printer, iterator based progress bars and implicit argument handling.
+*****
Click
------
+*****
-`click `_ is an upcoming Python package for creating
-command-line interfaces in a composable way with as little code as
-possible. This “Command-line Interface Creation Kit” is highly
-configurable but comes with good defaults out of the box.
+`click `_ is a Python package for creating
+command-line interfaces in a composable way with as little code as possible.
+This “Command-Line Interface Creation Kit” is highly configurable but comes
+with good defaults out of the box.
+
+******
docopt
-------
+******
`docopt `_ is a lightweight, highly Pythonic package that
allows creating command-line interfaces easily and intuitively, by parsing
POSIX-style usage instructions.
+
+****
Plac
-------
+****
-`Plac `_ is a simple wrapper
+`Plac `_ is a simple wrapper
over the Python standard library `argparse `_,
which hides most of its complexity by using a declarative interface: the
-argument parser is inferred rather than written down by imperatively. This
-module targets especially unsophisticated users, programmers, sys-admins,
-scientists and in general people writing throw-away scripts for themselves,
+argument parser is inferred rather than written down imperatively. This
+module targets unsophisticated users, programmers, sysadmins,
+scientists, and in general people writing throw-away scripts for themselves,
who choose to create a command-line interface because it is quick and simple.
+
+*****
Cliff
-------
+*****
`Cliff `_ is a framework for
building command-line programs. It uses setuptools entry points to provide
subcommands, output formatters, and other extensions. The framework is meant
-to be used to create multi-level commands such as subversion and git, where
+to be used to create multi-level commands such as ``svn`` and ``git``, where
the main program handles some basic argument parsing and then invokes a
sub-command to do the work.
+
+
+******
+Cement
+******
+
+`Cement `_ is an advanced CLI Application
+Framework. Its goal is to introduce a standard and feature-full platform for
+both simple and complex command line applications as well as support rapid
+development needs without sacrificing quality. Cement is flexible, and its use
+cases span from the simplicity of a micro-framework to the complexity of a
+mega-framework.
+
+
+***********
+Python Fire
+***********
+
+`Python Fire `_ is a library for
+automatically generating command-line interfaces from absolutely any Python
+object. It can help debug Python code more easily from the command line,
+create CLI interfaces to existing code, allow you to interactively explore
+code in a REPL, and simplify transitioning between Python and Bash (or any
+other shell).
diff --git a/docs/scenarios/clibs.rst b/docs/scenarios/clibs.rst
index bae907b6b..13c9e4802 100644
--- a/docs/scenarios/clibs.rst
+++ b/docs/scenarios/clibs.rst
@@ -1,14 +1,20 @@
+
+################################
Interfacing with C/C++ Libraries
-================================
+################################
+
+.. image:: /_static/photos/34725951345_c8f5959a2e_k_d.jpg
+
+****************************
C Foreign Function Interface
-----------------------------
+****************************
-`CFFI `_ provides a simple to use
+`CFFI `_ provides a simple to use
mechanism for interfacing with C from both CPython and PyPy. It supports two
-modes: an inline ABI compatibility mode (example provided below), which allows
+modes: an inline `ABI `_ compatibility mode (example provided below), which allows
you to dynamically load and run functions from executable modules (essentially
-exposing the same functionality as LoadLibrary or dlopen), and an API mode,
+exposing the same functionality as `LoadLibrary `_ or `dlopen `_), and an API mode,
which allows you to build C extension modules.
ABI Interaction
@@ -25,21 +31,23 @@ ABI Interaction
# prints: 23
print("{}".format(length))
+
+******
ctypes
-------
+******
-`ctypes `_ is the de facto
+`ctypes `_ is the de facto standard
library for interfacing with C/C++ from CPython, and it provides not only
full access to the native C interface of most major operating systems (e.g.,
kernel32 on Windows, or libc on \*nix), but also provides support for loading
-and interfacing with dynamic libraries, such as DLLs or shared objects at
-runtime. It does bring along with it a whole host of types for interacting
+and interfacing with dynamic libraries, such as DLLs or shared objects, at
+runtime. It brings along with it a whole host of types for interacting
with system APIs, and allows you to rather easily define your own complex
types, such as structs and unions, and allows you to modify things such as
padding and alignment, if needed. It can be a bit crufty to use, but in
-conjunction with the `struct `_
+conjunction with the `struct `_
module, you are essentially provided full control over how your data types get
-translated into something usable by a pure C(++) method.
+translated into something usable by a pure C/C++ method.
Struct Equivalents
~~~~~~~~~~~~~~~~~~
@@ -64,15 +72,17 @@ Struct Equivalents
_fields_ = [("a", c_int),
("b", c_int)]
+
+****
SWIG
-----
+****
`SWIG `_, though not strictly Python focused (it supports a
large number of scripting languages), is a tool for generating bindings for
interpreted languages from C/C++ header files. It is extremely simple to use:
the consumer simply needs to define an interface file (detailed in the
tutorial and documentations), include the requisite C/C++ headers, and run
-the build tool against them. While it does have some limits, (it currently
+the build tool against them. While it does have some limits (it currently
seems to have issues with a small subset of newer C++ features, and getting
template-heavy code to work can be a bit verbose), it provides a great deal
of power and exposes lots of features to Python with little effort.
@@ -96,9 +106,10 @@ Example: Overloading __repr__
std::string getName();
};
+
:file:`myclass.i`
-.. code-block:: c++
+.. code-block:: idl
:linenos:
%include "string.i"
@@ -119,11 +130,12 @@ Example: Overloading __repr__
%include "MyClass.h"
+************
Boost.Python
-------------
+************
`Boost.Python `_
requires a bit more manual work to expose C++ object functionality, but
it is capable of providing all the same features SWIG does and then some,
-to include providing wrappers to access PyObjects in C++, extracting SWIG-
+to include providing wrappers to access PyObjects in C++, extracting SWIG
wrapper objects, and even embedding bits of Python into your C++ code.
diff --git a/docs/scenarios/client.rst b/docs/scenarios/client.rst
index 2dc7748cd..c2d5c289b 100644
--- a/docs/scenarios/client.rst
+++ b/docs/scenarios/client.rst
@@ -1,10 +1,14 @@
+
+####################
Network Applications
-====================
+####################
+.. image:: /_static/photos/34364815780_bea6614025_k_d.jpg
+****
HTTP
-::::
+****
The Hypertext Transfer Protocol (HTTP) is an application protocol for
distributed, collaborative, hypermedia information systems. HTTP is the
@@ -24,13 +28,14 @@ your URLs, or to form-encode your POST data. Keep-alive and HTTP connection
pooling are 100% automatic, powered by urllib3, which is embedded within
Requests.
-- `Documentation `_
-- `PyPi `_
+- `Documentation `_
+- `PyPi `_
- `GitHub `_
+*******************
Distributed Systems
-::::::::::::::::::::
+*******************
ZeroMQ
diff --git a/docs/scenarios/crypto.rst b/docs/scenarios/crypto.rst
index e4b103a9d..262337204 100644
--- a/docs/scenarios/crypto.rst
+++ b/docs/scenarios/crypto.rst
@@ -1,16 +1,22 @@
-Cryptography
-============
+############
Cryptography
-------------
+############
+
+.. image:: /_static/photos/33907152824_bf91078cc1_k_d.jpg
+
-`Cryptography `_ is an actively developed
-library that provides cryptographic recipes and primitives. It supports
-Python 2.6-2.7, Python 3.3+ and PyPy.
+************
+cryptography
+************
+`cryptography `_ is an actively developed
+library that provides cryptographic recipes and primitives. It supports
+Python 2.6-2.7, Python 3.3+, and PyPy.
-Cryptography is divided into two layers of recipes and hazardous materials
-(hazmat). The recipes layer provides simple API for proper symmetric
+
+cryptography is divided into two layers of recipes and hazardous materials
+(hazmat). The recipes layer provides a simple API for proper symmetric
encryption and the hazmat layer provides low-level cryptographic primitives.
@@ -36,32 +42,50 @@ Example code using high level symmetric encryption recipe:
plain_text = cipher_suite.decrypt(cipher_text)
+**************
+GPGME bindings
+**************
+
+The `GPGME Python bindings `_ provide Pythonic access to `GPG Made Easy `_, a C API for the entire GNU Privacy Guard suite of projects, including GPG, libgcrypt, and gpgsm (the S/MIME engine). It supports Python 2.6, 2.7, 3.4, and above. Depends on the SWIG C interface for Python as well as the GnuPG software and libraries.
-PyCrypto
---------
+A more comprehensive `GPGME Python Bindings HOWTO `_ is available with the source, and an HTML version is available `at http://files.au.adversary.org `_. Python 3 sample scripts from the examples in the HOWTO are also provided with the source and are accessible `at gnupg.org `_.
-`PyCrypto `_ is another library,
-which provides secure hash functions and various encryption algorithms. It
-supports Python version 2.1 through 3.3.
+Available under the same terms as the rest of the GnuPG Project: GPLv2 and LGPLv2.1, both with the "or any later version" clause.
Installation
~~~~~~~~~~~~
-.. code-block:: console
-
- $ pip install pycrypto
+Included by default when compiling GPGME if the configure script locates a supported python version (which it will if it's in $PATH during configuration).
Example
~~~~~~~
-.. code-block:: python
-
- from Crypto.Cipher import AES
- # Encryption
- encryption_suite = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456')
- cipher_text = encryption_suite.encrypt("A really secret message. Not for prying eyes.")
-
- # Decryption
- decryption_suite = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456')
- plain_text = decryption_suite.decrypt(cipher_text)
+.. code-block:: python3
+
+ import gpg
+
+ # Encryption to public key specified in rkey.
+ a_key = input("Enter the fingerprint or key ID to encrypt to: ")
+ filename = input("Enter the filename to encrypt: ")
+ with open(filename, "rb") as afile:
+ text = afile.read()
+ c = gpg.core.Context(armor=True)
+ rkey = list(c.keylist(pattern=a_key, secret=False))
+ ciphertext, result, sign_result = c.encrypt(text, recipients=rkey,
+ always_trust=True,
+ add_encrypt_to=True)
+ with open("{0}.asc".format(filename), "wb") as bfile:
+ bfile.write(ciphertext)
+ # Decryption with corresponding secret key
+ # invokes gpg-agent and pinentry.
+ with open("{0}.asc".format(filename), "rb") as cfile:
+ plaintext, result, verify_result = gpg.Context().decrypt(cfile)
+ with open("new-{0}".format(filename), "wb") as dfile:
+ dfile.write(plaintext)
+ # Matching the data.
+ # Also running a diff on filename and the new filename should match.
+ if text == plaintext:
+ print("Hang on ... did you say *all* of GnuPG? Yep.")
+ else:
+ pass
diff --git a/docs/scenarios/db.rst b/docs/scenarios/db.rst
index 3ed11914f..21217477a 100644
--- a/docs/scenarios/db.rst
+++ b/docs/scenarios/db.rst
@@ -1,20 +1,28 @@
+
+#########
Databases
-=========
+#########
+
+.. image:: /_static/photos/33907152464_a99fdcc8de_k_d.jpg
+
+******
DB-API
-------
+******
The Python Database API (DB-API) defines a standard interface for Python
database access modules. It's documented in :pep:`249`.
-Nearly all Python database modules such as `sqlite3`, `psycopg` and
+Nearly all Python database modules such as `sqlite3`, `psycopg`, and
`mysql-python` conform to this interface.
Tutorials that explain how to work with modules that conform to this interface can be found
`here `__ and
`here `__.
+
+**********
SQLAlchemy
-----------
+**********
`SQLAlchemy `_ is a commonly used database toolkit.
Unlike many database libraries it not only provides an ORM layer but also a
@@ -24,12 +32,14 @@ generalized API for writing database-agnostic code without SQL.
$ pip install sqlalchemy
+
+*******
Records
--------
+*******
`Records `_ is minimalist SQL library,
designed for sending raw SQL queries to various databases. Data can be used
-programmatically, or exported to a number of useful data formats.
+programmatically or exported to a number of useful data formats.
.. code-block:: console
@@ -37,10 +47,25 @@ programmatically, or exported to a number of useful data formats.
Also included is a command-line tool for exporting SQL data.
+
+******
+PugSQL
+******
+
+`PugSQL `_ is a simple Python interface for organizing
+and using parameterized, handwritten SQL. It is an anti-ORM that is
+philosophically lo-fi, but it still presents a clean interface in Python.
+
+.. code-block:: console
+
+ $ pip install pugsql
+
+
+**********
Django ORM
-----------
+**********
-The Django ORM is the interface used by `Django `_
+The Django ORM is the interface used by `Django `_
to provide database access.
It's based on the idea of
@@ -54,36 +79,40 @@ The basics:
- Django gives you an automatically-generated database-access API; see
`Making queries `__.
+
+******
peewee
-------
+******
`peewee `_ is another ORM with a focus
on being lightweight with support for Python 2.6+ and 3.2+ which supports
-SQLite, MySQL and Postgres by default. The
-`model layer `_
+SQLite, MySQL, and PostgreSQL by default. The
+`model layer `_
is similar to that of the Django ORM and it has
-`SQL-like methods `_
-to query data. While SQLite, MySQL and Postgres are supported out-of-the-box,
-there is a `collection of add-ons `_
+`SQL-like methods `_
+to query data. While SQLite, MySQL, and PostgreSQL are supported out-of-the-box,
+there is a `collection of add-ons `_
available.
+
+*******
PonyORM
--------
+*******
`PonyORM `_ is an ORM that takes a different approach to
querying the database. Instead of writing an SQL-like language or boolean
-expressions, Python's generator syntax is used. There's also an graphical
+expressions, Python's generator syntax is used. There's also a graphical
schema editor that can generate PonyORM entities for you. It supports Python
-2.6+ and Python 3.3+ and can connect to SQLite, MySQL, Postgres & Oracle
-
+2.6+ and Python 3.3+ and can connect to SQLite, MySQL, PostgreSQL, and Oracle.
+*********
SQLObject
----------
+*********
`SQLObject `_ is yet another ORM. It supports a wide
-variety of databases: Common database systems MySQL, Postgres and SQLite and
-more exotic systems like SAP DB, SyBase and MSSQL. It only supports Python 2
+variety of databases: common database systems like MySQL, PostgreSQL, and SQLite and
+more exotic systems like SAP DB, SyBase, and Microsoft SQL Server. It only supports Python 2
from Python 2.6 upwards.
-.. There's no official information on this on their page, this information was gathered by looking at their source code
+.. There's no official information on this on their page; this information was gathered by looking at their source code.
diff --git a/docs/scenarios/gui.rst b/docs/scenarios/gui.rst
index 854fb826c..3ad0af364 100644
--- a/docs/scenarios/gui.rst
+++ b/docs/scenarios/gui.rst
@@ -1,92 +1,162 @@
+
+################
GUI Applications
-================
+################
+
+.. image:: /_static/photos/33907143624_cd621b535c_k_d.jpg
+
Alphabetical list of GUI Applications.
+
+*******
Camelot
--------
+*******
+
`Camelot `_ provides components for building
-applications on top of Python, SQLAlchemy and Qt. It is inspired by
-the Django admin interface.
+applications on top of Python, SQLAlchemy, and Qt. It is inspired by the Django
+admin interface.
The main resource for information is the website:
http://www.python-camelot.com
-and the mailing list https://groups.google.com/forum/#!forum/project-camelot
+and the mailing list https://groups.google.com/forum/#!forum/project-camelot.
+
+*****
Cocoa
------
+*****
+
.. note:: The Cocoa framework is only available on OS X. Don't pick this if you're writing a cross-platform application!
+
+***
GTk
----
-PyGTK provides Python bindings for the GTK+ toolkit. Like the GTK+ library
-itself, it is currently licensed under the GNU LGPL. It is worth noting that
-PyGTK only currently supports the Gtk-2.X API (NOT Gtk-3.0). It is currently
-recommended that PyGTK not be used for new projects and that existing
-applications be ported from PyGTK to PyGObject.
+***
+
+.. note:: PyGTK provides Python bindings for the GTK+ toolkit. However, it has been superseded by PyGObject. PyGTK should not be used for new projects and existing projects should be ported to PyGObject.
+
+********************
PyGObject aka (PyGi)
---------------------
-`PyGObject `_ provides Python bindings, which gives access to the entire GNOME software platform.
-It is fully compatible with GTK+ 3. Here is a tutorial to get started with `Python GTK+ 3 Tutorial `_.
+********************
+
+`PyGObject `_ provides Python
+bindings which gives access to the entire GNOME software platform. It is fully
+compatible with GTK+ 3. Here is a tutorial to get started with `Python GTK+ 3
+Tutorial `_.
`API Reference `_
+
+****
Kivy
-----
+****
+
`Kivy `_ is a Python library for development of multi-touch
enabled media rich applications. The aim is to allow for quick and easy
-interaction design and rapid prototyping, while making your code reusable
-and deployable.
+interaction design and rapid prototyping, while making your code reusable and
+deployable.
-Kivy is written in Python, based on OpenGL and supports different input devices
-such as: Mouse, Dual Mouse, TUIO, WiiMote, WM_TOUCH, HIDtouch, Apple's products
+Kivy is written in Python, based on OpenGL, and supports different input devices
+such as: Mouse, Dual Mouse, TUIO, WiiMote, WM_TOUCH, HIDtouch, Apple's products,
and so on.
Kivy is actively being developed by a community and is free to use. It operates
-on all major platforms (Linux, OSX, Windows, Android).
+on all major platforms (Linux, OS X, Windows, Android).
The main resource for information is the website: http://kivy.org
+
+******
PyObjC
-------
+******
+
.. note:: Only available on OS X. Don't pick this if you're writing a cross-platform application.
+
+******
PySide
-------
+******
+
PySide is a Python binding of the cross-platform GUI toolkit Qt.
+The package name depends on the major Qt version (`PySide` for Qt4,
+`PySide2` for Qt5, and `PySide6` for Qt6).
+This set of bindings is developed by `The Qt Company `_.
+
+.. code-block:: console
+
+ $ pip install pyside6
- pip install pyside
+https://pyside.org
-https://wiki.qt.io/Category:LanguageBindings::PySide::Downloads
+****
PyQt
-----
+****
+
.. note:: If your software does not fully comply with the GPL you will need a commercial license!
PyQt provides Python bindings for the Qt Framework (see below).
http://www.riverbankcomputing.co.uk/software/pyqt/download
-PyjamasDesktop (pyjs Desktop)
------------------------------
-PyjamasDesktop is a port of Pyjamas. PyjamasDesktop is application widget set
-for desktop and a cross-platform framework. (After release v0.6 PyjamasDesktop
-is a part of Pyjamas (Pyjs)). Briefly, it allows the exact same Python web
-application source code to be executed as a standalone desktop application.
-`Python Wiki for PyjamasDesktop `_.
+***************************************
+Pyjs Desktop (formerly Pyjamas Desktop)
+***************************************
+
+Pyjs Desktop is a application widget set for desktop and a cross-platform
+framework. It allows the exact same Python web application source code to be
+executed as a standalone desktop application.
+
-The main website; `pyjs Desktop `_.
+The main website: `pyjs `_.
+
+**
Qt
---
-`Qt `_ is a cross-platform application framework that
-is widely used for developing software with a GUI but can also be used for
-non-GUI applications.
+**
+
+`Qt `_ is a cross-platform application framework that is
+widely used for developing software with a GUI but can also be used for non-GUI
+applications.
+
+
+***********
+PySimpleGUI
+***********
+
+`PySimpleGUI `_ is a wrapper for Tkinter
+and Qt (others on the way). The amount of code required to implement custom
+GUIs is much shorter using PySimpleGUI than if the same GUI were written
+directly using Tkinter or Qt. PySimpleGUI code can be "ported" between GUI
+frameworks by changing import statements.
+
+.. code-block:: console
+
+ $ pip install pysimplegui
+PySimpleGUI is contained in a single PySimpleGUI.py file. Should pip
+installation be impossible, copying the PySimpleGUI.py file into a project's
+folder is all that's required to import and begin using.
+
+
+****
+Toga
+****
+
+`Toga `_ is a Python native, OS native,
+cross platform GUI toolkit. Toga consists of a library of base components with a
+shared interface to simplify platform-agnostic GUI development.
+
+Toga is available on macOS, Windows, Linux (GTK), and mobile platforms such as
+Android and iOS.
+
+
+**
Tk
---
+**
+
Tkinter is a thin object-oriented layer on top of Tcl/Tk. **It has the advantage
of being included with the Python standard library, making it the most
convenient and compatible toolkit to program with.**
@@ -95,18 +165,21 @@ Both Tk and Tkinter are available on most Unix platforms, as well as on Windows
and Macintosh systems. Starting with the 8.0 release, Tk offers native look and
feel on all platforms.
-There's a good multi-language Tk tutorial with Python examples at
-`TkDocs `_. There's more information
+There's a good multi-language Tk tutorial with Python examples at `TkDocs
+`_. There's more information
available on the `Python Wiki `_.
+
+********
wxPython
---------
-wxPython is a GUI toolkit for the Python programming language. It allows
-Python programmers to create programs with a robust, highly functional
-graphical user interface, simply and easily. It is implemented as a Python
-extension module (native code) that wraps the popular wxWidgets cross platform
-GUI library, which is written in C++.
+********
+
+wxPython is a GUI toolkit for the Python programming language. It allows Python
+programmers to create programs with a robust, highly functional graphical user
+interface, simply and easily. It is implemented as a Python extension module
+(native code) that wraps the popular wxWidgets cross platform GUI library, which
+is written in C++.
**Install (Stable) wxPython**
-*go to http://www.wxpython.org/download.php#stable and download the appropriate
+*go to https://www.wxpython.org/pages/downloads/ and download the appropriate
package for your OS.*
diff --git a/docs/scenarios/imaging.rst b/docs/scenarios/imaging.rst
index 23ecece49..8fe7e08fe 100644
--- a/docs/scenarios/imaging.rst
+++ b/docs/scenarios/imaging.rst
@@ -1,30 +1,35 @@
-==================
+
+##################
Image Manipulation
-==================
+##################
+
+.. image:: /_static/photos/34575689432_3de8e9a348_k_d.jpg
Most image processing and manipulation techniques can be carried out
-effectively using two libraries: Python Imaging Library (PIL) and OpenSource
+effectively using two libraries: Python Imaging Library (PIL) and Open Source
Computer Vision (OpenCV).
A brief description of both is given below.
+
+**********************
Python Imaging Library
-----------------------
+**********************
The `Python Imaging Library `_, or PIL
for short, is one of the core libraries for image manipulation in Python. Unfortunately,
its development has stagnated, with its last release in 2009.
Luckily for you, there's an actively-developed fork of PIL called
-`Pillow `_ - it's easier to install, runs on
-all operating systems, and supports Python 3.
+`Pillow `_ -- it's easier to install, runs on
+all major operating systems, and supports Python 3.
Installation
~~~~~~~~~~~~
Before installing Pillow, you'll have to install Pillow's prerequisites. Find
the instructions for your platform in the
-`Pillow installation instructions `_.
+`Pillow installation instructions `_.
After that, it's straightforward:
@@ -57,13 +62,14 @@ Example
exif_data
There are more examples of the Pillow library in the
-`Pillow tutorial `_.
+`Pillow tutorial `_.
-OpenSource Computer Vision
---------------------------
+***************************
+Open Source Computer Vision
+***************************
-OpenSource Computer Vision, more commonly known as OpenCV, is a more advanced
+Open Source Computer Vision, more commonly known as OpenCV, is a more advanced
image manipulation and processing software than PIL. It has been implemented
in several languages and is widely used.
@@ -87,8 +93,7 @@ Example
.. code-block:: python
- from cv2 import *
- import numpy as np
+ import cv2
#Read Image
img = cv2.imread('testimg.jpg')
#Display Image
@@ -104,4 +109,4 @@ Example
There are more Python-implemented examples of OpenCV in this `collection of
tutorials
-`_.
+`_.
diff --git a/docs/scenarios/json.rst b/docs/scenarios/json.rst
index b5145fc4c..1c3663777 100644
--- a/docs/scenarios/json.rst
+++ b/docs/scenarios/json.rst
@@ -1,12 +1,18 @@
+
+####
JSON
-====
+####
+
+.. image:: /_static/photos/33928819683_97b5c6a184_k_d.jpg
-The `json `_ library can parse
+The `json `_ library can parse
JSON from strings or files. The library parses JSON into a Python dictionary
or list. It can also convert Python dictionaries or lists into JSON strings.
+
+************
Parsing JSON
-------------
+************
Take the following string containing JSON data:
@@ -40,26 +46,3 @@ You can also convert the following to JSON:
print(json.dumps(d))
'{"first_name": "Guido", "last_name": "Rossum", "titles": ["BDFL", "Developer"]}'
-
-
-simplejson
-----------
-
-The JSON library was added to Python in version 2.6.
-If you're using an earlier version of Python, the
-`simplejson `_ library is
-available via PyPI.
-
-simplejson mimics the json standard library. It is available so that developers
-that use older versions of Python can use the latest features available in the
-json lib.
-
-You can start using simplejson when the json library is not available by
-importing simplejson under a different name:
-
-.. code-block:: python
-
- import simplejson as json
-
-After importing simplejson as json, the above examples will all work as if you
-were using the standard json library.
diff --git a/docs/scenarios/ml.rst b/docs/scenarios/ml.rst
new file mode 100644
index 000000000..b3655c819
--- /dev/null
+++ b/docs/scenarios/ml.rst
@@ -0,0 +1,125 @@
+
+
+################
+Machine Learning
+################
+
+.. image:: /_static/photos/34018729885_002ced9b54_k_d.jpg
+
+Python has a vast number of libraries for data analysis, statistics, and Machine Learning itself, making it a language of choice for many data scientists.
+
+Some widely used packages for Machine Learning and other data science applications are listed below.
+
+
+***********
+SciPy Stack
+***********
+
+The SciPy stack consists of a bunch of core helper packages used in data science for statistical analysis and visualising data. Because of its huge number of functionalities and ease of use, the Stack is considered a must-have for most data science applications.
+
+The Stack consists of the following packages (link to documentation given):
+
+1. `NumPy `_
+2. `SciPy library `_
+3. `Matplotlib `_
+4. `IPython `_
+5. `pandas `_
+6. `Sympy `_
+7. `nose `_
+
+The stack also comes with Python bundled in, but has been excluded from the above list.
+
+Installation
+~~~~~~~~~~~~
+
+For installing the full stack, or individual packages, you can refer to the instructions given `here `_.
+
+**NB:** `Anaconda `_ is highly preferred and recommended for installing and maintaining data science packages seamlessly.
+
+
+************
+scikit-learn
+************
+
+Scikit is a free and open source machine learning library for Python. It offers off-the-shelf functions to implement many algorithms like linear regression, classifiers, SVMs, k-means, Neural Networks, etc. It also has a few sample datasets which can be directly used for training and testing.
+
+Because of its speed, robustness, and ease of, it's one of the most widely-used libraries for many Machine Learning applications.
+
+Installation
+~~~~~~~~~~~~
+
+Through PyPI:
+
+.. code-block:: python
+
+ pip install -U scikit-learn
+
+Through conda:
+
+.. code-block:: python
+
+ conda install scikit-learn
+
+scikit-learn also comes shipped with Anaconda (mentioned above). For more installation instructions, refer to `this link `_.
+
+Example
+~~~~~~~
+
+For this example, we train a simple classifier on the `Iris dataset `_, which comes bundled in with scikit-learn.
+
+The dataset takes four features of flowers: sepal length, sepal width, petal length, and petal width, and classifies them into three flower species (labels): setosa, versicolor, or virginica. The labels have been represented as numbers in the dataset: 0 (setosa), 1 (versicolor), and 2 (virginica).
+
+We shuffle the Iris dataset and divide it into separate training and testing sets, keeping the last 10 data points for testing and rest for training. We then train the classifier on the training set and predict on the testing set.
+
+.. code-block:: python
+
+ from sklearn.datasets import load_iris
+ from sklearn import tree
+ from sklearn.metrics import accuracy_score
+ import numpy as np
+
+ #loading the iris dataset
+ iris = load_iris()
+
+ x = iris.data #array of the data
+ y = iris.target #array of labels (i.e answers) of each data entry
+
+ #getting label names i.e the three flower species
+ y_names = iris.target_names
+
+ #taking random indices to split the dataset into train and test
+ test_ids = np.random.permutation(len(x))
+
+ #splitting data and labels into train and test
+ #keeping last 10 entries for testing, rest for training
+
+ x_train = x[test_ids[:-10]]
+ x_test = x[test_ids[-10:]]
+
+ y_train = y[test_ids[:-10]]
+ y_test = y[test_ids[-10:]]
+
+ #classifying using decision tree
+ clf = tree.DecisionTreeClassifier()
+
+ #training (fitting) the classifier with the training set
+ clf.fit(x_train, y_train)
+
+ #predictions on the test dataset
+ pred = clf.predict(x_test)
+
+ print pred #predicted labels i.e flower species
+ print y_test #actual labels
+ print (accuracy_score(pred, y_test))*100 #prediction accuracy
+
+Since we're splitting randomly and the classifier trains on every iteration, the accuracy may vary. Running the above code gives:
+
+.. code-block:: python
+
+ [0 1 1 1 0 2 0 2 2 2]
+ [0 1 1 1 0 2 0 2 2 2]
+ 100.0
+
+The first line contains the labels (i.e. flower species) of the testing data as predicted by our classifier, and the second line contains the actual flower species as given in the dataset. We thus get an accuracy of 100% this time.
+
+More on scikit-learn can be read in the `documentation `_.
diff --git a/docs/scenarios/network.rst b/docs/scenarios/network.rst
index dd8bfa67d..bb751b78e 100644
--- a/docs/scenarios/network.rst
+++ b/docs/scenarios/network.rst
@@ -1,20 +1,28 @@
+
+##########
Networking
-==========
+##########
+
+.. image:: /_static/photos/34151833832_6bdfd930af_k_d.jpg
+
+*******
Twisted
--------
+*******
-`Twisted `_ is an event-driven networking
+`Twisted `_ is an event-driven networking
engine. It can be used to build applications around many different networking
-protocols, including http servers and clients, applications using SMTP, POP3,
-IMAP or SSH protocols, instant messaging
-and `much more `_.
+protocols, including HTTP servers and clients, applications using SMTP, POP3,
+IMAP, or SSH protocols, instant messaging,
+and `much more `_.
+
+*****
PyZMQ
------
+*****
-`PyZMQ `_ is the Python binding for
-`ZeroMQ `_, which is a high-performance asynchronous
+`PyZMQ `_ is the Python binding for
+`ZeroMQ `_, which is a high-performance asynchronous
messaging library. One great advantage of ZeroMQ is that it can be used for
message queuing without a message broker. The basic patterns for this are:
@@ -22,15 +30,17 @@ message queuing without a message broker. The basic patterns for this are:
remote procedure call and task distribution pattern.
- publish-subscribe: connects a set of publishers to a set of subscribers.
This is a data distribution pattern.
-- push-pull (or pipeline): connects nodes in a fan-out / fan-in pattern that
- can have multiple steps, and loops. This is a parallel task distribution
+- push-pull (or pipeline): connects nodes in a fan-out/fan-in pattern that
+ can have multiple steps and loops. This is a parallel task distribution
and collection pattern.
For a quick start, read the `ZeroMQ guide `_.
+
+******
gevent
-------
+******
`gevent `_ is a coroutine-based Python networking
library that uses greenlets to provide a high-level synchronous API on top of
-the libev event loop.
+the libev event loop.
diff --git a/docs/scenarios/scientific.rst b/docs/scenarios/scientific.rst
index 8600bc9b2..bb0547323 100644
--- a/docs/scenarios/scientific.rst
+++ b/docs/scenarios/scientific.rst
@@ -1,9 +1,14 @@
-=======================
+
+#######################
Scientific Applications
-=======================
+#######################
+
+.. image:: /_static/photos/33925223870_97e44f5629_k_d.jpg
+
+*******
Context
-:::::::
+*******
Python is frequently used for high-performance scientific applications. It
is widely used in academia and scientific projects because it is easy to write
@@ -11,15 +16,16 @@ and performs well.
Due to its high performance nature, scientific computing in Python often
utilizes external libraries, typically written in faster languages (like C, or
-FORTRAN for matrix operations). The main libraries used are `NumPy`_, `SciPy`_
+Fortran for matrix operations). The main libraries used are `NumPy`_, `SciPy`_
and `Matplotlib`_. Going into detail about these libraries is beyond the scope
of the Python guide. However, a comprehensive introduction to the scientific
Python ecosystem can be found in the `Python Scientific Lecture Notes
-`_
+`_.
+*****
Tools
-:::::
+*****
IPython
-------
@@ -29,19 +35,20 @@ which provides features of great interest to scientists. The `inline mode`
allows graphics and plots to be displayed in the terminal (Qt based version).
Moreover, the `notebook` mode supports literate programming and reproducible
science generating a web-based Python notebook. This notebook allows you to
-store chunks of Python code along side the results and additional comments
+store chunks of Python code alongside the results and additional comments
(HTML, LaTeX, Markdown). The notebook can then be shared and exported in various
file formats.
+*********
Libraries
-:::::::::
+*********
NumPy
-----
`NumPy `_ is a low level library written in C (and
-FORTRAN) for high level mathematical functions. NumPy cleverly overcomes the
+Fortran) for high level mathematical functions. NumPy cleverly overcomes the
problem of running slower algorithms on Python by using multidimensional arrays
and functions that operate on arrays. Any algorithm can then be expressed as a
function on arrays, allowing the algorithms to be run quickly.
@@ -50,7 +57,7 @@ NumPy is part of the SciPy project, and is released as a separate library so
people who only need the basic requirements can use it without installing the
rest of SciPy.
-NumPy is compatible with Python versions 2.4 through to 2.7.2 and 3.1+.
+NumPy is compatible with Python versions 2.4 through 2.7.2 and 3.1+.
Numba
-----
@@ -68,7 +75,7 @@ SciPy
functions. SciPy uses NumPy arrays as the basic data structure, and comes
with modules for various commonly used tasks in scientific programming,
including linear algebra, integration (calculus), ordinary differential equation
-solving and signal processing.
+solving, and signal processing.
Matplotlib
----------
@@ -77,24 +84,35 @@ Matplotlib
library for creating interactive 2D and 3D plots that can also be saved as
manuscript-quality figures. The API in many ways reflects that of `MATLAB
`_, easing transition of MATLAB
-users to Python. Many examples, along with the source code to re-create them,
+users to Python. Many examples, along with the source code to recreate them,
are available in the `matplotlib gallery
`_.
Pandas
------
-`Pandas `_ is data manipulation library
-based on Numpy which provides many useful functions for accessing,
-indexing, merging and grouping data easily. The main data structure (DataFrame)
+`Pandas `_ is a data manipulation library
+based on NumPy which provides many useful functions for accessing,
+indexing, merging, and grouping data easily. The main data structure (DataFrame)
is close to what could be found in the R statistical package; that is,
-heterogeneous data tables with name indexing, time series operations and
+heterogeneous data tables with name indexing, time series operations, and
auto-alignment of data.
+xarray
+------
+
+`xarray `_ is similar to Pandas, but it
+is intended for wrapping multidimensional scientific data. By labelling the
+data with dimensions, coordinates, and attributes, it makes complex
+multidimensional operations clearer and more intuitive. It also wraps
+matplotlib for quick plotting, and can apply most operations in parallel using
+`dask `_.
+
+
Rpy2
----
-`Rpy2 `_ is a Python binding for the R
+`Rpy2 `_ is a Python binding for the R
statistical package allowing the execution of R functions from Python and
passing data back and forth between the two environments. Rpy2 is the object
oriented implementation of the `Rpy `_
@@ -105,12 +123,13 @@ PsychoPy
`PsychoPy `_ is a library for cognitive scientists
allowing the creation of cognitive psychology and neuroscience experiments.
-The library handles presentation of stimuli, scripting of experimental design
+The library handles presentation of stimuli, scripting of experimental design,
and data collection.
+*********
Resources
-:::::::::
+*********
Installation of scientific Python packages can be troublesome, as many of
these packages are implemented as Python C extensions which need to be compiled.
@@ -123,7 +142,7 @@ Unofficial Windows Binaries for Python Extension Packages
Many people who do scientific computing are on Windows, yet many of the
scientific computing packages are notoriously difficult to build and install on
-this platform. `Christoph Gohlke `_
+this platform. `Christoph Gohlke `_,
however, has compiled a list of Windows binaries for many useful Python
packages. The list of packages has grown from a mainly scientific Python
resource to a more general list. If you're on Windows, you may want to check it
@@ -132,17 +151,16 @@ out.
Anaconda
--------
-`Continuum Analytics `_ offers the `Anaconda
-Python Distribution `_ which
+The `Anaconda Python Distribution `_
includes all the common scientific Python packages as well as many packages
-related to data analytics and big data. Anaconda itself is free, and
-Continuum sells a number of proprietary add-ons. Free licenses for the
+related to data analytics and big data. Anaconda itself is free, and a number
+of proprietary add-ons are available for a fee. Free licenses for the
add-ons are available for academics and researchers.
Canopy
------
-`Canopy `_ is another scientific
+`Canopy `_ is another scientific
Python distribution, produced by `Enthought `_.
A limited 'Canopy Express' variant is available for free, but Enthought
charges for the full distribution. Free licenses are available for academics.
diff --git a/docs/scenarios/scrape.rst b/docs/scenarios/scrape.rst
index 889a93082..527719200 100644
--- a/docs/scenarios/scrape.rst
+++ b/docs/scenarios/scrape.rst
@@ -1,26 +1,34 @@
+
+#############
HTML Scraping
-=============
+#############
+
+.. image:: /_static/photos/34268661876_442428e122_k_d.jpg
+
+************
Web Scraping
-------------
+************
Web sites are written using HTML, which means that each web page is a
structured document. Sometimes it would be great to obtain some data from
them and preserve the structure while we're at it. Web sites don't always
-provide their data in comfortable formats such as ``csv`` or ``json``.
+provide their data in comfortable formats such as CSV or JSON.
This is where web scraping comes in. Web scraping is the practice of using a
computer program to sift through a web page and gather the data that you need
in a format most useful to you while at the same time preserving the structure
of the data.
+
+*****************
lxml and Requests
------------------
+*****************
`lxml `_ is a pretty extensive library written for parsing
XML and HTML documents very quickly, even handling messed up tags in the
process. We will also be using the
-`Requests `_ module instead of the
+`Requests `_ module instead of the
already built-in urllib2 module due to improvements in speed and readability.
You can easily install both using ``pip install lxml`` and
``pip install requests``.
@@ -33,7 +41,7 @@ Let's start with the imports:
import requests
Next we will use ``requests.get`` to retrieve the web page with our data,
-parse it using the ``html`` module and save the results in ``tree``:
+parse it using the ``html`` module, and save the results in ``tree``:
.. code-block:: python
@@ -49,15 +57,15 @@ will focus on the former.
XPath is a way of locating information in structured documents such as
HTML or XML documents. A good introduction to XPath is on
-`W3Schools `_ .
+`W3Schools `_ .
There are also various tools for obtaining the XPath of elements such as
FireBug for Firefox or the Chrome Inspector. If you're using Chrome, you
can right click an element, choose 'Inspect element', highlight the code,
-right click again and choose 'Copy XPath'.
+right click again, and choose 'Copy XPath'.
After a quick analysis, we see that in our page the data is contained in
-two elements - one is a div with title 'buyer-name' and the other is a
+two elements -- one is a div with title 'buyer-name' and the other is a
span with class 'item-price':
.. code-block:: html
@@ -79,8 +87,8 @@ Let's see what we got exactly:
.. code-block:: python
- print 'Buyers: ', buyers
- print 'Prices: ', prices
+ print('Buyers: ', buyers)
+ print('Prices: ', prices)
::
diff --git a/docs/scenarios/serialization.rst b/docs/scenarios/serialization.rst
index ac0494f14..3edc6aaa2 100644
--- a/docs/scenarios/serialization.rst
+++ b/docs/scenarios/serialization.rst
@@ -1,40 +1,237 @@
-==================
+
+##################
Data Serialization
-==================
+##################
+.. image:: /_static/photos/33467946364_3e59bd376a_k_d.jpg
+
+
+***************************
What is data serialization?
----------------------------
+***************************
+
+Data serialization is the process of converting structured data to a format
+that allows sharing or storage of the data in a form that allows recovery of its original
+structure. In some cases, the secondary intention of data
+serialization is to minimize the data's size which then
+reduces disk space or bandwidth requirements.
+
+********************
+Flat vs. Nested data
+********************
+
+Before beginning to serialize data, it is important to identify or decide how the
+data should be structured during data serialization - flat or nested.
+The differences in the two styles are shown in the below examples.
+
+Flat style:
+
+.. code-block:: python
+
+ { "Type" : "A", "field1": "value1", "field2": "value2", "field3": "value3" }
+
+
+Nested style:
+
+.. code-block:: python
+
+ {"A"
+ { "field1": "value1", "field2": "value2", "field3": "value3" } }
+
+
+For more reading on the two styles, please see the discussion on
+`Python mailing list `__,
+`IETF mailing list `__ and
+`in stackexchange `__.
+
+****************
+Serializing Text
+****************
+
+=======================
+Simple file (flat data)
+=======================
+
+If the data to be serialized is located in a file and contains flat data, Python offers two methods to serialize data.
+
+repr
+----
+
+The repr method in Python takes a single object parameter and returns a printable representation of the input:
+
+.. code-block:: python
+
+ # input as flat text
+ a = { "Type" : "A", "field1": "value1", "field2": "value2", "field3": "value3" }
+
+ # the same input can also be read from a file
+ a = open('/tmp/file.py', 'r')
+
+ # returns a printable representation of the input;
+ # the output can be written to a file as well
+ print(repr(a))
+
+ # write content to files using repr
+ with open('/tmp/file.py') as f:f.write(repr(a))
+
+
+ast.literal_eval
+----------------
+
+The literal_eval method safely parses and evaluates an expression for a Python datatype.
+Supported data types are: strings, numbers, tuples, lists, dicts, booleans, and None.
+
+.. code-block:: python
+
+ with open('/tmp/file.py', 'r') as f: inp = ast.literal_eval(f.read())
+
+====================
+CSV file (flat data)
+====================
+
+The CSV module in Python implements classes to read and write tabular
+data in CSV format.
+
+Simple example for reading:
+
+.. code-block:: python
+
+ # Reading CSV content from a file
+ import csv
+ with open('/tmp/file.csv', newline='') as f:
+ reader = csv.reader(f)
+ for row in reader:
+ print(row)
+
+Simple example for writing:
+
+.. code-block:: python
-Data serialization is the concept of converting structured data into a format
-that allows it to be shared or stored in such a way that its original
-structure to be recovered. In some cases, the secondary intention of data
-serialization is to minimize the size of the serialized data which then
-minimizes disk space or bandwidth requirements.
+ # Writing CSV content to a file
+ import csv
+ with open('/temp/file.csv', 'w', newline='') as f:
+ writer = csv.writer(f)
+ writer.writerows(iterable)
-Pickle
-------
-The native data serialization module for Python is called `Pickle
-`_.
+The module's contents, functions, and examples can be found
+`in the Python documentation `__.
+
+==================
+YAML (nested data)
+==================
+
+There are many third party modules to parse and read/write YAML file
+structures in Python. One such example is below.
+
+.. code-block:: python
+
+ # Reading YAML content from a file using the load method
+ import yaml
+ with open('/tmp/file.yaml', 'r', newline='') as f:
+ try:
+ print(yaml.load(f))
+ except yaml.YAMLError as ymlexcp:
+ print(ymlexcp)
+
+Documentation on the third party module can be found
+`in the PyYAML Documentation `__.
+
+=======================
+JSON file (nested data)
+=======================
+
+Python's JSON module can be used to read and write JSON files.
+Example code is below.
+
+Reading:
+
+.. code-block:: python
+
+ # Reading JSON content from a file
+ import json
+ with open('/tmp/file.json', 'r') as f:
+ data = json.load(f)
+
+Writing:
+
+.. code-block:: python
+
+ # Writing JSON content to a file using the dump method
+ import json
+ with open('/tmp/file.json', 'w') as f:
+ json.dump(data, f, sort_keys=True)
+
+=================
+XML (nested data)
+=================
+
+XML parsing in Python is possible using the `xml` package.
+
+Example:
+
+.. code-block:: python
+
+ # reading XML content from a file
+ import xml.etree.ElementTree as ET
+ tree = ET.parse('country_data.xml')
+ root = tree.getroot()
+
+More documentation on using the `xml.dom` and `xml.sax` packages can be found
+`in the Python XML library documentation `__.
+
+
+*******
+Binary
+*******
+
+=======================
+NumPy Array (flat data)
+=======================
+
+Python's NumPy array can be used to serialize and deserialize data to and from byte representation.
+
+Example:
+
+.. code-block:: python
+
+ import NumPy as np
+
+ # Converting NumPy array to byte format
+ byte_output = np.array([ [1, 2, 3], [4, 5, 6], [7, 8, 9] ]).tobytes()
+
+ # Converting byte format back to NumPy array
+ array_format = np.frombuffer(byte_output)
+
+
+
+====================
+Pickle (nested data)
+====================
+
+The native data serialization module for Python is called `Pickle
+`_.
Here's an example:
.. code-block:: python
-
+
import pickle
-
+
#Here's an example dict
grades = { 'Alice': 89, 'Bob': 72, 'Charles': 87 }
-
+
#Use dumps to convert the object to a serialized string
serial_grades = pickle.dumps( grades )
-
- #Use loads to de-serialize an object
+
+ #Use loads to de-serialize an object
received_grades = pickle.loads( serial_grades )
+
+********
Protobuf
---------
+********
-If you're looking for a serialization module that has support in multiple
-languages, Google's `Protobuf
-`_ library is an option.
+If you're looking for a serialization module that has support in multiple
+languages, Google's `Protobuf
+`_ library is an option.
diff --git a/docs/scenarios/speed.rst b/docs/scenarios/speed.rst
index fc038c1c7..5dc0cd845 100644
--- a/docs/scenarios/speed.rst
+++ b/docs/scenarios/speed.rst
@@ -1,5 +1,9 @@
+
+#####
Speed
-=====
+#####
+
+.. image:: /_static/photos/33175625804_e225b90f3e_k_d.jpg
CPython, the most commonly used implementation of Python, is slow for CPU bound
tasks. `PyPy`_ is fast.
@@ -33,8 +37,10 @@ and PyPy's processing.
1.54693889618
1.60109114647
+
+*******
Context
-:::::::
+*******
The GIL
@@ -61,14 +67,16 @@ The GIL
`Special care`_ must be taken when writing C extensions to make sure you
register your threads with the interpreter.
+
+************
C Extensions
-::::::::::::
+************
Cython
------
-`Cython `_ implements a superset of the Python language
+`Cython `_ implements a superset of the Python language
with which you are able to write C and C++ modules for Python. Cython also
allows you to call functions from compiled C libraries. Using Cython allows
you to take advantage of Python's strong typing of variables and operations.
@@ -160,25 +168,23 @@ What's the difference in speed? Let's try it!
.. code-block:: python
import time
- #activate pyx compiler
+ # Activate pyx compiler
import pyximport
- pyximport.install()
- #primes implemented with Cython
- import primesCy
- #primes implemented with Python
- import primes
-
- print "Cython:"
- t1= time.time()
- print primesCy.primes(500)
- t2= time.time()
- print "Cython time: %s" %(t2-t1)
- print ""
- print "Python"
- t1= time.time()
- print primes.primes(500)
- t2= time.time()
- print "Python time: %s" %(t2-t1)
+ pyximport.install()
+ import primesCy # primes implemented with Cython
+ import primes # primes implemented with Python
+
+ print("Cython:")
+ t1 = time.time()
+ print(primesCy.primes(500))
+ t2 = time.time()
+ print("Cython time: %s" % (t2 - t1))
+ print("")
+ print("Python")
+ t1 = time.time()
+ print(primes.primes(500))
+ t2 = time.time()
+ print("Python time: %s" % (t2 - t1))
These lines both need a remark:
@@ -192,8 +198,8 @@ These lines both need a remark:
The `pyximport` module allows you to import :file:`*.pyx` files (e.g.,
:file:`primesCy.pyx`) with the Cython-compiled version of the `primes`
function. The `pyximport.install()` command allows the Python interpreter to
-start the Cython compiler directly to generate C-code, which is automatically
-compiled to a :file:`*.so` C-library. Cython is then able to import this
+start the Cython compiler directly to generate C code, which is automatically
+compiled to a :file:`*.so` C library. Cython is then able to import this
library for you in your Python code, easily and efficiently. With the
`time.time()` function you are able to compare the time between these 2
different calls to find 500 prime numbers. On a standard notebook (dual core
@@ -206,7 +212,7 @@ AMD E-450 1.6 GHz), the measured values are:
Python time: 0.0566 seconds
-And here the output of an embedded `ARM beaglebone `_ machine:
+And here is the output of an embedded `ARM beaglebone `_ machine:
.. code-block:: console
@@ -222,12 +228,10 @@ Pyrex
Shedskin?
---------
-Numba
------
-.. todo:: Write about Numba and the autojit compiler for NumPy
+***********
Concurrency
-:::::::::::
+***********
Concurrent.futures
@@ -236,18 +240,18 @@ Concurrent.futures
The `concurrent.futures`_ module is a module in the standard library that
provides a "high-level interface for asynchronously executing callables". It
abstracts away a lot of the more complicated details about using multiple
-threads or processes for concurrency, and allows the user to focus on
+threads or processes for concurrency, and allows the user to focus on
accomplishing the task at hand.
The `concurrent.futures`_ module exposes two main classes, the
`ThreadPoolExecutor` and the `ProcessPoolExecutor`. The ThreadPoolExecutor
will create a pool of worker threads that a user can submit jobs to. These jobs
will then be executed in another thread when the next worker thread becomes
-available.
+available.
The ProcessPoolExecutor works in the same way, except instead of using multiple
threads for its workers, it will use multiple processes. This makes it possible
-to side-step the GIL, however because of the way things are passed to worker
+to side-step the GIL; however, because of the way things are passed to worker
processes, only picklable objects can be executed and returned.
Because of the way the GIL works, a good rule of thumb is to use a
@@ -258,7 +262,7 @@ executor when the task is computationally expensive.
There are two main ways of executing things in parallel using the two
Executors. One way is with the `map(func, iterables)` method. This works
almost exactly like the builtin `map()` function, except it will execute
-everything in parallel. :
+everything in parallel.
.. code-block:: python
@@ -277,7 +281,7 @@ everything in parallel. :
# Do something with the result
print(page.text)
-For even more control, the `submit(func, *args, **kwargs)` method will schedule
+For even more control, the `submit(func, *args, **kwargs)` method will schedule
a callable to be executed ( as `func(*args, **kwargs)`) and returns a `Future`_
object that represents the execution of the callable.
@@ -298,7 +302,7 @@ result()
the scheduled callable returns by default.
exception()
Return the exception raised by the call. If no exception was raised then
- this returns `None`. Note that this will block just like `result()`.
+ this returns None. Note that this will block just like `result()`.
add_done_callback(fn)
Attach a callback function that will be executed (as `fn(future)`) when the
scheduled callable returns.
@@ -351,14 +355,14 @@ futures provided have completed.
For more information, on using the `concurrent.futures`_ module, consult the
official documentation.
-Threading
+threading
---------
The standard library comes with a `threading`_ module that allows a user to
work with multiple threads manually.
Running a function in another thread is as simple as passing a callable and
-it's arguments to `Thread`'s constructor and then calling `start()`:
+its arguments to `Thread`'s constructor and then calling `start()`:
.. code-block:: python
@@ -391,9 +395,9 @@ still alive (because the join call timed out):
Because multiple threads have access to the same section of memory, sometimes
there might be situations where two or more threads are trying to write to the
same resource at the same time or where the output is dependent on the sequence
-or timing of certain events. This is called a `data race`_ or race condition.
+or timing of certain events. This is called a `data race`_ or race condition.
When this happens, the output will be garbled or you may encounter problems
-which are difficult to debug. A good example is this `stackoverflow post`_.
+which are difficult to debug. A good example is this `Stack Overflow post`_.
The way this can be avoided is by using a `Lock`_ that each thread needs to
acquire before writing to a shared resource. Locks can be acquired and released
@@ -414,7 +418,7 @@ through either the contextmanager protocol (`with` statement), or by using
def monitor_website(some_website):
"""
- Monitor a website and then if there are any changes,
+ Monitor a website and then if there are any changes,
log them to disk.
"""
while True:
@@ -431,7 +435,7 @@ Here, we have a bunch of threads checking for changes on a list of sites and
whenever there are any changes, they attempt to write those changes to a file
by calling `log(changes)`. When `log()` is called, it will wait to acquire
the lock with `with file_lock:`. This ensures that at any one time, only one
-thread is writing to the file.
+thread is writing to the file.
Spawning Processes
------------------
@@ -442,13 +446,14 @@ Multiprocessing
.. _`PyPy`: http://pypy.org
-.. _`The GIL`: http://wiki.python.org/moin/GlobalInterpreterLock
+.. _`The GIL`: https://wiki.python.org/moin/GlobalInterpreterLock
.. _`guide`: http://www.dabeaz.com/python/UnderstandingGIL.pdf
.. _`New GIL`: http://www.dabeaz.com/python/NewGIL.pdf
-.. _`Special care`: http://docs.python.org/c-api/init.html#threads
+.. _`Special care`: https://docs.python.org/c-api/init.html#threads
.. _`David Beazley's`: http://www.dabeaz.com/GIL/gilvis/measure2.py
.. _`concurrent.futures`: https://docs.python.org/3/library/concurrent.futures.html
.. _`Future`: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future
.. _`threading`: https://docs.python.org/3/library/threading.html
-.. _`stackoverflow post`: http://stackoverflow.com/questions/26688424/python-threads-are-printing-at-the-same-time-messing-up-the-text-output
+.. _`Stack Overflow post`: https://stackoverflow.com/questions/26688424/python-threads-are-printing-at-the-same-time-messing-up-the-text-output
.. _`data race`: https://en.wikipedia.org/wiki/Race_condition
+.. _`Lock`: https://docs.python.org/3/library/threading.html#lock-objects
diff --git a/docs/scenarios/web.rst b/docs/scenarios/web.rst
index 601e880a1..a2c84818b 100644
--- a/docs/scenarios/web.rst
+++ b/docs/scenarios/web.rst
@@ -1,13 +1,19 @@
-================
-Web Applications
-================
+
+#############################
+Web Applications & Frameworks
+#############################
+
+.. image:: /_static/photos/34309496175_b82d104282_k_d.jpg
As a powerful scripting language adapted to both fast prototyping
and bigger projects, Python is widely used in web application
development.
+
+*******
Context
-:::::::
+*******
+
WSGI
@@ -21,8 +27,9 @@ can be deployed in any :ref:`WSGI-compliant web server `.
WSGI is documented in :pep:`3333`.
+**********
Frameworks
-::::::::::
+**********
Broadly speaking, a web framework consists of a set of libraries and a main
handler within which you can build custom code to implement a web application
@@ -34,7 +41,7 @@ URL Routing
be invoked
Request and Response Objects
- Encapsulate the information received from or sent to a user's browser
+ Encapsulates the information received from or sent to a user's browser
Template Engine
Allows for separating Python code implementing an application's logic from
@@ -48,7 +55,7 @@ Development Web Server
Django
------
-`Django `_ is a "batteries included" web
+`Django `_ is a "batteries included" web
application framework, and is an excellent choice for creating content-oriented
websites. By providing many utilities and patterns out of the box, Django aims
to make it possible to build complex, database-backed web applications quickly,
@@ -59,7 +66,7 @@ modules `_ that can be incorporated into a new
project as-is, or customized to fit your needs.
There are annual Django conferences `in the United States
-`_ and `in Europe `_.
+`_, `Europe `_, and `Australia `_.
The majority of new Python web applications today are built with Django.
@@ -88,11 +95,24 @@ suit your needs. Or, you can easily use any library you want yourself!
Flask is default choice for any Python web application that isn't a good
fit for Django.
+Falcon
+------
+
+`Falcon `_ is a good choice when your goal is
+to build RESTful API microservices that are fast and scalable.
+
+It is a reliable, high-performance Python web framework for building large-scale
+app backends and microservices. Falcon encourages the REST architectural style of
+mapping URIs to resources, trying to do as little as possible while remaining highly effective.
+
+Falcon highlights four main focuses: speed, reliability, flexibility, and debuggability.
+It implements HTTP through "responders" such as ``on_get()``, ``on_put()``, etc.
+These responders receive intuitive request and response objects.
Tornado
--------
-`Tornado `_ is an asyncronous web framework
+`Tornado `_ is an asynchronous web framework
for Python that has its own event loop. This allows it to natively support
WebSockets, for example. Well-written Tornado applications are known to
have excellent performance characteristics.
@@ -104,14 +124,48 @@ Pyramid
`Pyramid `_ is a very flexible framework with a heavy
focus on modularity. It comes with a small number of libraries ("batteries")
-built-in, and encourages users to extend its base functionality.
+built-in, and encourages users to extend its base functionality. A set of
+provided cookiecutter templates helps making new project decisions for users.
+It powers one of the most important parts of python infrastructure
+`PyPI `_.
Pyramid does not have a large user base, unlike Django and Flask. It's a
capable framework, but not a very popular choice for new Python web
applications today.
+Masonite
+--------
+
+`Masonite `_ is a modern and developer centric, "batteries included", web framework.
+
+The Masonite framework follows the MVC (Model-View-Controller) architecture pattern and is heavily inspired by frameworks such as Rails and Laravel, so if you are coming to Python from a Ruby or PHP background then you will feel right at home!
+
+Masonite comes with a lot of functionality out of the box including a powerful IOC container with auto resolving dependency injection, craft command line tools, and the Orator active record style ORM.
+
+Masonite is perfect for beginners or experienced developers alike and works hard to be fast and easy from install through to deployment. Try it once and you’ll fall in love.
+
+FastAPI
+-------
+
+`FastAPI `_ is a modern web framework for building
+APIs with Python 3.6+.
+
+It has very high performance as it is based on `Starlette `_
+and `Pydantic `_.
+
+FastAPI takes advantage of standard Python type declarations in function parameters
+to declare request parameters and bodies, perform data conversion (serialization,
+parsing), data validation, and automatic API documentation with **OpenAPI 3**
+(including **JSON Schema**).
+
+It includes tools and utilities for security and authentication (including OAuth2 with JWT
+tokens), a dependency injection system, automatic generation of interactive API
+documentation, and other features.
+
+
+***********
Web Servers
-:::::::::::
+***********
.. _nginx-ref:
@@ -119,7 +173,7 @@ Nginx
-----
`Nginx `_ (pronounced "engine-x") is a web server and
-reverse-proxy for HTTP, SMTP and other protocols. It is known for its
+reverse-proxy for HTTP, SMTP, and other protocols. It is known for its
high performance, relative simplicity, and compatibility with many
application servers (like WSGI servers). It also includes handy features
like load-balancing, basic authentication, streaming, and others. Designed
@@ -128,20 +182,22 @@ to serve high-load websites, Nginx is gradually becoming quite popular.
.. _wsgi-servers-ref:
+
+************
WSGI Servers
-::::::::::::
+************
Stand-alone WSGI servers typically use less resources than traditional web
-servers and provide top performance [3]_.
+servers and provide top performance [1]_.
.. _gunicorn-ref:
Gunicorn
--------
-`Gunicorn `_ (Green Unicorn) is a pure-python WSGI
+`Gunicorn `_ (Green Unicorn) is a pure-Python WSGI
server used to serve Python applications. Unlike other Python web servers,
-it has a thoughtful user-interface, and is extremely easy to use and
+it has a thoughtful user interface, and is extremely easy to use and
configure.
Gunicorn has sane and reasonable defaults for configurations. However, some
@@ -154,7 +210,7 @@ Gunicorn is the recommended choice for new Python web applications today.
Waitress
--------
-`Waitress `_ is a pure-python WSGI server
+`Waitress `_ is a pure-Python WSGI server
that claims "very acceptable performance". Its documentation is not very
detailed, but it does offer some nice functionality that Gunicorn doesn't have
(e.g. HTTP request buffering).
@@ -166,26 +222,27 @@ Waitress is gaining popularity within the Python web development community.
uWSGI
-----
-`uWSGI `_ is a full stack for building
+`uWSGI `_ is a full stack for building
hosting services. In addition to process management, process monitoring,
and other functionality, uWSGI acts as an application server for various
-programming languages and protocols - including Python and WSGI. uWSGI can
+programming languages and protocols -- including Python and WSGI. uWSGI can
either be run as a stand-alone web router, or be run behind a full web
server (such as Nginx or Apache). In the latter case, a web server can
configure uWSGI and an application's operation over the
-`uwsgi protocol `_.
+`uwsgi protocol `_.
uWSGI's web server support allows for dynamically configuring
-Python, passing environment variables and further tuning. For full details,
+Python, passing environment variables, and further tuning. For full details,
see `uWSGI magic
-variables `_.
+variables `_.
I do not recommend using uWSGI unless you know why you need it.
.. _server-best-practices-ref:
+*********************
Server Best Practices
-:::::::::::::::::::::
+*********************
The majority of self-hosted Python applications today are hosted with a WSGI
server such as :ref:`Gunicorn `, either directly or behind a
@@ -195,8 +252,10 @@ The WSGI servers serve the Python applications while the web server handles
tasks better suited for it such as static file serving, request routing, DDoS
protection, and basic authentication.
+
+*******
Hosting
-:::::::
+*******
Platform-as-a-Service (PaaS) is a type of cloud computing infrastructure
which abstracts and manages infrastructure, routing, and scaling of web
@@ -207,39 +266,27 @@ details.
Heroku
------
-`Heroku `_ offers first-class support for
+`Heroku `_ offers first-class support for
Python 2.7–3.5 applications.
Heroku supports all types of Python web applications, servers, and frameworks.
Applications can be developed on Heroku for free. Once your application is
ready for production, you can upgrade to a Hobby or Professional application.
-Heroku maintains `detailed articles `_
+Heroku maintains `detailed articles `_
on using Python with Heroku, as well as `step-by-step instructions
`_ on
how to set up your first application.
Heroku is the recommended PaaS for deploying Python web applications today.
-Gondor
-------
-
-`Gondor `_ is a PaaS specialized for deploying Django
-and Pinax applications. Gondor recommends Django version 1.6 and supports any
-WSGI application on Python version 2.7. Gondor can automatically configure your
-Django site if you use :file:`local_settings.py` for site-specific configuration
-information.
-
-Gondor has a guide on deploying `Django projects `_.
-
-Gondor is run by a small company and focuses on helping businesses find success with
-Python and Django.
+**********
Templating
-::::::::::
+**********
Most WSGI applications are responding to HTTP requests to serve content in HTML
-or other markup languages. Instead of generating directly textual content from
+or other markup languages. Instead of directly generating textual content from
Python, the concept of separation of concerns advises us to use templates. A
template engine manages a suite of template files, with a system of hierarchy
and inclusion to avoid unnecessary repetition, and is in charge of rendering
@@ -277,10 +324,10 @@ Jinja2
`Jinja2 `_ is a very well-regarded template engine.
It uses a text-based template language and can thus be used to generate any
-type markup, not just HTML. It allows customization of filters, tags, tests
+type of markup, not just HTML. It allows customization of filters, tags, tests,
and globals. It features many improvements over Django's templating system.
-Here some important html tags in Jinja2:
+Here some important HTML tags in Jinja2:
.. code-block:: html
@@ -300,7 +347,7 @@ Here some important html tags in Jinja2:
{% endfor %}
-The next listings is an example of a web site in combination with the Tornado
+The next listings are an example of a web site in combination with the Tornado
web server. Tornado is not very complicated to use.
.. code-block:: python
@@ -372,7 +419,7 @@ into the corresponding block in the :file:`base.html` page.
.. code-block:: html
-
@@ -393,17 +440,17 @@ Jinja2 is the recommended templating library for new Python web applications.
Chameleon
---------
-`Chameleon `_ Page Templates are an HTML/XML template
-engine implementation of the `Template Attribute Language (TAL) `_,
-`TAL Expression Syntax (TALES) `_,
-and `Macro Expansion TAL (Metal) `_ syntaxes.
+`Chameleon `_ Page Templates are an HTML/XML template
+engine implementation of the `Template Attribute Language (TAL) `_,
+`TAL Expression Syntax (TALES) `_,
+and `Macro Expansion TAL (Metal) `_ syntaxes.
-Chameleon is available for Python 2.5 and up (including 3.x and pypy), and
-is commonly used by the `Pyramid Framework `_.
+Chameleon is available for Python 2.5 and up (including 3.x and PyPy), and
+is commonly used by the `Pyramid Framework `_.
Page Templates add within your document structure special element attributes
and text markup. Using a set of simple language constructs, you control the
-document flow, element repetition, text replacement and translation. Because
+document flow, element repetition, text replacement, and translation. Because
of the attribute-based syntax, unrendered page templates are valid HTML and can
be viewed in a browser and even edited in WYSIWYG editors. This can make
round-trip collaboration with designers and prototyping with static files in a
@@ -457,14 +504,14 @@ Mako
----
`Mako `_ is a template language that compiles to Python
-for maximum performance. Its syntax and api is borrowed from the best parts of other
+for maximum performance. Its syntax and API are borrowed from the best parts of other
templating languages like Django and Jinja2 templates. It is the default template
language included with the `Pylons and Pyramid `_ web
frameworks.
An example template in Mako looks like:
-.. code-block:: html
+.. code-block:: mako
<%inherit file="base.html"/>
<%
@@ -495,6 +542,4 @@ Mako is well respected within the Python web community.
.. rubric:: References
-.. [1] `The mod_python project is now officially dead `_
-.. [2] `mod_wsgi vs mod_python `_
-.. [3] `Benchmark of Python WSGI Servers `_
+.. [1] `Benchmark of Python WSGI Servers `_
diff --git a/docs/scenarios/xml.rst b/docs/scenarios/xml.rst
index 0cc3cdc4c..24dab5869 100644
--- a/docs/scenarios/xml.rst
+++ b/docs/scenarios/xml.rst
@@ -1,8 +1,14 @@
+
+###########
XML parsing
-===========
+###########
+
+.. image:: /_static/photos/33888714601_a1f7d020a2_k_d.jpg
+
+********
untangle
---------
+********
`untangle `_ is a simple library which
takes an XML document and returns a Python object which mirrors the nodes and
@@ -24,18 +30,20 @@ can be loaded like this:
import untangle
obj = untangle.parse('path/to/file.xml')
-and then you can get the child elements name like this:
+and then you can get the child element's name attribute like this:
.. code-block:: python
obj.root.child['name']
-untangle also supports loading XML from a string or an URL.
+untangle also supports loading XML from a string or a URL.
+
+*********
xmltodict
----------
+*********
-`xmltodict `_ is another simple
+`xmltodict `_ is another simple
library that aims at making XML feel like working with JSON.
An XML file like this:
@@ -61,7 +69,7 @@ can be loaded into a Python dict like this:
with open('path/to/file.xml') as fd:
doc = xmltodict.parse(fd.read())
-and then you can access elements, attributes and values like this:
+and then you can access elements, attributes, and values like this:
.. code-block:: python
@@ -71,5 +79,32 @@ and then you can access elements, attributes and values like this:
doc['mydocument']['plus']['#text'] # == u'element as well'
xmltodict also lets you roundtrip back to XML with the unparse function,
-has a streaming mode suitable for handling files that don't fit in memory
-and supports namespaces.
+has a streaming mode suitable for handling files that don't fit in memory,
+and supports XML namespaces.
+
+**********
+xmlschema
+**********
+
+`xmlschema `_ provides support for using XSD-Schemas in Python.
+Unlike other XML libraries, automatic type parsing is available, so f.e. if the schema defines an element to be of type ``int``, the parsed ``dict`` will contain also an ``int`` value for that element.
+Moreover the library supports automatic and explicit validation of XML documents against a schema.
+
+.. code-block:: python
+
+ from xmlschema import XMLSchema, etree_tostring
+
+ # load a XSD schema file
+ schema = XMLSchema("your_schema.xsd")
+
+ # validate against the schema
+ schema.validate("your_file.xml")
+
+ # or
+ schema.is_valid("your_file.xml")
+
+ # decode a file
+ data = schmema.decode("your_file.xml")
+
+ # encode to string
+ s = etree_tostring(schema.encode(data))
diff --git a/docs/shipping/freezing.rst b/docs/shipping/freezing.rst
index d42f2d61b..841fa2040 100644
--- a/docs/shipping/freezing.rst
+++ b/docs/shipping/freezing.rst
@@ -1,31 +1,36 @@
.. _freezing-your-code-ref:
-==================
+
+##################
Freezing Your Code
-==================
+##################
+
+.. image:: /_static/photos/33907151034_e0a9e53402_k_d.jpg
-"Freezing" your code is creating a single-file executable file to distribute
-to end-users, that contains all of your application code as well as the
+"Freezing" your code is creating a single-file executable file to distribute
+to end-users, that contains all of your application code as well as the
Python interpreter.
-Applications such as 'Dropbox', 'Eve Online', 'Civilisation IV', and
+Applications such as 'Dropbox', 'Eve Online', 'Civilization IV', and
BitTorrent clients do this.
The advantage of distributing this way is that your application will "just work",
-even if the user doesn't already have the required version of Python (or any)
+even if the user doesn't already have the required version of Python (or any)
installed. On Windows, and even on many Linux distributions and OS X, the right
version of Python will not already be installed.
-Besides, end-user software should always be in an executable format. Files
-ending in ``.py`` are for software engineers and system administrators.
+Besides, end-user software should always be in an executable format. Files
+ending in ``.py`` are for software engineers and system administrators.
+
+One disadvantage of freezing is that it will increase the size of your
+distribution by about 2–12 MB. Also, you will be responsible for shipping
+updated versions of your application when security vulnerabilities to
+Python are patched.
-One disadvantage of freezing is that it will increase the size of your
-distribution by about 2–12MB. Also, you will be responsible for shipping
-updated versions of your application when security vulnerabilities to
-Python are patched.
+************************
Alternatives to Freezing
-------------------------
+************************
:ref:`Packaging your code ` is for distributing
libraries or tools to other developers.
@@ -37,45 +42,88 @@ On Linux, an alternative to freezing is to
.. todo:: Fill in "Freezing Your Code" stub
+****************************
Comparison of Freezing Tools
-----------------------------
+****************************
+Date of this writing: Oct 5, 2019
Solutions and platforms/features supported:
-=========== ======= ===== ==== ======== ======= ============= ============== ==== =====================
-Solution Windows Linux OS X Python 3 License One-file mode Zipfile import Eggs pkg_resources support
-=========== ======= ===== ==== ======== ======= ============= ============== ==== =====================
-bbFreeze yes yes yes no MIT no yes yes yes
-py2exe yes no no yes MIT yes yes no no
-pyInstaller yes yes yes yes GPL yes no yes no
-cx_Freeze yes yes yes yes PSF no yes yes no
-py2app no no yes yes MIT no yes yes yes
-=========== ======= ===== ==== ======== ======= ============= ============== ==== =====================
+=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== =====================
+Solution Windows Linux OS X Python 3 License One-file mode Zipfile import Eggs pkg_resources support Latest release date
+=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== =====================
+bbFreeze yes yes yes no MIT no yes yes yes Jan 20, 2014
+py2exe yes no no yes MIT yes yes no no Oct 21, 2014
+pyInstaller yes yes yes yes GPL yes no yes no Jul 9, 2019
+cx_Freeze yes yes yes yes PSF no yes yes no Aug 29, 2019
+py2app no no yes yes MIT no yes yes yes Mar 25, 2019
+=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== =====================
.. note::
Freezing Python code on Linux into a Windows executable was only once
- supported in PyInstaller `and later dropped.
- `_.
+ supported in PyInstaller `and later dropped
+ `_.
.. note::
- All solutions need MS Visual C++ dll to be installed on target machine, except py2app.
- Only Pyinstaller makes self-executable exe that bundles the dll when
- passing :option:`--onefile` to :file:`Configure.py`.
+ All solutions need a Microsoft Visual C++ to be installed on the target machine, except py2app.
+ Only PyInstaller makes a self-executable exe that bundles the appropriate DLL when
+ passing ``--onefile`` to :file:`Configure.py`.
+
+*******
Windows
--------
+*******
bbFreeze
~~~~~~~~
Prerequisite is to install :ref:`Python, Setuptools and pywin32 dependency on Windows `.
-.. todo:: Write steps for most basic .exe
+1. Install :code:`bbfreeze`:
+
+.. code-block:: console
+
+ $ pip install bbfreeze
+
+2. Write most basic :file:`bb_setup.py`
+
+.. code-block:: python
+
+ from bbfreeze import Freezer
+
+ freezer = Freezer(distdir='dist')
+ freezer.addScript('foobar.py', gui_only=True)
+ freezer()
+
+.. note::
+
+ This will work for the most basic one file scripts. For more advanced freezing you will have to provide
+ include and exclude paths like so:
+
+ .. code-block:: python
+
+ freezer = Freezer(distdir='dist', includes=['my_code'], excludes=['docs'])
+
+3. (Optionally) include icon
+
+.. code-block:: python
+
+ freezer.setIcon('my_awesome_icon.ico')
+
+4. Provide the Microsoft Visual C++ runtime DLL for the freezer. It might be possible to append your :code:`sys.path`
+with the Microsoft Visual Studio path but I find it easier to drop :file:`msvcp90.dll` in the same folder where your script
+resides.
+
+5. Freeze!
+
+.. code-block:: console
+
+ $ python bb_setup.py
py2exe
~~~~~~
-Prerequisite is to install :ref:`Python on Windows `.
+Prerequisite is to install :ref:`Python on Windows `. The last release of py2exe is from the year 2014. There is not active development.
1. Download and install http://sourceforge.net/projects/py2exe/files/py2exe/
@@ -92,7 +140,7 @@ Prerequisite is to install :ref:`Python on Windows `.
3. (Optionally) `include icon