- This opinionated guide exists to provide both novice and expert Python developers a best-practice handbook to the installation, configuration, and usage of Python on a daily basis.
+
- If you enjoy this guide, consider supporting the author on Gittip:
-
+
+
+
+
+
+
+
-
+ This opinionated guide exists to provide both novice and expert Python developers a best practice handbook to the installation, configuration, and usage of Python on a daily basis.
-
Feedback
+
+
+
+
+
+
O'Reilly Book
+
+
This guide is now available in tangible book form!
+
+
+
+
All proceeds are being directly donated to the DjangoGirls organization.
+
+
Contributors
- Feedback is greatly appreciated. If you have any questions, comments,
- random praise, or anonymous threats,
- shoot me an email.
+ This guide is the result of the collaboration of
+ hundreds of people
+ around the world, and your contributions
+ are welcome!
- This opinionated guide exists to provide both novice and expert Python developers a best-practice handbook to the installation, configuration, and usage of Python on a daily basis.
+
- If you enjoy this guide, consider supporting the author on Gittip:
+ This opinionated guide exists to provide both novice and expert Python developers a best practice handbook to the installation, configuration, and usage of Python on a daily basis.
-
-
-
\ No newline at end of file
+
+
+
+
+
+
+
O'Reilly Book
+
+
This guide is now available in tangible book form!
+
+
+
+
All proceeds are being directly donated to the DjangoGirls organization.
diff --git a/docs/_themes/README.rst b/docs/_themes/README.rst
deleted file mode 100644
index 2e875d46e..000000000
--- a/docs/_themes/README.rst
+++ /dev/null
@@ -1,25 +0,0 @@
-krTheme Sphinx Style
-====================
-
-This repository contains sphinx styles Kenneth Reitz uses in most of
-his projects. It is a derivative of Mitsuhiko's themes for Flask and Flask related
-projects. To use this style in your Sphinx documentation, follow
-this guide:
-
-1. put this folder as _themes into your docs folder. Alternatively
- you can also use git submodules to check out the contents there.
-
-2. add this to your conf.py: ::
-
- sys.path.append(os.path.abspath('_themes'))
- html_theme_path = ['_themes']
- html_theme = 'flask'
-
-The following themes exist:
-
-**kr**
- the standard flask documentation theme for large projects
-
-**kr_small**
- small one-page theme. Intended to be used by very small addon libraries.
-
diff --git a/docs/_themes/kr/layout.html b/docs/_themes/kr/layout.html
deleted file mode 100644
index 54f270d0b..000000000
--- a/docs/_themes/kr/layout.html
+++ /dev/null
@@ -1,50 +0,0 @@
-{%- extends "basic/layout.html" %}
-{%- block extrahead %}
- {{ super() }}
- {% if theme_touch_icon %}
-
- {% endif %}
-
-{% endblock %}
-{%- block relbar2 %}{% endblock %}
-{%- block footer %}
-
-
-
-
-
-
-
-
-
-
-
-{%- endblock %}
diff --git a/docs/_themes/kr/relations.html b/docs/_themes/kr/relations.html
deleted file mode 100644
index 3bbcde85b..000000000
--- a/docs/_themes/kr/relations.html
+++ /dev/null
@@ -1,19 +0,0 @@
-
- {% endif %}
-{% endblock %}
-{# do not display relbars #}
-{% block relbar1 %}{% endblock %}
-{% block relbar2 %}
- {% if theme_github_fork %}
-
- {% endif %}
-{% endblock %}
-{% block sidebar1 %}{% endblock %}
-{% block sidebar2 %}{% endblock %}
diff --git a/docs/_themes/kr_small/static/flasky.css_t b/docs/_themes/kr_small/static/flasky.css_t
deleted file mode 100644
index fe2141c56..000000000
--- a/docs/_themes/kr_small/static/flasky.css_t
+++ /dev/null
@@ -1,287 +0,0 @@
-/*
- * flasky.css_t
- * ~~~~~~~~~~~~
- *
- * Sphinx stylesheet -- flasky theme based on nature theme.
- *
- * :copyright: Copyright 2007-2010 by the Sphinx team, see AUTHORS.
- * :license: BSD, see LICENSE for details.
- *
- */
-
-@import url("http://github.com/basic.css");
-
-/* -- page layout ----------------------------------------------------------- */
-
-body {
- font-family: 'Georgia', serif;
- font-size: 17px;
- color: #000;
- background: white;
- margin: 0;
- padding: 0;
-}
-
-div.documentwrapper {
- float: left;
- width: 100%;
-}
-
-div.bodywrapper {
- margin: 40px auto 0 auto;
- width: 700px;
-}
-
-hr {
- border: 1px solid #B1B4B6;
-}
-
-div.body {
- background-color: #ffffff;
- color: #3E4349;
- padding: 0 30px 30px 30px;
-}
-
-img.floatingflask {
- padding: 0 0 10px 10px;
- float: right;
-}
-
-div.footer {
- text-align: right;
- color: #888;
- padding: 10px;
- font-size: 14px;
- width: 650px;
- margin: 0 auto 40px auto;
-}
-
-div.footer a {
- color: #888;
- text-decoration: underline;
-}
-
-div.related {
- line-height: 32px;
- color: #888;
-}
-
-div.related ul {
- padding: 0 0 0 10px;
-}
-
-div.related a {
- color: #444;
-}
-
-/* -- body styles ----------------------------------------------------------- */
-
-a {
- color: #004B6B;
- text-decoration: underline;
-}
-
-a:hover {
- color: #6D4100;
- text-decoration: underline;
-}
-
-div.body {
- padding-bottom: 40px; /* saved for footer */
-}
-
-div.body h1,
-div.body h2,
-div.body h3,
-div.body h4,
-div.body h5,
-div.body h6 {
- font-family: 'Garamond', 'Georgia', serif;
- font-weight: normal;
- margin: 30px 0px 10px 0px;
- padding: 0;
-}
-
-{% if theme_index_logo %}
-div.indexwrapper h1 {
- text-indent: -999999px;
- background: url(http://github.com/%7B%7B%20theme_index_logo%20%7D%7D) no-repeat center center;
- height: {{ theme_index_logo_height }};
-}
-{% endif %}
-
-div.body h2 { font-size: 180%; }
-div.body h3 { font-size: 150%; }
-div.body h4 { font-size: 130%; }
-div.body h5 { font-size: 100%; }
-div.body h6 { font-size: 100%; }
-
-a.headerlink {
- color: white;
- padding: 0 4px;
- text-decoration: none;
-}
-
-a.headerlink:hover {
- color: #444;
- background: #eaeaea;
-}
-
-div.body p, div.body dd, div.body li {
- line-height: 1.4em;
-}
-
-div.admonition {
- background: #fafafa;
- margin: 20px -30px;
- padding: 10px 30px;
- border-top: 1px solid #ccc;
- border-bottom: 1px solid #ccc;
-}
-
-div.admonition p.admonition-title {
- font-family: 'Garamond', 'Georgia', serif;
- font-weight: normal;
- font-size: 24px;
- margin: 0 0 10px 0;
- padding: 0;
- line-height: 1;
-}
-
-div.admonition p.last {
- margin-bottom: 0;
-}
-
-div.highlight{
- background-color: white;
-}
-
-dt:target, .highlight {
- background: #FAF3E8;
-}
-
-div.note {
- background-color: #eee;
- border: 1px solid #ccc;
-}
-
-div.seealso {
- background-color: #ffc;
- border: 1px solid #ff6;
-}
-
-div.topic {
- background-color: #eee;
-}
-
-div.warning {
- background-color: #ffe4e4;
- border: 1px solid #f66;
-}
-
-p.admonition-title {
- display: inline;
-}
-
-p.admonition-title:after {
- content: ":";
-}
-
-pre, tt {
- font-family: 'Consolas', 'Menlo', 'Deja Vu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
- font-size: 0.85em;
-}
-
-img.screenshot {
-}
-
-tt.descname, tt.descclassname {
- font-size: 0.95em;
-}
-
-tt.descname {
- padding-right: 0.08em;
-}
-
-img.screenshot {
- -moz-box-shadow: 2px 2px 4px #eee;
- -webkit-box-shadow: 2px 2px 4px #eee;
- box-shadow: 2px 2px 4px #eee;
-}
-
-table.docutils {
- border: 1px solid #888;
- -moz-box-shadow: 2px 2px 4px #eee;
- -webkit-box-shadow: 2px 2px 4px #eee;
- box-shadow: 2px 2px 4px #eee;
-}
-
-table.docutils td, table.docutils th {
- border: 1px solid #888;
- padding: 0.25em 0.7em;
-}
-
-table.field-list, table.footnote {
- border: none;
- -moz-box-shadow: none;
- -webkit-box-shadow: none;
- box-shadow: none;
-}
-
-table.footnote {
- margin: 15px 0;
- width: 100%;
- border: 1px solid #eee;
-}
-
-table.field-list th {
- padding: 0 0.8em 0 0;
-}
-
-table.field-list td {
- padding: 0;
-}
-
-table.footnote td {
- padding: 0.5em;
-}
-
-dl {
- margin: 0;
- padding: 0;
-}
-
-dl dd {
- margin-left: 30px;
-}
-
-pre {
- padding: 0;
- margin: 15px -30px;
- padding: 8px;
- line-height: 1.3em;
- padding: 7px 30px;
- background: #eee;
- border-radius: 2px;
- -moz-border-radius: 2px;
- -webkit-border-radius: 2px;
-}
-
-dl pre {
- margin-left: -60px;
- padding-left: 60px;
-}
-
-tt {
- background-color: #ecf0f3;
- color: #222;
- /* padding: 1px 2px; */
-}
-
-tt.xref, a tt {
- background-color: #FBFBFB;
-}
-
-a:hover tt {
- background: #EEE;
-}
diff --git a/docs/_themes/kr_small/theme.conf b/docs/_themes/kr_small/theme.conf
deleted file mode 100644
index 542b46251..000000000
--- a/docs/_themes/kr_small/theme.conf
+++ /dev/null
@@ -1,10 +0,0 @@
-[theme]
-inherit = basic
-stylesheet = flasky.css
-nosidebar = true
-pygments_style = flask_theme_support.FlaskyStyle
-
-[options]
-index_logo = ''
-index_logo_height = 120px
-github_fork = ''
diff --git a/docs/conf.py b/docs/conf.py
index fc31228e7..d732ac29f 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -11,7 +11,9 @@
# All configuration values have a default; values that are commented out
# serve to show the default.
-import sys, os
+import datetime
+import os
+import sys
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
@@ -45,8 +47,11 @@
master_doc = 'index'
# General information about the project.
+current_year = datetime.datetime.now().year
project = u'pythonguide'
-copyright = u'2013. A Kenneth Reitz Project. Creative Commons Share-Alike 3.0.'
+copyright = (u'2011-{} Kenneth Reitz'
+ ' & Real Python.'
+ ' CC BY-NC-SA 3.0').format(current_year)
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
@@ -69,7 +74,10 @@
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
-exclude_patterns = ['_build']
+exclude_patterns = [
+ '_build',
+ '_themes/*.rst', # Excluded due to README.rst in _themes/
+]
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
@@ -96,12 +104,19 @@
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
-html_theme = 'kr'
+html_theme = 'alabaster'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
-#html_theme_options = {}
+html_theme_options = {
+ 'show_powered_by': False,
+ 'github_user': 'realpython',
+ 'github_repo': 'python-guide',
+ 'github_banner': True,
+ 'show_related': False,
+ 'note_bg': '#FFF59C',
+}
# Add any paths that contain custom themes here, relative to this directory.
html_theme_path = ['_themes']
@@ -137,9 +152,9 @@
# Custom sidebar templates, maps document names to template names.
html_sidebars = {
- 'index': ['sidebarintro.html', 'sourcelink.html', 'searchbox.html'],
+ 'index': ['sidebarintro.html', 'sourcelink.html', 'searchbox.html', 'hacks.html'],
'**': ['sidebarlogo.html', 'localtoc.html', 'relations.html',
- 'sourcelink.html', 'searchbox.html']
+ 'sourcelink.html', 'searchbox.html', 'hacks.html']
}
# Additional templates that should be rendered to pages, maps page names to
@@ -156,7 +171,7 @@
#html_split_index = False
# If true, links to the reST sources are added to the pages.
-html_show_sourcelink = True
+html_show_sourcelink = False
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
html_show_sphinx = False
@@ -231,7 +246,7 @@
epub_title = u'pythonguide'
epub_author = u'Kenneth Reitz'
epub_publisher = u'Kenneth Reitz'
-epub_copyright = u'2010, Kenneth Reitz'
+epub_copyright = u'2011–{}, Kenneth Reitz & Real Python'.format(current_year)
# The language of the text. It defaults to the language option
# or en if the language is not set.
@@ -251,12 +266,14 @@
# The format is a list of tuples containing the path and title.
#epub_pre_files = []
-# HTML files shat should be inserted after the pages created by sphinx.
+# HTML files that should be inserted after the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_post_files = []
# A list of files that should not be packed into the epub file.
-#epub_exclude_files = []
+epub_exclude_files = [
+ ('search.html', 'Search'),
+]
# The depth of the table of contents in toc.ncx.
#epub_tocdepth = 3
@@ -267,5 +284,5 @@
todo_include_todos = True
intersphinx_mapping = {
- 'python': ('http://docs.python.org/', None),
+ 'python': ('https://docs.python.org/3', None),
}
diff --git a/docs/contents.rst.inc b/docs/contents.rst.inc
index 7887307a7..29b076197 100644
--- a/docs/contents.rst.inc
+++ b/docs/contents.rst.inc
@@ -1,28 +1,55 @@
-Getting Started
----------------
+Getting Started with Python
+---------------------------
-This part of the guide focuses on setting up your Python environment.
+New to Python? Let's properly setup up your Python environment:
.. toctree::
:maxdepth: 2
starting/which-python
-- Properly Install Python
+- Properly Install Python on your system:
.. toctree::
:maxdepth: 1
+ starting/installation
+ starting/install3/osx
+ starting/install3/win
+ starting/install3/linux
starting/install/osx
starting/install/win
starting/install/linux
+- Using Virtualenvs with Pipenv:
+
+ .. toctree::
+ :maxdepth: 2
+
+ dev/virtualenvs
+
+
+Python Development Environments
+-------------------------------
+
+This part of the guide focuses on the Python development environment,
+and the best-practice tools that are available for writing Python code.
+
+.. toctree::
+ :maxdepth: 2
+
+ dev/env
+ dev/virtualenvs
+ dev/pip-virtualenv
-Writing Great Code
-------------------
-This part of the guide focuses on best practices for writing Python code.
+
+
+Writing Great Python Code
+-------------------------
+
+This part of the guide focuses on the best-practices for writing Python code.
.. toctree::
:maxdepth: 2
@@ -32,13 +59,14 @@ This part of the guide focuses on best practices for writing Python code.
writing/reading
writing/documentation
writing/tests
+ writing/logging
writing/gotchas
writing/license
-Scenario Guide
---------------
+Scenario Guide for Python Applications
+--------------------------------------
This part of the guide focuses on tool and module advice based on
different scenarios.
@@ -58,38 +86,32 @@ different scenarios.
scenarios/speed
scenarios/scientific
scenarios/imaging
+ scenarios/serialization
scenarios/xml
+ scenarios/json
+ scenarios/crypto
+ scenarios/ml
+ scenarios/clibs
-Shipping Great Code
--------------------
+Shipping Great Python Code
+--------------------------
-This part of the guide focuses on deploying your Python code.
+This part of the guide focuses on sharing and deploying your Python code.
.. toctree::
:maxdepth: 2
+ shipping/publishing
shipping/packaging
shipping/freezing
-Development Environment
------------------------
-
-.. toctree::
- :maxdepth: 2
-
- dev/env
- dev/virtualenvs
-
-
-
-
Additional Notes
----------------
This part of the guide, which is mostly prose, begins with some
-background information about Python, then focuses on next steps.
+background information about Python, and then focuses on next steps.
.. toctree::
:maxdepth: 2
@@ -100,12 +122,15 @@ background information about Python, then focuses on next steps.
intro/documentation
intro/news
-
+.. note::
+ Notes defined within all diatonic and chromatic musical scales have been
+ intentionally excluded from this list of additional notes. Additionally,
+ this note.
--------------------------------------
-Contribution notes and legal information are here (for those interested).
+Contribution notes and legal information (for those interested).
.. toctree::
:maxdepth: 2
@@ -113,8 +138,3 @@ Contribution notes and legal information are here (for those interested).
notes/contribute
notes/license
notes/styleguide
-
-
-
-
-
diff --git a/docs/dev/env.rst b/docs/dev/env.rst
index 56301fc9f..7da63c0e6 100644
--- a/docs/dev/env.rst
+++ b/docs/dev/env.rst
@@ -1,59 +1,61 @@
Your Development Environment
============================
+.. image:: /_static/photos/33175624924_7febc46cc4_k_d.jpg
+
Text Editors
::::::::::::
-Just about anything which can edit plain text will work for writing Python code,
+Just about anything that can edit plain text will work for writing Python code;
however, using a more powerful editor may make your life a bit easier.
-VIM
+Vim
---
Vim is a text editor which uses keyboard shortcuts for editing instead of menus
-or icons. There exist a couple of plugins and settings for the VIM editor to
-aid python development. If you only develop in Python, a good start is to set
+or icons. There are a couple of plugins and settings for the Vim editor to
+aid Python development. If you only develop in Python, a good start is to set
the default settings for indentation and line-wrapping to values compliant with
-`PEP 8 `_. In your home directory,
-open a file called `.vimrc` and add the following lines:::
+:pep:`8`. In your home directory, open a file called :file:`.vimrc` and add the
+following lines::
set textwidth=79 " lines longer than 79 columns will be broken
set shiftwidth=4 " operation >> indents 4 columns; << unindents 4 columns
- set tabstop=4 " an hard TAB displays as 4 columns
+ set tabstop=4 " a hard TAB displays as 4 columns
set expandtab " insert spaces when hitting TABs
set softtabstop=4 " insert/delete 4 spaces when hitting a TAB/BACKSPACE
set shiftround " round indent to multiple of 'shiftwidth'
set autoindent " align the new line indent with the previous line
With these settings, newlines are inserted after 79 characters and indentation
-is set to 4 spaces per tab. If you also use VIM for other languages, there is a
-handy plugin at indent_, which handles indentation settings for python source
-files.
-
-There is also a handy syntax plugin at syntax_ featuring some improvements over
-the syntax file included in VIM 6.1.
-
-These plugins supply you with a basic environment for developing in Python.
-To get the most out of Vim, you should continually check your code for syntax
-errors and PEP8 compliance. Luckily PEP8_ and Pyflakes_ will do this for you.
-If your VIM is compiled with `+python` you can also utilize some very handy
-plugins to do these checks from within the editor.
-
-For PEP8 checking, install the vim-pep8_ plugin, and for pyflakes you can
-install vim-pyflakes_. Now you can map the functions `Pep8()` or `Pyflakes()`
-to any hotkey or action you want in Vim. Both plugins will display errors at
-the bottom of the screen, and provide an easy way to jump to the corresponding
-line. It's very handy to call these functions whenever you save a file. In
-order to do this, add the following lines to your `vimrc`::
-
- autocmd BufWritePost *.py call Pyflakes()
- autocmd BufWritePost *.py call Pep8()
-
-If you are already using syntastic_ you can enable it to run Pyflakes on write
+is set to 4 spaces per tab. If you also use Vim for other languages, there is a
+handy plugin called indent_, which handles indentation settings for Python
+source files.
+
+There is also a handy syntax plugin called syntax_ featuring some improvements
+over the syntax file included in Vim 6.1.
+
+These plugins supply you with a basic environment for developing in Python. To
+get the most out of Vim, you should continually check your code for syntax
+errors and PEP8 compliance. Luckily pycodestyle_ and Pyflakes_ will do this
+for you. If your Vim is compiled with ``+python`` you can also utilize some
+very handy plugins to do these checks from within the editor.
+
+For PEP8 checking and pyflakes, you can install vim-flake8_. Now you can map the
+function ``Flake8`` to any hotkey or action you want in Vim. The plugin will
+display errors at the bottom of the screen, and provide an easy way to jump to
+the corresponding line. It's very handy to call this function whenever you save
+a file. In order to do this, add the following line to your
+:file:`.vimrc`::
+
+ autocmd BufWritePost *.py call Flake8()
+
+If you are already using syntastic_, you can set it to run Pyflakes on write
and show errors and warnings in the quickfix window. An example configuration
-to do that which also shows status and warning messages in the statusbar would be::
+to do that which also shows status and warning messages in the statusbar would
+be::
set statusline+=%#warningmsg#
set statusline+=%{SyntasticStatuslineFlag()}
@@ -61,64 +63,89 @@ to do that which also shows status and warning messages in the statusbar would b
let g:syntastic_auto_loc_list=1
let g:syntastic_loc_list_height=5
+
Python-mode
^^^^^^^^^^^
-Python-mode_ is a complex solution in VIM for working with python code.
+Python-mode_ is a complex solution for working with Python code in Vim.
It has:
-- Async python code checking (pylint, pyflakes, pep8, mccabe) in any combination
+- Asynchronous Python code checking (``pylint``, ``pyflakes``, ``pycodestyle``, ``mccabe``) in any combination
- Code refactoring and autocompletion with Rope
-- Fast python folding
+- Fast Python folding
- Virtualenv support
-- Search by python documentation and run python code
-- Auto pep8 error fixes
+- Search through Python documentation and run Python code
+- Auto pycodestyle_ error fixes
And more.
+SuperTab
+^^^^^^^^
+
+SuperTab_ is a small Vim plugin that makes code completion more convenient by
+using ```` key or any other customized keys.
+
.. _indent: http://www.vim.org/scripts/script.php?script_id=974
.. _syntax: http://www.vim.org/scripts/script.php?script_id=790
-.. _Pyflakes: http://pypi.python.org/pypi/pyflakes/
-.. _vim-pyflakes: https://github.com/nvie/vim-pyflakes
-.. _PEP8: http://pypi.python.org/pypi/pep8/
-.. _vim-pep8: https://github.com/nvie/vim-pep8
-.. _syntastic: https://github.com/scrooloose/syntastic
-.. _Python-mode: https://github.com/klen/python-mode
-
-.. todo:: add supertab notes
+.. _Pyflakes: http://pypi.org/project/pyflakes/
+.. _pycodestyle: https://pypi.org/project/pycodestyle/
+.. _syntastic: https://github.com/vim-syntastic/syntastic
+.. _Python-mode: https://github.com/python-mode/python-mode
+.. _SuperTab: http://www.vim.org/scripts/script.php?script_id=1643
+.. _vim-flake8: https://github.com/nvie/vim-flake8
Emacs
-----
-Emacs is a powerful text editor. It's fully programmable (lisp), but
-it can be some work to wire up correctly. A good start if you're
-already an Emacs user is `Python Programming in Emacs`_ at EmacsWiki.
+Emacs is another powerful text editor. It is fully programmable (Lisp), but
+it can be some work to wire up correctly. A good start if you're already an
+Emacs user is `Python Programming in Emacs`_ at EmacsWiki.
-1. Emacs itself comes with a python mode.
-2. Python ships with an alternate version:
- `python-mode.el `_
-3. Fabián Ezequiel Gallina's `python.el `_
- provides nice functionality and behavior out of the box
+1. Emacs itself comes with a Python mode.
-.. _Python Programming in Emacs: http://emacswiki.org/emacs/PythonProgrammingInEmacs
+.. _Python Programming in Emacs: https://www.emacswiki.org/emacs/PythonProgrammingInEmacs
TextMate
--------
-"`TextMate `_ brings Apple's approach to operating
-systems into the world of text editors. By bridging UNIX underpinnings and GUI,
-TextMate cherry-picks the best of both worlds to the benefit of expert
-scripters and novice users alike."
+ `TextMate `_ brings Apple's approach to operating
+ systems into the world of text editors. By bridging Unix underpinnings and
+ GUI, TextMate cherry-picks the best of both worlds to the benefit of expert
+ scripters and novice users alike.
Sublime Text
------------
-"`Sublime Text `_ is a sophisticated text editor
-for code, html and prose. You'll love the slick user interface and
-extraordinary features."
+ `Sublime Text `_ is a sophisticated text
+ editor for code, markup, and prose. You'll love the slick user interface,
+ extraordinary features, and amazing performance.
Sublime Text has excellent support for editing Python code and uses Python for
-its plugin API.
+its plugin API. It also has a diverse variety of plugins,
+`some of which `_ allow for
+in-editor PEP8 checking and code "linting".
+
+Atom
+----
+
+ `Atom `_ is a hackable text editor for the 21st century,
+ built on atom-shell, and based on everything we love about our favorite
+ editors.
+
+Atom is web native (HTML, CSS, JS), focusing on modular design and easy plugin
+development. It comes with native package control and a plethora of packages.
+Recommended for Python development is
+`Linter `_ combined with
+`linter-flake8 `_.
+
+Python (on Visual Studio Code)
+------------------------------
+
+`Python for Visual Studio `_ is an extension for the `Visual Studio Code `_.
+This is a free, lightweight, open source code editor, with support for Mac, Windows, and Linux.
+Built using open source technologies such as Node.js and Python, with compelling features such as Intellisense (autocompletion), local and remote debugging, linting, and the like.
+
+MIT licensed.
IDEs
::::
@@ -128,38 +155,51 @@ PyCharm / IntelliJ IDEA
`PyCharm `_ is developed by JetBrains, also
known for IntelliJ IDEA. Both share the same code base and most of PyCharm's
-features can be brought to IntelliJ with the free `Python Plug-In `_.
+features can be brought to IntelliJ with the free
+`Python Plug-In `_. There are two
+versions of PyCharm: Professional Edition (Free 30-day trial) and Community
+Edition (Apache 2.0 License) with fewer features.
+Enthought Canopy
+----------------
+`Enthought Canopy `_ is a Python
+IDE which is focused towards Scientists and Engineers as it provides pre
+installed libraries for data analysis.
+
Eclipse
-------
The most popular Eclipse plugin for Python development is Aptana's
-`PyDev `_.
+`PyDev `_.
Komodo IDE
----------
-`Komodo IDE `_ is developed by
-ActiveState and is a commercial IDE for Windows, Mac and Linux.
+
+`Komodo IDE `_ is developed by
+ActiveState and is a commercial IDE for Windows, Mac, and Linux.
+`KomodoEdit `_ is the open source
+alternative.
Spyder
------
-`Spyder `_ is an IDE specifically geared
-toward working with scientific python libraries (namely `Scipy `_).
-It includes integration with pyflakes_, `pylint `_,
-and `rope `_.
+`Spyder `_ is an IDE specifically geared
+toward working with scientific Python libraries (namely
+`SciPy `_). It includes integration with pyflakes_,
+`pylint `_ and
+`rope `_.
-Spyder is open-source (free), offers code completion, syntax highlighting,
-class and function browser, and object inspection.
+Spyder is open source (free), offers code completion, syntax highlighting,
+a class and function browser, and object inspection.
WingIDE
-------
-`WingIDE `_ is a python specific IDE. It runs on Linux,
+`WingIDE `_ is a Python specific IDE. It runs on Linux,
Windows, and Mac (as an X11 application, which frustrates some Mac users).
WingIDE offers code completion, syntax highlighting, source browser, graphical
@@ -171,112 +211,75 @@ NINJA-IDE
`NINJA-IDE `_ (from the recursive acronym: "Ninja-IDE
Is Not Just Another IDE") is a cross-platform IDE, specially designed to build
-Python applications, and runs on Linux/X11, Mac OS X and Windows desktop operating
-systems. Installers for these platforms can be downloaded from the website.
-
-NINJA-IDE is open-source software (GPLv3 licence) and is developed in Python and
-Qt. The source files can be downloaded from `GitHub `_.
-
-Interpreter Tools
-:::::::::::::::::
-
-
-virtualenv
-----------
+Python applications, and runs on Linux/X11, Mac OS X, and Windows desktop
+operating systems. Installers for these platforms can be downloaded from the
+website.
-Virtualenv is a tool to keep the dependencies required by different projects
-in separate places, by creating virtual Python environments for them.
-It solves the "Project X depends on version 1.x but, Project Y needs 4.x"
-dilemma and keeps your global site-packages directory clean and manageable.
+NINJA-IDE is open source software (GPLv3 licence) and is developed
+in Python and Qt. The source files can be downloaded from
+`GitHub `_.
-`virtualenv `_ creates
-a folder which contains all the necessary executables to contain the
-packages that a Python project would need. An example workflow is given.
-Install virtualenv::
+Eric (The Eric Python IDE)
+--------------------------
- $ pip install virtualenv
+`Eric `_ is a full featured Python IDE
+offering source code autocompletion, syntax highlighting, support for version
+control systems, Python 3 support, integrated web browser, python shell,
+integrated debugger, and a flexible plug-in system. Written in Python, it is
+based on the Qt GUI toolkit, integrating the Scintilla editor control. Eric
+is an open source software project (GPLv3 licence) with more than ten years of
+active development.
+Mu
+--
-Create a virtual environment for a project::
+`Mu `_ is a minimalist Python IDE which can run Python 3 code
+locally and can also deploy code to the BBC micro:bit and to Adafruit boards running
+CircuitPython.
- $ cd my_project
- $ virtualenv venv
+Intended for beginners, mu includes a Python 3 interpreter, and is easy to install
+on Windows, OS/X and Linux. It runs well on the Raspberry Pi.
-``virtualenv venv`` will create a folder in the current directory
-which will contain the Python executable files, and a copy of the ``pip``
-library which you can use to install other packages. The name of the
-virtual environment (in this case, it was ``venv``) can be anything;
-omitting the name will place the files in the current directory instead.
+There's an active support community on gitter.
-To start using the virtual environment, run::
- $ source venv/bin/activate
-
-
-The name of the current virtual environment will now appear on the left
-of the prompt (e.g. ``(venv)Your-Computer:your_project UserName$``) to
-let you know that it's active. From now on, any package that you install
-using ``pip`` will be placed in the venv folder, isolated from the global
-Python installation. Install packages as usual::
-
- $ pip install requests
-
-To stop using an environment simply type ``deactivate``. To remove the
-environment, just remove the directory it was installed into. (In this
-case, it would be ``rm -rf venv``).
-
-Other Notes
-^^^^^^^^^^^
-
-Running ``virtualenv`` with the option ``--no-site-packages`` will not
-include the packages that are installed globally. This can be useful
-for keeping the package list clean in case it needs to be accessed later.
-[This is the default behavior for ``virtualenv`` 1.7 and later.]
-
-In order to keep your environment consistent, it's a good idea to "freeze"
-the current state of the environment packages. To do this, run
-
-::
-
- $ pip freeze > requirements.txt
-
-This will create a ``requirements.txt`` file, which contains a simple
-list of all the packages in the current environment, and their respective
-versions. Later, when a different developer (or you, if you need to re-
-create the environment) can install the same packages, with the same
-versions by running
-
-::
-
- $ pip install -r requirements.txt
-
-This can help ensure consistency across installations, across deployments,
-and across developers.
-
-Lastly, remember to exclude the virtual environment folder from source
-control by adding it to the ignore list.
-
-virtualenvwrapper
------------------
-
-`Virtualenvwrapper `_ makes
-virtualenv a pleasure to use by wrapping the command line API with a nicer CLI.
+Interpreter Tools
+:::::::::::::::::
-::
- $ pip install virtualenvwrapper
+Virtual Environments
+--------------------
+Virtual Environments provide a powerful way to isolate project package dependencies. This means that you can use packages particular to a Python project without installing them system wide and thus avoiding potential version conflicts.
-Put this into your `~/.bash_profile` (Linux/Mac) file:
+To start using and see more information:
+`Virtual Environments `_ docs.
-::
- $ export VIRTUALENVWRAPPER_VIRTUALENV_ARGS='--no-site-packages'
+pyenv
+-----
-This will prevent your virtualenvs from relying on your (global) site packages
-directory, so that they are completely separate..
-[note: This is the default behavior for ``virtualenv`` 1.7 and later]
+`pyenv `_ is a tool to allow multiple versions
+of the Python interpreter to be installed at the same time. This solves the
+problem of having different projects requiring different versions of Python.
+For example, it becomes very easy to install Python 2.7 for compatibility in
+one project, while still using Python 3.4 as the default interpreter.
+pyenv isn't just limited to the CPython versions – it will also install PyPy,
+Anaconda, miniconda, stackless, Jython, and IronPython interpreters.
+
+pyenv works by filling a ``shims`` directory with fake versions of the Python
+interpreter (plus other tools like ``pip`` and ``2to3``). When the system
+looks for a program named ``python``, it looks inside the ``shims`` directory
+first, and uses the fake version, which in turn passes the command on to
+pyenv. pyenv then works out which version of Python should be run based on
+environment variables, ``.python-version`` files, and the global default.
+
+pyenv isn't a tool for managing virtual environments, but there is the plugin
+`pyenv-virtualenv `_ which automates
+the creation of different environments, and also makes it possible to use the
+existing pyenv tools to switch to different environments based on environment
+variables or ``.python-version`` files.
Other Tools
:::::::::::
@@ -284,12 +287,11 @@ Other Tools
IDLE
----
-`IDLE `_ is an integrated
-development environment that is part of Python standard library. It is
-completely written in Python and uses the Tkinter GUI toolkit. Though IDLE
-is not suited for full-blown development using Python, it is quite
-helpful to try out small Python snippets and experiment with different
-features in Python.
+:ref:`IDLE ` is an integrated development environment that is
+part of the Python standard distribution. It is completely written in Python and uses
+the Tkinter GUI toolkit. Though IDLE is not suited for full-blown development
+using Python, it is quite helpful to try out small Python snippets and
+experiment with different features in Python.
It provides the following features:
@@ -304,35 +306,60 @@ IPython
`IPython `_ provides a rich toolkit to help you make the
most out of using Python interactively. Its main components are:
-* Powerful Python shells (terminal- and Qt-based).
+* Powerful Python shells (terminal- and Qt-based)
* A web-based notebook with the same core features but support for rich media,
- text, code, mathematical expressions and inline plots.
-* Support for interactive data visualization and use of GUI toolkits.
-* Flexible, embeddable interpreters to load into your own projects.
-* Tools for high level and interactive parallel computing.
+ text, code, mathematical expressions and inline plots
+* Support for interactive data visualization and use of GUI toolkits
+* Flexible, embeddable interpreters to load into your own projects
+* Tools for high level and interactive parallel computing
-::
+.. code-block:: console
$ pip install ipython
+To download and install IPython with all its optional dependencies for the notebook, qtconsole, tests, and other functionalities:
+.. code-block:: console
+
+ $ pip install ipython[all]
BPython
-------
-`bpython `_ is an alternative interface to the
-Python interpreter for Unix-like operating systems. It has the following features:
+`bpython `_ is an alternative interface to the
+Python interpreter for Unix-like operating systems. It has the following
+features:
-* In-line syntax highlighting.
-* Readline-like autocomplete with suggestions displayed as you type.
-* Expected parameter list for any Python function.
-* "Rewind" function to pop the last line of code from memory and re-evaluate.
-* Send entered code off to a pastebin.
-* Save entered code to a file.
-* Auto-indentation.
-* Python 3 support.
+* In-line syntax highlighting
+* Readline-like autocomplete with suggestions displayed as you type
+* Expected parameter list for any Python function
+* "Rewind" function to pop the last line of code from memory and re-evaluate
+* Send entered code off to a pastebin
+* Save entered code to a file
+* Auto-indentation
+* Python 3 support
-::
+.. code-block:: console
$ pip install bpython
+ptpython
+--------
+
+`ptpython `_ is a REPL build
+on top of the `prompt_toolkit `_
+library. It is considered to be an alternative to BPython_. Features include:
+
+* Syntax highlighting
+* Autocompletion
+* Multiline editing
+* Emacs and Vim Modes
+* Embedding REPL inside of your code
+* Syntax validation
+* Tab pages
+* Support for integrating with IPython_'s shell, by installing IPython
+ (``pip install ipython``) and running ``ptipython``.
+
+.. code-block:: console
+
+ $ pip install ptpython
diff --git a/docs/dev/pip-virtualenv.rst b/docs/dev/pip-virtualenv.rst
new file mode 100644
index 000000000..667f30c7a
--- /dev/null
+++ b/docs/dev/pip-virtualenv.rst
@@ -0,0 +1,137 @@
+.. _pip-virtualenv:
+
+Further Configuration of pip and Virtualenv
+===========================================
+
+.. image:: /_static/photos/34018732105_f0e6758859_k_d.jpg
+
+Requiring an active virtual environment for ``pip``
+---------------------------------------------------
+
+By now it should be clear that using virtual environments is a great way to
+keep your development environment clean and keeping different projects'
+requirements separate.
+
+When you start working on many different projects, it can be hard to remember to
+activate the related virtual environment when you come back to a specific
+project. As a result of this, it is very easy to install packages globally
+while thinking that you are actually installing the package for the virtual
+environment of the project. Over time this can result in a messy global package
+list.
+
+In order to make sure that you install packages to your active virtual
+environment when you use ``pip install``, consider adding the following
+line to your :file:`~/.bashrc` file:
+
+.. code-block:: console
+
+ export PIP_REQUIRE_VIRTUALENV=true
+
+After saving this change and sourcing the :file:`~/.bashrc` file with
+``source ~/.bashrc``, pip will no longer let you install packages if you are not
+in a virtual environment. If you try to use ``pip install`` outside of a
+virtual environment pip will gently remind you that an activated virtual
+environment is needed to install packages.
+
+.. code-block:: console
+
+ $ pip install requests
+ Could not find an activated virtualenv (required).
+
+You can also do this configuration by editing your :file:`pip.conf` or
+:file:`pip.ini` file. :file:`pip.conf` is used by Unix and Mac OS X operating
+systems and it can be found at:
+
+.. code-block:: console
+
+ $HOME/.pip/pip.conf
+
+Similarly, the :file:`pip.ini` file is used by Windows operating systems and it
+can be found at:
+
+.. code-block:: console
+
+ %USERPROFILE%\pip\pip.ini
+
+If you don't have a :file:`pip.conf` or :file:`pip.ini` file at these locations,
+you can create a new file with the correct name for your operating system.
+
+If you already have a configuration file, just add the following line under the
+``[global]`` settings to require an active virtual environment:
+
+.. code-block:: console
+
+ require-virtualenv = true
+
+If you did not have a configuration file, you will need to create a new one and
+add the following lines to this new file:
+
+.. code-block:: console
+
+ [global]
+ require-virtualenv = true
+
+
+You will of course need to install some packages globally (usually ones that
+you use across different projects consistently) and this can be accomplished by
+adding the following to your :file:`~/.bashrc` file:
+
+.. code-block:: console
+
+ gpip() {
+ PIP_REQUIRE_VIRTUALENV=false pip "$@"
+ }
+
+After saving the changes and sourcing your :file:`~/.bashrc` file you can now
+install packages globally by running ``gpip install``. You can change the name
+of the function to anything you like, just keep in mind that you will have to
+use that name when trying to install packages globally with pip.
+
+Caching packages for future use
+-------------------------------
+
+Every developer has preferred libraries and when you are working on a lot of
+different projects, you are bound to have some overlap between the libraries
+that you use. For example, you may be using the ``requests`` library in a lot
+of different projects.
+
+It is surely unnecessary to re-download the same packages/libraries each time
+you start working on a new project (and in a new virtual environment as a
+result). Fortunately, starting with version 6.0, pip provides an `on-by-default
+caching mechanism
+`_ that doesn't
+need any configuration.
+
+When using older versions, you can configure pip in such a way that it tries to
+reuse already installed packages, too.
+
+On Unix systems, you can add the following line to your :file:`.bashrc` or
+:file:`.bash_profile` file.
+
+.. code-block:: console
+
+ export PIP_DOWNLOAD_CACHE=$HOME/.pip/cache
+
+You can set the path to anywhere you like (as long as you have write
+access). After adding this line, ``source`` your :file:`.bashrc`
+(or :file:`.bash_profile`) file and you will be all set.
+
+Another way of doing the same configuration is via the :file:`pip.conf` or
+:file:`pip.ini` files, depending on your system. If you are on Windows, you can
+add the following line to your :file:`pip.ini` file under ``[global]`` settings:
+
+.. code-block:: console
+
+ download-cache = %USERPROFILE%\pip\cache
+
+Similarly, on Unix systems you should simply add the following line to your
+:file:`pip.conf` file under ``[global]`` settings:
+
+.. code-block:: console
+
+ download-cache = $HOME/.pip/cache
+
+Even though you can use any path you like to store your cache, it is recommended
+that you create a new folder *in* the folder where your :file:`pip.conf` or
+:file:`pip.ini` file lives. If you don't trust yourself with all of this path
+voodoo, just use the values provided here and you will be fine.
diff --git a/docs/dev/virtualenvs.rst b/docs/dev/virtualenvs.rst
index ee6ace244..fe6bd3f9e 100644
--- a/docs/dev/virtualenvs.rst
+++ b/docs/dev/virtualenvs.rst
@@ -1,36 +1,257 @@
-Virtual Environments
-====================
+.. _virtualenvironments-ref:
-A Virtual Environment, put simply, is an isolated working copy of Python which
-allows you to work on a specific project without worry of affecting other
-projects.
+Pipenv & Virtual Environments
+=============================
-For example, you can work on a project which requires Django 1.3 while also
-maintaining a project which requires Django 1.0.
+.. image:: /_static/photos/35294660055_42c02b2316_k_d.jpg
-virtualenv
+This tutorial walks you through installing and using Python packages.
+
+It will show you how to install and use the necessary tools and make strong
+recommendations on best practices. Keep in mind that Python is used for a great
+many different purposes, and precisely how you want to manage your dependencies
+may change based on how you decide to publish your software. The guidance
+presented here is most directly applicable to the development and deployment of
+network services (including web applications), but is also very well suited to
+managing development and testing environments for any kind of project.
+
+.. Note:: This guide is written for Python 3, however, these instructions
+ should work fine on Python 2.7—if you are still using it, for some reason.
+
+
+Make sure you've got Python & pip
+---------------------------------
+
+Before you go any further, make sure you have Python and that it's available
+from your command line. You can check this by simply running:
+
+.. code-block:: console
+
+ $ python --version
+
+You should get some output like ``3.6.2``. If you do not have Python, please
+install the latest 3.x version from `python.org`_ or refer to the
+`Installing Python`_ section of this guide.
+
+.. Note:: If you're newcomer and you get an error like this:
+
+ .. code-block:: python
+
+ >>> python
+ Traceback (most recent call last):
+ File "", line 1, in
+ NameError: name 'python' is not defined
+
+ It's because this command is intended to be run in a *shell* (also called
+ a *terminal* or *console*). See the Python for Beginners
+ `getting started tutorial`_ for an introduction to using your operating
+ system's shell and interacting with Python.
+
+Additionally, you'll need to make sure you have `pip`_ available. You can
+check this by running:
+
+.. code-block:: console
+
+ $ pip --version
+
+If you installed Python from source, with an installer from `python.org`_, or
+via `Homebrew`_ you should already have pip. If you're on Linux and installed
+using your OS package manager, you may have to `install pip `_ separately.
+
+.. _getting started tutorial: https://opentechschool.github.io/python-beginners/en/getting_started.html#what-is-python-exactly
+.. _python.org: https://python.org
+.. _pip: https://pypi.org/project/pip/
+.. _Homebrew: https://brew.sh
+.. _Installing Python: https://docs.python-guide.org/starting/installation/
+
+
+Installing Pipenv
+-----------------
+
+`Pipenv`_ is a dependency manager for Python projects. If you're familiar
+with Node.js' `npm`_ or Ruby's `bundler`_, it is similar in spirit to those
+tools. While `pip`_ can install Python packages, Pipenv is recommended as
+it's a higher-level tool that simplifies dependency management for common use
+cases.
+
+Use ``pip`` to install Pipenv:
+
+.. code-block:: console
+
+ $ pip install --user pipenv
+
+
+.. Note:: This does a `user installation`_ to prevent breaking any system-wide
+ packages. If ``pipenv`` isn't available in your shell after installation,
+ you'll need to add the `user base`_'s binary directory to your ``PATH``.
+
+ On Linux and macOS you can find the user base binary directory by running
+ ``python -m site --user-base`` and adding ``bin`` to the end. For example,
+ this will typically print ``~/.local`` (with ``~`` expanded to the
+ absolute path to your home directory) so you'll need to add
+ ``~/.local/bin`` to your ``PATH``. You can set your ``PATH`` permanently by
+ `modifying ~/.profile`_.
+
+ On Windows you can find the user base binary directory by running
+ ``py -m site --user-site`` and replacing ``site-packages`` with
+ ``Scripts``. For example, this could return
+ ``C:\Users\Username\AppData\Roaming\Python36\site-packages`` so you would
+ need to set your ``PATH`` to include
+ ``C:\Users\Username\AppData\Roaming\Python36\Scripts``. You can set your
+ user ``PATH`` permanently in the `Control Panel`_. You may need to log
+ out for the ``PATH`` changes to take effect.
+
+.. _Pipenv: https://pipenv.kennethreitz.org/
+.. _npm: https://www.npmjs.com/
+.. _bundler: http://bundler.io/
+.. _user base: https://docs.python.org/3/library/site.html#site.USER_BASE
+.. _user installation: https://pip.pypa.io/en/stable/user_guide/#user-installs
+.. _modifying ~/.profile: https://stackoverflow.com/a/14638025
+.. _Control Panel: https://msdn.microsoft.com/en-us/library/windows/desktop/bb776899(v=vs.85).aspx
+
+Installing packages for your project
+------------------------------------
+
+Pipenv manages dependencies on a per-project basis. To install packages,
+change into your project's directory (or just an empty directory for this
+tutorial) and run:
+
+.. code-block:: console
+
+ $ cd project_folder
+ $ pipenv install requests
+
+Pipenv will install the excellent `Requests`_ library and create a ``Pipfile``
+for you in your project's directory. The `Pipfile`_ is used to track which
+dependencies your project needs in case you need to re-install them, such as
+when you share your project with others. You should get output similar to this
+(although the exact paths shown will vary):
+
+.. _Pipfile: https://github.com/pypa/pipfile
+
+.. code-block:: text
+
+ Creating a Pipfile for this project...
+ Creating a virtualenv for this project...
+ Using base prefix '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6'
+ New python executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python3.6
+ Also creating executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python
+ Installing setuptools, pip, wheel...done.
+
+ Virtualenv location: ~/.local/share/virtualenvs/tmp-agwWamBd
+ Installing requests...
+ Collecting requests
+ Using cached requests-2.18.4-py2.py3-none-any.whl
+ Collecting idna<2.7,>=2.5 (from requests)
+ Using cached idna-2.6-py2.py3-none-any.whl
+ Collecting urllib3<1.23,>=1.21.1 (from requests)
+ Using cached urllib3-1.22-py2.py3-none-any.whl
+ Collecting chardet<3.1.0,>=3.0.2 (from requests)
+ Using cached chardet-3.0.4-py2.py3-none-any.whl
+ Collecting certifi>=2017.4.17 (from requests)
+ Using cached certifi-2017.7.27.1-py2.py3-none-any.whl
+ Installing collected packages: idna, urllib3, chardet, certifi, requests
+ Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22
+
+ Adding requests to Pipfile's [packages]...
+ P.S. You have excellent taste! ✨ 🍰 ✨
+
+.. _Requests: https://requests.readthedocs.io/en/latest/
+
+
+Using installed packages
+------------------------
+
+Now that Requests is installed you can create a simple ``main.py`` file to
+use it:
+
+.. code-block:: python
+
+ import requests
+
+ response = requests.get('https://httpbin.org/ip')
+
+ print('Your IP is {0}'.format(response.json()['origin']))
+
+Then you can run this script using ``pipenv run``:
+
+.. code-block:: console
+
+ $ pipenv run python main.py
+
+You should get output similar to this:
+
+.. code-block:: text
+
+ Your IP is 8.8.8.8
+
+Using ``$ pipenv run`` ensures that your installed packages are available to
+your script. It's also possible to spawn a new shell that ensures all commands
+have access to your installed packages with ``$ pipenv shell``.
+
+
+Next steps
----------
-`virtualenv `_ is a tool to create
-isolated Python environments.
+Congratulations, you now know how to install and use Python packages! ✨ 🍰 ✨
+
+
-Install it via pip:
+Lower level: virtualenv
+=======================
+
+`virtualenv `_ is a tool to create
+isolated Python environments. virtualenv creates a folder which contains all the
+necessary executables to use the packages that a Python project would need.
+
+It can be used standalone, in place of Pipenv.
+
+Install virtualenv via pip:
.. code-block:: console
$ pip install virtualenv
+Test your installation:
+
+.. code-block:: console
+
+ $ virtualenv --version
+
Basic Usage
-~~~~~~~~~~~
+-----------
-1. Create a virtual environment:
+1. Create a virtual environment for a project:
.. code-block:: console
+ $ cd project_folder
$ virtualenv venv
+``virtualenv venv`` will create a folder in the current directory which will
+contain the Python executable files, and a copy of the ``pip`` library which you
+can use to install other packages. The name of the virtual environment (in this
+case, it was ``venv``) can be anything; omitting the name will place the files
+in the current directory instead.
+
+.. note::
+ 'venv' is the general convention used globally. As it is readily available in ignore files (eg: .gitignore')
+
This creates a copy of Python in whichever directory you ran the command in,
-placing it in a folder named ``venv``.
+placing it in a folder named :file:`venv`.
+
+You can also use the Python interpreter of your choice (like
+``python2.7``).
+
+.. code-block:: console
+
+ $ virtualenv -p /usr/bin/python2.7 venv
+
+or change the interpreter globally with an env variable in ``~/.bashrc``:
+
+.. code-block:: console
+
+ $ export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python2.7
2. To begin using the virtual environment, it needs to be activated:
@@ -38,29 +259,83 @@ placing it in a folder named ``venv``.
$ source venv/bin/activate
-You can then begin installing any new modules without affecting the system
-default Python or other virtual environments.
+The name of the current virtual environment will now appear on the left of
+the prompt (e.g. ``(venv)Your-Computer:project_folder UserName$``) to let you know
+that it's active. From now on, any package that you install using pip will be
+placed in the ``venv`` folder, isolated from the global Python installation.
+
+For Windows, the same command mentioned in step 1 can be used to create a virtual environment. However, activating the environment requires a slightly different command.
+
+Assuming that you are in your project directory:
+
+.. code-block:: console
+
+ C:\Users\SomeUser\project_folder> venv\Scripts\activate
+
+Install packages using the ``pip`` command:
+
+.. code-block:: console
+
+ $ pip install requests
3. If you are done working in the virtual environment for the moment, you can
deactivate it:
.. code-block:: console
- $ deactivate
+ $ deactivate
This puts you back to the system's default Python interpreter with all its
installed libraries.
-To delete a virtual environment, just delete its folder.
+To delete a virtual environment, just delete its folder. (In this case,
+it would be ``rm -rf venv``.)
After a while, though, you might end up with a lot of virtual environments
-littered across your system, and its possible you'll forget their names or
+littered across your system, and it's possible you'll forget their names or
where they were placed.
+.. note::
+ Python has included venv module from version 3.3. For more details: `venv `_.
+
+Other Notes
+-----------
+
+Running ``virtualenv`` with the option ``--no-site-packages`` will not
+include the packages that are installed globally. This can be useful
+for keeping the package list clean in case it needs to be accessed later.
+[This is the default behavior for ``virtualenv`` 1.7 and later.]
+
+In order to keep your environment consistent, it's a good idea to "freeze"
+the current state of the environment packages. To do this, run:
+
+.. code-block:: console
+
+ $ pip freeze > requirements.txt
+
+This will create a :file:`requirements.txt` file, which contains a simple
+list of all the packages in the current environment, and their respective
+versions. You can see the list of installed packages without the requirements
+format using ``pip list``. Later it will be easier for a different developer
+(or you, if you need to re-create the environment) to install the same packages
+using the same versions:
+
+.. code-block:: console
+
+ $ pip install -r requirements.txt
+
+This can help ensure consistency across installations, across deployments,
+and across developers.
+
+Lastly, remember to exclude the virtual environment folder from source
+control by adding it to the ignore list (see :ref:`Version Control Ignores`).
+
+.. _virtualenvwrapper-ref:
+
virtualenvwrapper
-----------------
-`virtualenvwrapper `_
+`virtualenvwrapper `_
provides a set of commands which makes working with virtual environments much
more pleasant. It also places all your virtual environments in one place.
@@ -72,18 +347,17 @@ To install (make sure **virtualenv** is already installed):
$ export WORKON_HOME=~/Envs
$ source /usr/local/bin/virtualenvwrapper.sh
-(`Full virtualenvwrapper install instructions `_.)
+(`Full virtualenvwrapper install instructions `_.)
-For Windows, you can use the `virtualenvwrapper-powershell `_ clone.
+For Windows, you can use the `virtualenvwrapper-win `_.
To install (make sure **virtualenv** is already installed):
.. code-block:: console
- PS> pip install virtualenvwrapper-powershell
- PS> $env:WORKON_HOME="~/Envs"
- PS> mkdir $env:WORKON_HOME
- PS> import-module virtualenvwrapper
+ $ pip install virtualenvwrapper-win
+
+In Windows, the default path for WORKON_HOME is %USERPROFILE%\\Envs
Basic Usage
~~~~~~~~~~~
@@ -92,19 +366,28 @@ Basic Usage
.. code-block:: console
- $ mkvirtualenv venv
+ $ mkvirtualenv project_folder
-This creates the ``venv`` folder inside ``~/Envs``.
+This creates the :file:`project_folder` folder inside :file:`~/Envs`.
2. Work on a virtual environment:
.. code-block:: console
- $ workon venv
+ $ workon project_folder
+
+Alternatively, you can make a project, which creates the virtual environment,
+and also a project directory inside ``$WORKON_HOME``, which is ``cd``-ed into
+when you ``workon project_folder``.
+
+.. code-block:: console
+
+ $ mkproject project_folder
**virtualenvwrapper** provides tab-completion on environment names. It really
helps when you have a lot of environments and have trouble remembering their
names.
+
``workon`` also deactivates whatever environment you are currently in, so you
can quickly switch between environments.
@@ -128,30 +411,31 @@ Other useful commands
``cdvirtualenv``
Navigate into the directory of the currently activated virtual environment,
- so you can browse its ``site-packages``, for example.
+ so you can browse its :file:`site-packages`, for example.
``cdsitepackages``
- Like the above, but directly into ``site-packages`` directory.
+ Like the above, but directly into :file:`site-packages` directory.
``lssitepackages``
- Shows contents of ``site-packages`` directory.
+ Shows contents of :file:`site-packages` directory.
+
+`Full list of virtualenvwrapper commands `_.
+
+virtualenv-burrito
+------------------
-`Full list of virtualenvwrapper commands `_.
+With `virtualenv-burrito `_, you
+can have a working virtualenv + virtualenvwrapper environment in a single command.
-autoenv
+direnv
-------
-When you ``cd`` into a directory containing a ``.env`` `autoenv `_
+When you ``cd`` into a directory containing a :file:`.env`, `direnv `_
automagically activates the environment.
Install it on Mac OS X using ``brew``:
.. code-block:: console
- $ brew install autoenv
-
-And on Linux:
-
-.. code-block:: console
+ $ brew install direnv
- $ git clone git://github.com/kennethreitz/autoenv.git ~/.autoenv
- $ echo 'source ~/.autoenv/activate.sh' >> ~/.bashrc
+On Linux follow the instructions at `direnv.net `_
diff --git a/docs/index.rst b/docs/index.rst
index 19e167222..df7fcb8ed 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,17 +1,33 @@
-.. osxpython documentation master file, created by
+.. pythonguide documentation master file, created by
sphinx-quickstart on Wed Aug 4 22:51:11 2010.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
+.. meta::
+ :description: An opinionated guide to the Python programming language and a best practice handbook for the installation, configuration, and usage of Python on a daily basis.
+
+
+#################################
The Hitchhiker's Guide to Python!
-=================================
+#################################
-Welcome to The Hitchhiker's Guide to Python.
+Greetings, Earthling! Welcome to The Hitchhiker's Guide to Python.
-**This guide is currently under heavy active development.** If you'd like to help, `fork us on GitHub `_!
+**This is a living, breathing guide.** If you'd like to contribute,
+`fork us on GitHub `_!
-This *opinionated* guide exists to provide both novice and expert Python
-developers a best-practice handbook to the installation, configuration, and
+This handcrafted guide exists to provide both novice and expert Python
+developers a best practice handbook for the installation, configuration, and
usage of Python on a daily basis.
+This guide is **opinionated** in a way that is almost, but not quite, entirely
+*unlike* Python's official documentation. You won't find a list of every Python web framework
+available here. Rather, you'll find a nice concise list of highly recommended
+options.
+
+.. note:: The use of **Python 3** is *highly* recommended over Python 2. Consider upgrading your applications and infrastructures if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste.
+ —*Kenneth Reitz*
+
+Let's get started! But first, let's make sure you know where your towel is.
+
.. include:: contents.rst.inc
diff --git a/docs/intro/community.rst b/docs/intro/community.rst
index 8ae20192d..4ce563658 100644
--- a/docs/intro/community.rst
+++ b/docs/intro/community.rst
@@ -1,18 +1,25 @@
.. _the-community:
+
+#############
The Community
-=============
+#############
+
+.. image:: /_static/photos/34689432801_78d97ecec9_k_d.jpg
+
+****
BDFL
-----
+****
Guido van Rossum, the creator of Python, is often referred to as the BDFL — the
Benevolent Dictator For Life.
-
+**************************
Python Software Foundation
---------------------------
+**************************
+
The mission of the Python Software Foundation is to promote, protect, and
advance the Python programming language, and to support and facilitate the
@@ -21,13 +28,14 @@ growth of a diverse and international community of Python programmers.
`Learn More about the PSF `_.
+****
PEPs
-----
+****
PEPs are *Python Enhancement Proposals*. They describe changes to Python itself,
or the standards around it.
-There are three different types of PEPs (as defined by `PEP1 `_):
+There are three different types of PEPs (as defined by :pep:`1`):
**Standards**
Describes a new feature or implementation.
@@ -45,13 +53,13 @@ Notable PEPs
There are a few PEPs that could be considered required reading:
-- `PEP8 `_: The Python Style Guide.
+- :pep:`8`: The Python Style Guide.
Read this. All of it. Follow it.
-- `PEP20 `_: The Zen of Python.
+- :pep:`20`: The Zen of Python.
A list of 19 statements that briefly explain the philosophy behind Python.
-- `PEP257 `_: Docstring Conventions.
+- :pep:`257`: Docstring Conventions.
Gives guidelines for semantics and conventions associated with Python
docstrings.
@@ -60,27 +68,48 @@ You can read more at `The PEP Index `_.
Submitting a PEP
~~~~~~~~~~~~~~~~
- PEPs are peer-reviewed and accepted/rejected after much discussion. Anyone
- can write and submit a PEP for review.
+PEPs are peer-reviewed and accepted/rejected after much discussion. Anyone
+can write and submit a PEP for review.
- Here's an overview of the PEP acceptance workflow:
+Here's an overview of the PEP acceptance workflow:
- .. image:: http://www.python.org/dev/peps/pep-0001/pep-0001-1.png
+.. image:: ../_static/pep-0001-1.png
+******************
Python Conferences
---------------------------
+******************
The major events for the Python community are developer conferences. The two
most notable conferences are PyCon, which is held in the US, and its European
sibling, EuroPython.
-A comprehensive list of conferences is maintained `at pycon.org `_.
+A comprehensive list of conferences is maintained at `pycon.org `_.
+******************
Python User Groups
---------------------------
+******************
User Groups are where a bunch of Python developers meet to present or talk
about Python topics of interest. A list of local user groups is maintained at
the `Python Software Foundation Wiki `_.
+
+
+******************
+Online Communities
+******************
+
+`PythonistaCafe `_ is an invite-only, online community
+of Python and software development enthusiasts helping each other succeed and grow.
+Think of it as a club of mutual improvement for Pythonistas where a broad range of
+programming questions, career advice, and other topics are discussed every day.
+
+
+*****************
+Python Job Boards
+*****************
+
+`Python Jobs HQ `_ is a Python job board, by Python Developers
+for Python Developers. The site aggregates Python job postings from across the web and
+also allows employers to post Python job openings directly on the site.
diff --git a/docs/intro/documentation.rst b/docs/intro/documentation.rst
index cdc796dc7..887d0982b 100644
--- a/docs/intro/documentation.rst
+++ b/docs/intro/documentation.rst
@@ -1,21 +1,49 @@
+
+
+#############
Documentation
-=============
+#############
+
+.. image:: /_static/photos/33928823133_2f3d32cf32_k_d.jpg
+
+**********************
Official Documentation
-----------------------
+**********************
The official Python Language and Library documentation can be found here:
- - `Python 2.x `_
- - `Python 3.x `_
+ - `Python 2.x `_
+ - `Python 3.x `_
+*************
Read the Docs
--------------
+*************
+
+Read the Docs is a popular community project that hosts documentation
+for open source software. It holds documentation for many Python modules,
+both popular and exotic.
+
+ `Read the Docs `_
+
+
+*****
+pydoc
+*****
+
+:program:`pydoc` is a utility that is installed when you install Python.
+It allows you to quickly retrieve and search for documentation from your
+shell. For example, if you needed a quick refresher on the
+:mod:`time` module, pulling up documentation would be as simple as:
+
+ .. code-block:: console
-Read the Docs is a popular community project, providing a single location for
-all documentation of popular and even more exotic Python modules.
+ $ pydoc time
- `Read the Docs `_
+The above command is essentially equivalent to opening the Python REPL
+and running:
+ .. code-block:: pycon
+ >>> help(time)
diff --git a/docs/intro/duction.rst b/docs/intro/duction.rst
index 7b0cf7dda..16baebc0d 100644
--- a/docs/intro/duction.rst
+++ b/docs/intro/duction.rst
@@ -1,5 +1,10 @@
+
+
+############
Introduction
-============
+############
+
+.. image:: /_static/photos/34725946825_0f85497e60_k_d.jpg
From the `official Python website `_:
@@ -7,30 +12,30 @@ Python is a general-purpose, high-level programming language similar
to Tcl, Perl, Ruby, Scheme, or Java. Some of its main key features
include:
-* very clear, readable syntax
+* **very clear, readable syntax**
Python's philosophy focuses on readability, from code blocks
delineated with significant whitespace to intuitive keywords in
- place of inscrutable punctuation
+ place of inscrutable punctuation.
-* extensive standard libraries and third party modules for virtually
- any task
+* **extensive standard libraries and third party modules for virtually
+ any task**
Python is sometimes described with the words "batteries included"
- for its extensive
+ because of its extensive
`standard library `_, which includes
modules for regular expressions, file IO, fraction handling,
object serialization, and much more.
Additionally, the
- `Python Package Index `_ is available
+ `Python Package Index `_ is available
for users to submit their packages for widespread use, similar to
- Perl's `CPAN `_. There is a thriving community
+ Perl's `CPAN `_. There is a thriving community
of very powerful Python frameworks and tools like
- the `Django `_ web framework and the
+ the `Django `_ web framework and the
`NumPy `_ set of math routines.
-* integration with other systems
+* **integration with other systems**
Python can integrate with `Java libraries `_,
enabling it to be used with the rich Java environment that corporate
@@ -38,13 +43,13 @@ include:
`extended by C or C++ modules `_
when speed is of the essence.
-* ubiquity on computers
+* **ubiquity on computers**
Python is available on Windows, \*nix, and Mac. It runs wherever the
Java virtual machine runs, and the reference implementation CPython
can help bring Python to wherever there is a working C compiler.
-* friendly community
+* **friendly community**
Python has a vibrant and large :ref:`community `
which maintains wikis, conferences, countless repositories,
@@ -54,14 +59,16 @@ include:
.. _about-ref:
+
+****************
About This Guide
-----------------
+****************
Purpose
~~~~~~~
The Hitchhiker's Guide to Python exists to provide both novice and expert
-Python developers a best-practice handbook to the installation, configuration,
+Python developers a best practice handbook for the installation, configuration,
and usage of Python on a daily basis.
@@ -77,12 +84,12 @@ For the Community
All contributions to the Guide are welcome, from Pythonistas of all levels.
If you think there's a gap in what the Guide covers, fork the Guide on
-GitHub and submit a pull request. Contributions are welcome from everyone,
-whether they're an old hand or a first-time Pythonista, and the authors to
-the Guide will gladly help if you have any questions about the
-appropriateness, completeness, or accuracy of a contribution.
-
-To get started working on The Hitchhiker's Guide, see
-the: doc:`/notes/contribute` page.
+GitHub and submit a pull request.
+Contributions are welcome from everyone, whether they're an old hand or a
+first-time Pythonista, and the authors to the Guide will gladly help if you
+have any questions about the appropriateness, completeness, or accuracy of
+a contribution.
+To get started working on The Hitchhiker's Guide,
+see the :doc:`/notes/contribute` page.
diff --git a/docs/intro/learning.rst b/docs/intro/learning.rst
index 5425d84f3..b0c957398 100644
--- a/docs/intro/learning.rst
+++ b/docs/intro/learning.rst
@@ -1,26 +1,114 @@
+
+
+###############
Learning Python
-===============
+###############
+
+.. image:: /_static/photos/32800783863_11a00db52c_k_d.jpg
+
+********
Beginner
---------
+********
+
+The Python Tutorial
+~~~~~~~~~~~~~~~~~~~~
+
+This is the official tutorial. It covers all the basics, and offers a tour of
+the language and the standard library. Recommended for those who need a
+quick-start guide to the language.
+
+ `The Python Tutorial `_
+
+Real Python
+~~~~~~~~~~~
+
+Real Python is a repository of free and in-depth Python tutorials created by a diverse team of professional Python developers. At Real Python you can learn all things Python from the ground up. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond.
+
+ `Real Python `_
+
+Python Basics
+~~~~~~~~~~~~~
+
+pythonbasics.org is an introductory tutorial for beginners. The tutorial includes exercises. It covers the basics and there are also in-depth lessons like object oriented programming and regular expressions.
+
+ `Python basics `_
+
+Python for Beginners
+~~~~~~~~~~~~~~~~~~~~
+
+thepythonguru.com is a tutorial focused on beginner programmers. It covers many Python concepts
+in depth. It also teaches you some advanced constructs of Python like lambda expressions and regular expressions.
+And last it finishes off with the tutorial "How to access MySQL db using Python"
+
+ `Python for Beginners `_
Learn Python Interactive Tutorial
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Learnpython.org is an easy non-intimidating way to get introduced to python.
-The website takes the same approach used on the popular `Try Ruby `_
-website, it has an interactive python interpreter built into the site that
-allows you to go through the lessons without having to install Python locally.
+Learnpython.org is an easy non-intimidating way to get introduced to Python.
+The website takes the same approach used on the popular
+`Try Ruby `_ website. It has an interactive Python
+interpreter built into the site that allows you to go through the lessons
+without having to install Python locally.
`Learn Python `_
+Python for You and Me
+~~~~~~~~~~~~~~~~~~~~~
+
+If you want a more traditional book, *Python For You and Me* is an excellent
+resource for learning all aspects of the language.
+
+ `Python for You and Me `_
+
+Learn Python Step by Step
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Techbeamers.com provides step-by-step tutorials to teach Python. Each tutorial is supplemented with logically added coding snippets and equips with a follow-up quiz on the subject learned. There is a section for `Python interview questions `_ to help job seekers. You can also read essential `Python tips `_ and learn `best coding practices `_ for writing quality code. Here, you'll get the right platform to learn Python quickly.
+
+`Learn Python Basic to Advanced `_
+
+
+Online Python Tutor
+~~~~~~~~~~~~~~~~~~~
+
+Online Python Tutor gives you a visual step-by-step
+representation of how your program runs. Python Tutor
+helps people overcome a fundamental barrier to learning
+programming by understanding what happens as the computer
+executes each line of a program's source code.
+
+ `Online Python Tutor `_
+
+Invent Your Own Computer Games with Python
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This beginner's book is for those with no programming experience at all. Each
+chapter has the source code to a small game, using these example programs
+to demonstrate programming concepts to give the reader an idea of what
+programs "look like".
+
+ `Invent Your Own Computer Games with Python `_
+
+
+Hacking Secret Ciphers with Python
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This book teaches Python programming and basic cryptography for absolute
+beginners. The chapters provide the source code for various ciphers, as well
+as programs that can break them.
+
+ `Hacking Secret Ciphers with Python `_
+
+
Learn Python the Hard Way
~~~~~~~~~~~~~~~~~~~~~~~~~
This is an excellent beginner programmer's guide to Python. It covers "hello
world" from the console to the web.
- `Learn Python the Hard Way `_
+ `Learn Python the Hard Way `_
Crash into Python
@@ -29,7 +117,7 @@ Crash into Python
Also known as *Python for Programmers with 3 Hours*, this guide gives
experienced developers from other languages a crash course on Python.
- `Crash into Python `_
+ `Crash into Python `_
Dive Into Python 3
@@ -39,22 +127,23 @@ Dive Into Python 3 is a good book for those ready to jump in to Python 3. It's
a good read if you are moving from Python 2 to 3 or if you already have some
experience programming in another language.
- `Dive Into Python 3 `_
+ `Dive Into Python 3 `_
+
Think Python: How to Think Like a Computer Scientist
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Think Python attempts to give an introduction to basic concepts in computer
-science through the use of the python language. The focus was to create a book
-with plenty of exercises, minimal jargon and a section in each chapter devoted
+science through the use of the Python language. The focus was to create a book
+with plenty of exercises, minimal jargon, and a section in each chapter devoted
to the subject of debugging.
-While exploring the various features available in the python language the
+While exploring the various features available in the Python language the
author weaves in various design patterns and best practices.
The book also includes several case studies which have the reader explore the
topics discussed in the book in greater detail by applying those topics to
-real-world examples. Case studies include assignments in GUI and Markov
+real-world examples. Case studies include assignments in GUI programming and Markov
Analysis.
`Think Python `_
@@ -64,38 +153,90 @@ Python Koans
~~~~~~~~~~~~
Python Koans is a port of Edgecase's Ruby Koans. It uses a test-driven
-approach, q.v. TEST DRIVEN DESIGN SECTION to provide an interactive tutorial
-teaching basic python concepts. By fixing assertion statements that fail in a
-test script, this provides sequential steps to learning python.
+approach to provide an interactive tutorial
+teaching basic Python concepts. By fixing assertion statements that fail in a
+test script, this provides sequential steps to learning Python.
For those used to languages and figuring out puzzles on their own, this can be
-a fun, attractive option. For those new to python and programming, having an
+a fun, attractive option. For those new to Python and programming, having an
additional resource or reference will be helpful.
- `Python Koans `_
+ `Python Koans `_
More information about test driven development can be found at these resources:
- `Test Driven Development `_
+ `Test Driven Development `_
+
A Byte of Python
~~~~~~~~~~~~~~~~
-A free introductory book that teaches python at the beginner level, it assumes no
-previous programming experience.
+A free introductory book that teaches Python at the beginner level, it assumes
+no previous programming experience.
`A Byte of Python for Python 2.x `_
- `A Byte of Python for Python 3.x `_
+ `A Byte of Python for Python 3.x `_
+
+Computer Science Path on Codecademy
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A Codecademy course for the absolute Python beginner. This free and interactive
+course provides and teaches the basics (and beyond) of Python programming while
+testing the user's knowledge in between progress.
+This course also features a built-in interpreter for receiving instant feedback on your learning.
+
+ `Computer Science Path on Codecademy `_
+
+
+Code the blocks
+~~~~~~~~~~~~~~~
+
+*Code the blocks* provides free and interactive Python tutorials for
+beginners. It combines Python programming with a 3D environment where
+you "place blocks" and construct structures. The tutorials teach you
+how to use Python to create progressively more elaborate 3D structures,
+making the process of learning Python fun and engaging.
+
+ `Code the blocks `_
+
+
+************
+Intermediate
+************
+
+Python Tricks: The Book
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Discover Python's best practices with simple examples and start writing even more beautiful + Pythonic code. *Python Tricks: The Book* shows you exactly how.
+
+You’ll master intermediate and advanced-level features in Python with practical examples and a clear narrative.
+
+ `Python Tricks: The Book `_
+
+Effective Python
+~~~~~~~~~~~~~~~~
+This book contains 59 specific ways to improve writing Pythonic code. At 227
+pages, it is a very brief overview of some of the most common adaptations
+programmers need to make to become efficient intermediate level Python
+programmers.
+
+ `Effective Python `_
+
+
+********
Advanced
---------
+********
Pro Python
~~~~~~~~~~
-This book is for intermediate to advanced Python programmers who are looking to understand how
-and why Python works the way it does and how they can take their code to the next level.
+This book is for intermediate to advanced Python programmers who are looking to
+understand how and why Python works the way it does and how they can take their
+code to the next level.
+
+ `Pro Python `_
Expert Python Programming
@@ -104,39 +245,168 @@ Expert Python Programming deals with best practices in programming Python and
is focused on the more advanced crowd.
It starts with topics like decorators (with caching, proxy, and context manager
-case-studies), method resolution order, using super() and meta-programming, and
-general PEP8 best practices.
+case studies), method resolution order, using super() and meta-programming, and
+general :pep:`8` best practices.
It has a detailed, multi-chapter case study on writing and releasing a package
and eventually an application, including a chapter on using zc.buildout. Later
-chapters detail best practices with writing documentation, test-driven
-development, version control, and optimization/profiling.
+chapters detail best practices such as writing documentation, test-driven
+development, version control, optimization, and profiling.
- `Expert Python Programming `_
+ `Expert Python Programming `_
-The Python Tutorial
-~~~~~~~~~~~~~~~~~~~~
-This is the official tutorial, it covers all the basics, and offers a tour of the
-language and the standard library, recommended for those who need a quickstart
-guide to the language.
+A Guide to Python's Magic Methods
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a collection of blog posts by Rafe Kettler which explain 'magic methods'
+in Python. Magic methods are surrounded by double underscores (i.e. __init__)
+and can make classes and objects behave in different and magical ways.
- `The Python Tutorial `_
+ `A Guide to Python's Magic Methods `_
+
+.. note:: Rafekettler.com is currently down; you can go to their GitHub version directly. Here you can find a PDF version:
+ `A Guide to Python's Magic Methods (repo on GitHub) `_
+
+
+****************************
+For Engineers and Scientists
+****************************
+
+A Primer on Scientific Programming with Python
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A Primer on Scientific Programming with Python, written by Hans Petter
+Langtangen, mainly covers Python's usage in the scientific field. In the book,
+examples are chosen from mathematics and the natural sciences.
+
+ `A Primer on Scientific Programming with Python `_
+
+Numerical Methods in Engineering with Python
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Numerical Methods in Engineering with Python, written by Jaan Kiusalaas,
+puts the emphasis on numerical methods and how to implement them in Python.
+
+ `Numerical Methods in Engineering with Python `_
+
+
+********************
+Miscellaneous Topics
+********************
+
+Problem Solving with Algorithms and Data Structures
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Problem Solving with Algorithms and Data Structures covers a range of data
+structures and algorithms. All concepts are illustrated with Python code along
+with interactive samples that can be run directly in the browser.
+
+ `Problem Solving with Algorithms and Data Structures
+ `_
+
+Programming Collective Intelligence
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Programming Collective Intelligence introduces a wide array of basic machine
+learning and data mining methods. The exposition is not very mathematically
+formal, but rather focuses on explaining the underlying intuition and shows
+how to implement the algorithms in Python.
+
+ `Programming Collective Intelligence `_
+
+
+Transforming Code into Beautiful, Idiomatic Python
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Transforming Code into Beautiful, Idiomatic Python is a video by Raymond Hettinger.
+Learn to take better advantage of Python's best features and improve existing code
+through a series of code transformations: "When you see this, do that instead."
+
+ `Transforming Code into Beautiful, Idiomatic Python `_
+
+
+Fullstack Python
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Fullstack Python offers a complete top-to-bottom resource for web development
+using Python.
+
+From setting up the web server, to designing the front-end, choosing a database,
+optimizing/scaling, etc.
+
+As the name suggests, it covers everything you need to build and run a complete
+web app from scratch.
+
+ `Fullstack Python `_
+
+
+PythonistaCafe
+~~~~~~~~~~~~~~
+
+PythonistaCafe is an invite-only, online community of Python and software development enthusiasts helping each other succeed and grow. Think of it as a club of mutual improvement for Pythonistas where a broad range of programming questions, career advice, and other topics are discussed every day.
+
+ `PythonistaCafe `_
+
+
+**********
References
-----------
+**********
Python in a Nutshell
~~~~~~~~~~~~~~~~~~~~
Python in a Nutshell, written by Alex Martelli, covers most cross-platform
-python's usage, from its syntax to built-in libraries to advanced topics such
+Python usage, from its syntax to built-in libraries to advanced topics such
as writing C extensions.
+ `Python in a Nutshell `_
+
The Python Language Reference
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-This is Python's reference manual, it covers the syntax and the core semantics of the
-language.
+This is Python's reference manual. It covers the syntax and the core semantics
+of the language.
`The Python Language Reference `_
+
+Python Essential Reference
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Python Essential Reference, written by David Beazley, is the definitive reference
+guide to Python. It concisely explains both the core language and the most essential
+parts of the standard library. It covers Python 3 and 2.6 versions.
+
+ `Python Essential Reference `_
+
+Python Pocket Reference
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Python Pocket Reference, written by Mark Lutz, is an easy to use reference to
+the core language, with descriptions of commonly used modules and toolkits. It
+covers Python 3 and 2.6 versions.
+
+ `Python Pocket Reference `_
+
+Python Cookbook
+~~~~~~~~~~~~~~~
+
+Python Cookbook, written by David Beazley and Brian K. Jones, is packed with
+practical recipes. This book covers the core Python language as well as tasks
+common to a wide variety of application domains.
+
+ `Python Cookbook `_
+
+Writing Idiomatic Python
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Writing Idiomatic Python, written by Jeff Knupp, contains the most common and
+important Python idioms in a format that maximizes identification and
+understanding. Each idiom is presented as a recommendation of a way to write
+some commonly used piece of code, followed by an explanation of why the idiom
+is important. It also contains two code samples for each idiom: the "Harmful"
+way to write it and the "Idiomatic" way.
+
+ `For Python 2.7.3+ `_
+
+ `For Python 3.3+ `_
diff --git a/docs/intro/news.rst b/docs/intro/news.rst
index 74a5b44ab..d1cda93be 100644
--- a/docs/intro/news.rst
+++ b/docs/intro/news.rst
@@ -1,33 +1,102 @@
+
+
+####
News
-====
+####
+
+.. image:: /_static/photos/33573767786_eececc5d27_k_d.jpg
+
+
+****************
+PyCoder’s Weekly
+****************
+
+PyCoder’s Weekly is a free weekly Python newsletter for Python developers
+by Python developers (Projects, Articles, News, and Jobs).
+
+ `PyCoder’s Weekly `_
+
+***********
+Real Python
+***********
+
+At Real Python you can learn all things Python from the ground up, with weekly free and in-depth tutorials. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond.
+
+ `Real Python `_
+
+
+*************
Planet Python
-~~~~~~~~~~~~~
+*************
This is an aggregate of Python news from a growing number of developers.
- `Planet Python `_
+ `Planet Python `_
+
+*********
/r/python
-~~~~~~~~~
+*********
/r/python is the Reddit Python community where users contribute and vote on
Python-related news.
- `/r/python `_
+ `/r/python `_
+
+
+*******************
+Talk Python Podcast
+*******************
+
+The #1 Python-focused podcast covering the people and ideas in Python.
-Pycoder's Weekly
-~~~~~~~~~~~~~~~~
+ `Talk Python To Me `_
-Pycoder's Weekly is a free weekly python newsletter for Python developers
-by Python developers (Project, Articles, News, and Jobs).
- `Pycoder's Weekly `_
+********************
+Python Bytes Podcast
+********************
+A short-form Python podcast covering recent developer headlines.
+
+ `Python Bytes `_
+
+
+*************
Python Weekly
-~~~~~~~~~~~~~
+*************
Python Weekly is a free weekly newsletter featuring curated news, articles,
new releases, jobs, etc. related to Python.
- `Python Weekly `_
+ `Python Weekly `_
+
+
+***********
+Python News
+***********
+
+Python News is the news section in the official Python web site
+(www.python.org). It briefly highlights the news from the Python community.
+
+ `Python News `_
+
+
+********************
+Import Python Weekly
+********************
+
+Weekly Python Newsletter containing Python Articles, Projects, Videos, and Tweets
+delivered in your inbox. Keep Your Python Programming Skills Updated.
+
+ `Import Python Weekly Newsletter `_
+
+
+*************************
+Awesome Python Newsletter
+*************************
+
+A weekly overview of the most popular Python news, articles, and packages.
+
+ `Awesome Python Newsletter `_
diff --git a/docs/make.bat b/docs/make.bat
index 39f2a6893..115651a00 100644
--- a/docs/make.bat
+++ b/docs/make.bat
@@ -7,8 +7,10 @@ if "%SPHINXBUILD%" == "" (
)
set BUILDDIR=_build
set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% .
+set I18NSPHINXOPTS=%SPHINXOPTS% .
if NOT "%PAPER%" == "" (
set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS%
+ set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS%
)
if "%1" == "" goto help
@@ -28,7 +30,11 @@ if "%1" == "help" (
echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter
echo. text to make text files
echo. man to make manual pages
+ echo. texinfo to make Texinfo files
+ echo. gettext to make PO message catalogs
echo. changes to make an overview over all changed/added/deprecated items
+ echo. xml to make Docutils-native XML files
+ echo. pseudoxml to make pseudoxml-XML files for display purposes
echo. linkcheck to check all external links for integrity
echo. doctest to run all doctests embedded in the documentation if enabled
goto end
@@ -40,8 +46,34 @@ if "%1" == "clean" (
goto end
)
+
+REM Check if sphinx-build is available and fallback to Python version if any
+%SPHINXBUILD% 2> nul
+if errorlevel 9009 goto sphinx_python
+goto sphinx_ok
+
+:sphinx_python
+
+set SPHINXBUILD=python -m sphinx.__init__
+%SPHINXBUILD% 2> nul
+if errorlevel 9009 (
+ echo.
+ echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+ echo.installed, then set the SPHINXBUILD environment variable to point
+ echo.to the full path of the 'sphinx-build' executable. Alternatively you
+ echo.may add the Sphinx directory to PATH.
+ echo.
+ echo.If you don't have Sphinx installed, grab it from
+ echo.http://sphinx-doc.org/
+ exit /b 1
+)
+
+:sphinx_ok
+
+
if "%1" == "html" (
%SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished. The HTML pages are in %BUILDDIR%/html.
goto end
@@ -49,6 +81,7 @@ if "%1" == "html" (
if "%1" == "dirhtml" (
%SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml.
goto end
@@ -56,6 +89,7 @@ if "%1" == "dirhtml" (
if "%1" == "singlehtml" (
%SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml.
goto end
@@ -63,6 +97,7 @@ if "%1" == "singlehtml" (
if "%1" == "pickle" (
%SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished; now you can process the pickle files.
goto end
@@ -70,6 +105,7 @@ if "%1" == "pickle" (
if "%1" == "json" (
%SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished; now you can process the JSON files.
goto end
@@ -77,6 +113,7 @@ if "%1" == "json" (
if "%1" == "htmlhelp" (
%SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished; now you can run HTML Help Workshop with the ^
.hhp project file in %BUILDDIR%/htmlhelp.
@@ -85,17 +122,19 @@ if "%1" == "htmlhelp" (
if "%1" == "qthelp" (
%SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished; now you can run "qcollectiongenerator" with the ^
.qhcp project file in %BUILDDIR%/qthelp, like this:
- echo.^> qcollectiongenerator %BUILDDIR%\qthelp\osxpython.qhcp
+ echo.^> qcollectiongenerator %BUILDDIR%\qthelp\pythonguide.qhcp
echo.To view the help file:
- echo.^> assistant -collectionFile %BUILDDIR%\qthelp\osxpython.ghc
+ echo.^> assistant -collectionFile %BUILDDIR%\qthelp\pythonguide.ghc
goto end
)
if "%1" == "devhelp" (
%SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished.
goto end
@@ -103,6 +142,7 @@ if "%1" == "devhelp" (
if "%1" == "epub" (
%SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished. The epub file is in %BUILDDIR%/epub.
goto end
@@ -110,13 +150,35 @@ if "%1" == "epub" (
if "%1" == "latex" (
%SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished; the LaTeX files are in %BUILDDIR%/latex.
goto end
)
+if "%1" == "latexpdf" (
+ %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
+ cd %BUILDDIR%/latex
+ make all-pdf
+ cd %BUILDDIR%/..
+ echo.
+ echo.Build finished; the PDF files are in %BUILDDIR%/latex.
+ goto end
+)
+
+if "%1" == "latexpdfja" (
+ %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
+ cd %BUILDDIR%/latex
+ make all-pdf-ja
+ cd %BUILDDIR%/..
+ echo.
+ echo.Build finished; the PDF files are in %BUILDDIR%/latex.
+ goto end
+)
+
if "%1" == "text" (
%SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished. The text files are in %BUILDDIR%/text.
goto end
@@ -124,13 +186,31 @@ if "%1" == "text" (
if "%1" == "man" (
%SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man
+ if errorlevel 1 exit /b 1
echo.
echo.Build finished. The manual pages are in %BUILDDIR%/man.
goto end
)
+if "%1" == "texinfo" (
+ %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo
+ if errorlevel 1 exit /b 1
+ echo.
+ echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo.
+ goto end
+)
+
+if "%1" == "gettext" (
+ %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale
+ if errorlevel 1 exit /b 1
+ echo.
+ echo.Build finished. The message catalogs are in %BUILDDIR%/locale.
+ goto end
+)
+
if "%1" == "changes" (
%SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes
+ if errorlevel 1 exit /b 1
echo.
echo.The overview file is in %BUILDDIR%/changes.
goto end
@@ -138,6 +218,7 @@ if "%1" == "changes" (
if "%1" == "linkcheck" (
%SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck
+ if errorlevel 1 exit /b 1
echo.
echo.Link check complete; look for any errors in the above output ^
or in %BUILDDIR%/linkcheck/output.txt.
@@ -146,10 +227,27 @@ or in %BUILDDIR%/linkcheck/output.txt.
if "%1" == "doctest" (
%SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest
+ if errorlevel 1 exit /b 1
echo.
echo.Testing of doctests in the sources finished, look at the ^
results in %BUILDDIR%/doctest/output.txt.
goto end
)
+if "%1" == "xml" (
+ %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml
+ if errorlevel 1 exit /b 1
+ echo.
+ echo.Build finished. The XML files are in %BUILDDIR%/xml.
+ goto end
+)
+
+if "%1" == "pseudoxml" (
+ %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml
+ if errorlevel 1 exit /b 1
+ echo.
+ echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml.
+ goto end
+)
+
:end
diff --git a/docs/notes/contribute.rst b/docs/notes/contribute.rst
index 0d709f29f..b6c72bb4f 100644
--- a/docs/notes/contribute.rst
+++ b/docs/notes/contribute.rst
@@ -1,5 +1,8 @@
+##########
Contribute
-~~~~~~~~~~
+##########
+
+.. image:: /_static/photos/33573769116_49c1ef51e7_k_d.jpg
Python-guide is under active development, and contributors are welcome.
@@ -8,20 +11,24 @@ issue on GitHub_. To submit patches, please send a pull request on GitHub_.
Once your changes get merged back in, you'll automatically be added to the
`Contributors List `_.
+
+***********
Style Guide
------------
+***********
For all contributions, please follow the :ref:`guide-style-guide`.
.. _todo-list-ref:
+
+*********
Todo List
----------
+*********
If you'd like to contribute, there's plenty to do. Here's a short todo_ list.
.. include:: ../../TODO.rst
-.. _GitHub: http://github.com/kennethreitz/python-guide/
+.. _GitHub: https://github.com/kennethreitz/python-guide/
.. _todo: https://github.com/kennethreitz/python-guide/blob/master/TODO.rst
diff --git a/docs/notes/license.rst b/docs/notes/license.rst
index d26b3c78c..1dd3d340a 100644
--- a/docs/notes/license.rst
+++ b/docs/notes/license.rst
@@ -1,4 +1,7 @@
+#######
License
--------
+#######
-.. todo:: Determine License
\ No newline at end of file
+.. image:: /_static/photos/32800805573_568d6b72fd_k_d.jpg
+
+The Guide is licensed under the `Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license `_.
diff --git a/docs/notes/styleguide.rst b/docs/notes/styleguide.rst
index 958382c18..5a715332e 100644
--- a/docs/notes/styleguide.rst
+++ b/docs/notes/styleguide.rst
@@ -1,8 +1,10 @@
.. _guide-style-guide:
-=====================
+#####################
The Guide Style Guide
-=====================
+#####################
+
+.. image:: /_static/photos/33573755856_7f43d43adf_k_d.jpg
As with all documentation, having a consistent format helps make the
document more understandable. In order to make The Guide easier to digest,
@@ -17,8 +19,10 @@ The Guide is written as :ref:`restructuredtext-ref`.
.. note:: On any page of the rendered HTML you can click "Show Source" to
see how authors have styled the page.
+
+*********
Relevancy
----------
+*********
Strive to keep any contributions relevant to the :ref:`purpose of The Guide
`.
@@ -27,63 +31,92 @@ Strive to keep any contributions relevant to the :ref:`purpose of The Guide
relate to Python development.
* Prefer to link to other sources if the information is already out there.
Be sure to describe what and why you are linking.
-* `Cite `_
+* `Cite `_
references where needed.
* If a subject isn't directly relevant to Python, but useful in conjunction
- with Python (ex: Git, Github, Databases), reference by linking to useful
- resources and describe why it's useful to Python.
+ with Python (e.g., Git, GitHub, Databases), reference by linking to useful
+ resources, and describe why it's useful to Python.
* When in doubt, ask.
+
+********
Headings
---------
+********
Use the following styles for headings.
-Chapter title::
+Chapter title:
+
+.. code-block:: rest
#########
Chapter 1
#########
-Page title::
+Page title:
- ===================
+.. code-block:: rest
+
+ *******************
Time is an Illusion
- ===================
+ *******************
-Section headings::
+Section headings:
+
+.. code-block:: rest
Lunchtime Doubly So
- -------------------
+ ===================
-Sub section headings::
+Sub section headings:
+
+.. code-block:: rest
Very Deep
- ~~~~~~~~~
+ ---------
+
+*****
Prose
------
+*****
Wrap text lines at 78 characters. Where necessary, lines may exceed 78
characters, especially if wrapping would make the source text more difficult
to read.
+Use Standard American English, not British English.
+
+Use of the `serial comma `_
+(also known as the Oxford comma) is 100% non-optional. Any attempt to
+submit content with a missing serial comma will result in permanent banishment
+from this project, due to complete and total lack of taste.
+
+Banishment? Is this a joke? Hopefully we will never have to find out.
+
+*************
Code Examples
--------------
+*************
Wrap all code examples at 70 characters to avoid horizontal scrollbars.
-Command line examples::
+Command line examples:
+
+.. code-block:: rest
.. code-block:: console
$ run command --help
$ ls ..
-Be sure to include the ``$`` prefix before each line.
+Be sure to include the ``$`` prefix before each line for Unix console examples.
+
+For Windows console examples, use ``doscon`` or ``powershell`` instead of
+``console``, and omit the ``$`` prefix.
-Python interpreter examples::
+Python interpreter examples:
+
+.. code-block:: rest
Label the example::
@@ -91,7 +124,9 @@ Python interpreter examples::
>>> import this
-Python examples::
+Python examples:
+
+.. code-block:: rest
Descriptive title::
@@ -100,65 +135,82 @@ Python examples::
def get_answer():
return 42
+
+******************
Externally Linking
-------------------
+******************
+
+* Prefer labels for well known subjects (e.g. proper nouns) when linking:
-* Prefer labels for well known subjects (ex: proper nouns) when linking::
+ .. code-block:: rest
- Sphinx_ is used to document Python.
+ Sphinx_ is used to document Python.
- .. _Sphinx: http://sphinx.pocoo.org
+ .. _Sphinx: https://www.sphinx-doc.org
* Prefer to use descriptive labels with inline links instead of leaving bare
- links::
+ links:
+
+ .. code-block:: rest
- Read the `Sphinx Tutorial `_
+ Read the `Sphinx Tutorial `_
-* Avoid using labels such as "click here", "this", etc. preferring
+* Avoid using labels such as "click here", "this", etc., preferring
descriptive labels (SEO worthy) instead.
+
+********************************
Linking to Sections in The Guide
---------------------------------
+********************************
To cross-reference other parts of this documentation, use the `:ref:
-`_
+`_
keyword and labels.
-To make reference labels more clear and unique, always add a ``-ref`` suffix::
+To make reference labels more clear and unique, always add a ``-ref`` suffix:
+
+.. code-block:: rest
.. _some-section-ref:
Some Section
------------
+
+******************
Notes and Warnings
-------------------
+******************
Make use of the appropriate `admonitions directives
-`_ when making notes.
+`_ when making notes.
+
+Notes:
-Notes::
+.. code-block:: rest
.. note::
The Hitchhiker’s Guide to the Galaxy has a few things to say
on the subject of towels. A towel, it says, is about the most
massively useful thing an interstellar hitch hiker can have.
-Warnings::
+Warnings:
+
+.. code-block:: rest
.. warning:: DON'T PANIC
+
+*****
TODOs
------
+*****
Please mark any incomplete areas of The Guide with a `todo directive
-`_. To
+`_. To
avoid cluttering the :ref:`todo-list-ref`, use a single ``todo`` for stub
documents or large incomplete sections.
-::
+.. code-block:: rest
.. todo::
Learn the Ultimate Answer to the Ultimate Question
of Life, The Universe, and Everything
-
diff --git a/docs/scenarios/admin.rst b/docs/scenarios/admin.rst
index 3397d6632..f6433c213 100644
--- a/docs/scenarios/admin.rst
+++ b/docs/scenarios/admin.rst
@@ -1,26 +1,33 @@
+
+######################
Systems Administration
-======================
+######################
+
+.. image:: /_static/photos/34435690580_3afec7d4cd_k_d.jpg
+
+******
Fabric
-------
+******
-`Fabric `_ is a library for simplifying system administration tasks. While Chef
-and Puppet tend to focus on managing servers and system libraries,
-fabric is more focused on application level tasks such as deployment.
+`Fabric `_ is a library for simplifying system
+administration tasks. While Chef and Puppet tend to focus on managing servers
+and system libraries, Fabric is more focused on application level tasks such
+as deployment.
Install Fabric:
-.. code-block:: bash
+.. code-block:: console
$ pip install fabric
The following code will create two tasks that we can use: ``memory_usage`` and
``deploy``. The former will output the memory usage on each machine. The
-latter will ssh into each server, cd to our project directory, activate the
+latter will SSH into each server, cd to our project directory, activate the
virtual environment, pull the newest codebase, and restart the application
server.
-::
+.. code-block:: python
from fabric.api import cd, env, prefix, run, task
@@ -37,10 +44,10 @@ server.
run('git pull')
run('touch app.wsgi')
-With the previous code saved in a file named fabfile.py, we can check memory
-usage with:
+With the previous code saved in a file named :file:`fabfile.py`, we can check
+memory usage with:
-.. code-block:: bash
+.. code-block:: console
$ fab memory_usage
[my_server1] Executing task 'memory'
@@ -59,7 +66,7 @@ usage with:
and we can deploy with:
-.. code-block:: bash
+.. code-block:: console
$ fab deploy
@@ -68,44 +75,50 @@ programs, and host grouping.
`Fabric Documentation `_
+
+****
Salt
-----
+****
-`Salt `_ is an open source infrastructure management tool.
-It supports remote command execution from a central point (master host) to multiple
-hosts (minions). It also supports system states which can be used to configure
-multiple servers using simple template files.
+`Salt `_ is an open source infrastructure management
+tool. It supports remote command execution from a central point (master host)
+to multiple hosts (minions). It also supports system states which can be used
+to configure multiple servers using simple template files.
-Salt supports python versions 2.6 and 2.7 and can be installed via pip:
+Salt supports Python versions 2.6 and 2.7 and can be installed via pip:
-.. code-block:: bash
+.. code-block:: console
$ pip install salt
-After configuring a master server and any number of minion hosts, we can run arbitrary
-shell commands or use pre-built modules of complex commands on our minions.
+After configuring a master server and any number of minion hosts, we can run
+arbitrary shell commands or use pre-built modules of complex commands on our
+minions.
The following command lists all available minion hosts, using the ping module.
-.. code-block:: bash
+.. code-block:: console
$ salt '*' test.ping
-The host filtering is accomplished by matching the minion id, or using the grains system.
-The `grains `_ system
-uses static host information like the operating system version or the CPU architecture to
-provide a host taxonomy for the salt modules.
+The host filtering is accomplished by matching the minion id
+or using the grains system. The
+`grains `_
+system uses static host information like the operating system version or the
+CPU architecture to provide a host taxonomy for the Salt modules.
-The following command lists all available minions running CentOS using the grains system:
+The following command lists all available minions running CentOS using the
+grains system:
-.. code-block:: bash
+.. code-block:: console
$ salt -G 'os:CentOS' test.ping
-Salt also provides a state system. States can be used to configure the minion hosts.
+Salt also provides a state system. States can be used to configure the minion
+hosts.
-For example, when a minion host is ordered to read the following state file, it will install
-and start the Apache server:
+For example, when a minion host is ordered to read the following state file,
+it will install and start the Apache server:
.. code-block:: yaml
@@ -114,35 +127,271 @@ and start the Apache server:
- installed
service:
- running
+ - enable: True
+ - require:
+ - pkg: apache
+
+State files can be written using YAML, the Jinja2 template system, or pure Python.
+
+ `Salt Documentation `_
+
+
+******
+Psutil
+******
+
+`Psutil `_ is an interface to different
+system information (e.g. CPU, memory, disks, network, users, and processes).
+
+Here is an example to be aware of some server overload. If any of the
+tests (net, CPU) fail, it will send an email.
+
+.. code-block:: python
+
+ # Functions to get system values:
+ from psutil import cpu_percent, net_io_counters
+ # Functions to take a break:
+ from time import sleep
+ # Package for email services:
+ import smtplib
+ import string
+ MAX_NET_USAGE = 400000 # bytes per seconds
+ MAX_ATTACKS = 4
+ attack = 0
+ while attack <= MAX_ATTACKS:
+ sleep(4)
+
+ # Check the net usage wit named tuples
+ neti1 = net_io_counters().bytes_recv
+ neto1 = net_io_counters().bytes_sent
+ sleep(1)
+ neti2 = net_io_counters().bytes_recv
+ neto2 = net_io_counters().bytes_sent
+
+ # Calculate the bytes per second
+ net = ((neti2+neto2) - (neti1+neto1))/2
+
+ # Check the net and cpu usage
+ if (net > MAX_NET_USAGE) or (cpu_percent(interval = 1) > 70):
+ attack+=1
+ elif attack > 1:
+ attack-=1
+
+ # Write a very important email if attack is higher than 4
+ TO = "you@your_email.com"
+ FROM = "webmaster@your_domain.com"
+ SUBJECT = "Your domain is out of system resources!"
+ text = "Go and fix your server!"
+ string="\r\n"
+ BODY = string.join(("From: %s" %FROM,"To: %s" %TO,
+ "Subject: %s" %SUBJECT, "",text))
+ server = smtplib.SMTP('127.0.0.1')
+ server.sendmail(FROM, [TO], BODY)
+ server.quit()
+
+
+A full terminal application like a widely extended top is `Glance `_, which is based on psutil and has the ability for client-server monitoring.
+
+*******
+Ansible
+*******
+
+`Ansible `_ is an open source system automation tool.
+Its biggest advantage over Puppet or Chef is that it does not require an agent on
+the client machine. Playbooks are Ansible’s configuration, deployment, and
+orchestration language and are written in YAML with Jinja2 for templating.
+
+Ansible supports Python versions 2.6 and 2.7 and can be installed via pip:
+
+.. code-block:: console
+
+ $ pip install ansible
+
+Ansible requires an inventory file that describes the hosts to which it has
+access. Below is an example of a host and playbook that will ping all the
+hosts in the inventory file.
+
+Here is an example inventory file:
+:file:`hosts.yml`
+
+.. code-block:: yaml
+
+ [server_name]
+ 127.0.0.1
+
+Here is an example playbook:
+:file:`ping.yml`
+
+.. code-block:: yaml
+
+ ---
+ - hosts: all
+
+ tasks:
+ - name: ping
+ action: ping
+
+To run the playbook:
+
+.. code-block:: console
-State files can be written using YAML, the Jinja2 template system or pure python.
+ $ ansible-playbook ping.yml -i hosts.yml --ask-pass
- `Salt Documentation `_
+The Ansible playbook will ping all of the servers in the :file:`hosts.yml` file.
+You can also select groups of servers using Ansible. For more information
+about Ansible, read the `Ansible Docs `_.
+`An Ansible tutorial `_ is also a
+great and detailed introduction to getting started with Ansible.
+
+
+****
Chef
-----
+****
+
+`Chef `_ is a systems and cloud infrastructure automation
+framework that makes it easy to deploy servers and applications to any physical,
+virtual, or cloud location. In case this is your choice for configuration management,
+you will primarily use Ruby to write your infrastructure code.
+
+Chef clients run on every server that is part of your infrastructure and these regularly
+check with your Chef server to ensure your system is always aligned and represents the
+desired state. Since each individual server has its own distinct Chef client, each server
+configures itself and this distributed approach makes Chef a scalable automation platform.
+
+Chef works by using custom recipes (configuration elements), implemented in cookbooks. Cookbooks, which are basically
+packages for infrastructure choices, are usually stored in your Chef server.
+Read the `DigitalOcean tutorial series
+`_
+on Chef to learn how to create a simple Chef Server.
+
+To create a simple cookbook the `knife `_ command is used:
+
+.. code-block:: console
+
+ knife cookbook create cookbook_name
+
+`Getting started with Chef `_
+is a good starting point for Chef Beginners and many community maintained cookbooks that can
+serve as a good reference or tweaked to serve your infrastructure configuration needs can be
+found on the `Chef Supermarket `_.
-.. todo:: Write about Chef
+- `Chef Documentation `_
- `Chef Documentation
- `_
+******
Puppet
-------
+******
-.. todo:: Write about Puppet
+`Puppet `_ is IT Automation and configuration management
+software from Puppet Labs that allows System Administrators to define the state
+of their IT Infrastructure, thereby providing an elegant way to manage their
+fleet of physical and virtual machines.
- `Puppet Labs Documentation `_
+Puppet is available both as an Open Source and an Enterprise variant. Modules
+are small, shareable units of code written to automate or define the state of a
+system. `Puppet Forge `_ is a repository for
+modules written by the community for Open Source and Enterprise Puppet.
-Blueprint
----------
+Puppet Agents are installed on nodes whose state needs to be monitored or
+changed. A designated server known as the Puppet Master is responsible for
+orchestrating the agent nodes.
-.. todo:: Write about Blueprint
+Agent nodes send basic facts about the system such as the operating system,
+kernel, architecture, IP address, hostname, etc. to the Puppet Master.
+The Puppet Master then compiles a catalog with information provided by the
+agents on how each node should be configured and sends it to the agent. The
+agent enforces the change as prescribed in the catalog and sends a report back
+to the Puppet Master.
-Buildout
---------
+Facter is an interesting tool that ships with Puppet that pulls basic facts
+about the system. These facts can be referenced as a variable while writing
+your Puppet modules.
+
+.. code-block:: console
+
+ $ facter kernel
+ Linux
+.. code-block:: console
+
+ $ facter operatingsystem
+ Ubuntu
+
+Writing Modules in Puppet is pretty straight forward. Puppet Manifests together
+form Puppet Modules. Puppet manifests end with an extension of ``.pp``.
+Here is an example of 'Hello World' in Puppet.
+
+.. code-block:: puppet
-.. todo:: Write about Buildout
+ notify { 'This message is getting logged into the agent node':
- `Buildout Website `_
+ #As nothing is specified in the body the resource title
+ #the notification message by default.
+ }
+Here is another example with system based logic. Note how the operating system
+fact is being used as a variable prepended with the ``$`` sign. Similarly, this
+holds true for other facts such as hostname which can be referenced by
+``$hostname``.
+
+.. code-block:: puppet
+
+ notify{ 'Mac Warning':
+ message => $operatingsystem ? {
+ 'Darwin' => 'This seems to be a Mac.',
+ default => 'I am a PC.',
+ },
+ }
+
+There are several resource types for Puppet but the package-file-service
+paradigm is all you need for undertaking the majority of the configuration
+management. The following Puppet code makes sure that the OpenSSH-Server
+package is installed in a system and the sshd service is notified to restart
+every time the sshd configuration file is changed.
+
+.. code-block:: puppet
+
+ package { 'openssh-server':
+ ensure => installed,
+ }
+
+ file { '/etc/ssh/sshd_config':
+ source => 'puppet:///modules/sshd/sshd_config',
+ owner => 'root',
+ group => 'root',
+ mode => '640',
+ notify => Service['sshd'], # sshd will restart
+ # whenever you edit this
+ # file
+ require => Package['openssh-server'],
+
+ }
+
+ service { 'sshd':
+ ensure => running,
+ enable => true,
+ hasstatus => true,
+ hasrestart=> true,
+ }
+
+For more information, refer to the `Puppet Labs Documentation `_
+
+
+*********
+Blueprint
+*********
+
+.. todo:: Write about Blueprint
+
+
+********
+Buildout
+********
+
+`Buildout `_ is an open source software build tool.
+Buildout is created using the Python programming language. It implements a
+principle of separation of configuration from the scripts that do the setting
+up. Buildout is primarily used to download and set up dependencies in `Python
+eggs `_
+format of the software being developed or deployed. Recipes for build tasks in
+any environment can be created, and many are already available.
diff --git a/docs/scenarios/ci.rst b/docs/scenarios/ci.rst
index 3a80c70b3..4296679c7 100644
--- a/docs/scenarios/ci.rst
+++ b/docs/scenarios/ci.rst
@@ -1,12 +1,20 @@
+
+######################
Continuous Integration
-======================
+######################
+
+.. image:: /_static/photos/33907150594_9abba7ad0a_k_d.jpg
+
+.. note::
+ For advice on writing your tests, see :doc:`/writing/tests`.
+****
Why?
-----
+****
Martin Fowler, who first wrote about `Continuous Integration `_
-(short: CI) together with Kent Beck, describes the CI as follows:
+(short: CI) together with Kent Beck, describes CI as follows:
Continuous Integration is a software development practice where members of
a team integrate their work frequently, usually each person integrates at
@@ -16,30 +24,28 @@ Martin Fowler, who first wrote about `Continuous Integration `_ is an extensible continuous integration
-engine. Use it.
+*******
+Jenkins
+*******
+`Jenkins CI `_ is an extensible Continuous Integration engine. Use it.
+********
Buildbot
---------
-`Buildbot `_ is a Python system to
-automate the compile/test cycle to validate code changes.
-
+********
-Mule?
------
+`Buildbot `_ is a Python system to
+automate the compile/test cycle to validate code changes.
-.. todo:: Write about Mule
+***
Tox
----
+***
-`tox `_ is an automation tool providing
-packaging, testing and deployment of Python software right from the console or
+`tox `_ is an automation tool providing
+packaging, testing, and deployment of Python software right from the console or
CI server. It is a generic virtualenv management and test command line tool
which provides the following features:
@@ -47,29 +53,30 @@ which provides the following features:
interpreters
* Running tests in each of the environments, configuring your test tool of
choice
-* Acting as a frontend to Continuous Integration servers, reducing boilerplate
- and merging CI and shell-based testing.
+* Acting as a front-end to Continuous Integration servers, reducing boilerplate
+ and merging CI and shell-based testing
+*********
Travis-CI
----------
-`Travis-CI `_ is a distributed CI server which builds tests
-for open source projects for free. It provides multiple workers to run Python tests
-on and seamlessly integrates with Github. You can even have it comment on your Pull
-Requests whether this particular changeset breaks the build or not. So if you are
-hosting your code on Github, travis-ci is a great and easy way to get started with
-Continuous Integration.
+*********
+
+`Travis-CI `_ is a distributed CI server which builds
+tests for open source projects for free. It provides multiple workers to run
+Python tests on and seamlessly integrates with GitHub. You can even have it
+comment on your Pull Requests whether this particular changeset breaks the
+build or not. So, if you are hosting your code on GitHub, Travis-CI is a great
+and easy way to get started with Continuous Integration.
-In order to get started, add a ``.travis.yml`` file to your repository with this
-example content::
+In order to get started, add a :file:`.travis.yml` file to your repository with
+this example content::
language: python
python:
- - "2.5"
- "2.6"
- "2.7"
- - "3.1"
- "3.2"
+ - "3.3"
# command to install dependencies
script: python tests/test_all_of_the_units.py
branches:
@@ -77,12 +84,13 @@ example content::
- master
-This will get your project tested on all the listed Python versions by running the given
-script and only build the master branch. There are a lot more options you can enable, like
-notifications, before and after steps and much more. The
-`travis-ci docs `_ explain all of those and are very
-thorough.
+This will get your project tested on all the listed Python versions by
+running the given script, and will only build the ``master`` branch. There are a
+lot more options you can enable, like notifications, before and after steps,
+and much more. The `Travis-CI docs `_
+explain all of these options, and are very thorough.
-In order to activate testing for your project, go to `the travis-ci site `_
-and login with your Github account. Then activate your project in your profile settings and that's
-it. From now on, your project's tests will be run on every push to Github.
+In order to activate testing for your project, go to `the Travis-CI site `_
+and login with your GitHub account. Then activate your project in your
+profile settings and you're ready to go. From now on, your project's tests
+will be run on every push to GitHub.
diff --git a/docs/scenarios/cli.rst b/docs/scenarios/cli.rst
index bc952c74f..59f311e83 100644
--- a/docs/scenarios/cli.rst
+++ b/docs/scenarios/cli.rst
@@ -1,9 +1,91 @@
-Command Line Applications
-=========================
-.. todo:: Explain "Command Line Applications"
+#########################
+Command-line Applications
+#########################
-Clint
------
+.. image:: /_static/photos/34435690330_11930b5987_k_d.jpg
-.. todo:: Write about Clint
+Command-line applications, also referred to as `Console Applications
+`_, are computer programs
+designed to be used from a text interface, such as a `shell
+`_. Command-line applications
+usually accept various inputs as arguments, often referred to as parameters or
+sub-commands, as well as options, often referred to as flags or switches.
+
+Some popular command-line applications include:
+
+* `grep `_ - A plain-text data search utility
+* `curl `_ - A tool for data transfer with URL syntax
+* `httpie `_ - A command-line HTTP
+ client, a user-friendly cURL replacement
+* `Git `_ - A distributed version control system
+* `Mercurial `_ - A distributed version control
+ system primarily written in Python
+
+
+*****
+Click
+*****
+
+`click `_ is a Python package for creating
+command-line interfaces in a composable way with as little code as possible.
+This “Command-Line Interface Creation Kit” is highly configurable but comes
+with good defaults out of the box.
+
+
+******
+docopt
+******
+
+`docopt `_ is a lightweight, highly Pythonic package that
+allows creating command-line interfaces easily and intuitively, by parsing
+POSIX-style usage instructions.
+
+
+****
+Plac
+****
+
+`Plac `_ is a simple wrapper
+over the Python standard library `argparse `_,
+which hides most of its complexity by using a declarative interface: the
+argument parser is inferred rather than written down imperatively. This
+module targets unsophisticated users, programmers, sysadmins,
+scientists, and in general people writing throw-away scripts for themselves,
+who choose to create a command-line interface because it is quick and simple.
+
+
+*****
+Cliff
+*****
+
+`Cliff `_ is a framework for
+building command-line programs. It uses setuptools entry points to provide
+subcommands, output formatters, and other extensions. The framework is meant
+to be used to create multi-level commands such as ``svn`` and ``git``, where
+the main program handles some basic argument parsing and then invokes a
+sub-command to do the work.
+
+
+******
+Cement
+******
+
+`Cement `_ is an advanced CLI Application
+Framework. Its goal is to introduce a standard and feature-full platform for
+both simple and complex command line applications as well as support rapid
+development needs without sacrificing quality. Cement is flexible, and its use
+cases span from the simplicity of a micro-framework to the complexity of a
+mega-framework.
+
+
+***********
+Python Fire
+***********
+
+`Python Fire `_ is a library for
+automatically generating command-line interfaces from absolutely any Python
+object. It can help debug Python code more easily from the command line,
+create CLI interfaces to existing code, allow you to interactively explore
+code in a REPL, and simplify transitioning between Python and Bash (or any
+other shell).
diff --git a/docs/scenarios/clibs.rst b/docs/scenarios/clibs.rst
new file mode 100644
index 000000000..13c9e4802
--- /dev/null
+++ b/docs/scenarios/clibs.rst
@@ -0,0 +1,141 @@
+
+################################
+Interfacing with C/C++ Libraries
+################################
+
+.. image:: /_static/photos/34725951345_c8f5959a2e_k_d.jpg
+
+
+****************************
+C Foreign Function Interface
+****************************
+
+`CFFI `_ provides a simple to use
+mechanism for interfacing with C from both CPython and PyPy. It supports two
+modes: an inline `ABI `_ compatibility mode (example provided below), which allows
+you to dynamically load and run functions from executable modules (essentially
+exposing the same functionality as `LoadLibrary `_ or `dlopen `_), and an API mode,
+which allows you to build C extension modules.
+
+ABI Interaction
+~~~~~~~~~~~~~~~
+
+.. code-block:: python
+ :linenos:
+
+ from cffi import FFI
+ ffi = FFI()
+ ffi.cdef("size_t strlen(const char*);")
+ clib = ffi.dlopen(None)
+ length = clib.strlen("String to be evaluated.")
+ # prints: 23
+ print("{}".format(length))
+
+
+******
+ctypes
+******
+
+`ctypes `_ is the de facto standard
+library for interfacing with C/C++ from CPython, and it provides not only
+full access to the native C interface of most major operating systems (e.g.,
+kernel32 on Windows, or libc on \*nix), but also provides support for loading
+and interfacing with dynamic libraries, such as DLLs or shared objects, at
+runtime. It brings along with it a whole host of types for interacting
+with system APIs, and allows you to rather easily define your own complex
+types, such as structs and unions, and allows you to modify things such as
+padding and alignment, if needed. It can be a bit crufty to use, but in
+conjunction with the `struct `_
+module, you are essentially provided full control over how your data types get
+translated into something usable by a pure C/C++ method.
+
+Struct Equivalents
+~~~~~~~~~~~~~~~~~~
+
+:file:`MyStruct.h`
+
+.. code-block:: c
+ :linenos:
+
+ struct my_struct {
+ int a;
+ int b;
+ };
+
+:file:`MyStruct.py`
+
+.. code-block:: python
+ :linenos:
+
+ import ctypes
+ class my_struct(ctypes.Structure):
+ _fields_ = [("a", c_int),
+ ("b", c_int)]
+
+
+****
+SWIG
+****
+
+`SWIG `_, though not strictly Python focused (it supports a
+large number of scripting languages), is a tool for generating bindings for
+interpreted languages from C/C++ header files. It is extremely simple to use:
+the consumer simply needs to define an interface file (detailed in the
+tutorial and documentations), include the requisite C/C++ headers, and run
+the build tool against them. While it does have some limits (it currently
+seems to have issues with a small subset of newer C++ features, and getting
+template-heavy code to work can be a bit verbose), it provides a great deal
+of power and exposes lots of features to Python with little effort.
+Additionally, you can easily extend the bindings SWIG creates (in the
+interface file) to overload operators and built-in methods, effectively re-
+cast C++ exceptions to be catchable by Python, etc.
+
+Example: Overloading __repr__
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:file:`MyClass.h`
+
+.. code-block:: c++
+ :linenos:
+
+ #include
+ class MyClass {
+ private:
+ std::string name;
+ public:
+ std::string getName();
+ };
+
+
+:file:`myclass.i`
+
+.. code-block:: idl
+ :linenos:
+
+ %include "string.i"
+
+ %module myclass
+ %{
+ #include
+ #include "MyClass.h"
+ %}
+
+ %extend MyClass {
+ std::string __repr__()
+ {
+ return $self->getName();
+ }
+ }
+
+ %include "MyClass.h"
+
+
+************
+Boost.Python
+************
+
+`Boost.Python `_
+requires a bit more manual work to expose C++ object functionality, but
+it is capable of providing all the same features SWIG does and then some,
+to include providing wrappers to access PyObjects in C++, extracting SWIG
+wrapper objects, and even embedding bits of Python into your C++ code.
diff --git a/docs/scenarios/client.rst b/docs/scenarios/client.rst
index 8a4fdde4b..c2d5c289b 100644
--- a/docs/scenarios/client.rst
+++ b/docs/scenarios/client.rst
@@ -1,10 +1,14 @@
+
+####################
Network Applications
-====================
+####################
+.. image:: /_static/photos/34364815780_bea6614025_k_d.jpg
+****
HTTP
-::::
+****
The Hypertext Transfer Protocol (HTTP) is an application protocol for
distributed, collaborative, hypermedia information systems. HTTP is the
@@ -24,13 +28,14 @@ your URLs, or to form-encode your POST data. Keep-alive and HTTP connection
pooling are 100% automatic, powered by urllib3, which is embedded within
Requests.
-- `Documentation `_
-- `PyPi `_
+- `Documentation `_
+- `PyPi `_
- `GitHub `_
+*******************
Distributed Systems
-::::::::::::::::::::
+*******************
ZeroMQ
@@ -45,5 +50,11 @@ library is designed to have a familiar socket-style API.
RabbitMQ
--------
-.. todo:: Write about RabbitMQ
+RabbitMQ is an open source message broker software that implements the Advanced
+Message Queuing Protocol (AMQP). The RabbitMQ server is written in the Erlang
+programming language and is built on the Open Telecom Platform framework for
+clustering and failover. Client libraries to interface with the broker are
+available for all major programming languages.
+- `Homepage `_
+- `GitHub Organization `_
diff --git a/docs/scenarios/crypto.rst b/docs/scenarios/crypto.rst
new file mode 100644
index 000000000..262337204
--- /dev/null
+++ b/docs/scenarios/crypto.rst
@@ -0,0 +1,91 @@
+
+############
+Cryptography
+############
+
+.. image:: /_static/photos/33907152824_bf91078cc1_k_d.jpg
+
+
+************
+cryptography
+************
+
+`cryptography `_ is an actively developed
+library that provides cryptographic recipes and primitives. It supports
+Python 2.6-2.7, Python 3.3+, and PyPy.
+
+
+cryptography is divided into two layers of recipes and hazardous materials
+(hazmat). The recipes layer provides a simple API for proper symmetric
+encryption and the hazmat layer provides low-level cryptographic primitives.
+
+
+
+Installation
+~~~~~~~~~~~~
+
+.. code-block:: console
+
+ $ pip install cryptography
+
+Example
+~~~~~~~
+
+Example code using high level symmetric encryption recipe:
+
+.. code-block:: python
+
+ from cryptography.fernet import Fernet
+ key = Fernet.generate_key()
+ cipher_suite = Fernet(key)
+ cipher_text = cipher_suite.encrypt(b"A really secret message. Not for prying eyes.")
+ plain_text = cipher_suite.decrypt(cipher_text)
+
+
+**************
+GPGME bindings
+**************
+
+The `GPGME Python bindings `_ provide Pythonic access to `GPG Made Easy `_, a C API for the entire GNU Privacy Guard suite of projects, including GPG, libgcrypt, and gpgsm (the S/MIME engine). It supports Python 2.6, 2.7, 3.4, and above. Depends on the SWIG C interface for Python as well as the GnuPG software and libraries.
+
+
+A more comprehensive `GPGME Python Bindings HOWTO `_ is available with the source, and an HTML version is available `at http://files.au.adversary.org `_. Python 3 sample scripts from the examples in the HOWTO are also provided with the source and are accessible `at gnupg.org `_.
+
+Available under the same terms as the rest of the GnuPG Project: GPLv2 and LGPLv2.1, both with the "or any later version" clause.
+
+Installation
+~~~~~~~~~~~~
+
+Included by default when compiling GPGME if the configure script locates a supported python version (which it will if it's in $PATH during configuration).
+
+Example
+~~~~~~~
+
+.. code-block:: python3
+
+ import gpg
+
+ # Encryption to public key specified in rkey.
+ a_key = input("Enter the fingerprint or key ID to encrypt to: ")
+ filename = input("Enter the filename to encrypt: ")
+ with open(filename, "rb") as afile:
+ text = afile.read()
+ c = gpg.core.Context(armor=True)
+ rkey = list(c.keylist(pattern=a_key, secret=False))
+ ciphertext, result, sign_result = c.encrypt(text, recipients=rkey,
+ always_trust=True,
+ add_encrypt_to=True)
+ with open("{0}.asc".format(filename), "wb") as bfile:
+ bfile.write(ciphertext)
+ # Decryption with corresponding secret key
+ # invokes gpg-agent and pinentry.
+ with open("{0}.asc".format(filename), "rb") as cfile:
+ plaintext, result, verify_result = gpg.Context().decrypt(cfile)
+ with open("new-{0}".format(filename), "wb") as dfile:
+ dfile.write(plaintext)
+ # Matching the data.
+ # Also running a diff on filename and the new filename should match.
+ if text == plaintext:
+ print("Hang on ... did you say *all* of GnuPG? Yep.")
+ else:
+ pass
diff --git a/docs/scenarios/db.rst b/docs/scenarios/db.rst
index d3c398f82..21217477a 100644
--- a/docs/scenarios/db.rst
+++ b/docs/scenarios/db.rst
@@ -1,35 +1,118 @@
+
+#########
Databases
-=========
+#########
+
+.. image:: /_static/photos/33907152464_a99fdcc8de_k_d.jpg
+
+******
DB-API
-------
+******
The Python Database API (DB-API) defines a standard interface for Python
-database access modules. It's documented in `PEP 249 `_.
-Nearly all Python database modules such as `sqlite3`, `psycopg` and
+database access modules. It's documented in :pep:`249`.
+Nearly all Python database modules such as `sqlite3`, `psycopg`, and
`mysql-python` conform to this interface.
Tutorials that explain how to work with modules that conform to this interface can be found
`here `__ and
-`here `__.
+`here `__.
+
+**********
SQLAlchemy
-----------
+**********
`SQLAlchemy `_ is a commonly used database toolkit.
Unlike many database libraries it not only provides an ORM layer but also a
generalized API for writing database-agnostic code without SQL.
-::
+.. code-block:: console
+
+ $ pip install sqlalchemy
+
+
+*******
+Records
+*******
+
+`Records `_ is minimalist SQL library,
+designed for sending raw SQL queries to various databases. Data can be used
+programmatically or exported to a number of useful data formats.
+
+.. code-block:: console
+
+ $ pip install records
+
+Also included is a command-line tool for exporting SQL data.
+
+
+******
+PugSQL
+******
+
+`PugSQL `_ is a simple Python interface for organizing
+and using parameterized, handwritten SQL. It is an anti-ORM that is
+philosophically lo-fi, but it still presents a clean interface in Python.
- pip install sqlalchemy
+.. code-block:: console
+ $ pip install pugsql
+
+
+**********
Django ORM
-----------
+**********
-The Django ORM is the interface used by `Django `_
+The Django ORM is the interface used by `Django `_
to provide database access.
-It's based on the idea of `models `_, an abstraction that makes it easier to
-manipulate data in Python.
+It's based on the idea of
+`models `_,
+an abstraction that makes it easier to manipulate data in Python.
+
+The basics:
+
+- Each model is a Python class that subclasses django.db.models.Model.
+- Each attribute of the model represents a database field.
+- Django gives you an automatically-generated database-access API; see
+ `Making queries `__.
+
+
+******
+peewee
+******
+
+`peewee `_ is another ORM with a focus
+on being lightweight with support for Python 2.6+ and 3.2+ which supports
+SQLite, MySQL, and PostgreSQL by default. The
+`model layer `_
+is similar to that of the Django ORM and it has
+`SQL-like methods `_
+to query data. While SQLite, MySQL, and PostgreSQL are supported out-of-the-box,
+there is a `collection of add-ons `_
+available.
+
+
+*******
+PonyORM
+*******
+
+`PonyORM `_ is an ORM that takes a different approach to
+querying the database. Instead of writing an SQL-like language or boolean
+expressions, Python's generator syntax is used. There's also a graphical
+schema editor that can generate PonyORM entities for you. It supports Python
+2.6+ and Python 3.3+ and can connect to SQLite, MySQL, PostgreSQL, and Oracle.
+
+
+*********
+SQLObject
+*********
+
+`SQLObject `_ is yet another ORM. It supports a wide
+variety of databases: common database systems like MySQL, PostgreSQL, and SQLite and
+more exotic systems like SAP DB, SyBase, and Microsoft SQL Server. It only supports Python 2
+from Python 2.6 upwards.
+.. There's no official information on this on their page; this information was gathered by looking at their source code.
diff --git a/docs/scenarios/gui.rst b/docs/scenarios/gui.rst
index 49dd0ad5e..3ad0af364 100644
--- a/docs/scenarios/gui.rst
+++ b/docs/scenarios/gui.rst
@@ -1,69 +1,185 @@
+
+################
GUI Applications
-================
+################
+.. image:: /_static/photos/33907143624_cd621b535c_k_d.jpg
+
+
+Alphabetical list of GUI Applications.
+
+
+*******
+Camelot
+*******
+
+`Camelot `_ provides components for building
+applications on top of Python, SQLAlchemy, and Qt. It is inspired by the Django
+admin interface.
+
+The main resource for information is the website:
+http://www.python-camelot.com
+and the mailing list https://groups.google.com/forum/#!forum/project-camelot.
+
+
+*****
+Cocoa
+*****
+
+.. note:: The Cocoa framework is only available on OS X. Don't pick this if you're writing a cross-platform application!
+
+
+***
+GTk
+***
+
+.. note:: PyGTK provides Python bindings for the GTK+ toolkit. However, it has been superseded by PyGObject. PyGTK should not be used for new projects and existing projects should be ported to PyGObject.
+
+
+********************
+PyGObject aka (PyGi)
+********************
+
+`PyGObject `_ provides Python
+bindings which gives access to the entire GNOME software platform. It is fully
+compatible with GTK+ 3. Here is a tutorial to get started with `Python GTK+ 3
+Tutorial `_.
+
+`API Reference `_
+
+
+****
+Kivy
+****
+
+`Kivy `_ is a Python library for development of multi-touch
+enabled media rich applications. The aim is to allow for quick and easy
+interaction design and rapid prototyping, while making your code reusable and
+deployable.
+
+Kivy is written in Python, based on OpenGL, and supports different input devices
+such as: Mouse, Dual Mouse, TUIO, WiiMote, WM_TOUCH, HIDtouch, Apple's products,
+and so on.
+
+Kivy is actively being developed by a community and is free to use. It operates
+on all major platforms (Linux, OS X, Windows, Android).
+
+The main resource for information is the website: http://kivy.org
+
+
+******
+PyObjC
+******
+
+.. note:: Only available on OS X. Don't pick this if you're writing a cross-platform application.
-Qt
---
-Qt is a cross-platform application framework that is widely used for developing
-software with a GUI but can also be used for non-GUI applications.
+******
PySide
-~~~~~~
+******
+
PySide is a Python binding of the cross-platform GUI toolkit Qt.
+The package name depends on the major Qt version (`PySide` for Qt4,
+`PySide2` for Qt5, and `PySide6` for Qt6).
+This set of bindings is developed by `The Qt Company `_.
+
+.. code-block:: console
+
+ $ pip install pyside6
+
+https://pyside.org
-http://developer.qt.nokia.com/wiki/PySideDownloads/
+****
PyQt
-~~~~
+****
+
.. note:: If your software does not fully comply with the GPL you will need a commercial license!
+PyQt provides Python bindings for the Qt Framework (see below).
+
http://www.riverbankcomputing.co.uk/software/pyqt/download
-Cocoa
------
-.. note:: The Cocoa framework is only available on Mac OSX. Don't pick this if you're writing a cross-platform application!
-PyObjC
-~~~~~~
-.. note:: Only available on Mac OSX. Don't pick this if you're writing a cross-platform application.
+***************************************
+Pyjs Desktop (formerly Pyjamas Desktop)
+***************************************
-wxPython
---------
-wxPython is a GUI toolkit for the Python programming language. It allows Python programmers to create programs with a robust, highly functional graphical user interface, simply and easily. It is implemented as a Python extension module (native code) that wraps the popular wxWidgets cross platform GUI library, which is written in C++.
+Pyjs Desktop is a application widget set for desktop and a cross-platform
+framework. It allows the exact same Python web application source code to be
+executed as a standalone desktop application.
-Install (Stable)
-~~~~~~~~~~~~~~~~
-*Go to http://www.wxpython.org/download.php#stable and download the appropriate
-package for your OS.*
-Gtk
----
-PyGTK provides Python bindings for the GTK+ toolkit. Like the GTK+ library
-itself, it is currently licensed under the GNU LGPL. It is worth noting that
-PyGTK only currently supports the Gtk-2.X API (NOT Gtk-3.0). It is currently
-recommended that PyGTK not be used for new projects and existing applications
-be ported from PyGTK to PyGObject.
+The main website: `pyjs `_.
+
+
+**
+Qt
+**
+
+`Qt `_ is a cross-platform application framework that is
+widely used for developing software with a GUI but can also be used for non-GUI
+applications.
+
+***********
+PySimpleGUI
+***********
+
+`PySimpleGUI `_ is a wrapper for Tkinter
+and Qt (others on the way). The amount of code required to implement custom
+GUIs is much shorter using PySimpleGUI than if the same GUI were written
+directly using Tkinter or Qt. PySimpleGUI code can be "ported" between GUI
+frameworks by changing import statements.
+
+.. code-block:: console
+
+ $ pip install pysimplegui
+
+PySimpleGUI is contained in a single PySimpleGUI.py file. Should pip
+installation be impossible, copying the PySimpleGUI.py file into a project's
+folder is all that's required to import and begin using.
+
+
+****
+Toga
+****
+
+`Toga `_ is a Python native, OS native,
+cross platform GUI toolkit. Toga consists of a library of base components with a
+shared interface to simplify platform-agnostic GUI development.
+
+Toga is available on macOS, Windows, Linux (GTK), and mobile platforms such as
+Android and iOS.
+
+
+**
Tk
---
-Tkinter is a thin object-oriented layer on top of Tcl/Tk. It has the advantage
+**
+
+Tkinter is a thin object-oriented layer on top of Tcl/Tk. **It has the advantage
of being included with the Python standard library, making it the most
-convenient and compatible toolkit to program with.
+convenient and compatible toolkit to program with.**
Both Tk and Tkinter are available on most Unix platforms, as well as on Windows
and Macintosh systems. Starting with the 8.0 release, Tk offers native look and
feel on all platforms.
-There's a good multi-language Tk tutorial with Python examples at
-`TkDocs `_. There's more information
+There's a good multi-language Tk tutorial with Python examples at `TkDocs
+`_. There's more information
available on the `Python Wiki `_.
-Kivy
-----
-`Kivy `_ is a Python library for development of multi-touch enabled media rich applications. The aim is to allow for quick and easy interaction design and rapid prototyping, while making your code reusable and deployable.
-Kivy is written in Python, based on OpenGL and supports different input devices such as: Mouse, Dual Mouse, TUIO, WiiMote, WM_TOUCH, HIDtouch, Apple's products and so on.
+********
+wxPython
+********
-Kivy is actively being developed by a community and free to use. It operates on all major platforms (Linux, OSX, Windows, Android).
+wxPython is a GUI toolkit for the Python programming language. It allows Python
+programmers to create programs with a robust, highly functional graphical user
+interface, simply and easily. It is implemented as a Python extension module
+(native code) that wraps the popular wxWidgets cross platform GUI library, which
+is written in C++.
-The main resource for information is the website: http://kivy.org
+**Install (Stable) wxPython**
+*go to https://www.wxpython.org/pages/downloads/ and download the appropriate
+package for your OS.*
diff --git a/docs/scenarios/imaging.rst b/docs/scenarios/imaging.rst
index 8defa0b73..8fe7e08fe 100644
--- a/docs/scenarios/imaging.rst
+++ b/docs/scenarios/imaging.rst
@@ -1,74 +1,112 @@
-==================
+
+##################
Image Manipulation
-==================
+##################
+
+.. image:: /_static/photos/34575689432_3de8e9a348_k_d.jpg
+
+Most image processing and manipulation techniques can be carried out
+effectively using two libraries: Python Imaging Library (PIL) and Open Source
+Computer Vision (OpenCV).
+
+A brief description of both is given below.
-.. todo::
- Add introduction about image manipulation and its Python libraries.
+**********************
Python Imaging Library
-----------------------
+**********************
The `Python Imaging Library `_, or PIL
-for short, is *the* library for image manipulation in Python.
+for short, is one of the core libraries for image manipulation in Python. Unfortunately,
+its development has stagnated, with its last release in 2009.
-It works with Python 1.5.2 and above, including 2.5, 2.6 and 2.7. Unfortunately,
-it doesn't work with 3.0+ yet.
+Luckily for you, there's an actively-developed fork of PIL called
+`Pillow `_ -- it's easier to install, runs on
+all major operating systems, and supports Python 3.
Installation
~~~~~~~~~~~~
-PIL has a reputation of not being very straightforward to install. Listed below
-are installation notes on various systems.
+Before installing Pillow, you'll have to install Pillow's prerequisites. Find
+the instructions for your platform in the
+`Pillow installation instructions `_.
-Also, there's a fork named `Pillow `_ which is easier
-to install. It has good setup instructions for all platforms.
+After that, it's straightforward:
-Installing on Linux
-~~~~~~~~~~~~~~~~~~~
+.. code-block:: console
-Arch Linux
-``````````
+ $ pip install Pillow
-PIL is maintained in the official community repository, and installed with the system installer as:
+Example
+~~~~~~~
-.. code-block:: bash
+.. code-block:: python
- $ sudo pacman -S python2-imaging
+ from PIL import Image, ImageFilter
+ #Read image
+ im = Image.open( 'image.jpg' )
+ #Display image
+ im.show()
-Ubuntu 12.10
-````````````
+ #Applying a filter to the image
+ im_sharp = im.filter( ImageFilter.SHARPEN )
+ #Saving the filtered image to a new file
+ im_sharp.save( 'image_sharpened.jpg', 'JPEG' )
-Can be installed on the command line as:
+ #Splitting the image into its respective bands, i.e. Red, Green,
+ #and Blue for RGB
+ r,g,b = im_sharp.split()
-.. code-block:: bash
+ #Viewing EXIF data embedded in image
+ exif_data = im._getexif()
+ exif_data
- $ sudo apt-get install python-imaging
+There are more examples of the Pillow library in the
+`Pillow tutorial `_.
-Installing on Mac OS X
-~~~~~~~~~~~~~~~~~~~~~~
+***************************
+Open Source Computer Vision
+***************************
+
+Open Source Computer Vision, more commonly known as OpenCV, is a more advanced
+image manipulation and processing software than PIL. It has been implemented
+in several languages and is widely used.
+
+Installation
+~~~~~~~~~~~~
-PIP doesn't know about the Mac OS X Freetype paths. To rectify that:
+In Python, image processing using OpenCV is implemented using the ``cv2`` and
+``NumPy`` modules. The `installation instructions for OpenCV
+`_
+should guide you through configuring the project for yourself.
-.. code-block:: bash
+NumPy can be downloaded from the Python Package Index(PyPI):
- $ ln -s /usr/X11/include/freetype2 /usr/local/include/
- $ ln -s /usr/X11/include/ft2build.h /usr/local/include/
- $ ln -s /usr/X11/lib/libfreetype.6.dylib /usr/local/lib/
- $ ln -s /usr/X11/lib/libfreetype.6.dylib /usr/local/lib/libfreetype.dylib
+.. code-block:: console
-then:
+ $ pip install numpy
-.. code-block:: bash
- $ brew install libjpeg
- $ pip install PIL
+Example
+~~~~~~~
+.. code-block:: python
-Installing on Windows
-~~~~~~~~~~~~~~~~~~~~~
+ import cv2
+ #Read Image
+ img = cv2.imread('testimg.jpg')
+ #Display Image
+ cv2.imshow('image',img)
+ cv2.waitKey(0)
+ cv2.destroyAllWindows()
-.. todo::
- Notes on installing on Windows machines
+ #Applying Grayscale filter to image
+ gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
+ #Saving filtered image to new file
+ cv2.imwrite('graytest.jpg',gray)
+There are more Python-implemented examples of OpenCV in this `collection of
+tutorials
+`_.
diff --git a/docs/scenarios/json.rst b/docs/scenarios/json.rst
new file mode 100644
index 000000000..1c3663777
--- /dev/null
+++ b/docs/scenarios/json.rst
@@ -0,0 +1,48 @@
+
+####
+JSON
+####
+
+.. image:: /_static/photos/33928819683_97b5c6a184_k_d.jpg
+
+The `json `_ library can parse
+JSON from strings or files. The library parses JSON into a Python dictionary
+or list. It can also convert Python dictionaries or lists into JSON strings.
+
+
+************
+Parsing JSON
+************
+
+Take the following string containing JSON data:
+
+.. code-block:: python
+
+ json_string = '{"first_name": "Guido", "last_name":"Rossum"}'
+
+It can be parsed like this:
+
+.. code-block:: python
+
+ import json
+ parsed_json = json.loads(json_string)
+
+and can now be used as a normal dictionary:
+
+.. code-block:: python
+
+ print(parsed_json['first_name'])
+ "Guido"
+
+You can also convert the following to JSON:
+
+.. code-block:: python
+
+ d = {
+ 'first_name': 'Guido',
+ 'second_name': 'Rossum',
+ 'titles': ['BDFL', 'Developer'],
+ }
+
+ print(json.dumps(d))
+ '{"first_name": "Guido", "last_name": "Rossum", "titles": ["BDFL", "Developer"]}'
diff --git a/docs/scenarios/ml.rst b/docs/scenarios/ml.rst
new file mode 100644
index 000000000..b3655c819
--- /dev/null
+++ b/docs/scenarios/ml.rst
@@ -0,0 +1,125 @@
+
+
+################
+Machine Learning
+################
+
+.. image:: /_static/photos/34018729885_002ced9b54_k_d.jpg
+
+Python has a vast number of libraries for data analysis, statistics, and Machine Learning itself, making it a language of choice for many data scientists.
+
+Some widely used packages for Machine Learning and other data science applications are listed below.
+
+
+***********
+SciPy Stack
+***********
+
+The SciPy stack consists of a bunch of core helper packages used in data science for statistical analysis and visualising data. Because of its huge number of functionalities and ease of use, the Stack is considered a must-have for most data science applications.
+
+The Stack consists of the following packages (link to documentation given):
+
+1. `NumPy `_
+2. `SciPy library `_
+3. `Matplotlib `_
+4. `IPython `_
+5. `pandas `_
+6. `Sympy `_
+7. `nose `_
+
+The stack also comes with Python bundled in, but has been excluded from the above list.
+
+Installation
+~~~~~~~~~~~~
+
+For installing the full stack, or individual packages, you can refer to the instructions given `here `_.
+
+**NB:** `Anaconda `_ is highly preferred and recommended for installing and maintaining data science packages seamlessly.
+
+
+************
+scikit-learn
+************
+
+Scikit is a free and open source machine learning library for Python. It offers off-the-shelf functions to implement many algorithms like linear regression, classifiers, SVMs, k-means, Neural Networks, etc. It also has a few sample datasets which can be directly used for training and testing.
+
+Because of its speed, robustness, and ease of, it's one of the most widely-used libraries for many Machine Learning applications.
+
+Installation
+~~~~~~~~~~~~
+
+Through PyPI:
+
+.. code-block:: python
+
+ pip install -U scikit-learn
+
+Through conda:
+
+.. code-block:: python
+
+ conda install scikit-learn
+
+scikit-learn also comes shipped with Anaconda (mentioned above). For more installation instructions, refer to `this link `_.
+
+Example
+~~~~~~~
+
+For this example, we train a simple classifier on the `Iris dataset `_, which comes bundled in with scikit-learn.
+
+The dataset takes four features of flowers: sepal length, sepal width, petal length, and petal width, and classifies them into three flower species (labels): setosa, versicolor, or virginica. The labels have been represented as numbers in the dataset: 0 (setosa), 1 (versicolor), and 2 (virginica).
+
+We shuffle the Iris dataset and divide it into separate training and testing sets, keeping the last 10 data points for testing and rest for training. We then train the classifier on the training set and predict on the testing set.
+
+.. code-block:: python
+
+ from sklearn.datasets import load_iris
+ from sklearn import tree
+ from sklearn.metrics import accuracy_score
+ import numpy as np
+
+ #loading the iris dataset
+ iris = load_iris()
+
+ x = iris.data #array of the data
+ y = iris.target #array of labels (i.e answers) of each data entry
+
+ #getting label names i.e the three flower species
+ y_names = iris.target_names
+
+ #taking random indices to split the dataset into train and test
+ test_ids = np.random.permutation(len(x))
+
+ #splitting data and labels into train and test
+ #keeping last 10 entries for testing, rest for training
+
+ x_train = x[test_ids[:-10]]
+ x_test = x[test_ids[-10:]]
+
+ y_train = y[test_ids[:-10]]
+ y_test = y[test_ids[-10:]]
+
+ #classifying using decision tree
+ clf = tree.DecisionTreeClassifier()
+
+ #training (fitting) the classifier with the training set
+ clf.fit(x_train, y_train)
+
+ #predictions on the test dataset
+ pred = clf.predict(x_test)
+
+ print pred #predicted labels i.e flower species
+ print y_test #actual labels
+ print (accuracy_score(pred, y_test))*100 #prediction accuracy
+
+Since we're splitting randomly and the classifier trains on every iteration, the accuracy may vary. Running the above code gives:
+
+.. code-block:: python
+
+ [0 1 1 1 0 2 0 2 2 2]
+ [0 1 1 1 0 2 0 2 2 2]
+ 100.0
+
+The first line contains the labels (i.e. flower species) of the testing data as predicted by our classifier, and the second line contains the actual flower species as given in the dataset. We thus get an accuracy of 100% this time.
+
+More on scikit-learn can be read in the `documentation `_.
diff --git a/docs/scenarios/network.rst b/docs/scenarios/network.rst
index b01c0efe5..bb751b78e 100644
--- a/docs/scenarios/network.rst
+++ b/docs/scenarios/network.rst
@@ -1,32 +1,46 @@
+
+##########
Networking
-==========
+##########
+
+.. image:: /_static/photos/34151833832_6bdfd930af_k_d.jpg
+
+*******
Twisted
--------
+*******
-`Twisted `_ is an event-driven networking engine. It can be
-used to build applications around many different networking protocols, including http servers
-and clients, applications using SMTP, POP3, IMAP or SSH protocols, instant messaging and
-`many more `_.
+`Twisted `_ is an event-driven networking
+engine. It can be used to build applications around many different networking
+protocols, including HTTP servers and clients, applications using SMTP, POP3,
+IMAP, or SSH protocols, instant messaging,
+and `much more `_.
+
+*****
PyZMQ
------
+*****
-`PyZMQ `_ is the Python binding for `ZeroMQ `_,
-which is a high-performance asynchronous messaging library. One great advantage is that ZeroMQ
-can be used for message queuing without a message broker. The basic patterns for this are:
+`PyZMQ `_ is the Python binding for
+`ZeroMQ `_, which is a high-performance asynchronous
+messaging library. One great advantage of ZeroMQ is that it can be used for
+message queuing without a message broker. The basic patterns for this are:
-- request-reply: connects a set of clients to a set of services. This is a remote procedure call
- and task distribution pattern.
-- publish-subscribe: connects a set of publishers to a set of subscribers. This is a data
- distribution pattern.
-- push-pull (or pipeline): connects nodes in a fan-out / fan-in pattern that can have multiple
- steps, and loops. This is a parallel task distribution and collection pattern.
+- request-reply: connects a set of clients to a set of services. This is a
+ remote procedure call and task distribution pattern.
+- publish-subscribe: connects a set of publishers to a set of subscribers.
+ This is a data distribution pattern.
+- push-pull (or pipeline): connects nodes in a fan-out/fan-in pattern that
+ can have multiple steps and loops. This is a parallel task distribution
+ and collection pattern.
For a quick start, read the `ZeroMQ guide `_.
+
+******
gevent
-------
-`gevent `_ is a coroutine-based Python networking library
-that uses greenlets and libevent event loop.
+******
+`gevent `_ is a coroutine-based Python networking
+library that uses greenlets to provide a high-level synchronous API on top of
+the libev event loop.
diff --git a/docs/scenarios/scientific.rst b/docs/scenarios/scientific.rst
index 031721091..bb0547323 100644
--- a/docs/scenarios/scientific.rst
+++ b/docs/scenarios/scientific.rst
@@ -1,75 +1,138 @@
-=======================
+
+#######################
Scientific Applications
-=======================
+#######################
+
+.. image:: /_static/photos/33925223870_97e44f5629_k_d.jpg
+
+*******
Context
-:::::::
+*******
-Python is frequently used for high-performance scientific applications. Python
-is widely used in academia and scientific projects because it is easy to write,
-and it performs really well.
+Python is frequently used for high-performance scientific applications. It
+is widely used in academia and scientific projects because it is easy to write
+and performs well.
-Due to its high performance nature, scientific computing in python often refers
-to external libraries, typically written in faster languages (like C, or
-FORTRAN for matrix operations). The main libraries used are `NumPy`_, `SciPy`_
+Due to its high performance nature, scientific computing in Python often
+utilizes external libraries, typically written in faster languages (like C, or
+Fortran for matrix operations). The main libraries used are `NumPy`_, `SciPy`_
and `Matplotlib`_. Going into detail about these libraries is beyond the scope
of the Python guide. However, a comprehensive introduction to the scientific
Python ecosystem can be found in the `Python Scientific Lecture Notes
-`_
+`_.
+
+
+*****
+Tools
+*****
+
+IPython
+-------
+
+`IPython `_ is an enhanced version of Python interpreter,
+which provides features of great interest to scientists. The `inline mode`
+allows graphics and plots to be displayed in the terminal (Qt based version).
+Moreover, the `notebook` mode supports literate programming and reproducible
+science generating a web-based Python notebook. This notebook allows you to
+store chunks of Python code alongside the results and additional comments
+(HTML, LaTeX, Markdown). The notebook can then be shared and exported in various
+file formats.
+
+*********
Libraries
-:::::::::
+*********
NumPy
-----
`NumPy `_ is a low level library written in C (and
-FORTRAN) for high level mathematical functions. NumPy cleverly overcomes the
+Fortran) for high level mathematical functions. NumPy cleverly overcomes the
problem of running slower algorithms on Python by using multidimensional arrays
and functions that operate on arrays. Any algorithm can then be expressed as a
function on arrays, allowing the algorithms to be run quickly.
-
NumPy is part of the SciPy project, and is released as a separate library so
-people who only need the basic requirements can just use NumPy.
+people who only need the basic requirements can use it without installing the
+rest of SciPy.
-NumPy is compatible with Python versions 2.4 through to 2.7.2 and 3.1+.
+NumPy is compatible with Python versions 2.4 through 2.7.2 and 3.1+.
Numba
-----
-Numba is an Numpy aware Python compiler (just-in-time (JIT) specializing
-compiler) which compiles annotated Python (and Numpy) code to LLVM (Low Level
-Virtual Machine) (through special decorators).
-Briefly, Numba using system that compiles Python code with LLVM to code which
-can be natively executed at runtime.
-.. todo:: Write about Numba
+`Numba `_ is a NumPy aware Python compiler
+(just-in-time (JIT) specializing compiler) which compiles annotated Python (and
+NumPy) code to LLVM (Low Level Virtual Machine) through special decorators.
+Briefly, Numba uses a system that compiles Python code with LLVM to code which
+can be natively executed at runtime.
SciPy
-----
-`SciPy `_ is a library that uses Numpy for more mathematical
-functions. SciPy uses NumPy arrays as the basic data structure. SciPy comes
-with modules for various commonly used tasks in scientific programing, for
-example: linear algebra, integration (calculus), ordinary differential equation
-solvers and signal processing.
+`SciPy `_ is a library that uses NumPy for more mathematical
+functions. SciPy uses NumPy arrays as the basic data structure, and comes
+with modules for various commonly used tasks in scientific programming,
+including linear algebra, integration (calculus), ordinary differential equation
+solving, and signal processing.
Matplotlib
----------
`Matplotlib `_ is a flexible plotting
library for creating interactive 2D and 3D plots that can also be saved as
-manuscript-quality figures. The API in many ways reflects that of `MATLAB
+manuscript-quality figures. The API in many ways reflects that of `MATLAB
`_, easing transition of MATLAB
-users to Python. Many examples, along with the source code to re-create them,
-can be browsed at the `matplotlib gallery
+users to Python. Many examples, along with the source code to recreate them,
+are available in the `matplotlib gallery
`_.
+Pandas
+------
+
+`Pandas `_ is a data manipulation library
+based on NumPy which provides many useful functions for accessing,
+indexing, merging, and grouping data easily. The main data structure (DataFrame)
+is close to what could be found in the R statistical package; that is,
+heterogeneous data tables with name indexing, time series operations, and
+auto-alignment of data.
+
+xarray
+------
+
+`xarray `_ is similar to Pandas, but it
+is intended for wrapping multidimensional scientific data. By labelling the
+data with dimensions, coordinates, and attributes, it makes complex
+multidimensional operations clearer and more intuitive. It also wraps
+matplotlib for quick plotting, and can apply most operations in parallel using
+`dask `_.
+
+
+Rpy2
+----
+
+`Rpy2 `_ is a Python binding for the R
+statistical package allowing the execution of R functions from Python and
+passing data back and forth between the two environments. Rpy2 is the object
+oriented implementation of the `Rpy `_
+bindings.
+
+PsychoPy
+--------
+
+`PsychoPy `_ is a library for cognitive scientists
+allowing the creation of cognitive psychology and neuroscience experiments.
+The library handles presentation of stimuli, scripting of experimental design,
+and data collection.
+
+
+*********
Resources
-:::::::::
+*********
-Installation of scientific Python packages can be troublesome. Many of these
-packages are implemented as Python C extensions which need to be compiled.
+Installation of scientific Python packages can be troublesome, as many of
+these packages are implemented as Python C extensions which need to be compiled.
This section lists various so-called scientific Python distributions which
provide precompiled and easy-to-install collections of scientific Python
packages.
@@ -77,31 +140,27 @@ packages.
Unofficial Windows Binaries for Python Extension Packages
---------------------------------------------------------
-Many people who do scientific computing are on Windows. And yet many of the
-scientific computing packages are notoriously difficult to build and install.
-`Christoph Gohlke `_ however, has
-compiled a list of Windows binaries for many useful Python packages. The list
-of packages has grown from a mainly scientific python resource to a more
-general list. It might be a good idea to check it out if you're on Windows.
-
-Enthought Python Distribution (EPD)
------------------------------------
-
-Installing NumPy and SciPy can be a daunting task. Which is why the
-`Enthought Python distribution `_ was created. With
-Enthought, scientific python has never been easier (one click to install about
-100 scientific python packages). The Enthought Python Distribution comes in two
-variants: a free version `EPD Free `_
-and a paid version with various `pricing options.
-`_
+Many people who do scientific computing are on Windows, yet many of the
+scientific computing packages are notoriously difficult to build and install on
+this platform. `Christoph Gohlke `_,
+however, has compiled a list of Windows binaries for many useful Python
+packages. The list of packages has grown from a mainly scientific Python
+resource to a more general list. If you're on Windows, you may want to check it
+out.
Anaconda
--------
-`Continuum Analytics `_ offers the `Anaconda
-Python Distribution `_ which
-includes all the common scientific python packages and additionally many
-packages related to data analytics and big data. Anaconda comes in two
-flavors, a paid for version and a completely free and open source community
-edition, Anaconda CE, which contains a slightly reduced feature set. Free
-licenses for the paid-for version are available for academics and researchers.
+The `Anaconda Python Distribution `_
+includes all the common scientific Python packages as well as many packages
+related to data analytics and big data. Anaconda itself is free, and a number
+of proprietary add-ons are available for a fee. Free licenses for the
+add-ons are available for academics and researchers.
+
+Canopy
+------
+
+`Canopy `_ is another scientific
+Python distribution, produced by `Enthought `_.
+A limited 'Canopy Express' variant is available for free, but Enthought
+charges for the full distribution. Free licenses are available for academics.
diff --git a/docs/scenarios/scrape.rst b/docs/scenarios/scrape.rst
index b4f10b2fa..527719200 100644
--- a/docs/scenarios/scrape.rst
+++ b/docs/scenarios/scrape.rst
@@ -1,62 +1,74 @@
+
+#############
HTML Scraping
-=============
+#############
+
+.. image:: /_static/photos/34268661876_442428e122_k_d.jpg
+
+************
Web Scraping
-------------
+************
Web sites are written using HTML, which means that each web page is a
structured document. Sometimes it would be great to obtain some data from
them and preserve the structure while we're at it. Web sites don't always
-provide their data in comfortable formats such as ``csv`` or ``json``.
+provide their data in comfortable formats such as CSV or JSON.
This is where web scraping comes in. Web scraping is the practice of using a
computer program to sift through a web page and gather the data that you need
in a format most useful to you while at the same time preserving the structure
of the data.
+
+*****************
lxml and Requests
------------------
+*****************
`lxml `_ is a pretty extensive library written for parsing
-XML and HTML documents really fast. It even handles messed up tags. We will
-also be using the `Requests `_
-module instead of the already built-in urlib2 due to improvements in speed and
-readability. You can easily install both using ``pip install lxml`` and
+XML and HTML documents very quickly, even handling messed up tags in the
+process. We will also be using the
+`Requests `_ module instead of the
+already built-in urllib2 module due to improvements in speed and readability.
+You can easily install both using ``pip install lxml`` and
``pip install requests``.
-Lets start with the imports:
+Let's start with the imports:
.. code-block:: python
from lxml import html
import requests
-Next we will use ``requests.get`` to retrieve the web page with our data
-and parse it using the ``html`` module and save the results in ``tree``:
+Next we will use ``requests.get`` to retrieve the web page with our data,
+parse it using the ``html`` module, and save the results in ``tree``:
.. code-block:: python
page = requests.get('http://econpy.pythonanywhere.com/ex/001.html')
- tree = html.fromstring(page.text)
+ tree = html.fromstring(page.content)
+
+(We need to use ``page.content`` rather than ``page.text`` because
+``html.fromstring`` implicitly expects ``bytes`` as input.)
``tree`` now contains the whole HTML file in a nice tree structure which
-we can go over two different ways: XPath and CSSSelect. In this example, I
+we can go over two different ways: XPath and CSSSelect. In this example, we
will focus on the former.
XPath is a way of locating information in structured documents such as
HTML or XML documents. A good introduction to XPath is on
-`W3Schools `_ .
+`W3Schools `_ .
There are also various tools for obtaining the XPath of elements such as
FireBug for Firefox or the Chrome Inspector. If you're using Chrome, you
can right click an element, choose 'Inspect element', highlight the code,
-right click again and choose 'Copy XPath'.
+right click again, and choose 'Copy XPath'.
After a quick analysis, we see that in our page the data is contained in
-two elements - one is a div with title 'buyer-name' and the other is a
+two elements -- one is a div with title 'buyer-name' and the other is a
span with class 'item-price':
-::
+.. code-block:: html
Carson Busses
$29.95
@@ -71,12 +83,12 @@ Knowing this we can create the correct XPath query and use the lxml
#This will create a list of prices
prices = tree.xpath('//span[@class="item-price"]/text()')
-Lets see what we got exactly:
+Let's see what we got exactly:
.. code-block:: python
- print 'Buyers: ', buyers
- print 'Prices: ', prices
+ print('Buyers: ', buyers)
+ print('Prices: ', prices)
::
@@ -96,6 +108,6 @@ a web page using lxml and Requests. We have it stored in memory as two
lists. Now we can do all sorts of cool stuff with it: we can analyze it
using Python or we can save it to a file and share it with the world.
-A cool idea to think about is modifying this script to iterate through
-the rest of the pages of this example dataset or rewriting this
+Some more cool ideas to think about are modifying this script to iterate
+through the rest of the pages of this example dataset, or rewriting this
application to use threads for improved speed.
diff --git a/docs/scenarios/serialization.rst b/docs/scenarios/serialization.rst
new file mode 100644
index 000000000..3edc6aaa2
--- /dev/null
+++ b/docs/scenarios/serialization.rst
@@ -0,0 +1,237 @@
+
+##################
+Data Serialization
+##################
+
+.. image:: /_static/photos/33467946364_3e59bd376a_k_d.jpg
+
+
+***************************
+What is data serialization?
+***************************
+
+Data serialization is the process of converting structured data to a format
+that allows sharing or storage of the data in a form that allows recovery of its original
+structure. In some cases, the secondary intention of data
+serialization is to minimize the data's size which then
+reduces disk space or bandwidth requirements.
+
+********************
+Flat vs. Nested data
+********************
+
+Before beginning to serialize data, it is important to identify or decide how the
+data should be structured during data serialization - flat or nested.
+The differences in the two styles are shown in the below examples.
+
+Flat style:
+
+.. code-block:: python
+
+ { "Type" : "A", "field1": "value1", "field2": "value2", "field3": "value3" }
+
+
+Nested style:
+
+.. code-block:: python
+
+ {"A"
+ { "field1": "value1", "field2": "value2", "field3": "value3" } }
+
+
+For more reading on the two styles, please see the discussion on
+`Python mailing list `__,
+`IETF mailing list `__ and
+`in stackexchange `__.
+
+****************
+Serializing Text
+****************
+
+=======================
+Simple file (flat data)
+=======================
+
+If the data to be serialized is located in a file and contains flat data, Python offers two methods to serialize data.
+
+repr
+----
+
+The repr method in Python takes a single object parameter and returns a printable representation of the input:
+
+.. code-block:: python
+
+ # input as flat text
+ a = { "Type" : "A", "field1": "value1", "field2": "value2", "field3": "value3" }
+
+ # the same input can also be read from a file
+ a = open('/tmp/file.py', 'r')
+
+ # returns a printable representation of the input;
+ # the output can be written to a file as well
+ print(repr(a))
+
+ # write content to files using repr
+ with open('/tmp/file.py') as f:f.write(repr(a))
+
+
+ast.literal_eval
+----------------
+
+The literal_eval method safely parses and evaluates an expression for a Python datatype.
+Supported data types are: strings, numbers, tuples, lists, dicts, booleans, and None.
+
+.. code-block:: python
+
+ with open('/tmp/file.py', 'r') as f: inp = ast.literal_eval(f.read())
+
+====================
+CSV file (flat data)
+====================
+
+The CSV module in Python implements classes to read and write tabular
+data in CSV format.
+
+Simple example for reading:
+
+.. code-block:: python
+
+ # Reading CSV content from a file
+ import csv
+ with open('/tmp/file.csv', newline='') as f:
+ reader = csv.reader(f)
+ for row in reader:
+ print(row)
+
+Simple example for writing:
+
+.. code-block:: python
+
+ # Writing CSV content to a file
+ import csv
+ with open('/temp/file.csv', 'w', newline='') as f:
+ writer = csv.writer(f)
+ writer.writerows(iterable)
+
+
+The module's contents, functions, and examples can be found
+`in the Python documentation `__.
+
+==================
+YAML (nested data)
+==================
+
+There are many third party modules to parse and read/write YAML file
+structures in Python. One such example is below.
+
+.. code-block:: python
+
+ # Reading YAML content from a file using the load method
+ import yaml
+ with open('/tmp/file.yaml', 'r', newline='') as f:
+ try:
+ print(yaml.load(f))
+ except yaml.YAMLError as ymlexcp:
+ print(ymlexcp)
+
+Documentation on the third party module can be found
+`in the PyYAML Documentation `__.
+
+=======================
+JSON file (nested data)
+=======================
+
+Python's JSON module can be used to read and write JSON files.
+Example code is below.
+
+Reading:
+
+.. code-block:: python
+
+ # Reading JSON content from a file
+ import json
+ with open('/tmp/file.json', 'r') as f:
+ data = json.load(f)
+
+Writing:
+
+.. code-block:: python
+
+ # Writing JSON content to a file using the dump method
+ import json
+ with open('/tmp/file.json', 'w') as f:
+ json.dump(data, f, sort_keys=True)
+
+=================
+XML (nested data)
+=================
+
+XML parsing in Python is possible using the `xml` package.
+
+Example:
+
+.. code-block:: python
+
+ # reading XML content from a file
+ import xml.etree.ElementTree as ET
+ tree = ET.parse('country_data.xml')
+ root = tree.getroot()
+
+More documentation on using the `xml.dom` and `xml.sax` packages can be found
+`in the Python XML library documentation `__.
+
+
+*******
+Binary
+*******
+
+=======================
+NumPy Array (flat data)
+=======================
+
+Python's NumPy array can be used to serialize and deserialize data to and from byte representation.
+
+Example:
+
+.. code-block:: python
+
+ import NumPy as np
+
+ # Converting NumPy array to byte format
+ byte_output = np.array([ [1, 2, 3], [4, 5, 6], [7, 8, 9] ]).tobytes()
+
+ # Converting byte format back to NumPy array
+ array_format = np.frombuffer(byte_output)
+
+
+
+====================
+Pickle (nested data)
+====================
+
+The native data serialization module for Python is called `Pickle
+`_.
+
+Here's an example:
+
+.. code-block:: python
+
+ import pickle
+
+ #Here's an example dict
+ grades = { 'Alice': 89, 'Bob': 72, 'Charles': 87 }
+
+ #Use dumps to convert the object to a serialized string
+ serial_grades = pickle.dumps( grades )
+
+ #Use loads to de-serialize an object
+ received_grades = pickle.loads( serial_grades )
+
+
+********
+Protobuf
+********
+
+If you're looking for a serialization module that has support in multiple
+languages, Google's `Protobuf
+`_ library is an option.
diff --git a/docs/scenarios/speed.rst b/docs/scenarios/speed.rst
index 6aaa96bbd..5dc0cd845 100644
--- a/docs/scenarios/speed.rst
+++ b/docs/scenarios/speed.rst
@@ -1,16 +1,20 @@
+
+#####
Speed
-=====
+#####
+
+.. image:: /_static/photos/33175625804_e225b90f3e_k_d.jpg
CPython, the most commonly used implementation of Python, is slow for CPU bound
tasks. `PyPy`_ is fast.
-Using a slightly modified version of `David Beazleys`_ CPU bound test code
+Using a slightly modified version of `David Beazley's`_ CPU bound test code
(added loop for multiple tests), you can see the difference between CPython
and PyPy's processing.
-::
+.. code-block:: console
- PyPy
+ # PyPy
$ ./pypy -V
Python 2.7.1 (7773f8fc4223, Nov 18 2011, 18:47:10)
[PyPy 1.7.0 with GCC 4.4.3]
@@ -21,9 +25,9 @@ and PyPy's processing.
0.0440690517426
0.0695300102234
-::
+.. code-block:: console
- CPython
+ # CPython
$ ./python -V
Python 2.7.1
$ ./python measure2.py
@@ -33,8 +37,10 @@ and PyPy's processing.
1.54693889618
1.60109114647
+
+*******
Context
-:::::::
+*******
The GIL
@@ -61,13 +67,159 @@ The GIL
`Special care`_ must be taken when writing C extensions to make sure you
register your threads with the interpreter.
+
+************
C Extensions
-::::::::::::
+************
Cython
------
+`Cython `_ implements a superset of the Python language
+with which you are able to write C and C++ modules for Python. Cython also
+allows you to call functions from compiled C libraries. Using Cython allows
+you to take advantage of Python's strong typing of variables and operations.
+
+Here's an example of strong typing with Cython:
+
+.. code-block:: cython
+
+ def primes(int kmax):
+ """Calculation of prime numbers with additional
+ Cython keywords"""
+
+ cdef int n, k, i
+ cdef int p[1000]
+ result = []
+ if kmax > 1000:
+ kmax = 1000
+ k = 0
+ n = 2
+ while k < kmax:
+ i = 0
+ while i < k and n % p[i] != 0:
+ i = i + 1
+ if i == k:
+ p[k] = n
+ k = k + 1
+ result.append(n)
+ n = n + 1
+ return result
+
+
+This implementation of an algorithm to find prime numbers has some additional
+keywords compared to the next one, which is implemented in pure Python:
+
+.. code-block:: python
+
+ def primes(kmax):
+ """Calculation of prime numbers in standard Python syntax"""
+
+ p = range(1000)
+ result = []
+ if kmax > 1000:
+ kmax = 1000
+ k = 0
+ n = 2
+ while k < kmax:
+ i = 0
+ while i < k and n % p[i] != 0:
+ i = i + 1
+ if i == k:
+ p[k] = n
+ k = k + 1
+ result.append(n)
+ n = n + 1
+ return result
+
+Notice that in the Cython version you declare integers and integer arrays
+to be compiled into C types while also creating a Python list:
+
+
+.. code-block:: cython
+
+ def primes(int kmax):
+ """Calculation of prime numbers with additional
+ Cython keywords"""
+
+ cdef int n, k, i
+ cdef int p[1000]
+ result = []
+
+
+.. code-block:: python
+
+ def primes(kmax):
+ """Calculation of prime numbers in standard Python syntax"""
+
+ p = range(1000)
+ result = []
+
+What is the difference? In the upper Cython version you can see the
+declaration of the variable types and the integer array in a similar way as
+in standard C. For example `cdef int n,k,i` in line 3. This additional type
+declaration (i.e. integer) allows the Cython compiler to generate more
+efficient C code from the second version. While standard Python code is saved
+in :file:`*.py` files, Cython code is saved in :file:`*.pyx` files.
+
+What's the difference in speed? Let's try it!
+
+.. code-block:: python
+
+ import time
+ # Activate pyx compiler
+ import pyximport
+ pyximport.install()
+ import primesCy # primes implemented with Cython
+ import primes # primes implemented with Python
+
+ print("Cython:")
+ t1 = time.time()
+ print(primesCy.primes(500))
+ t2 = time.time()
+ print("Cython time: %s" % (t2 - t1))
+ print("")
+ print("Python")
+ t1 = time.time()
+ print(primes.primes(500))
+ t2 = time.time()
+ print("Python time: %s" % (t2 - t1))
+
+
+These lines both need a remark:
+
+.. code-block:: python
+
+ import pyximport
+ pyximport.install()
+
+
+The `pyximport` module allows you to import :file:`*.pyx` files (e.g.,
+:file:`primesCy.pyx`) with the Cython-compiled version of the `primes`
+function. The `pyximport.install()` command allows the Python interpreter to
+start the Cython compiler directly to generate C code, which is automatically
+compiled to a :file:`*.so` C library. Cython is then able to import this
+library for you in your Python code, easily and efficiently. With the
+`time.time()` function you are able to compare the time between these 2
+different calls to find 500 prime numbers. On a standard notebook (dual core
+AMD E-450 1.6 GHz), the measured values are:
+
+.. code-block:: console
+
+ Cython time: 0.0054 seconds
+
+ Python time: 0.0566 seconds
+
+
+And here is the output of an embedded `ARM beaglebone `_ machine:
+
+.. code-block:: console
+
+ Cython time: 0.0196 seconds
+
+ Python time: 0.3302 seconds
+
Pyrex
-----
@@ -76,17 +228,214 @@ Pyrex
Shedskin?
---------
-Numba
------
-.. todo:: Write about Numba and the autojit compiler for NumPy
-Threading
-:::::::::
+***********
+Concurrency
+***********
-Threading
+Concurrent.futures
+------------------
+
+The `concurrent.futures`_ module is a module in the standard library that
+provides a "high-level interface for asynchronously executing callables". It
+abstracts away a lot of the more complicated details about using multiple
+threads or processes for concurrency, and allows the user to focus on
+accomplishing the task at hand.
+
+The `concurrent.futures`_ module exposes two main classes, the
+`ThreadPoolExecutor` and the `ProcessPoolExecutor`. The ThreadPoolExecutor
+will create a pool of worker threads that a user can submit jobs to. These jobs
+will then be executed in another thread when the next worker thread becomes
+available.
+
+The ProcessPoolExecutor works in the same way, except instead of using multiple
+threads for its workers, it will use multiple processes. This makes it possible
+to side-step the GIL; however, because of the way things are passed to worker
+processes, only picklable objects can be executed and returned.
+
+Because of the way the GIL works, a good rule of thumb is to use a
+ThreadPoolExecutor when the task being executed involves a lot of blocking
+(i.e. making requests over the network) and to use a ProcessPoolExecutor
+executor when the task is computationally expensive.
+
+There are two main ways of executing things in parallel using the two
+Executors. One way is with the `map(func, iterables)` method. This works
+almost exactly like the builtin `map()` function, except it will execute
+everything in parallel.
+
+.. code-block:: python
+
+ from concurrent.futures import ThreadPoolExecutor
+ import requests
+
+ def get_webpage(url):
+ page = requests.get(url)
+ return page
+
+ pool = ThreadPoolExecutor(max_workers=5)
+
+ my_urls = ['http://google.com/']*10 # Create a list of urls
+
+ for page in pool.map(get_webpage, my_urls):
+ # Do something with the result
+ print(page.text)
+
+For even more control, the `submit(func, *args, **kwargs)` method will schedule
+a callable to be executed ( as `func(*args, **kwargs)`) and returns a `Future`_
+object that represents the execution of the callable.
+
+The Future object provides various methods that can be used to check on the
+progress of the scheduled callable. These include:
+
+cancel()
+ Attempt to cancel the call.
+cancelled()
+ Return True if the call was successfully cancelled.
+running()
+ Return True if the call is currently being executed and cannot be
+ cancelled.
+done()
+ Return True if the call was successfully cancelled or finished running.
+result()
+ Return the value returned by the call. Note that this call will block until
+ the scheduled callable returns by default.
+exception()
+ Return the exception raised by the call. If no exception was raised then
+ this returns None. Note that this will block just like `result()`.
+add_done_callback(fn)
+ Attach a callback function that will be executed (as `fn(future)`) when the
+ scheduled callable returns.
+
+
+.. code-block:: python
+
+ from concurrent.futures import ProcessPoolExecutor, as_completed
+
+ def is_prime(n):
+ if n % 2 == 0:
+ return n, False
+
+ sqrt_n = int(n**0.5)
+ for i in range(3, sqrt_n + 1, 2):
+ if n % i == 0:
+ return n, False
+ return n, True
+
+ PRIMES = [
+ 112272535095293,
+ 112582705942171,
+ 112272535095293,
+ 115280095190773,
+ 115797848077099,
+ 1099726899285419]
+
+ futures = []
+ with ProcessPoolExecutor(max_workers=4) as pool:
+ # Schedule the ProcessPoolExecutor to check if a number is prime
+ # and add the returned Future to our list of futures
+ for p in PRIMES:
+ fut = pool.submit(is_prime, p)
+ futures.append(fut)
+
+ # As the jobs are completed, print out the results
+ for number, result in as_completed(futures):
+ if result:
+ print("{} is prime".format(number))
+ else:
+ print("{} is not prime".format(number))
+
+The `concurrent.futures`_ module contains two helper functions for working with
+Futures. The `as_completed(futures)` function returns an iterator over the list
+of futures, yielding the futures as they complete.
+
+The `wait(futures)` function will simply block until all futures in the list of
+futures provided have completed.
+
+For more information, on using the `concurrent.futures`_ module, consult the
+official documentation.
+
+threading
---------
+The standard library comes with a `threading`_ module that allows a user to
+work with multiple threads manually.
+
+Running a function in another thread is as simple as passing a callable and
+its arguments to `Thread`'s constructor and then calling `start()`:
+
+.. code-block:: python
+
+ from threading import Thread
+ import requests
+
+ def get_webpage(url):
+ page = requests.get(url)
+ return page
+
+ some_thread = Thread(get_webpage, 'http://google.com/')
+ some_thread.start()
+
+To wait until the thread has terminated, call `join()`:
+
+.. code-block:: python
+
+ some_thread.join()
+
+After calling `join()`, it is always a good idea to check whether the thread is
+still alive (because the join call timed out):
+
+.. code-block:: python
+
+ if some_thread.is_alive():
+ print("join() must have timed out.")
+ else:
+ print("Our thread has terminated.")
+
+Because multiple threads have access to the same section of memory, sometimes
+there might be situations where two or more threads are trying to write to the
+same resource at the same time or where the output is dependent on the sequence
+or timing of certain events. This is called a `data race`_ or race condition.
+When this happens, the output will be garbled or you may encounter problems
+which are difficult to debug. A good example is this `Stack Overflow post`_.
+
+The way this can be avoided is by using a `Lock`_ that each thread needs to
+acquire before writing to a shared resource. Locks can be acquired and released
+through either the contextmanager protocol (`with` statement), or by using
+`acquire()` and `release()` directly. Here is a (rather contrived) example:
+
+
+.. code-block:: python
+
+ from threading import Lock, Thread
+
+ file_lock = Lock()
+
+ def log(msg):
+ with file_lock:
+ open('website_changes.log', 'w') as f:
+ f.write(changes)
+
+ def monitor_website(some_website):
+ """
+ Monitor a website and then if there are any changes,
+ log them to disk.
+ """
+ while True:
+ changes = check_for_changes(some_website)
+ if changes:
+ log(changes)
+
+ websites = ['http://google.com/', ... ]
+ for website in websites:
+ t = Thread(monitor_website, website)
+ t.start()
+
+Here, we have a bunch of threads checking for changes on a list of sites and
+whenever there are any changes, they attempt to write those changes to a file
+by calling `log(changes)`. When `log()` is called, it will wait to acquire
+the lock with `with file_lock:`. This ensures that at any one time, only one
+thread is writing to the file.
Spawning Processes
------------------
@@ -97,8 +446,14 @@ Multiprocessing
.. _`PyPy`: http://pypy.org
-.. _`The GIL`: http://wiki.python.org/moin/GlobalInterpreterLock
+.. _`The GIL`: https://wiki.python.org/moin/GlobalInterpreterLock
.. _`guide`: http://www.dabeaz.com/python/UnderstandingGIL.pdf
.. _`New GIL`: http://www.dabeaz.com/python/NewGIL.pdf
-.. _`Special care`: http://docs.python.org/c-api/init.html#threads
-.. _`David Beazleys`: http://www.dabeaz.com/GIL/gilvis/measure2.py
+.. _`Special care`: https://docs.python.org/c-api/init.html#threads
+.. _`David Beazley's`: http://www.dabeaz.com/GIL/gilvis/measure2.py
+.. _`concurrent.futures`: https://docs.python.org/3/library/concurrent.futures.html
+.. _`Future`: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future
+.. _`threading`: https://docs.python.org/3/library/threading.html
+.. _`Stack Overflow post`: https://stackoverflow.com/questions/26688424/python-threads-are-printing-at-the-same-time-messing-up-the-text-output
+.. _`data race`: https://en.wikipedia.org/wiki/Race_condition
+.. _`Lock`: https://docs.python.org/3/library/threading.html#lock-objects
diff --git a/docs/scenarios/web.rst b/docs/scenarios/web.rst
index 1520b4bbc..a2c84818b 100644
--- a/docs/scenarios/web.rst
+++ b/docs/scenarios/web.rst
@@ -1,13 +1,19 @@
-================
-Web Applications
-================
+
+#############################
+Web Applications & Frameworks
+#############################
+
+.. image:: /_static/photos/34309496175_b82d104282_k_d.jpg
As a powerful scripting language adapted to both fast prototyping
-and bigger projects, Python is widely used in Web applications
+and bigger projects, Python is widely used in web application
development.
+
+*******
Context
-:::::::
+*******
+
WSGI
@@ -18,13 +24,14 @@ interface between web servers and Python web application frameworks. By
standardizing behavior and communication between web servers and Python web
frameworks, WSGI makes it possible to write portable Python web code that
can be deployed in any :ref:`WSGI-compliant web server `.
-WSGI is documented in `PEP-3333 `_.
+WSGI is documented in :pep:`3333`.
+**********
Frameworks
-::::::::::
+**********
-Broadly speaking, a web framework consist of a set of libraries and a main
+Broadly speaking, a web framework consists of a set of libraries and a main
handler within which you can build custom code to implement a web application
(i.e. an interactive web site). Most web frameworks include patterns and
utilities to accomplish at least the following:
@@ -34,7 +41,7 @@ URL Routing
be invoked
Request and Response Objects
- Encapsulate the information received from or sent to a user's browser
+ Encapsulates the information received from or sent to a user's browser
Template Engine
Allows for separating Python code implementing an application's logic from
@@ -48,67 +55,117 @@ Development Web Server
Django
------
-`Django `_ is a "batteries included" web
-application framework. By providing many utilities and patterns out of the
-box, Django aims to make it possible to build complex, database-backed web
-applications quickly, while encouraging best practices in code written using
-it.
+`Django `_ is a "batteries included" web
+application framework, and is an excellent choice for creating content-oriented
+websites. By providing many utilities and patterns out of the box, Django aims
+to make it possible to build complex, database-backed web applications quickly,
+while encouraging best practices in code written using it.
Django has a large and active community, and many pre-built `re-usable
modules `_ that can be incorporated into a new
project as-is, or customized to fit your needs.
There are annual Django conferences `in the United States
-`_ and `in Europe `_.
+`_, `Europe `_, and `Australia `_.
+The majority of new Python web applications today are built with Django.
Flask
-----
-`Flask `_ is a "microframework" for Python. Rather
-than aiming to provide everything you could possibly need, Flask implements
-the most commonly-used core components of a web application framework, like
-URL routing, request and response objects, and templates. As a user of
-Flask, it is therefore up to you to choose and integrate other components
-you may need, such as database access or form generation and validation. For
-many popular modules, `Extensions `_ may
-already exist to suit your needs.
+`Flask `_ is a "microframework" for Python, and is
+an excellent choice for building smaller applications, APIs, and web services.
-**Support** for flask can best be found in its mailing list. Just shoot an
-email to flask@librelist.com and reply to the confirmation email.
+Building an app with Flask is a lot like writing standard Python modules,
+except some functions have routes attached to them. It's really beautiful.
+Rather than aiming to provide everything you could possibly need, Flask
+implements the most commonly-used core components of a web application
+framework, like URL routing, request and response objects, and templates.
-Werkzeug
---------
+If you use Flask, it is up to you to choose other components for your
+application, if any. For example, database access or form generation and
+validation are not built-in functions of Flask.
+
+This is great, because many web applications don't need those features.
+For those that do, there are many
+`Extensions `_ available that may
+suit your needs. Or, you can easily use any library you want yourself!
+
+Flask is default choice for any Python web application that isn't a good
+fit for Django.
+
+Falcon
+------
-`Werkzeug `_ is not actually a real framework, but
-rather a very powerful set of tools for building web applications. It provides
-URL routing utilities, request and response objects and a basic development
-server. It is mostly used where users need bigger flexibility for their
-application that is not commonly found in other web frameworks.
+`Falcon `_ is a good choice when your goal is
+to build RESTful API microservices that are fast and scalable.
-Support can be found on its `mailing list `_.
+It is a reliable, high-performance Python web framework for building large-scale
+app backends and microservices. Falcon encourages the REST architectural style of
+mapping URIs to resources, trying to do as little as possible while remaining highly effective.
+Falcon highlights four main focuses: speed, reliability, flexibility, and debuggability.
+It implements HTTP through "responders" such as ``on_get()``, ``on_put()``, etc.
+These responders receive intuitive request and response objects.
+
+Tornado
+--------
+
+`Tornado `_ is an asynchronous web framework
+for Python that has its own event loop. This allows it to natively support
+WebSockets, for example. Well-written Tornado applications are known to
+have excellent performance characteristics.
+
+I do not recommend using Tornado unless you think you need it.
Pyramid
--------
-`Pyramid `_ lies somewhere between a big
-framework like Django and the microframeworks: It comes with a lot of libraries
-and functionality and can thus not be considered lightweight. On the other
-hand, it does not provide all the functionality Django does. Instead Pyramid
-brings basic support for most regular tasks and provides a great deal of
-extensibility. Additionally, Pyramid has a huge focus on complete
-`documentation `__. As
-a little extra it comes with the Werkzeug Debugger which allows you to debug a
-running web application in the browser.
+`Pyramid `_ is a very flexible framework with a heavy
+focus on modularity. It comes with a small number of libraries ("batteries")
+built-in, and encourages users to extend its base functionality. A set of
+provided cookiecutter templates helps making new project decisions for users.
+It powers one of the most important parts of python infrastructure
+`PyPI `_.
+
+Pyramid does not have a large user base, unlike Django and Flask. It's a
+capable framework, but not a very popular choice for new Python web
+applications today.
+
+Masonite
+--------
+
+`Masonite `_ is a modern and developer centric, "batteries included", web framework.
-**Support** can also be found in the
-`documentation `__.
+The Masonite framework follows the MVC (Model-View-Controller) architecture pattern and is heavily inspired by frameworks such as Rails and Laravel, so if you are coming to Python from a Ruby or PHP background then you will feel right at home!
+Masonite comes with a lot of functionality out of the box including a powerful IOC container with auto resolving dependency injection, craft command line tools, and the Orator active record style ORM.
+Masonite is perfect for beginners or experienced developers alike and works hard to be fast and easy from install through to deployment. Try it once and you’ll fall in love.
+
+FastAPI
+-------
+
+`FastAPI `_ is a modern web framework for building
+APIs with Python 3.6+.
+
+It has very high performance as it is based on `Starlette `_
+and `Pydantic `_.
+
+FastAPI takes advantage of standard Python type declarations in function parameters
+to declare request parameters and bodies, perform data conversion (serialization,
+parsing), data validation, and automatic API documentation with **OpenAPI 3**
+(including **JSON Schema**).
+
+It includes tools and utilities for security and authentication (including OAuth2 with JWT
+tokens), a dependency injection system, automatic generation of interactive API
+documentation, and other features.
+
+
+***********
Web Servers
-:::::::::::
+***********
.. _nginx-ref:
@@ -116,7 +173,7 @@ Nginx
-----
`Nginx `_ (pronounced "engine-x") is a web server and
-reverse-proxy for HTTP, SMTP and other protocols. It is known for its
+reverse-proxy for HTTP, SMTP, and other protocols. It is known for its
high performance, relative simplicity, and compatibility with many
application servers (like WSGI servers). It also includes handy features
like load-balancing, basic authentication, streaming, and others. Designed
@@ -125,33 +182,69 @@ to serve high-load websites, Nginx is gradually becoming quite popular.
.. _wsgi-servers-ref:
+
+************
WSGI Servers
-::::::::::::
+************
Stand-alone WSGI servers typically use less resources than traditional web
-servers and provide top performance [3]_.
+servers and provide top performance [1]_.
.. _gunicorn-ref:
Gunicorn
--------
-`Gunicorn `_ (Green Unicorn) is a WSGI server used
-to serve Python applications. It is a Python interpretation of the Ruby
-`Unicorn `_ server. Unicorn is designed to be
-lightweight, easy to use, and uses many UNIX idioms. Gunicorn is not designed
-to face the internet -- it was designed to run behind Nginx which buffers
-slow requests and takes care of other important considerations. A sample
-setup for Nginx + Gunicorn can be found in the
-`Gunicorn help `_.
+`Gunicorn `_ (Green Unicorn) is a pure-Python WSGI
+server used to serve Python applications. Unlike other Python web servers,
+it has a thoughtful user interface, and is extremely easy to use and
+configure.
+
+Gunicorn has sane and reasonable defaults for configurations. However, some
+other servers, like uWSGI, are tremendously more customizable, and therefore,
+are much more difficult to effectively use.
+
+Gunicorn is the recommended choice for new Python web applications today.
+
+
+Waitress
+--------
+
+`Waitress `_ is a pure-Python WSGI server
+that claims "very acceptable performance". Its documentation is not very
+detailed, but it does offer some nice functionality that Gunicorn doesn't have
+(e.g. HTTP request buffering).
+
+Waitress is gaining popularity within the Python web development community.
.. _uwsgi-ref:
+uWSGI
+-----
+
+`uWSGI `_ is a full stack for building
+hosting services. In addition to process management, process monitoring,
+and other functionality, uWSGI acts as an application server for various
+programming languages and protocols -- including Python and WSGI. uWSGI can
+either be run as a stand-alone web router, or be run behind a full web
+server (such as Nginx or Apache). In the latter case, a web server can
+configure uWSGI and an application's operation over the
+`uwsgi protocol `_.
+uWSGI's web server support allows for dynamically configuring
+Python, passing environment variables, and further tuning. For full details,
+see `uWSGI magic
+variables `_.
+I do not recommend using uWSGI unless you know why you need it.
+
+.. _server-best-practices-ref:
+
+
+*********************
Server Best Practices
-:::::::::::::::::::::
+*********************
-The majority of self hosted Python applications today are hosted with a WSGI
+The majority of self-hosted Python applications today are hosted with a WSGI
server such as :ref:`Gunicorn `, either directly or behind a
lightweight web server such as :ref:`nginx `.
@@ -159,102 +252,57 @@ The WSGI servers serve the Python applications while the web server handles
tasks better suited for it such as static file serving, request routing, DDoS
protection, and basic authentication.
-Hosting
-:::::::
-Platform-as-a-Service
----------------------
+*******
+Hosting
+*******
Platform-as-a-Service (PaaS) is a type of cloud computing infrastructure
which abstracts and manages infrastructure, routing, and scaling of web
-applications. When using PaaS, application developers can focus on writing
+applications. When using a PaaS, application developers can focus on writing
application code rather than needing to be concerned with deployment
details.
-Most PaaS services offer a command-line interface that developers can use to
-set up and interrogate configuration, and to deploy new releases of an
-application to the service.
-
-PaaS services and their partners offer add-on functionality which is well
-integrated into the platform, such as database hosting, email services,
-logging, scheduled and background tasks, billing and payment, etc.
-
-
Heroku
-~~~~~~
-
-`Heroku `_'s
-`Cedar stack `_ offers first class
-support for Python 2.7 applications.
-
-Heroku allows you to run as many Python web applications as you like, 24/7 and
-free of charge. Heroku is best described as a horizontal scaling platform. They
-start to charge you once you "scale" your application to run on more than one
-Dyno (abstracted servers) at a time.
-
-Heroku publishes `step-by-step instructions
-`_ on how to set up your first
-application for use in Heroku, and maintains a list of `example applications
-`_.
-
-
-DotCloud
-~~~~~~~~
-
-`DotCloud `_ supports WSGI applications and
-background/worker tasks natively on their platform. Web applications run
-Python version 2.6, use :ref:`nginx ` and :ref:`uWSGI
-`, and allow custom configuration of both for advanced users.
-
-DotCloud uses a custom command-line API client which can work with
-applications managed in git repositories or any other version control
-system.
-
-DotCloud has a free plan with limited database size, and without extra
-services (caching…).
-
-See the `DotCloud documentation on Python
-`_ for more information and help
-getting started.
+------
+`Heroku `_ offers first-class support for
+Python 2.7–3.5 applications.
-Gondor
-~~~~~~
+Heroku supports all types of Python web applications, servers, and frameworks.
+Applications can be developed on Heroku for free. Once your application is
+ready for production, you can upgrade to a Hobby or Professional application.
-`Gondor `_ is a PaaS specialized for deploying Django
-and Pinax applications. Gondor supports Django versions 1.2 and 1.3 on
-Python version 2.7, and can automatically configure your Django site if you
-use ``local_settings.py`` for site-specific configuration information.
+Heroku maintains `detailed articles `_
+on using Python with Heroku, as well as `step-by-step instructions
+`_ on
+how to set up your first application.
-Gondor publishes guides to deploying `Django projects
-`_ and `Pinax projects
-`_ on their platform.
+Heroku is the recommended PaaS for deploying Python web applications today.
+**********
Templating
-::::::::::
-
-Most WSGI applications are responding to HTTP requests to serve
-content in HTML or other markup languages. Instead of generating directly
-textual content from Python, the concept of separation of concerns
-advises us to use templates. A template engine manages a suite of
-template files, with a system of hierarchy and inclusion to
-avoid unnecessary repetition, and is in charge of rendering
-(generating) the actual content, filling the static content
-of the templates with the dynamic content generated by the
-application.
+**********
+
+Most WSGI applications are responding to HTTP requests to serve content in HTML
+or other markup languages. Instead of directly generating textual content from
+Python, the concept of separation of concerns advises us to use templates. A
+template engine manages a suite of template files, with a system of hierarchy
+and inclusion to avoid unnecessary repetition, and is in charge of rendering
+(generating) the actual content, filling the static content of the templates
+with the dynamic content generated by the application.
As template files are
-sometimes written by designers or front-end developers,
-it can be difficult to handle increasing complexity.
+sometimes written by designers or front-end developers, it can be difficult to
+handle increasing complexity.
-Some general good practices apply to the part of the
-application passing dynamic content to the template engine,
-and to the templates themselves.
+Some general good practices apply to the part of the application passing
+dynamic content to the template engine, and to the templates themselves.
- Template files should be passed only the dynamic
content that is needed for rendering the template. Avoid
- to be tempted to pass additional content "just in case":
+ the temptation to pass additional content "just in case":
it is easier to add some missing variable when needed than to remove
a likely unused variable later.
@@ -262,15 +310,236 @@ and to the templates themselves.
or assignments in the template itself, and many
allow some Python code to be evaluated in the
templates. This convenience can lead to uncontrolled
- increase in complexity, and often harder to find bugs.
+ increase in complexity, and often make it harder to find bugs.
-- It is often necessary to mix javascript templates with
+- It is often necessary to mix JavaScript templates with
HTML templates. A sane approach to this design is to isolate
the parts where the HTML template passes some variable content
- to the javascript code.
+ to the JavaScript code.
+
+
+
+Jinja2
+------
+`Jinja2 `_ is a very well-regarded template engine.
+
+It uses a text-based template language and can thus be used to generate any
+type of markup, not just HTML. It allows customization of filters, tags, tests,
+and globals. It features many improvements over Django's templating system.
+
+Here some important HTML tags in Jinja2:
+
+.. code-block:: html
+
+ {# This is a comment #}
+
+ {# The next tag is a variable output: #}
+ {{title}}
+
+ {# Tag for a block, can be replaced through inheritance with other html code #}
+ {% block head %}
+
This is the head!
+ {% endblock %}
+
+ {# Output of an array as an iteration #}
+ {% for item in list %}
+
{{ item }}
+ {% endfor %}
+
+
+The next listings are an example of a web site in combination with the Tornado
+web server. Tornado is not very complicated to use.
+
+.. code-block:: python
+
+ # import Jinja2
+ from jinja2 import Environment, FileSystemLoader
+
+ # import Tornado
+ import tornado.ioloop
+ import tornado.web
+
+ # Load template file templates/site.html
+ TEMPLATE_FILE = "site.html"
+ templateLoader = FileSystemLoader( searchpath="templates/" )
+ templateEnv = Environment( loader=templateLoader )
+ template = templateEnv.get_template(TEMPLATE_FILE)
+
+ # List for famous movie rendering
+ movie_list = [[1,"The Hitchhiker's Guide to the Galaxy"],[2,"Back to future"],[3,"Matrix"]]
+
+ # template.render() returns a string which contains the rendered html
+ html_output = template.render(list=movie_list,
+ title="Here is my favorite movie list")
+
+ # Handler for main page
+ class MainHandler(tornado.web.RequestHandler):
+ def get(self):
+ # Returns rendered template string to the browser request
+ self.write(html_output)
+
+ # Assign handler to the server root (127.0.0.1:PORT/)
+ application = tornado.web.Application([
+ (r"/", MainHandler),
+ ])
+ PORT=8884
+ if __name__ == "__main__":
+ # Setup the server
+ application.listen(PORT)
+ tornado.ioloop.IOLoop.instance().start()
+
+The :file:`base.html` file can be used as base for all site pages which are
+for example implemented in the content block.
+
+.. code-block:: html
+
+
+
+
+
+
+ {{title}} - My Webpage
+
+
+
+ {# In the next line the content from the site.html template will be added #}
+ {% block content %}{% endblock %}
+
+
+
+
+
+The next listing is our site page (:file:`site.html`) loaded in the Python
+app which extends :file:`base.html`. The content block is automatically set
+into the corresponding block in the :file:`base.html` page.
+
+.. code-block:: html
+
+ {% extends "base.html" %}
+ {% block content %}
+
+
+
{{title}}
+
{{ list_title }}
+
+ {% for item in list %}
+
{{ item[0]}} : {{ item[1]}}
+ {% endfor %}
+
+
+
+ {% endblock %}
+
+
+Jinja2 is the recommended templating library for new Python web applications.
+
+Chameleon
+---------
+
+`Chameleon `_ Page Templates are an HTML/XML template
+engine implementation of the `Template Attribute Language (TAL) `_,
+`TAL Expression Syntax (TALES) `_,
+and `Macro Expansion TAL (Metal) `_ syntaxes.
+
+Chameleon is available for Python 2.5 and up (including 3.x and PyPy), and
+is commonly used by the `Pyramid Framework `_.
+
+Page Templates add within your document structure special element attributes
+and text markup. Using a set of simple language constructs, you control the
+document flow, element repetition, text replacement, and translation. Because
+of the attribute-based syntax, unrendered page templates are valid HTML and can
+be viewed in a browser and even edited in WYSIWYG editors. This can make
+round-trip collaboration with designers and prototyping with static files in a
+browser easier.
+
+The basic TAL language is simple enough to grasp from an example:
+
+.. code-block:: html
+
+
+
+
Hello, World!
+
+
+
+
+
+
+
+
+
+
+
+The `` pattern for text insertion is common
+enough that if you do not require strict validity in your unrendered templates,
+you can replace it with a more terse and readable syntax that uses the pattern
+`${expression}`, as follows:
+
+.. code-block:: html
+
+
+
+
Hello, ${world}!
+
+
+
+ ${row.capitalize()} ${col}
+
+
+
+
+
+
+
+But keep in mind that the full `Default Text`
+syntax also allows for default content in the unrendered template.
+
+Being from the Pyramid world, Chameleon is not widely used.
+
+Mako
+----
+
+`Mako `_ is a template language that compiles to Python
+for maximum performance. Its syntax and API are borrowed from the best parts of other
+templating languages like Django and Jinja2 templates. It is the default template
+language included with the `Pylons and Pyramid `_ web
+frameworks.
+
+An example template in Mako looks like:
+
+.. code-block:: mako
+
+ <%inherit file="base.html"/>
+ <%
+ rows = [[v for v in range(0,10)] for row in range(0,10)]
+ %>
+
+ % for row in rows:
+ ${makerow(row)}
+ % endfor
+
+
+ <%def name="makerow(row)">
+
+ % for name in row:
+
${name}
\
+ % endfor
+
+ %def>
+
+To render a very basic template, you can do the following:
+
+.. code-block:: python
+
+ from mako.template import Template
+ print(Template("hello ${data}!").render(data="world"))
+
+Mako is well respected within the Python web community.
.. rubric:: References
-.. [1] `The mod_python project is now officially dead `_
-.. [2] `mod_wsgi vs mod_python `_
-.. [3] `Benchmark of Python WSGI Servers `_
+.. [1] `Benchmark of Python WSGI Servers `_
diff --git a/docs/scenarios/xml.rst b/docs/scenarios/xml.rst
index df89d580a..24dab5869 100644
--- a/docs/scenarios/xml.rst
+++ b/docs/scenarios/xml.rst
@@ -1,14 +1,20 @@
+
+###########
XML parsing
-===========
+###########
+
+.. image:: /_static/photos/33888714601_a1f7d020a2_k_d.jpg
+
+********
untangle
---------
+********
-`untangle `_ is a simple library which takes
-an XML document and returns a Python object which mirrors the nodes and
+`untangle `_ is a simple library which
+takes an XML document and returns a Python object which mirrors the nodes and
attributes in its structure.
-For example, an xml file like this:
+For example, an XML file like this:
.. code-block:: xml
@@ -24,11 +30,81 @@ can be loaded like this:
import untangle
obj = untangle.parse('path/to/file.xml')
-and then you can get the child elements name like this:
+and then you can get the child element's name attribute like this:
.. code-block:: python
obj.root.child['name']
-untangle also supports loading XML from a string or an URL.
+untangle also supports loading XML from a string or a URL.
+
+
+*********
+xmltodict
+*********
+
+`xmltodict `_ is another simple
+library that aims at making XML feel like working with JSON.
+
+An XML file like this:
+
+.. code-block:: xml
+
+
+
+ elements
+ more elements
+
+
+ element as well
+
+
+
+can be loaded into a Python dict like this:
+
+.. code-block:: python
+
+ import xmltodict
+
+ with open('path/to/file.xml') as fd:
+ doc = xmltodict.parse(fd.read())
+
+and then you can access elements, attributes, and values like this:
+
+.. code-block:: python
+
+ doc['mydocument']['@has'] # == u'an attribute'
+ doc['mydocument']['and']['many'] # == [u'elements', u'more elements']
+ doc['mydocument']['plus']['@a'] # == u'complex'
+ doc['mydocument']['plus']['#text'] # == u'element as well'
+
+xmltodict also lets you roundtrip back to XML with the unparse function,
+has a streaming mode suitable for handling files that don't fit in memory,
+and supports XML namespaces.
+
+**********
+xmlschema
+**********
+
+`xmlschema `_ provides support for using XSD-Schemas in Python.
+Unlike other XML libraries, automatic type parsing is available, so f.e. if the schema defines an element to be of type ``int``, the parsed ``dict`` will contain also an ``int`` value for that element.
+Moreover the library supports automatic and explicit validation of XML documents against a schema.
+
+.. code-block:: python
+
+ from xmlschema import XMLSchema, etree_tostring
+ # load a XSD schema file
+ schema = XMLSchema("your_schema.xsd")
+
+ # validate against the schema
+ schema.validate("your_file.xml")
+
+ # or
+ schema.is_valid("your_file.xml")
+
+ # decode a file
+ data = schmema.decode("your_file.xml")
+
+ # encode to string
+ s = etree_tostring(schema.encode(data))
diff --git a/docs/shipping/freezing.rst b/docs/shipping/freezing.rst
index 2722c1250..841fa2040 100644
--- a/docs/shipping/freezing.rst
+++ b/docs/shipping/freezing.rst
@@ -1,63 +1,135 @@
+.. _freezing-your-code-ref:
+
+
+##################
Freezing Your Code
-==================
+##################
-An alternative to shipping your code is freezing it — shipping it as an
-executable with a bundled Python interpreter.
+.. image:: /_static/photos/33907151034_e0a9e53402_k_d.jpg
-Many applications you use every day do this:
+"Freezing" your code is creating a single-file executable file to distribute
+to end-users, that contains all of your application code as well as the
+Python interpreter.
-- Dropbox
-- BitTorrent
-- ...
+Applications such as 'Dropbox', 'Eve Online', 'Civilization IV', and
+BitTorrent clients do this.
-.. todo:: Fill in "Freezing Your Code" stub
+The advantage of distributing this way is that your application will "just work",
+even if the user doesn't already have the required version of Python (or any)
+installed. On Windows, and even on many Linux distributions and OS X, the right
+version of Python will not already be installed.
+Besides, end-user software should always be in an executable format. Files
+ending in ``.py`` are for software engineers and system administrators.
+One disadvantage of freezing is that it will increase the size of your
+distribution by about 2–12 MB. Also, you will be responsible for shipping
+updated versions of your application when security vulnerabilities to
+Python are patched.
-Comparison
-----------
-Solutions and platforms/features supported:
+************************
+Alternatives to Freezing
+************************
+
+:ref:`Packaging your code ` is for distributing
+libraries or tools to other developers.
+
+On Linux, an alternative to freezing is to
+:ref:`create a Linux distro package `
+(e.g. .deb files for Debian or Ubuntu, or .rpm files for Red Hat and SuSE.)
+
+.. todo:: Fill in "Freezing Your Code" stub
+
-=========== ======= ===== ==== ======== ======= ============= ============== ==== =====================
-Solution Windows Linux OS X Python 3 License One-file mode Zipfile import Eggs pkg_resources support
-=========== ======= ===== ==== ======== ======= ============= ============== ==== =====================
-bbFreeze yes yes yes no MIT no yes yes yes
-py2exe yes no no no MIT yes yes no no
-pyInstaller yes yes yes no GPL yes no yes no
-cx_Freeze yes yes yes yes PSF no yes yes no
-=========== ======= ===== ==== ======== ======= ============= ============== ==== =====================
+****************************
+Comparison of Freezing Tools
+****************************
-.. todo:: Add other solutions: py2app
+Date of this writing: Oct 5, 2019
+Solutions and platforms/features supported:
+
+=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== =====================
+Solution Windows Linux OS X Python 3 License One-file mode Zipfile import Eggs pkg_resources support Latest release date
+=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== =====================
+bbFreeze yes yes yes no MIT no yes yes yes Jan 20, 2014
+py2exe yes no no yes MIT yes yes no no Oct 21, 2014
+pyInstaller yes yes yes yes GPL yes no yes no Jul 9, 2019
+cx_Freeze yes yes yes yes PSF no yes yes no Aug 29, 2019
+py2app no no yes yes MIT no yes yes yes Mar 25, 2019
+=========== ======= ===== ==== ======== ======= ============= ============== ==== ===================== =====================
.. note::
Freezing Python code on Linux into a Windows executable was only once
- supported in PyInstaller `and later dropped.
- `_.
+ supported in PyInstaller `and later dropped
+ `_.
.. note::
- All solutions need MS Visual C++ dll to be installed on target machine.
- Only Pyinstaller makes self-executable exe that bundles the dll when
- passing ``--onefile`` to `Configure.py`.
+ All solutions need a Microsoft Visual C++ to be installed on the target machine, except py2app.
+ Only PyInstaller makes a self-executable exe that bundles the appropriate DLL when
+ passing ``--onefile`` to :file:`Configure.py`.
+
+*******
Windows
--------
+*******
bbFreeze
~~~~~~~~
-Prerequisite is to install :ref:`Python, Distribute and pywin32 dependency on Windows `.
+Prerequisite is to install :ref:`Python, Setuptools and pywin32 dependency on Windows `.
+
+1. Install :code:`bbfreeze`:
+
+.. code-block:: console
+
+ $ pip install bbfreeze
+
+2. Write most basic :file:`bb_setup.py`
+
+.. code-block:: python
+
+ from bbfreeze import Freezer
-.. todo:: Write steps for most basic .exe
+ freezer = Freezer(distdir='dist')
+ freezer.addScript('foobar.py', gui_only=True)
+ freezer()
+
+.. note::
+
+ This will work for the most basic one file scripts. For more advanced freezing you will have to provide
+ include and exclude paths like so:
+
+ .. code-block:: python
+
+ freezer = Freezer(distdir='dist', includes=['my_code'], excludes=['docs'])
+
+3. (Optionally) include icon
+
+.. code-block:: python
+
+ freezer.setIcon('my_awesome_icon.ico')
+
+4. Provide the Microsoft Visual C++ runtime DLL for the freezer. It might be possible to append your :code:`sys.path`
+with the Microsoft Visual Studio path but I find it easier to drop :file:`msvcp90.dll` in the same folder where your script
+resides.
+
+5. Freeze!
+
+.. code-block:: console
+
+ $ python bb_setup.py
py2exe
~~~~~~
-Prerequisite is to install :ref:`Python on Windows `.
+Prerequisite is to install :ref:`Python on Windows `. The last release of py2exe is from the year 2014. There is not active development.
1. Download and install http://sourceforge.net/projects/py2exe/files/py2exe/
-2. Write setup.py (`List of configuration options `_)::
+2. Write :file:`setup.py` (`List of configuration options `_):
+
+.. code-block:: python
from distutils.core import setup
import py2exe
@@ -68,25 +140,28 @@ Prerequisite is to install :ref:`Python on Windows `.
3. (Optionally) `include icon `_
-4. (Optionally) `one-file mode `_
+4. (Optionally) `one-file mode `_
-5. Generate `.exe` into `dist` directory::
+5. Generate :file:`.exe` into :file:`dist` directory:
+
+.. code-block:: console
$ python setup.py py2exe
-6. Provide the Microsoft Visual C runtime DLL. Two options: `globally install dll on target machine `_ or `distribute dll alongside with .exe `_.
+6. Provide the Microsoft Visual C++ runtime DLL. Two options: `globally install dll on target machine `_ or `distribute dll alongside with .exe `_.
PyInstaller
~~~~~~~~~~~
-Prerequisite is to have installed :ref:`Python, Distribute and pywin32 dependency on Windows `.
+Prerequisite is to have installed :ref:`Python, Setuptools and pywin32 dependency on Windows `.
- `Most basic tutorial `_
-- `Manual `_
+- `Manual `_
-OSX
----
+****
+OS X
+****
py2app
@@ -95,13 +170,97 @@ py2app
PyInstaller
~~~~~~~~~~~
+PyInstaller can be used to build Unix executables and windowed apps on Mac OS X 10.6 (Snow Leopard) or newer.
+
+To install PyInstaller, use pip:
+
+.. code-block:: console
+
+ $ pip install pyinstaller
+To create a standard Unix executable, from say :code:`script.py`, use:
+
+.. code-block:: console
+
+ $ pyinstaller script.py
+
+This creates:
+
+- a :code:`script.spec` file, analogous to a :code:`make` file
+- a :code:`build` folder, that holds some log files
+- a :code:`dist` folder, that holds the main executable :code:`script`, and some dependent Python libraries
+
+all in the same folder as :code:`script.py`. PyInstaller puts all the Python libraries used in :code:`script.py` into the :code:`dist` folder, so when distributing the executable, distribute the whole :code:`dist` folder.
+
+The :code:`script.spec` file can be edited to `customise the build `_, with options such as:
+
+- bundling data files with the executable
+- including run-time libraries (:code:`.dll` or :code:`.so` files) that PyInstaller can't infer automatically
+- adding Python run-time options to the executable
+
+Now :code:`script.spec` can be run with :code:`pyinstaller` (instead of using :code:`script.py` again):
+
+.. code-block:: console
+
+ $ pyinstaller script.spec
+
+To create a standalone windowed OS X application, use the :code:`--windowed` option:
+
+.. code-block:: console
+
+ $ pyinstaller --windowed script.spec
+
+This creates a :code:`script.app` in the :code:`dist` folder. Make sure to use GUI packages in your Python code, like `PyQt `_ or `PySide `_, to control the graphical parts of the app.
+
+There are several options in :code:`script.spec` related to Mac OS X app bundles `here `_. For example, to specify an icon for the app, use the :code:`icon=\path\to\icon.icns` option.
+
+
+*****
Linux
------
+*****
bbFreeze
~~~~~~~~
+.. warning:: bbFreeze will ONLY work in Python 2.x environment, since it's no longer being maintained as stated by it's former maintainer. If you're interested in it, check the repository in `here `_.
+
+bbFreeze can be used with all distributions that has Python installed along with pip2 and/or easy_install.
+
+For pip2, use the following:
+
+.. code-block:: console
+
+ $ pip2 install bbfreeze
+
+Or, for easy_install:
+
+.. code-block:: console
+
+ $ easy_install bbfreeze
+
+With bbFreeze installed, you're ready to freeze your applications.
+
+Let's assume you have a script, say, "hello.py" and a module called "module.py" and you have a function in it that's being used in your script.
+No need to worry, you can just ask to freeze the main entrypoint of your script and it should freeze entirely:
+
+.. code-block:: console
+
+ $ bbfreeze script.py
+
+With this, it creates a folder called dist/, of which contains the executable of the script and required .so (shared objects) files linked against libraries used within the Python script.
+
+Alternatively, you can create a script that does the freezing for you. An API for the freezer is available from the library within:
+
+.. code-block:: python
+
+ from bbfreeze import Freezer
+
+ freezer = Freezer(distdir='dist')
+ freezer.addScript('script.py', gui_only=True) # Enable gui_only kwarg for app that uses GUI packages.
+ freezer()
PyInstaller
~~~~~~~~~~~
+PyInstaller can be used in a similar fashion as in OS X. The installation goes in the same manner as shown in the OS X section.
+
+Don't forget to have dependencies such as Python and pip installed for usage.
diff --git a/docs/shipping/packaging.rst b/docs/shipping/packaging.rst
index e91bb33d6..34674cff2 100644
--- a/docs/shipping/packaging.rst
+++ b/docs/shipping/packaging.rst
@@ -1,35 +1,77 @@
+.. _packaging-your-code-ref:
+
+
+###################
Packaging Your Code
-===================
+###################
+
+.. image:: /_static/photos/36137234682_be6898bf57_k_d.jpg
+
+Package your code to share it with other developers. For example,
+to share a library for other developers to use in their application,
+or for development tools like 'py.test'.
+
+An advantage of this method of distribution is its well established ecosystem
+of tools such as PyPI and pip, which make it easy for other developers to
+download and install your package either for casual experiments, or as part of
+large, professional systems.
+
+It is a well-established convention for Python code to be shared this way.
+If your code isn't packaged on PyPI, then it will be harder
+for other developers to find it and to use it as part of their existing
+process. They will regard such projects with substantial suspicion of being
+either badly managed or abandoned.
+
+The downside of distributing code like this is that it relies on the
+recipient understanding how to install the required version of Python,
+and being able and willing to use tools such as pip to install your code's
+other dependencies. This is fine when distributing to other developers, but
+makes this method unsuitable for distributing applications to end-users.
-Packaging your code is important.
+The `Python Packaging Guide `_
+provides an extensive guide on creating and maintaining Python packages.
-You'll need to package your code first before sharing it with other developers.
+*************************
+Alternatives to Packaging
+*************************
+
+To distribute applications to end-users, you should
+:ref:`freeze your application `.
+
+On Linux, you may also want to consider
+:ref:`creating a Linux distro package `
+(e.g. a .deb file for Debian or Ubuntu.)
+
+
+*********************
For Python Developers
-:::::::::::::::::::::
+*********************
-If you're writing an open source Python module, `PyPI `_,
-more properly known as *The Cheeseshop*, is the place to host it.
+If you're writing an open source Python module, `PyPI `_
+, more properly known as *The Cheeseshop*, is the place to host it.
Pip vs. easy_install
--------------------
-Use `pip `_. More details `here `_
+Use `pip `_. More details
+`here `_.
Personal PyPI
-------------
-If you want to install packages from a source different from PyPI, (say, if
-your packages are *proprietary*), you can do it by hosting a simple http server,
-running from the directory which holds those packages which need to be installed.
+If you want to install packages from a source other than PyPI (say, if
+your packages are *proprietary*), you can do it by hosting a simple HTTP
+server, running from the directory which holds those packages which need to be
+installed.
**Showing an example is always beneficial**
-Say if you are after installing a package called MyPackage.tar.gz, and
-assuming this is your directory structure:
+For example, if you want to install a package called :file:`MyPackage.tar.gz`,
+and assuming this is your directory structure:
- archive
@@ -41,11 +83,11 @@ Go to your command prompt and type:
.. code-block:: console
$ cd archive
- $ python -m SimpleHTTPServer 9000
+ $ python -m http.server 9000
-This runs a simple http server running on port 9000 and will list all packages
-(like **MyPackage**). Now you can install **MyPackage** using any python
-package installer. Using Pip, you would do it like:
+This runs a simple HTTP server running on port 9000 and will list all packages
+(like **MyPackage**). Now you can install **MyPackage** using any Python
+package installer. Using pip, you would do it like:
.. code-block:: console
@@ -53,30 +95,102 @@ package installer. Using Pip, you would do it like:
Having a folder with the same name as the package name is **crucial** here.
I got fooled by that, one time. But if you feel that creating a folder called
-**MyPackage** and keeping **MyPackage.tar.gz** inside that, is *redundant*,
-you can still install MyPackage using:
+:file:`MyPackage` and keeping :file:`MyPackage.tar.gz` inside that is
+*redundant*, you can still install MyPackage using:
.. code-block:: console
$ pip install http://127.0.0.1:9000/MyPackage.tar.gz
-Chishop
-+++++++
+pypiserver
+++++++++++
+
+`pypiserver `_ is a minimal PyPI
+compatible server. It can be used to serve a set of packages to easy_install
+or pip. It includes helpful features like an administrative command
+(``-U``) which will update all its packages to their latest versions
+found on PyPI.
+
+
+S3-Hosted PyPi
+++++++++++++++
+
+One simple option for a personal PyPI server is to use Amazon S3. A
+prerequisite for this is that you have an Amazon AWS account with an S3 bucket.
+
+1. **Install all your requirements from PyPi or another source**
+2. **Install pip2pi**
+
+* :code:`pip install git+https://github.com/wolever/pip2pi.git`
+
+3. **Follow pip2pi README for pip2tgz and dir2pi commands**
+
+* :code:`pip2tgz packages/ YourPackage` (or :code:`pip2tgz packages/ -r requirements.txt`)
+* :code:`dir2pi packages/`
+
+4. **Upload the new files**
+
+* Use a client like Cyberduck to sync the entire :file:`packages` folder to your s3 bucket.
+* Make sure you upload :code:`packages/simple/index.html` as well as all new files and directories.
+
+5. **Fix new file permissions**
+
+* By default, when you upload new files to the S3 bucket, they will have the wrong permissions set.
+* Use the Amazon web console to set the READ permission of the files to EVERYONE.
+* If you get HTTP 403 when trying to install a package, make sure you've set the permissions correctly.
+
+6. **All done**
+
+* You can now install your package with :code:`pip install --index-url=http://your-s3-bucket/packages/simple/ YourPackage`.
+
+.. _packaging-for-linux-distributions-ref:
-`Chishop `_ is a simple PyPI server
-written in django which allows you to register/upload with distutils and
-install with easy_install/pip.
+***********************
For Linux Distributions
-::::::::::::::::::::::::
+***********************
+
+Creating a Linux distro package is arguably the "right way" to distribute code
+on Linux.
+
+Because a distribution package doesn't include the Python interpreter, it
+makes the download and install about 2-12 MB smaller than
+:ref:`freezing your application `.
+
+Also, if a distribution releases a new security update for Python, then your
+application will automatically start using that new version of Python.
+
+The bdist_rpm command makes `producing an RPM file `_
+for use by distributions like Red Hat or SuSE trivially easy.
+
+However, creating and maintaining the different configurations required for
+each distribution's format (e.g. .deb for Debian/Ubuntu, .rpm for Red
+Hat/Fedora, etc.) is a fair amount of work. If your code is an application that
+you plan to distribute on other platforms, then you'll also have to create and
+maintain the separate config required to freeze your application for Windows
+and OS X. It would be much less work to simply create and maintain a single
+config for one of the cross platform :ref:`freezing tools
+`, which will produce stand-alone executables for all
+distributions of Linux, as well as Windows and OS X.
+
+Creating a distribution package is also problematic if your code is for a
+version of Python that isn't currently supported by a distribution.
+Having to tell *some versions* of Ubuntu end-users that they need to add `the
+'dead-snakes' PPA `_
+using `sudo apt-repository` commands before they can install your .deb file
+makes for an extremely hostile user experience. Not only that, but you'd have
+to maintain a custom equivalent of these instructions for every distribution,
+and worse, have your users read, understand, and act on them.
+
+Having said all that, here's how to do it:
-* `Ubuntu `_
* `Fedora `_
-* `Debian `_
+* `Debian and Ubuntu `_
* `Arch `_
Useful Tools
------------
-- epm
-- alien
+- `fpm `_
+- `alien `_
+- `dh-virtualenv `_ (for APT/DEB omnibus packaging)
diff --git a/docs/shipping/publishing.rst b/docs/shipping/publishing.rst
new file mode 100644
index 000000000..4b480f801
--- /dev/null
+++ b/docs/shipping/publishing.rst
@@ -0,0 +1,66 @@
+.. _publishing-your-code-ref:
+
+
+####################
+Publishing Your Code
+####################
+
+.. todo:: Replace this kitten with the photo we want.
+
+.. image:: https://placekitten.com/800/600
+
+A healthy open source project needs a place to publish its code and project
+management stuff so other developers can collaborate with you. This lets your
+users gain a better understanding of your code, keep up with new developments,
+report bugs, and contribute code.
+
+This development web site should include the source code history itself, a bug
+tracker, a patch submission (aka "Pull Request") queue, and possibly additional
+developer-oriented documentation.
+
+There are several free open source project hosting sites (aka "forges"). These
+include GitHub, SourceForge, Bitbucket, and GitLab. GitHub is currently the best.
+Use GitHub.
+
+
+*********************************
+Creating a Project Repo on GitHub
+*********************************
+
+To publish your Python project on GitHub:
+
+1. Create a GitHub account if you don't already have one.
+
+2. Create a new repo for your project.
+
+ 1. Click on the "+" menu next to your avatar in the upper right of the page and choose "New repository".
+
+ 2. Name it after your project and give it an SEO-friendly description.
+
+ 3. If you don't have an existing project repo, choose the settings to add a
+ README, `.gitignore`, and license. Use the Python `.gitignore` option.
+
+3. On the newly created repo page, click "Manage topics" and add the tags "python" and "python3" and/or "python2" as appropriate.
+
+4. Include a link to your new GitHub repo in your project's README file so people who just have the project distribution know where to find it.
+
+If this is a brand new repo, clone it to your local machine and start working:
+
+.. code-block:: console
+
+ $ git clone https://github.com//
+
+Or, if you already have a project Git repo, add your new GitHub repo as a remote:
+
+.. code-block:: console
+
+ $ cd
+ $ git remote add origin https://github.com//
+ $ git push --tags
+
+***********************
+When Your Project Grows
+***********************
+
+For more information about managing an open source software project, see the book
+`Producing Open Source Software `_.
diff --git a/docs/starting/install/linux.rst b/docs/starting/install/linux.rst
index 1fd04cba4..4198be8be 100644
--- a/docs/starting/install/linux.rst
+++ b/docs/starting/install/linux.rst
@@ -1,89 +1,79 @@
.. _install-linux:
-Installing Python on Linux
-==========================
-The latest versions of Ubuntu and Fedora **come with Python 2.7 out of the box**.
+############################
+Installing Python 2 on Linux
+############################
-The latest versions of Redhat Enterprise (RHEL) and CentOS come with Python 2.6.
-Some older versions of RHEL and CentOS come with Python 2.4 which is
-unacceptable for modern Python development. Fortunately, there are
-`Extra Packages for Enterprise Linux`_ which include high
-quality additional packages based on their Fedora counterparts. This
-repository contains a Python 2.6 package specifically designed to install
-side-by-side with the system's Python 2.4 installation.
+.. image:: /_static/photos/34435688560_4cc2a7bcbb_k_d.jpg
-.. _Extra Packages for Enterprise Linux: http://fedoraproject.org/wiki/EPEL
+.. note::
+ Check out our :ref:`guide for installing Python 3 on Linux`.
-You do not need to install or configure anything else to use Python. Having
-said that, I would strongly recommend that you install the tools and libraries
-described in the next section before you start building Python applications
-for real-world use. In particular, you should always install Distribute, as
-it makes it much easier for you to use other third-party Python libraries.
+The latest versions of CentOS, Red Hat Enterprise Linux (RHEL) and Ubuntu
+**come with Python 2.7 out of the box**.
-Distribute & Pip
-----------------
+To see which version of Python you have installed, open a command prompt and run
-The most crucial third-party Python software of all is Distribute, which
-extends the packaging and installation facilities provided by the distutils
-in the standard library. Once you add Distribute to your Python system you can
-download and install any compliant Python software product with a single
-command. It also enables you to add this network installation capability to
-your own Python software with very little work.
-
-To obtain the latest version of Distribute for Linux, run the python script
-available here: `python-distribute `_
+.. code-block:: console
-The new``easy_install`` command you have available is considered by many to be
-deprecated, so we will install its replacement: **pip**. Pip allows for
-uninstallation of packages, and is actively maintained, unlike easy_install.
+ $ python2 --version
-To install pip, simply open a command prompt and run
+However, with the growing popularity of Python 3, some distributions, such as
+Fedora, don't come with Python 2 pre-installed. You can install the ``python2``
+package with your distribution package manager:
.. code-block:: console
- $ easy_install pip
+ $ sudo dnf install python2
+
+You do not need to install or configure anything else to use Python. Having
+said that, I would strongly recommend that you install the tools and libraries
+described in the next section before you start building Python applications
+for real-world use. In particular, you should always install Setuptools and pip, as
+it makes it much easier for you to use other third-party Python libraries.
+
+****************
+Setuptools & Pip
+****************
-Virtualenv
-----------
+The two most crucial third-party Python packages are `setuptools `_ and `pip `_.
-After Distribute & Pip, the next development tool that you should install is
-`virtualenv `_. Use pip
+Once installed, you can download, install and uninstall any compliant Python software
+product with a single command. It also enables you to add this network installation
+capability to your own Python software with very little work.
+
+Python 2.7.9 and later (on the python2 series), and Python 3.4 and later include
+pip by default.
+
+To see if pip is installed, open a command prompt and run
.. code-block:: console
- $ pip install virtualenv
+ $ command -v pip
-The virtualenv kit provides the ability to create virtual Python environments
-that do not interfere with either each other, or the main Python installation.
-If you install virtualenv before you begin coding then you can get into the
-habit of using it to create completely clean Python environments for each
-project. This is particularly important for Web development, where each
-framework and application will have many dependencies.
+To install pip, `follow the official pip installation guide `_ - this will automatically install the latest version of setuptools.
-To set up a new Python environment, change the working directory to where ever
-you want to store the environment, and run the virtualenv utility in your
-project's directory
-.. code-block:: console
+********************
+Virtual Environments
+********************
- $ virtualenv --distribute venv
+A Virtual Environment is a tool to keep the dependencies required by different projects
+in separate places, by creating virtual Python environments for them. It solves the
+"Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps
+your global site-packages directory clean and manageable.
-To use an environment, run ``source venv/bin/activate``. Your command prompt
-will change to show the active environment. Once you have finished working in
-the current virtual environment, run ``deactivate`` to restore your settings
-to normal.
+For example, you can work on a project which requires Django 1.10 while also
+maintaining a project which requires Django 1.8.
-Each new environment automatically includes a copy of ``pip``, so that you can
-setup the third-party libraries and tools that you want to use in that
-environment. Put your own code within a subdirectory of the environment,
-however you wish. When you no longer need a particular environment, simply
-copy your code out of it, and then delete the main directory for the environment.
+To start using this and see more information: :ref:`Virtual Environments ` docs.
+You can also use :ref:`virtualenvwrapper ` to make it easier to
+manage your virtual environments.
--------------------------------
-This page is a remixed version of `another guide `_,
+This page is a remixed version of `another guide `_,
which is available under the same license.
-
diff --git a/docs/starting/install/osx.rst b/docs/starting/install/osx.rst
index 2384a64ed..4a8a8e03b 100644
--- a/docs/starting/install/osx.rst
+++ b/docs/starting/install/osx.rst
@@ -1,129 +1,134 @@
.. _install-osx:
-Installing Python on Mac OS X
-=============================
-The latest version of Mac OS X, Mountain Lion, **comes with Python 2.7 out of the box**.
+###############################
+Installing Python 2 on Mac OS X
+###############################
-You do not need to install or configure anything else to use Python. Having
-said that, I would strongly recommend that you install the tools and libraries
-described in the next section before you start building Python applications
-for real-world use. In particular, you should always install Distribute, as it
-makes it much easier for you to use other third-party Python libraries.
+.. image:: /_static/photos/34435688560_4cc2a7bcbb_k_d.jpg
+
+.. note::
+ Check out our :ref:`guide for installing Python 3 on OS X`.
+
+**Mac OS X comes with Python 2.7 out of the box between versions 10.8 and 12.3.**
+
+If your Mac OS X version is between the above versions,
+you do not need to install or configure anything else to use Python. Having said
+that, I would strongly recommend that you install the tools and libraries
+described in the next section before you start building Python applications for
+real-world use. In particular, you should always install Setuptools, as it makes
+it much easier for you to install and manage other third-party Python libraries.
The version of Python that ships with OS X is great for learning, but it's not
-good for development. It's slightly out of date, and Apple has made significant
-changes that can cause hidden bugs.
+good for development. The version shipped with OS X may be out of date from the
+`official current Python release `_,
+which is considered the stable production version.
+
+**************
Doing it Right
---------------
+**************
Let's install a real version of Python.
-Before installing Python, you'll need to install GCC. GCC can be obtained
-by downloading `XCode `_, the smaller
-`Command Line Tools `_ (must have an
-Apple account) or the even smaller `OSX-GCC-Installer `_
+Before installing Python, you'll need to install a C compiler. The fastest way
+is to install the Xcode Command Line Tools by running
+``xcode-select --install``. You can also download the full version of
+`Xcode `_ from the Mac App Store, or the
+minimal but unofficial
+`OSX-GCC-Installer `_
package.
-While Lion comes with a large number of UNIX utilities, those familiar with
+.. note::
+ If you already have Xcode installed, do not install OSX-GCC-Installer.
+ In combination, the software can cause issues that are difficult to
+ diagnose.
+
+.. note::
+ If you perform a fresh install of Xcode, you will also need to add the
+ commandline tools by running ``xcode-select --install`` on the terminal.
+
+
+While OS X comes with a large number of Unix utilities, those familiar with
Linux systems will notice one key component missing: a decent package manager.
-`Homebrew `_ fills this void.
+`Homebrew `_ fills this void.
-To `install Homebrew `_,
-simply run
+To `install Homebrew `_, open :file:`Terminal` or
+your favorite OS X terminal emulator and run
.. code-block:: console
- $ ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
+ $ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
The script will explain what changes it will make and prompt you before the
installation begins.
Once you've installed Homebrew, insert the Homebrew directory at the top
-of your ``PATH`` environment variable. You can do this by adding the following
-line at the bottom of your ``~/.bashrc`` file
+of your :envvar:`PATH` environment variable. You can do this by adding the following
+line at the bottom of your :file:`~/.profile` file
.. code-block:: console
- export PATH=/usr/local/bin:$PATH
+ export PATH="/usr/local/bin:/usr/local/sbin:$PATH"
-Now, we can install Python 2.7: ::
-
- $ brew install python --framework
-
-This will take a minute or two. Once that's complete, you'll have to add the
-new Python scripts directory to your ``PATH``
+Now, we can install Python 2.7:
.. code-block:: console
- export PATH=/usr/local/share/python:$PATH
+ $ brew install python@2
-The ``--framework`` option tells Homebrew to compile a Framework-style Python
-build, rather than a UNIX-style build. The outdated version of Python that
-Snow Leopard comes packaged with is built as a Framework, so this helps avoid
-some future module installation bugs.
+Because ``python@2`` is a "keg", we need to update our ``PATH`` again, to point at our new installation:
+.. code-block:: console
-Distribute & Pip
-----------------
-
-The most crucial third-party Python software of all is Distribute, which
-extends the packaging and installation facilities provided by the distutils
-in the standard library. Once you add Distribute to your Python system you can
-download and install any compliant Python software product with a single
-command. It also enables you to add this network installation capability to
-your own Python software with very little work.
+ export PATH="/usr/local/opt/python@2/libexec/bin:$PATH"
-Homebrew already installed Distribute for you. Its ``easy_install`` command is
-considered by many to be deprecated, so we will install its replacement:
-**pip**. Pip allows for uninstallation of packages, and is actively maintained,
-unlike easy_install.
+Homebrew names the executable ``python2`` so that you can still run the system Python via the executable ``python``.
-To install pip, simply open a command prompt and run
.. code-block:: console
- $ easy_install pip
+ $ python -V # Homebrew installed Python 3 interpreter (if installed)
+ $ python2 -V # Homebrew installed Python 2 interpreter
+ $ python3 -V # Homebrew installed Python 3 interpreter (if installed)
-Virtualenv
-----------
+****************
+Setuptools & Pip
+****************
-After Distribute & Pip, the next development tool that you should install is
-`virtualenv `_. Use pip
+Homebrew installs Setuptools and ``pip`` for you.
-.. code-block:: console
+Setuptools enables you to download and install any compliant Python
+software over a network (usually the Internet) with a single command
+(``easy_install``). It also enables you to add this network installation
+capability to your own Python software with very little work.
- $ pip install virtualenv
+``pip`` is a tool for easily installing and managing Python packages,
+that is recommended over ``easy_install``. It is superior to ``easy_install``
+in `several ways `_,
+and is actively maintained.
-The virtualenv kit provides the ability to create virtual Python environments
-that do not interfere with either each other, or the main Python installation.
-If you install virtualenv before you begin coding then you can get into the
-habit of using it to create completely clean Python environments for each
-project. This is particularly important for Web development, where each
-framework and application will have many dependencies.
+.. code-block:: console
-To set up a new Python environment, change the working directory to where ever
-you want to store the environment, and run the virtualenv utility in your
-project's directory
+ $ pip2 -V # pip pointing to the Homebrew installed Python 2 interpreter
+ $ pip -V # pip pointing to the Homebrew installed Python 3 interpreter (if installed)
-.. code-block:: console
- $ virtualenv --distribute venv
+********************
+Virtual Environments
+********************
-To use an environment, run ``source venv/bin/activate``. Your command prompt
-will change to show the active environment. Once you have finished working in
-the current virtual environment, run ``deactivate`` to restore your settings
-to normal.
+A Virtual Environment (commonly referred to as a 'virtualenv') is a tool to keep the dependencies required by different projects
+in separate places, by creating virtual Python environments for them. It solves the
+"Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps
+your global site-packages directory clean and manageable.
-Each new environment automatically includes a copy of ``pip``, so that you can
-setup the third-party libraries and tools that you want to use in that
-environment. Put your own code within a subdirectory of the environment,
-however you wish. When you no longer need a particular environment, simply
-copy your code out of it, and then delete the main directory for the environment.
+For example, you can work on a project which requires Django 1.10 while also
+maintaining a project which requires Django 1.8.
+To start using this and see more information: :ref:`Virtual Environments ` docs.
--------------------------------
-This page is a remixed version of `another guide `_,
+This page is a remixed version of `another guide `_,
which is available under the same license.
diff --git a/docs/starting/install/win.rst b/docs/starting/install/win.rst
index 420957275..67f8885d6 100644
--- a/docs/starting/install/win.rst
+++ b/docs/starting/install/win.rst
@@ -1,112 +1,96 @@
.. _install-windows:
-Installing Python on Windows
-============================
-First, download the `latest version `_
-of Python 2.7 from the official Website. If you want to be sure you are installing a fully
-up-to-date version then use the "Windows Installer" link from the home page of the
-`Python.org web site `_ .
+##############################
+Installing Python 2 on Windows
+##############################
+
+.. image:: /_static/photos/34435688560_4cc2a7bcbb_k_d.jpg
+
+.. note::
+ Check out our :ref:`guide for installing Python 3 on Windows`.
+
+First, download the `latest version `_
+of Python 2.7 from the official website. If you want to be sure you are installing a fully
+up-to-date version, click the Downloads > Windows link from the home page of the
+`Python.org web site `_ .
The Windows version is provided as an MSI package. To install it manually, just
double-click the file. The MSI package format allows Windows administrators to
automate installation with their standard tools.
By design, Python installs to a directory with the version number embedded,
-e.g. Python version 2.7 will install at ``C:\Python27\``, so that you can
+e.g. Python version 2.7 will install at :file:`C:\\Python27\\`, so that you can
have multiple versions of Python on the
same system without conflicts. Of course, only one interpreter can be the
default application for Python file types. It also does not automatically
-modify the ``PATH`` environment variable, so that you always have control over
+modify the :envvar:`PATH` environment variable, so that you always have control over
which copy of Python is run.
Typing the full path name for a Python interpreter each time quickly gets
-tedious, so add the directories for your default Python version to the PATH.
-Assuming that your Python installation is in ``C:\Python27\``, add this to your
-PATH::
+tedious, so add the directories for your default Python version to the :envvar:`PATH`.
+Assuming that your Python installation is in :file:`C:\\Python27\\`, add this to your
+:envvar:`PATH`:
+
+.. code-block:: doscon
C:\Python27\;C:\Python27\Scripts\
-You can do this easily by running the following in ``powershell``::
+You can do this easily by running the following in ``powershell``:
+
+.. code-block:: powershell
[Environment]::SetEnvironmentVariable("Path", "$env:Path;C:\Python27\;C:\Python27\Scripts\", "User")
-The second (``Scripts``) directory receives command files when certain
+This is also an option during the installation process.
+
+The second (:file:`Scripts`) directory receives command files when certain
packages are installed, so it is a very useful addition.
You do not need to install or configure anything else to use Python. Having
said that, I would strongly recommend that you install the tools and libraries
described in the next section before you start building Python applications for
-real-world use. In particular, you should always install Distribute, as it
+real-world use. In particular, you should always install Setuptools, as it
makes it much easier for you to use other third-party Python libraries.
-Distribute + Pip
-----------------
-
-The most crucial third-party Python software of all is Distribute, which
-extends the packaging and installation facilities provided by the distutils in
-the standard library. Once you add Distribute to your Python system you can
-download and install any compliant Python software product with a single
-command. It also enables you to add this network installation capability to
-your own Python software with very little work.
-
-To obtain the latest version of Distribute for Windows, run the python script
-available here: `python-distribute `_
-
-
-You'll now have a new command available to you: **easy_install**. It is
-considered by many to be deprecated, so we will install its replacement:
-**pip**. Pip allows for uninstallation of packages, and is actively maintained,
-unlike easy_install.
-
-To install pip, simply open a command prompt and run
-
-.. code-block:: console
-
- > easy_install pip
+****************
+Setuptools + Pip
+****************
-Virtualenv
-----------
+The two most crucial third-party Python packages are `setuptools `_ and `pip `_.
-After Distribute & Pip, the next development tool that you should install is
-`virtualenv `_. Use pip
+Once installed, you can download, install and uninstall any compliant Python software
+product with a single command. It also enables you to add this network installation
+capability to your own Python software with very little work.
-.. code-block:: console
+Python 2.7.9 and later (on the python2 series), and Python 3.4 and later include
+pip by default.
- > pip install virtualenv
+To see if pip is installed, open a command prompt and run
-The virtualenv kit provides the ability to create virtual Python environments
-that do not interfere with either each other, or the main Python installation.
-If you install virtualenv before you begin coding then you can get into the
-habit of using it to create completely clean Python environments for each
-project. This is particularly important for Web development, where each
-framework and application will have many dependencies.
+.. code-block:: doscon
+ command -v pip
-To set up a new Python environment, change the working directory to where ever
-you want to store the environment, and run the virtualenv utility in your
-project's directory
+To install pip, `follow the official pip installation guide `_ - this will automatically install the latest version of setuptools.
-.. code-block:: console
- > virtualenv --distribute venv
+********************
+Virtual Environments
+********************
-To use an environment, run the ``activate.bat`` batch file in the ``Scripts``
-subdirectory of that environment. Your command prompt will change to show the
-active environment. Once you have finished working in the current virtual
-environment, run the ``deactivate.bat`` batch file to restore your settings to
-normal.
+A Virtual Environment is a tool to keep the dependencies required by different projects
+in separate places, by creating virtual Python environments for them. It solves the
+"Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps
+your global site-packages directory clean and manageable.
-Each new environment automatically includes a copy of ``pip`` in the
-``Scripts`` subdirectory, so that you can setup the third-party libraries and
-tools that you want to use in that environment. Put your own code within a
-subdirectory of the environment, however you wish. When you no longer need a
-particular environment, simply copy your code out of it, and then delete the
-main directory for the environment.
+For example, you can work on a project which requires Django 1.10 while also
+maintaining a project which requires Django 1.8.
+To start using this and see more information: :ref:`Virtual Environments ` docs.
--------------------------------
-This page is a remixed version of `another guide `_,
+This page is a remixed version of `another guide `_,
which is available under the same license.
diff --git a/docs/starting/install3/linux.rst b/docs/starting/install3/linux.rst
new file mode 100644
index 000000000..b02204bc0
--- /dev/null
+++ b/docs/starting/install3/linux.rst
@@ -0,0 +1,117 @@
+.. _install3-linux:
+
+
+############################
+Installing Python 3 on Linux
+############################
+
+.. image:: /_static/photos/34435689480_2e6f358510_k_d.jpg
+
+This document describes how to install Python 3.6 or 3.8 on Ubuntu Linux machines.
+
+To see which version of Python 3 you have installed, open a command prompt and run
+
+.. code-block:: console
+
+ $ python3 --version
+
+If you are using Ubuntu 16.10 or newer, then you can easily install Python 3.6 with the following commands::
+
+ $ sudo apt-get update
+ $ sudo apt-get install python3.6
+
+If you're using another version of Ubuntu (e.g. the latest LTS release) or you want to use a more current Python, we recommend using the `deadsnakes PPA `_ to install Python 3.8::
+
+ $ sudo apt-get install software-properties-common
+ $ sudo add-apt-repository ppa:deadsnakes/ppa
+ $ sudo apt-get update
+ $ sudo apt-get install python3.8
+
+If you are using other Linux distribution, chances are you already have Python 3
+pre-installed as well. If not, use your distribution's package manager.
+For example on Fedora, you would use `dnf`:
+
+.. code-block:: console
+
+ $ sudo dnf install python3
+
+Note that if the version of the ``python3`` package is not recent enough
+for you, there may be ways of installing more recent versions as well,
+depending on you distribution. For example installing the ``python3.9`` package
+on Fedora 32 to get Python 3.9. If you are a Fedora user, you might want
+to read about `multiple Python versions available in Fedora`_.
+
+.. _multiple Python versions available in Fedora: https://developer.fedoraproject.org/tech/languages/python/multiple-pythons.html
+
+
+*********************
+Working with Python 3
+*********************
+
+At this point, you may have system Python 2.7 available as well.
+
+.. code-block:: console
+
+ $ python
+
+This might launch the Python 2 interpreter.
+
+.. code-block:: console
+
+ $ python3
+
+This will always launch the Python 3 interpreter.
+
+
+****************
+Setuptools & Pip
+****************
+
+The two most crucial third-party Python packages are `setuptools `_ and `pip `_.
+
+Once installed, you can download, install and uninstall any compliant Python software
+product with a single command. It also enables you to add this network installation
+capability to your own Python software with very little work.
+
+Python 2.7.9 and later (on the python2 series), and Python 3.4 and later include
+pip by default.
+
+To see if pip is installed, open a command prompt and run
+
+.. code-block:: console
+
+ $ command -v pip
+
+To install pip, `follow the official pip installation guide `_ - this will automatically install the latest version of setuptools.
+
+Note that on some Linux distributions including Ubuntu and Fedora the ``pip``
+command is meant for Python 2, while the ``pip3`` command is meant for Python 3.
+
+.. code-block:: console
+
+ $ command -v pip3
+
+However, when using virtual environments (described below), you don't need to
+care about that.
+
+
+*****************************
+Pipenv & Virtual Environments
+*****************************
+
+The next step is to install Pipenv, so you can install dependencies and manage virtual environments.
+
+A Virtual Environment is a tool to keep the dependencies required by different projects
+in separate places, by creating virtual Python environments for them. It solves the
+"Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps
+your global site-packages directory clean and manageable.
+
+For example, you can work on a project which requires Django 1.10 while also
+maintaining a project which requires Django 1.8.
+
+So, onward! To the :ref:`Pipenv & Virtual Environments ` docs!
+
+--------------------------------
+
+This page is a remixed version of `another guide `_,
+which is available under the same license.
diff --git a/docs/starting/install3/osx.rst b/docs/starting/install3/osx.rst
new file mode 100644
index 000000000..fa90fb93a
--- /dev/null
+++ b/docs/starting/install3/osx.rst
@@ -0,0 +1,144 @@
+:orphan: This article should not be added to a toctree for now
+
+.. _install3-osx:
+
+
+###############################
+Installing Python 3 on Mac OS X
+###############################
+
+.. image:: /_static/photos/34435689480_2e6f358510_k_d.jpg
+
+**Mac OS X comes with Python 2.7 out of the box between versions 10.8 and 12.3.**
+
+If your Mac OS X version is between the above versions,
+you do not need to install or configure anything else to use Python 2. These
+instructions document the installation of Python 3.
+
+The version of Python that ships with OS X is great for learning, but it's not
+good for development. The version shipped with OS X may be out of date from the
+`official current Python release `_,
+which is considered the stable production version.
+
+
+**************
+Doing it Right
+**************
+
+Let's install a real version of Python.
+
+Before installing Python, you'll need to install GCC. GCC can be obtained
+by downloading `Xcode `_, the smaller
+`Command Line Tools `_ (must have an
+Apple account) or the even smaller `OSX-GCC-Installer `_
+package.
+
+.. note::
+ If you already have Xcode installed, do not install OSX-GCC-Installer.
+ In combination, the software can cause issues that are difficult to
+ diagnose.
+
+.. note::
+ If you perform a fresh install of Xcode, you will also need to add the
+ commandline tools by running ``xcode-select --install`` on the terminal.
+
+While OS X comes with a large number of Unix utilities, those familiar with
+Linux systems will notice one key component missing: a package manager.
+`Homebrew `_ fills this void.
+
+To `install Homebrew `_, open :file:`Terminal` or
+your favorite OS X terminal emulator and run
+
+.. code-block:: console
+
+ $ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
+
+The script will explain what changes it will make and prompt you before the
+installation begins.
+Once you've installed Homebrew, insert the Homebrew directory at the top
+of your :envvar:`PATH` environment variable. You can do this by adding the following
+line at the bottom of your :file:`~/.profile` file
+
+.. code-block:: console
+
+ export PATH="/usr/local/opt/python/libexec/bin:$PATH"
+
+If you have OS X 10.12 (Sierra) or older use this line instead
+
+.. code-block:: console
+
+ export PATH=/usr/local/bin:/usr/local/sbin:$PATH
+
+Now, we can install Python 3:
+
+.. code-block:: console
+
+ $ brew install python
+
+This will take a minute or two.
+
+***
+Pip
+***
+
+Homebrew installs ``pip`` pointing to the Homebrew'd Python 3 for you.
+
+
+*********************
+Working with Python 3
+*********************
+
+At this point, you have the system Python 2.7 available, potentially the
+:ref:`Homebrew version of Python 2 ` installed, and the Homebrew
+version of Python 3 as well.
+
+.. code-block:: console
+
+ $ python
+
+will launch the Homebrew-installed Python 3 interpreter.
+
+.. code-block:: console
+
+ $ python2
+
+will launch the Homebrew-installed Python 2 interpreter (if any).
+
+.. code-block:: console
+
+ $ python3
+
+will launch the Homebrew-installed Python 3 interpreter.
+
+If the Homebrew version of Python 2 is installed then ``pip2`` will point to Python 2.
+If the Homebrew version of Python 3 is installed then ``pip`` will point to Python 3.
+
+The rest of the guide will assume that ``python`` references Python 3.
+
+.. code-block:: console
+
+ # Do I have a Python 3 installed?
+ $ python --version
+ Python 3.7.1 # Success!
+
+
+*****************************
+Pipenv & Virtual Environments
+*****************************
+
+The next step is to install Pipenv, so you can install dependencies and manage virtual environments.
+
+A Virtual Environment is a tool to keep the dependencies required by different projects
+in separate places, by creating virtual Python environments for them. It solves the
+"Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps
+your global site-packages directory clean and manageable.
+
+For example, you can work on a project which requires Django 1.10 while also
+maintaining a project which requires Django 1.8.
+
+So, onward! To the :ref:`Pipenv & Virtual Environments ` docs!
+
+--------------------------------
+
+This page is a remixed version of `another guide `_,
+which is available under the same license.
diff --git a/docs/starting/install3/win.rst b/docs/starting/install3/win.rst
new file mode 100644
index 000000000..db78d9240
--- /dev/null
+++ b/docs/starting/install3/win.rst
@@ -0,0 +1,58 @@
+.. _install3-windows:
+
+
+##############################
+Installing Python 3 on Windows
+##############################
+
+.. image:: /_static/photos/34435689480_2e6f358510_k_d.jpg
+
+First, follow the installation instructions for `Chocolatey `_.
+It's a community system packager manager for Windows 7+. (It's very much like Homebrew on OS X.)
+
+Once done, installing Python 3 is very simple, because Chocolatey pushes Python 3 as the default.
+
+.. code-block:: doscon
+
+ choco install python
+
+Once you've run this command, you should be able to launch Python directly from to the console.
+(Chocolatey is fantastic and automatically adds Python to your path.)
+
+
+****************
+Setuptools + Pip
+****************
+
+The two most crucial third-party Python packages are `setuptools `_ and `pip `_,
+which let you download, install and uninstall any compliant Python software
+product with a single command. It also enables you to add this network installation
+capability to your own Python software with very little work.
+
+All supported versions of Python 3 include pip, so just make sure it's up to date:
+
+.. code-block:: doscon
+
+ python -m pip install -U pip
+
+
+*****************************
+Pipenv & Virtual Environments
+*****************************
+
+The next step is to install Pipenv, so you can install dependencies and manage virtual environments.
+
+A Virtual Environment is a tool to keep the dependencies required by different projects
+in separate places, by creating virtual Python environments for them. It solves the
+"Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps
+your global site-packages directory clean and manageable.
+
+For example, you can work on a project which requires Django 2.0 while also
+maintaining a project which requires Django 1.8.
+
+So, onward! To the :ref:`Pipenv & Virtual Environments ` docs!
+
+--------------------------------
+
+This page is a remixed version of `another guide `_,
+which is available under the same license.
diff --git a/docs/starting/installation.rst b/docs/starting/installation.rst
index 669817270..5b84a3ecf 100644
--- a/docs/starting/installation.rst
+++ b/docs/starting/installation.rst
@@ -1,5 +1,11 @@
+.. _installation:
+
+
+##########################
Properly Installing Python
-==========================
+##########################
+
+.. image:: /_static/photos/36137232412_fdcb0f84eb_k_d.jpg
There's a good chance that you already have Python on your operating system.
@@ -7,16 +13,29 @@ If so, you do not need to install or configure anything else to use Python.
Having said that, I would strongly recommend that you install the tools and
libraries described in the guides below before you start building Python
applications for real-world use. In particular, you should always install
-Distribute, Pip, and Virtualenv — they make it much easier for you to use
+Setuptools, Pip, and Virtualenv — they make it much easier for you to use
other third-party Python libraries.
+.. note:: The use of **Python 3** is *highly* preferred over Python 2. Consider upgrading your applications and infrastructure if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste.
+ —*Kenneth Reitz*
+
+*******************
Installation Guides
--------------------
+*******************
+
+These guides go over the proper installation of :ref:`Python `
+for development purposes, as well as setuptools, pip and virtualenv.
+
+Python 3 Installation Guides
+////////////////////////////
-These guides go over the proper installation of :ref:`Python 2.7 `
- for development purposes, as well as distribute, pip, and virtualenv setup.
+- :ref:`Python 3 on MacOS `.
+- :ref:`Python 3 on Windows `.
+- :ref:`Python 3 on Linux `.
-- :ref:`Mac OS X `.
-- :ref:`Microsoft Windows`.
-- :ref:`Ubuntu Linux `.
+Legacy Python 2 Installation Guides
+///////////////////////////////////
+- :ref:`Python 2 on MacOS `.
+- :ref:`Python 2 on Microsoft Windows `.
+- :ref:`Python 2 on Linux `.
diff --git a/docs/starting/which-python.rst b/docs/starting/which-python.rst
index e07ad4bc1..59d42352e 100644
--- a/docs/starting/which-python.rst
+++ b/docs/starting/which-python.rst
@@ -1,57 +1,91 @@
-Picking an Interpreter
-======================
+
+
+#####################################
+Picking a Python Interpreter (3 vs 2)
+#####################################
+
+.. image:: /_static/photos/34484834733_5b80f65ab1_k_d.jpg
.. _which-python:
-Today (Python 2)
-~~~~~~~~~~~~~~~~
-If you're choosing a Python interpreter to use, I *highly* recommend you use
-Python 2.7.x, unless you have a strong reason not to.
+***************************
+The State of Python (3 & 2)
+***************************
+
+When choosing a Python interpreter, one looming question is always present:
+"Should I choose Python 2 or Python 3"? The answer is a bit more subtle than
+one might think.
+
+
+The basic gist of the state of things is as follows:
+
+1. Most production applications today use Python 3.
+2. Python 3 is ready for the production deployment of applications today.
+3. Python 2 reached the end of its life on January 1, 2020 [#pep373_eol]_.
+4. The brand name "Python" encapsulates both Python 3 and Python 2.
+
+
+***************
+Recommendations
+***************
+
+
+.. note:: The use of **Python 3** is *highly* recommended over Python 2. Consider upgrading your applications and infrastructure if you find yourself *still* using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste.
+ —*Kenneth Reitz*
-Also use Python 2.7.x if you're starting to work on a new Python module. If you
-have managed to get it working on 2.7, you can add support for older 2.x
-versions.
+I'll be blunt:
-The Future (Python 3)
-~~~~~~~~~~~~~~~~~~~~~
+- Use Python 3 for new Python applications.
+- If you're learning Python for the first time, familiarizing yourself with Python 2.7 will be very
+ useful, but not more useful than learning Python 3.
+- Learn both. They are both "Python".
+
+
+*********
+So.... 3?
+*********
+
+If you're choosing a Python interpreter to use, I
+recommend you use the newest Python 3.x, since every version brings new and
+improved standard library modules, security and bug fixes.
+
+Given such, only use Python 2 if you have a strong reason to, such as a
+pre-existing code-base, a Python 2 exclusive library, simplicity/familiarity,
+or, of course, you absolutely love and are inspired by Python 2. No harm in that.
- Python 2.x is the status quo, Python 3.x is the shiny new thing.
`Further Reading `_
-Python 3, on the other hand, differs much more greatly from Python 2, so
-writing code that works both on Python 2 and Python 3 is a very complicated
-process.
+It is possible to `write code that works on Python 2.6, 2.7, and Python 3
+`_. This
+ranges from trivial to hard depending upon the kind of software
+you are writing; if you're a beginner there are far more important things to
+worry about.
-It is still possible to `write code that works on Python 2.6, 2.7 and 3.3
-`_. Depending on
-the kind of software you are writing, this might be either tricky or extremely
-hard, and if you're a beginner there are much more important things to worry
-about.
+***************
Implementations
-~~~~~~~~~~~~~~~
+***************
-There are several popular implementations of the Python programming language on
-different back-ends.
+When people speak of *Python* they often mean not just the language but also
+the CPython implementation. *Python* is actually a specification for a language
+that can be implemented in many different ways.
CPython
-------
`CPython `_ is the reference implementation of Python,
written in C. It compiles Python code to intermediate bytecode which is then
-interpreted by a virtual machine. When people speak of *Python* they often mean
-not just the language but also this implementation. It provides the highest
+interpreted by a virtual machine. CPython provides the highest
level of compatibility with Python packages and C extension modules.
-If you are writing open-source Python code and want to reach the widest possible
-audience, targeting CPython is your best bet. If you need to use any packages
-that rely on C extensions for their functionality (eg: numpy) then CPython
-is your only choice.
+If you are writing open source Python code and want to reach the widest possible
+audience, targeting CPython is best. To use packages which rely on C extensions
+to function, CPython is your only implementation option.
-Being the reference implementation, all versions of the Python language are
-available as CPython. Python 3 is only available in a CPython implementation.
+All versions of the Python language are implemented in C because CPython is the
+reference implementation.
PyPy
----
@@ -63,40 +97,63 @@ features a just-in-time compiler and supports multiple back-ends (C, CLI, JVM).
PyPy aims for maximum compatibility with the reference CPython implementation
while improving performance.
-If you are looking to squeeze more performance out of your Python code, it's
+If you are looking to increase performance of your Python code, it's
worth giving PyPy a try. On a suite of benchmarks, it's currently `over 5 times
faster than CPython `_.
-Currently PyPy supports Python 2.7. [#pypy_ver]_
+PyPy supports Python 2.7. PyPy3 [#pypy_ver]_, released in beta, targets Python 3.
Jython
------
`Jython `_ is a Python implementation that compiles
-Python code to Java byte code that then executes on a JVM. It has the additional
-advantage of being able to import and use any Java class the same as a Python
+Python code to Java bytecode which is then executed by the JVM (Java Virtual Machine).
+Additionally, it is able to import and use any Java class like a Python
module.
If you need to interface with an existing Java codebase or have other reasons to
need to write Python code for the JVM, Jython is the best choice.
-Currently Jython supports up to Python 2.5. [#jython_ver]_
+Jython currently supports up to Python 2.7. [#jython_ver]_
IronPython
----------
`IronPython `_ is an implementation of Python for the .NET
framework. It can use both Python and .NET framework libraries, and can also
-expose Python code to other .NET languages.
+expose Python code to other languages in the .NET framework.
`Python Tools for Visual Studio `_ integrates
-IronPython directly in to the Visual Studio development environment, making it
+IronPython directly into the Visual Studio development environment, making it
an ideal choice for Windows developers.
-IronPython supports Python 2.7. [#iron_ver]_
+IronPython supports Python 2.7. [#iron_ver]_ IronPython 3 [#iron_ver3]_
+is being developed, but is not ready for use as of September 2020.
+
+PythonNet
+---------
+
+`Python for .NET `_ is a package which
+provides near seamless integration of a natively installed Python
+installation with the .NET Common Language Runtime (CLR). This is the
+inverse approach to that taken by IronPython (see above), to which it
+is more complementary than competing with.
+
+In conjunction with Mono, pythonnet enables native Python
+installations on non-Windows operating systems, such as OS X and
+Linux, to operate within the .NET framework. It can be run in
+addition to IronPython without conflict.
+
+Pythonnet is compatible with Python 2.7 and 3.5-3.8. [#pythonnet_ver1]_
+
+.. [#pypy_ver] https://pypy.org/compat.html
+
+.. [#jython_ver] https://hg.python.org/jython/file/412a8f9445f7/NEWS
+
+.. [#iron_ver] https://ironpython.net/download/
-.. [#pypy_ver] http://pypy.org/compat.html
+.. [#iron_ver3] https://github.com/IronLanguages/ironpython3
-.. [#jython_ver] http://wiki.python.org/jython/JythonFaq/GeneralInfo#Is_Jython_the_same_language_as_Python.3F
+.. [#pythonnet_ver1] https://pythonnet.github.io/
-.. [#iron_ver] http://ironpython.codeplex.com/releases/view/81726
+.. [#pep373_eol] https://www.python.org/dev/peps/pep-0373/#id2
diff --git a/docs/writing/documentation.rst b/docs/writing/documentation.rst
index 7ff54654a..27d85405a 100644
--- a/docs/writing/documentation.rst
+++ b/docs/writing/documentation.rst
@@ -1,52 +1,60 @@
+
+
+#############
Documentation
-=============
+#############
+
+.. image:: /_static/photos/35620636012_f66aa88f93_k_d.jpg
Readability is a primary focus for Python developers, in both project
and code documentation. Following some simple best practices can save
both you and others a lot of time.
+
+*********************
Project Documentation
----------------------
-
-A ``README`` file at the root directory should give general
-information to the users and the maintainers. It should be raw text or
-written in some very easy to read markup, such as
-:ref:`reStructuredText-ref` and Markdown. It should contain a few
-lines explaining the purpose of the project or the library (without
-assuming the user knows anything about the project), the url of the
-main source for the software, and some basic credit information. This
-file is the main entry point for readers of the code.
-
-An ``INSTALL`` file is less necessary with python. The installation
+*********************
+
+A :file:`README` file at the root directory should give general information
+to both users and maintainers of a project. It should be raw text or
+written in some very easy to read markup, such as :ref:`reStructuredText-ref`
+or Markdown. It should contain a few lines explaining the purpose of the
+project or library (without assuming the user knows anything about the
+project), the URL of the main source for the software, and some basic credit
+information. This file is the main entry point for readers of the code.
+
+An :file:`INSTALL` file is less necessary with Python. The installation
instructions are often reduced to one command, such as ``pip install
-module`` or ``python setup.py install`` and added to the ``README``
+module`` or ``python setup.py install``, and added to the :file:`README`
file.
-A ``LICENSE`` file should *always* be present and specify the license under which the
-software is made available to the public.
+A :file:`LICENSE` file should *always* be present and specify the license
+under which the software is made available to the public.
-A ``TODO`` file or a ``TODO`` section in ``README`` should list the
+A :file:`TODO` file or a ``TODO`` section in :file:`README` should list the
planned development for the code.
-A ``CHANGELOG`` file or section in ``README`` should compile a short
+A :file:`CHANGELOG` file or section in :file:`README` should compile a short
overview of the changes in the code base for the latest versions.
+
+*******************
Project Publication
--------------------
+*******************
Depending on the project, your documentation might include some or all
of the following components:
-- An *introduction* should show a very short overview of what can be
+- An *introduction* should give a very short overview of what can be
done with the product, using one or two extremely simplified use
cases. This is the thirty-second pitch for your project.
-- A *tutorial* should show some primary use cases in more detail. The reader will
- follow a step-by-step procedure to set-up a working prototype.
+- A *tutorial* should show some primary use cases in more detail. The reader
+ will follow a step-by-step procedure to set-up a working prototype.
- An *API reference* is typically generated from the code (see
- :ref:`docstrings `). It will list all publicly available interfaces,
- parameters, and return values.
+ :ref:`docstrings `). It will list all publicly available
+ interfaces, parameters, and return values.
- *Developer documentation* is intended for potential contributors. This can
include code convention and general design strategy of the project.
@@ -56,7 +64,7 @@ of the following components:
Sphinx
~~~~~~
-Sphinx_ is far and away the most popular python documentation
+Sphinx_ is far and away the most popular Python documentation
tool. **Use it.** It converts :ref:`restructuredtext-ref` markup language
into a range of output formats including HTML, LaTeX (for printable
PDF versions), manual pages, and plain text.
@@ -66,13 +74,18 @@ There is also **great**, **free** hosting for your Sphinx_ docs:
your source repository so that rebuilding your documentation will
happen automatically.
+When run, Sphinx_ will import your code and using Python's introspection
+features it will extract all function, method, and class signatures. It will
+also extract the accompanying docstrings, and compile it all into well
+structured and easily readable documentation for your project.
+
.. note::
Sphinx is famous for its API generation, but it also works well
for general project documentation. This Guide is built with
Sphinx_ and is hosted on `Read The Docs`_
-.. _Sphinx: http://sphinx.pocoo.org
+.. _Sphinx: https://www.sphinx-doc.org
.. _Read The Docs: http://readthedocs.org
.. _restructuredtext-ref:
@@ -81,32 +94,36 @@ reStructuredText
~~~~~~~~~~~~~~~~
Most Python documentation is written with reStructuredText_. It's like
-Markdown with all the optional extensions built in.
+Markdown, but with all the optional extensions built in.
The `reStructuredText Primer`_ and the `reStructuredText Quick
Reference`_ should help you familiarize yourself with its syntax.
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
-.. _reStructuredText Primer: http://sphinx.pocoo.org/rest.html
+.. _reStructuredText Primer: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
.. _reStructuredText Quick Reference: http://docutils.sourceforge.net/docs/user/rst/quickref.html
+*************************
Code Documentation Advice
--------------------------
+*************************
-Comments clarify code and begin with a hash (``#``).
+Comments clarify the code and they are added with purpose of making the
+code easier to understand. In Python, comments begin with a hash
+(number sign) (``#``).
.. _docstring-ref:
-In Python, *docstrings* describe modules, classes, and functions: ::
+In Python, *docstrings* describe modules, classes, and functions:
+
+.. code-block:: python
def square_and_rooter(x):
- """Returns the square root of self times self."""
+ """Return the square root of self times self."""
...
-In general, follow the `comment section of PEP 0008`_ (the "Python Style Guide").
-
-.. _comment section of PEP 0008: http://www.python.org/dev/peps/pep-0008/#comments
+In general, follow the comment section of :pep:`8#comments` (the "Python Style
+Guide"). More information about docstrings can be found at :pep:`0257#specification` (The Docstring Conventions Guide).
Commenting Sections of Code
~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -125,25 +142,125 @@ Some tools use docstrings to embed more-than-documentation behavior,
such as unit test logic. Those can be nice, but you won't ever go
wrong with vanilla "here's what this does."
+Tools like Sphinx_ will parse your docstrings as reStructuredText and render it
+correctly as HTML. This makes it very easy to embed snippets of example code in
+a project's documentation.
+
+Additionally, Doctest_ will read all embedded docstrings that look like input
+from the Python commandline (prefixed with ">>>") and run them, checking to see
+if the output of the command matches the text on the following line. This
+allows developers to embed real examples and usage of functions alongside
+their source code. As a side effect, it also ensures that their code is
+tested and works.
+
+::
+
+ def my_function(a, b):
+ """
+ >>> my_function(2, 3)
+ 6
+ >>> my_function('a', 3)
+ 'aaa'
+ """
+ return a * b
+
+.. _Doctest: https://docs.python.org/3/library/doctest.html
+
Docstrings versus Block comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
These aren't interchangeable. For a function or class, the leading
comment block is a programmer's note. The docstring describes the
-operation of the function or class: ::
+*operation* of the function or class:
+
+.. code-block:: python
# This function slows down program execution for some reason.
def square_and_rooter(x):
"""Returns the square root of self times self."""
...
-.. seealso:: Further reading on docstrings: `PEP 0257`_
+Unlike block comments, docstrings are built into the Python language itself.
+This means you can use all of Python's powerful introspection capabilities to
+access docstrings at runtime, compared with comments which are optimized out.
+Docstrings are accessible from both the `__doc__` dunder attribute for almost
+every Python object, as well as with the built in `help()` function.
+
+While block comments are usually used to explain *what* a section of code is
+doing, or the specifics of an algorithm, docstrings are more intended towards
+explaining other users of your code (or you in 6 months time) *how* a
+particular function can be used and the general purpose of a function, class,
+or module.
-.. _PEP 0257: http://www.python.org/dev/peps/pep-0257/
+Writing Docstrings
+~~~~~~~~~~~~~~~~~~
+Depending on the complexity of the function, method, or class being written, a
+one-line docstring may be perfectly appropriate. These are generally used for
+really obvious cases, such as::
+ def add(a, b):
+ """Add two numbers and return the result."""
+ return a + b
+
+The docstring should describe the function in a way that is easy to understand.
+For simple cases like trivial functions and classes, simply embedding the
+function's signature (i.e. `add(a, b) -> result`) in the docstring is
+unnecessary. This is because with Python's `inspect` module, it is already
+quite easy to find this information if needed, and it is also readily available
+by reading the source code.
+
+In larger or more complex projects however, it is often a good idea to give
+more information about a function, what it does, any exceptions it may raise,
+what it returns, or relevant details about the parameters.
+
+For more detailed documentation of code a popular style used, is the one used by the
+NumPy project, often called `NumPy style`_ docstrings. While it can take up more
+lines than the previous example, it allows the developer to include a lot
+more information about a method, function, or class. ::
+
+ def random_number_generator(arg1, arg2):
+ """
+ Summary line.
+
+ Extended description of function.
+
+ Parameters
+ ----------
+ arg1 : int
+ Description of arg1
+ arg2 : str
+ Description of arg2
+
+ Returns
+ -------
+ int
+ Description of return value
+
+ """
+ return 42
+
+The `sphinx.ext.napoleon`_ plugin allows Sphinx to parse this style of
+docstrings, making it easy to incorporate NumPy style docstrings into your
+project.
+
+At the end of the day, it doesn't really matter what style is used for writing
+docstrings; their purpose is to serve as documentation for anyone who may need
+to read or make changes to your code. As long as it is correct, understandable,
+and gets the relevant points across then it has done the job it was designed to
+do.
+
+
+For further reading on docstrings, feel free to consult :pep:`257`
+
+.. _thomas-cokelaer.info: http://thomas-cokelaer.info/tutorials/sphinx/docstring_python.html
+.. _sphinx.ext.napoleon: https://sphinxcontrib-napoleon.readthedocs.io/
+.. _`NumPy style`: http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html
+
+
+***********
Other Tools
------------
+***********
You might see these in the wild. Use :ref:`sphinx-ref`.
@@ -152,11 +269,12 @@ Pycco_
and is a port of the node.js Docco_. It makes code into a
side-by-side HTML code and documentation.
-.. _Pycco: http://fitzgen.github.com/pycco
+.. _Pycco: https://pycco-docs.github.io/pycco/
.. _Docco: http://jashkenas.github.com/docco
Ronn_
- Ronn builds unix manuals. It converts human readable textfiles to roff for terminal display, and also to HTML for the web.
+ Ronn builds Unix manuals. It converts human readable textfiles to roff
+ for terminal display, and also to HTML for the web.
.. _Ronn: https://github.com/rtomayko/ronn
@@ -164,3 +282,9 @@ Epydoc_
Epydoc is discontinued. Use :ref:`sphinx-ref` instead.
.. _Epydoc: http://epydoc.sourceforge.net
+
+MkDocs_
+ MkDocs is a fast and simple static site generator that's geared towards
+ building project documentation with Markdown.
+
+.. _MkDocs: http://www.mkdocs.org/
diff --git a/docs/writing/gotchas.rst b/docs/writing/gotchas.rst
index 45c0e5605..f717858ca 100644
--- a/docs/writing/gotchas.rst
+++ b/docs/writing/gotchas.rst
@@ -1,20 +1,28 @@
+
+
+##############
Common Gotchas
-==============
+##############
+
+.. image:: /_static/photos/34435688380_b5a740762b_k_d.jpg
For the most part, Python aims to be a clean and consistent language that
-avoids surprises, but there are a few cases where newcomers to the language
-often get tripped up.
+avoids surprises. However, there are a few cases that can be confusing for
+newcomers.
-Some of these are intentional but potentially surprising. Some could arguably
-be considered language warts. In general though, what follows is a collection
-of potentially tricky behavior that might seem strange at first glance, but is
-generally sensible once you're aware of the underlying cause for the surprise.
+Some of these cases are intentional but can be potentially surprising. Some
+could arguably be considered language warts. In general, what follows
+is a collection of potentially tricky behavior that might seem strange at first
+glance, but are generally sensible, once you're aware of the underlying cause for
+the surprise.
.. _default_args:
+
+*************************
Mutable Default Arguments
--------------------------
+*************************
Seemingly the *most* common surprise new Python programmers encounter is
Python's treatment of mutable default arguments in function definitions.
@@ -22,7 +30,7 @@ Python's treatment of mutable default arguments in function definitions.
What You Wrote
~~~~~~~~~~~~~~
-.. testcode::
+.. code-block:: python
def append_to(element, to=[]):
to.append(element)
@@ -31,13 +39,13 @@ What You Wrote
What You Might Have Expected to Happen
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-.. testcode::
+.. code-block:: python
my_list = append_to(12)
- print my_list
+ print(my_list)
my_other_list = append_to(42)
- print my_other_list
+ print(my_other_list)
A new list is created each time the function is called if a second argument
isn't provided, so that the output is::
@@ -45,8 +53,8 @@ isn't provided, so that the output is::
[12]
[42]
-What Does Happen
-~~~~~~~~~~~~~~~~
+What Actually Happens
+~~~~~~~~~~~~~~~~~~~~~
.. testoutput::
@@ -65,7 +73,7 @@ What You Should Do Instead
~~~~~~~~~~~~~~~~~~~~~~~~~~
Create a new object each time the function is called, by using a default arg to
-signal that no argument was provided (``None`` is often a good choice).
+signal that no argument was provided (:py:data:`None` is often a good choice).
.. code-block:: python
@@ -75,21 +83,24 @@ signal that no argument was provided (``None`` is often a good choice).
to.append(element)
return to
+Do not forget, you are passing a *list* object as the second argument.
When the Gotcha Isn't a Gotcha
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Sometimes you specifically can "exploit" (read: use as intended) this behavior
+Sometimes you can specifically "exploit" (read: use as intended) this behavior
to maintain state between calls of a function. This is often done when writing
a caching function.
+****************************
Late Binding Closures
----------------------
+****************************
Another common source of confusion is the way Python binds its variables in
closures (or in the surrounding global scope).
+
What You Wrote
~~~~~~~~~~~~~~
@@ -104,7 +115,7 @@ What You Might Have Expected to Happen
.. testcode::
for multiplier in create_multipliers():
- print multiplier(2)
+ print(multiplier(2))
A list containing five functions that each have their own closed-over ``i``
variable that multiplies their argument, producing::
@@ -115,8 +126,8 @@ variable that multiplies their argument, producing::
6
8
-What Does Happen
-~~~~~~~~~~~~~~~~
+What Actually Happens
+~~~~~~~~~~~~~~~~~~~~~
.. testoutput::
@@ -126,7 +137,7 @@ What Does Happen
8
8
-Five functions are created, but all of them just multiply ``x`` by 4.
+Five functions are created; instead all of them just multiply ``x`` by 4.
Python's closures are *late binding*.
This means that the values of variables used in closures are looked
@@ -137,9 +148,10 @@ is looked up in the surrounding scope at call time. By then, the loop has
completed and ``i`` is left with its final value of 4.
What's particularly nasty about this gotcha is the seemingly prevalent
-misinformation that this has something to do with ``lambda``\s in Python.
-Functions created with a ``lambda`` expression are in no way special, and in
-fact the same exact behavior is exhibited by just using an ordinary ``def``:
+misinformation that this has something to do with :ref:`lambdas `
+in Python. Functions created with a ``lambda`` expression are in no way special,
+and in fact the same exact behavior is exhibited by just using an ordinary
+``def``:
.. code-block:: python
@@ -157,7 +169,7 @@ What You Should Do Instead
~~~~~~~~~~~~~~~~~~~~~~~~~~
The most general solution is arguably a bit of a hack. Due to Python's
-afformentioned behavior concerning evaluating default arguments to functions
+aforementioned behavior concerning evaluating default arguments to functions
(see :ref:`default_args`), you can create a closure that binds immediately to
its arguments by using a default arg like so:
@@ -166,9 +178,90 @@ its arguments by using a default arg like so:
def create_multipliers():
return [lambda x, i=i : i * x for i in range(5)]
+Alternatively, you can use the functools.partial function:
+
+.. code-block:: python
+
+ from functools import partial
+ from operator import mul
+
+ def create_multipliers():
+ return [partial(mul, i) for i in range(5)]
+
When the Gotcha Isn't a Gotcha
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Sometimes you want your closures to behave this way. Late binding is good in lots of
-situations. Looping to create unique functions is unfortunately a case where
-they can cause hiccups.
+Sometimes you want your closures to behave this way. Late binding is good in
+lots of situations. Looping to create unique functions is unfortunately a case
+where they can cause hiccups.
+
+
+*********************************
+Bytecode (.pyc) Files Everywhere!
+*********************************
+
+By default, when executing Python code from files, the Python interpreter
+will automatically write a bytecode version of that file to disk, e.g.
+``module.pyc``.
+
+These ``.pyc`` files should not be checked into your source code repositories.
+
+Theoretically, this behavior is on by default for performance reasons.
+Without these bytecode files, Python would re-generate the bytecode
+every time the file is loaded.
+
+
+Disabling Bytecode (.pyc) Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Luckily, the process of generating the bytecode is extremely fast, and isn't
+something you need to worry about while developing your code.
+
+Those files are annoying, so let's get rid of them!
+
+::
+
+ $ export PYTHONDONTWRITEBYTECODE=1
+
+With the ``$PYTHONDONTWRITEBYTECODE`` environment variable set, Python will
+no longer write these files to disk, and your development environment will
+remain nice and clean.
+
+I recommend setting this environment variable in your ``~/.profile``.
+
+Removing Bytecode (.pyc) Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Here's nice trick for removing all of these files, if they already exist::
+
+ $ find . -type f -name "*.py[co]" -delete -or -type d -name "__pycache__" -delete
+
+Run that from the root directory of your project, and all ``.pyc`` files
+will suddenly vanish. Much better.
+
+.. _version_control_ignores:
+
+Version Control Ignores
+~~~~~~~~~~~~~~~~~~~~~~~
+
+If you still need the ``.pyc`` files for performance reasons, you can always add them
+to the ignore files of your version control repositories. Popular version control
+systems have the ability to use wildcards defined in a file to apply special
+rules.
+
+An ignore file will make sure the matching files don't get checked into the repository.
+Git_ uses ``.gitignore`` while Mercurial_ uses ``.hgignore``.
+
+.. _Git: https://git-scm.com/
+.. _Mercurial: https://www.mercurial-scm.org/
+
+At the minimum your ignore files should look like this.
+
+::
+
+ syntax:glob # This line is not needed for .gitignore files.
+ *.py[cod] # Will match .pyc, .pyo and .pyd files.
+ __pycache__/ # Exclude the whole folder
+
+You may wish to include more files and directories depending on your needs.
+The next time you commit to the repository, these files will not be included.
diff --git a/docs/writing/license.rst b/docs/writing/license.rst
index 9e759baf6..820cc9a05 100644
--- a/docs/writing/license.rst
+++ b/docs/writing/license.rst
@@ -1,11 +1,15 @@
+
+
+##################
Choosing a License
-==================
+##################
+
+.. image:: /_static/photos/33907149294_82d7535a6c_k_d.jpg
-Your source publication *needs* a license. In the US, if no license is
-specified, users have no legal right to download, modify, or
-distribute. Furthermore, people can't contribute to your code unless
-you tell them what rules to play by. It's complicated, so here are
-some pointers:
+Your source publication *needs* a license. In the US, unless a license is
+specified, users have no legal right to download, modify, or distribute the product.
+Furthermore, people can't contribute to your code unless you tell them what
+rules to play by. Choosing a license is complicated, so here are some pointers:
Open source. There are plenty of `open source licenses
`_ available to choose
@@ -14,24 +18,24 @@ from.
In general, these licenses tend to fall into one of two categories:
1. licenses that focus more on the user's freedom to do with the
- software as they please (these are the more-permissive open
- source licenses such as the MIT, BSD, & Apache).
+ software as they please (these are the more permissive open
+ source licenses such as the MIT, BSD, and Apache)
2. licenses that focus more on making sure that the code itself —
including any changes made to it and distributed along with it —
- always remains free (these are the less-permissive free software
- licenses, for example, the GPL and LGPL).
+ always remains free (these are the less permissive free software
+ licenses such as the GPL and LGPL)
-The latter are less-permissive in the sense that they don't permit
+The latter are less permissive in the sense that they don't permit
someone to add code to the software and distribute it without also
including the source code for their changes.
-To help you choose one for your project, there's a `license chooser `_,
+To help you choose one for your project, there's a `license chooser `_;
**use it**.
-**More-Permissive**
+**More Permissive**
-- PSFL (Python Software Foundation License) -- for contributing to python itself
+- PSFL (Python Software Foundation License) -- for contributing to Python itself
- MIT / BSD / ISC
+ MIT (X11)
@@ -40,7 +44,7 @@ To help you choose one for your project, there's a `license chooser `_.
diff --git a/docs/writing/logging.rst b/docs/writing/logging.rst
new file mode 100644
index 000000000..b6110f7f7
--- /dev/null
+++ b/docs/writing/logging.rst
@@ -0,0 +1,202 @@
+
+
+#######
+Logging
+#######
+
+.. image:: /_static/photos/35254379756_c9fe23f843_k_d.jpg
+
+The :mod:`logging` module has been a part of Python's Standard Library since
+version 2.3. It is succinctly described in :pep:`282`. The documentation
+is notoriously hard to read, except for the `basic logging tutorial`_.
+
+As an alternative, `loguru `_ provides an approach for logging, nearly as simple as using a simple ``print`` statement.
+
+Logging serves two purposes:
+
+- **Diagnostic logging** records events related to the application's
+ operation. If a user calls in to report an error, for example, the logs
+ can be searched for context.
+- **Audit logging** records events for business analysis. A user's
+ transactions can be extracted and combined with other user details for
+ reports or to optimize a business goal.
+
+
+*************
+... or Print?
+*************
+
+The only time that ``print`` is a better option than logging is when
+the goal is to display a help statement for a command line application.
+Other reasons why logging is better than ``print``:
+
+- The `log record`_, which is created with every logging event, contains
+ readily available diagnostic information such as the file name, full path,
+ function, and line number of the logging event.
+- Events logged in included modules are automatically accessible via the root
+ logger to your application's logging stream, unless you filter them out.
+- Logging can be selectively silenced by using the method
+ :meth:`logging.Logger.setLevel` or disabled by setting the attribute
+ :attr:`logging.Logger.disabled` to ``True``.
+
+
+********************
+Logging in a Library
+********************
+
+Notes for `configuring logging for a library`_ are in the
+`logging tutorial`_. Because the *user*, not the library, should
+dictate what happens when a logging event occurs, one admonition bears
+repeating:
+
+.. note::
+ It is strongly advised that you do not add any handlers other than
+ NullHandler to your library’s loggers.
+
+
+Best practice when instantiating loggers in a library is to only create them
+using the ``__name__`` global variable: the :mod:`logging` module creates a
+hierarchy of loggers using dot notation, so using ``__name__`` ensures
+no name collisions.
+
+Here is an example of the best practice from the `requests source`_ -- place
+this in your ``__init__.py``:
+
+.. code-block:: python
+
+ import logging
+ logging.getLogger(__name__).addHandler(logging.NullHandler())
+
+
+*************************
+Logging in an Application
+*************************
+
+The `twelve factor app `_, an authoritative reference
+for good practice in application development, contains a section on
+`logging best practice `_. It emphatically
+advocates for treating log events as an event stream, and for
+sending that event stream to standard output to be handled by the
+application environment.
+
+
+There are at least three ways to configure a logger:
+
+- Using an INI-formatted file:
+ - **Pro**: possible to update configuration while running, using the
+ function :func:`logging.config.listen` to listen on a socket.
+ - **Con**: less control (e.g. custom subclassed filters or loggers)
+ than possible when configuring a logger in code.
+- Using a dictionary or a JSON-formatted file:
+ - **Pro**: in addition to updating while running, it is possible to load
+ from a file using the :mod:`json` module, in the standard library since
+ Python 2.6.
+ - **Con**: less control than when configuring a logger in code.
+- Using code:
+ - **Pro**: complete control over the configuration.
+ - **Con**: modifications require a change to the source code.
+
+
+Example Configuration via an INI File
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Let us say that the file is named ``logging_config.ini``.
+More details for the file format are in the `logging configuration`_
+section of the `logging tutorial`_.
+
+.. code-block:: ini
+
+ [loggers]
+ keys=root
+
+ [handlers]
+ keys=stream_handler
+
+ [formatters]
+ keys=formatter
+
+ [logger_root]
+ level=DEBUG
+ handlers=stream_handler
+
+ [handler_stream_handler]
+ class=StreamHandler
+ level=DEBUG
+ formatter=formatter
+ args=(sys.stderr,)
+
+ [formatter_formatter]
+ format=%(asctime)s %(name)-12s %(levelname)-8s %(message)s
+
+
+Then use :meth:`logging.config.fileConfig` in the code:
+
+.. code-block:: python
+
+ import logging
+ from logging.config import fileConfig
+
+ fileConfig('logging_config.ini')
+ logger = logging.getLogger()
+ logger.debug('often makes a very good meal of %s', 'visiting tourists')
+
+
+Example Configuration via a Dictionary
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As of Python 2.7, you can use a dictionary with configuration details.
+:pep:`391` contains a list of the mandatory and optional elements in
+the configuration dictionary.
+
+.. code-block:: python
+
+ import logging
+ from logging.config import dictConfig
+
+ logging_config = dict(
+ version = 1,
+ formatters = {
+ 'f': {'format':
+ '%(asctime)s %(name)-12s %(levelname)-8s %(message)s'}
+ },
+ handlers = {
+ 'h': {'class': 'logging.StreamHandler',
+ 'formatter': 'f',
+ 'level': logging.DEBUG}
+ },
+ root = {
+ 'handlers': ['h'],
+ 'level': logging.DEBUG,
+ },
+ )
+
+ dictConfig(logging_config)
+
+ logger = logging.getLogger()
+ logger.debug('often makes a very good meal of %s', 'visiting tourists')
+
+
+Example Configuration Directly in Code
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: python
+
+ import logging
+
+ logger = logging.getLogger()
+ handler = logging.StreamHandler()
+ formatter = logging.Formatter(
+ '%(asctime)s %(name)-12s %(levelname)-8s %(message)s')
+ handler.setFormatter(formatter)
+ logger.addHandler(handler)
+ logger.setLevel(logging.DEBUG)
+
+ logger.debug('often makes a very good meal of %s', 'visiting tourists')
+
+
+.. _basic logging tutorial: http://docs.python.org/3/howto/logging.html#logging-basic-tutorial
+.. _logging configuration: https://docs.python.org/3/howto/logging.html#configuring-logging
+.. _logging tutorial: http://docs.python.org/3/howto/logging.html
+.. _configuring logging for a library: https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library
+.. _log record: https://docs.python.org/3/library/logging.html#logrecord-attributes
+.. _requests source: https://github.com/kennethreitz/requests
diff --git a/docs/writing/reading.rst b/docs/writing/reading.rst
index e53b30c87..1e03a4def 100644
--- a/docs/writing/reading.rst
+++ b/docs/writing/reading.rst
@@ -1,18 +1,20 @@
+
+
+##################
Reading Great Code
-==================
+##################
-One of the core tenants behind the design of Python is creating
-readable code. The motivation behind this design is simple: The number
-one thing that Python programmers do is read code.
+.. image:: /_static/photos/34689452831_93d7fd0571_k_d.jpg
One of the secrets of becoming a great Python programmer is to read,
understand, and comprehend excellent code.
-Excellent code typically follows the guidelines outlined in :ref:`code_style`,
-and does its best to express a clear and concise intent to the reader.
+Excellent code typically follows the guidelines outlined in
+:ref:`code_style`, and does its best to express a clear and concise
+intent to the reader.
-Included below is a list of recommended Python projects for reading. Each of
-these projects are paragons of excellent Python code.
+Included below is a list of recommended Python projects for
+reading. Each one of these projects is a paragon of Python coding.
- `Howdoi `_
Howdoi is a code search tool, written in Python.
@@ -22,23 +24,29 @@ these projects are paragons of excellent Python code.
It's intended for getting started very quickly and was developed with
best intentions in mind.
+- `Diamond `_
+ Diamond is a Python daemon that collects metrics
+ and publishes them to Graphite or other backends.
+ It is capable of collecting CPU, memory, network, I/O, load, and disk metrics.
+ Additionally, it features an API for implementing custom collectors
+ for gathering metrics from almost any source.
+
- `Werkzeug `_
- Werkzeug started as simple collection of various utilities for WSGI
+ Werkzeug started as a simple collection of various utilities for WSGI
applications and has become one of the most advanced WSGI utility modules.
It includes a powerful debugger, full-featured request and response objects,
HTTP utilities to handle entity tags, cache control headers, HTTP dates,
- cookie handling, file uploads, a powerful URL routing system and a bunch
+ cookie handling, file uploads, a powerful URL routing system, and a bunch
of community-contributed addon modules.
- `Requests `_
Requests is an Apache2 Licensed HTTP library, written in Python,
for human beings.
-- `Tablib `_
+- `Tablib `_
Tablib is a format-agnostic tabular dataset library, written in Python.
-.. todo:: Embed and explain YouTube video showing python code reading: http://www.youtube.com/watch?v=Jc8M9-LoEuo This may require installing a Sphinx plugin. https://bitbucket.org/birkenfeld/sphinx-contrib/src/a09f29fc16970f34350ca36ac7f229e00b1b1674/youtube?at=default
.. todo:: Include code examples of exemplary code from each of the projects listed. Explain why it is excellent code. Use complex examples.
-.. todo:: Explain techniques to rapidly identify data structures, algorithms and determine what the code is doing.
+.. todo:: Explain techniques to rapidly identify data structures and algorithms and determine what the code is doing.
diff --git a/docs/writing/structure.rst b/docs/writing/structure.rst
index 9e56ac16f..5d9645acd 100644
--- a/docs/writing/structure.rst
+++ b/docs/writing/structure.rst
@@ -1,5 +1,10 @@
+
+
+########################
Structuring Your Project
-========================
+########################
+
+.. image:: /_static/photos/33907151224_0574e7dfc2_k_d.jpg
By "structure" we mean the decisions you make concerning
how your project best meets its objective. We need to consider how to
@@ -13,14 +18,306 @@ the project? What features and functions can be grouped together and
isolated? By answering questions like these you can begin to plan, in
a broad sense, what your finished product will look like.
-In this section we take a closer look at Python's module and import
-systems as they are the central element to enforcing structure in your
+In this section, we take a closer look at Python's modules and import
+systems as they are the central elements to enforcing structure in your
project. We then discuss various perspectives on how to build code which
can be extended and tested reliably.
-Structure is Key
-----------------
+***************************
+Structure of the Repository
+***************************
+
+It's Important.
+:::::::::::::::
+
+Just as Code Style, API Design, and Automation are essential for a
+healthy development cycle. Repository structure is a crucial part of
+your project's
+`architecture `__.
+
+When a potential user or contributor lands on your repository's page,
+they see a few things:
+
+- Project Name
+- Project Description
+- Bunch O' Files
+
+Only when they scroll below the fold will the user see your project's
+README.
+
+If your repo is a massive dump of files or a nested mess of directories,
+they might look elsewhere before even reading your beautiful
+documentation.
+
+ Dress for the job you want, not the job you have.
+
+Of course, first impressions aren't everything. You and your colleagues
+will spend countless hours working with this repository, eventually
+becoming intimately familiar with every nook and cranny. The layout is important.
+
+Sample Repository
+:::::::::::::::::
+
+**tl;dr**: This is what `Kenneth Reitz recommended in 2013 `__.
+
+This repository is `available on
+GitHub `__.
+
+::
+
+ README.rst
+ LICENSE
+ setup.py
+ requirements.txt
+ sample/__init__.py
+ sample/core.py
+ sample/helpers.py
+ docs/conf.py
+ docs/index.rst
+ tests/test_basic.py
+ tests/test_advanced.py
+
+Let's get into some specifics.
+
+The Actual Module
+:::::::::::::::::
+
+.. csv-table::
+ :widths: 20, 40
+
+ "Location", "``./sample/`` or ``./sample.py``"
+ "Purpose", "The code of interest"
+
+
+Your module package is the core focus of the repository. It should not
+be tucked away:
+
+::
+
+ ./sample/
+
+If your module consists of only a single file, you can place it directly
+in the root of your repository:
+
+::
+
+ ./sample.py
+
+Your library does not belong in an ambiguous src or python subdirectory.
+
+License
+:::::::
+
+
+.. csv-table::
+ :widths: 20, 40
+
+ "Location", "``./LICENSE``"
+ "Purpose", "Lawyering up."
+
+
+This is arguably the most important part of your repository, aside from
+the source code itself. The full license text and copyright claims
+should exist in this file.
+
+If you aren't sure which license you should use for your project, check
+out `choosealicense.com `_.
+
+Of course, you are also free to publish code without a license, but this
+would prevent many people from potentially using or contributing to your code.
+
+Setup.py
+::::::::
+
+.. csv-table::
+ :widths: 20, 40
+
+ "Location", "``./setup.py``"
+ "Purpose", "Package and distribution management."
+
+
+If your module package is at the root of your repository, this should
+obviously be at the root as well.
+
+Requirements File
+:::::::::::::::::
+
+.. csv-table::
+ :widths: 20, 40
+
+ "Location", "``./requirements.txt``"
+ "Purpose", "Development dependencies."
+
+
+A `pip requirements
+file `__
+should be placed at the root of the repository. It should specify the
+dependencies required to contribute to the project: testing, building,
+and generating documentation.
+
+If your project has no development dependencies, or if you prefer
+setting up a development environment via ``setup.py``, this file may be
+unnecessary.
+
+Documentation
+:::::::::::::
+
+
+.. csv-table::
+ :widths: 20, 40
+
+ "Location", "``./docs/``"
+ "Purpose", "Package reference documentation."
+
+There is little reason for this to exist elsewhere.
+
+Test Suite
+::::::::::
+
+
+*For advice on writing your tests, see* :doc:`/writing/tests`.
+
+.. csv-table::
+ :widths: 20, 40
+
+ "Location", "``./test_sample.py`` or ``./tests``"
+ "Purpose", "Package integration and unit tests."
+
+Starting out, a small test suite will often exist in a single file:
+
+::
+
+ ./test_sample.py
+
+Once a test suite grows, you can move your tests to a directory, like
+so:
+
+::
+
+ tests/test_basic.py
+ tests/test_advanced.py
+
+Obviously, these test modules must import your packaged module to test
+it. You can do this a few ways:
+
+- Expect the package to be installed in site-packages.
+- Use a simple (but *explicit*) path modification to resolve the
+ package properly.
+
+I highly recommend the latter. Requiring a developer to run
+``setup.py develop`` to test an actively changing
+codebase also requires them to have an isolated environment setup for
+each instance of the codebase.
+
+To give the individual tests import context, create a ``tests/context.py``
+file:
+
+::
+
+ import os
+ import sys
+ sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
+
+ import sample
+
+Then, within the individual test modules, import the module like so:
+
+::
+
+ from .context import sample
+
+This will always work as expected, regardless of installation method.
+
+Some people will assert that you should distribute your tests within
+your module itself -- I disagree. It often increases complexity for your
+users; many test suites often require additional dependencies and
+runtime contexts.
+
+Makefile
+::::::::
+
+
+.. csv-table::
+ :widths: 20, 40
+
+ "Location", "``./Makefile``"
+ "Purpose", "Generic management tasks."
+
+
+If you look at most of my projects or any Pocoo project, you'll notice a
+Makefile lying around. Why? These projects aren't written in C... In
+short, make is an incredibly useful tool for defining generic tasks for
+your project.
+
+**Sample Makefile:**
+
+::
+
+ init:
+ pip install -r requirements.txt
+
+ test:
+ py.test tests
+
+ .PHONY: init test
+
+Other generic management scripts (e.g. ``manage.py``
+or ``fabfile.py``) belong at the root of the repository as well.
+
+Regarding Django Applications
+:::::::::::::::::::::::::::::
+
+I've noticed a new trend in Django applications since the release of
+Django 1.4. Many developers are structuring their repositories poorly
+due to the new bundled application templates.
+
+How? Well, they go to their bare and fresh repository and run the
+following, as they always have:
+
+::
+
+ $ django-admin.py startproject samplesite
+
+The resulting repository structure looks like this:
+
+::
+
+ README.rst
+ samplesite/manage.py
+ samplesite/samplesite/settings.py
+ samplesite/samplesite/wsgi.py
+ samplesite/samplesite/sampleapp/models.py
+
+Don't do this.
+
+Repetitive paths are confusing for both your tools and your developers.
+Unnecessary nesting doesn't help anybody (unless they're nostalgic for
+monolithic SVN repos).
+
+Let's do it properly:
+
+::
+
+ $ django-admin.py startproject samplesite .
+
+Note the "``.``".
+
+The resulting structure:
+
+::
+
+ README.rst
+ manage.py
+ samplesite/settings.py
+ samplesite/wsgi.py
+ samplesite/sampleapp/models.py
+
+
+
+************************
+Structure of Code is Key
+************************
Thanks to the way imports and modules are handled in Python, it is
relatively easy to structure a Python project. Easy, here, means
@@ -33,46 +330,47 @@ Easy structuring of a project means it is also easy
to do it poorly. Some signs of a poorly structured project
include:
-- Multiple and messy circular dependencies: if your classes
- Table and Chair in furn.py need to import Carpenter from workers.py
- to answer a question such as table.isdoneby(),
- and if conversely the class Carpenter needs to import Table and Chair,
- to answer the question carpenter.whatdo(), then you
+- Multiple and messy circular dependencies: If the classes
+ Table and Chair in :file:`furn.py` need to import Carpenter from
+ :file:`workers.py` to answer a question such as ``table.isdoneby()``,
+ and if conversely the class Carpenter needs to import Table and Chair
+ to answer the question ``carpenter.whatdo()``, then you
have a circular dependency. In this case you will have to resort to
- fragile hacks such as using import statements inside
+ fragile hacks such as using import statements inside your
methods or functions.
-- Hidden coupling: each and every change in Table's implementation
+- Hidden coupling: Each and every change in Table's implementation
breaks 20 tests in unrelated test cases because it breaks Carpenter's code,
- which requires very careful surgery to adapt the change. This means
+ which requires very careful surgery to adapt to the change. This means
you have too many assumptions about Table in Carpenter's code or the
reverse.
-- Heavy usage of global state or context: instead of explicitly
+- Heavy usage of global state or context: Instead of explicitly
passing ``(height, width, type, wood)`` to each other, Table
and Carpenter rely on global variables that can be modified
and are modified on the fly by different agents. You need to
- scrutinize all access to these global variables to understand why
+ scrutinize all access to these global variables in order to understand why
a rectangular table became a square, and discover that remote
template code is also modifying this context, messing with
- table dimensions.
+ the table dimensions.
- Spaghetti code: multiple pages of nested if clauses and for loops
with a lot of copy-pasted procedural code and no
proper segmentation are known as spaghetti code. Python's
- meaningful indentation (one of its most controversial features) make
- it very hard to maintain this kind of code. So the good news is that
+ meaningful indentation (one of its most controversial features) makes
+ it very hard to maintain this kind of code. The good news is that
you might not see too much of it.
- Ravioli code is more likely in Python: it consists of hundreds of
similar little pieces of logic, often classes or objects, without
- proper structure. If you never can remember if you have to use
+ proper structure. If you never can remember, if you have to use
FurnitureTable, AssetTable or Table, or even TableNew for your
- task at hand, you might be swimming in ravioli code.
+ task at hand, then you might be swimming in ravioli code.
+*******
Modules
--------
+*******
Python modules are one of the main abstraction layers available and probably the
most natural one. Abstraction layers allow separating code into parts holding
@@ -83,58 +381,69 @@ while another would handle low-level manipulation of data. The most natural way
to separate these two layers is to regroup all interfacing functionality
in one file, and all low-level operations in another file. In this case,
the interface file needs to import the low-level file. This is done with the
-`import` and `from ... import` statements.
+``import`` and ``from ... import`` statements.
-As soon as you use `import` statements you use modules. These can be either built-in
-modules such as `os` and `sys`, third-party modules you have installed in your
-environment, or your project's internal modules.
+As soon as you use `import` statements, you use modules. These can be either
+built-in modules such as `os` and `sys`, third-party modules you have installed
+in your environment, or your project's internal modules.
To keep in line with the style guide, keep module names short, lowercase, and
be sure to avoid using special symbols like the dot (.) or question mark (?).
-So a file name like `my.spam.py` is one you should avoid! Naming this way
-will interfere with the way python looks for modules.
+A file name like :file:`my.spam.py` is the one you should avoid! Naming this way
+will interfere with the way Python looks for modules.
-In this example python expects to find a "spam.py" file in a folder named "my"
-which is not the case. There is an
+In the case of `my.spam.py` Python expects to find a :file:`spam.py` file in a
+folder named :file:`my` which is not the case. There is an
`example `_ of how the
-dot notation should be used in the python docs.
+dot notation should be used in the Python docs.
+
+If you like, you could name your module :file:`my_spam.py`, but even our trusty
+friend the underscore, should not be seen that often in module names. However, using other
+characters (spaces or hyphens) in module names will prevent importing
+(- is the subtract operator). Try to keep module names short so there is
+no need to separate words. And, most of all, don't namespace with underscores; use submodules instead.
-If you'd like you could name it as `my_spam.py` but even our friend the
-underscore should not be seen often in module names.
+.. code-block:: python
-Aside for some naming restrictions, nothing special is required for a Python file
-to be a module, but the import mechanism needs to be understood in order to use
-this concept properly and avoid some issues.
+ # OK
+ import library.plugin.foo
+ # not OK
+ import library.foo_plugin
-Concretely, the `import modu` statement will look for the proper file, which is
-`modu.py` in the same directory as the caller if it exists. If it is not
-found, the Python interpreter will search for `modu.py` in the "path"
-recursively and raise an ImportError exception if it is not found.
+Aside from some naming restrictions, nothing special is required for a Python
+file to be a module. But you need to understand the import mechanism in order
+to use this concept properly and avoid some issues.
-Once `modu.py` is found, the Python interpreter will execute the module in an
-isolated scope. Any top-level statement in `modu.py` will be executed,
+Concretely, the ``import modu`` statement will look for the proper file, which
+is :file:`modu.py` in the same directory as the caller, if it exists. If it is
+not found, the Python interpreter will search for :file:`modu.py` in the "path"
+recursively and raise an ImportError exception when it is not found.
+
+When :file:`modu.py` is found, the Python interpreter will execute the module in
+an isolated scope. Any top-level statement in :file:`modu.py` will be executed,
including other imports if any. Function and class definitions are stored in
the module's dictionary.
-Then, the module's variables, functions, and classes will be available to the caller
-through the module's namespace, a central concept in programming that is
+Then, the module's variables, functions, and classes will be available to the
+caller through the module's namespace, a central concept in programming that is
particularly helpful and powerful in Python.
-In many languages, an `include file` directive is used by the preprocessor to
+In many languages, an ``include file`` directive is used by the preprocessor to
take all code found in the file and 'copy' it into the caller's code. It is
different in Python: the included code is isolated in a module namespace, which
means that you generally don't have to worry that the included code could have
unwanted effects, e.g. override an existing function with the same name.
It is possible to simulate the more standard behavior by using a special syntax
-of the import statement: `from modu import *`. This is generally considered bad
-practice. **Using `import *` makes code harder to read and makes dependencies less
-compartmentalized**.
+of the import statement: ``from modu import *``. This is generally considered
+bad practice. **Using** ``import *`` **makes the code harder to read and makes
+dependencies less compartmentalized**.
-Using `from modu import func` is a way to pinpoint the function you want to
-import and put it in the global namespace. While much less harmful than `import
-*` because it shows explicitly what is imported in the global namespace, its
-advantage over a simpler `import modu` is only that it will save some typing.
+Using ``from modu import func`` is a way to pinpoint the function you want to
+import and put it in the local namespace. While much less harmful than ``import
+*`` because it shows explicitly what is imported in the local namespace, its
+only advantage over a simpler ``import modu`` is that it will save a little
+typing.
**Very bad**
@@ -161,101 +470,107 @@ advantage over a simpler `import modu` is only that it will save some typing.
[...]
x = modu.sqrt(4) # sqrt is visibly part of modu's namespace
-As said in the section about style, readability is one of the main features of
-Python. Readability means to avoid useless boilerplate text and clutter,
-therefore some efforts are spent trying to achieve a certain level of brevity.
-But terseness and obscurity are the limits where brevity should stop. Being
-able to tell immediately where a class or function comes from, as in the
-`modu.func` idiom, greatly improves code readability and understandability in
+As mentioned in the :ref:`code_style` section, readability is one of the main
+features of Python. Readability means to avoid useless boilerplate text and
+clutter; therefore some efforts are spent trying to achieve a certain level of
+brevity. But terseness and obscurity are the limits where brevity should stop.
+Being able to tell immediately where a class or function comes from, as in the
+``modu.func`` idiom, greatly improves code readability and understandability in
all but the simplest single file projects.
+********
Packages
---------
+********
Python provides a very straightforward packaging system, which is simply an
extension of the module mechanism to a directory.
-Any directory with an __init__.py file is considered a Python package. The
-different modules in the package are imported in a similar manner as plain
-modules, but with a special behavior for the __init__.py file, which is used to
-gather all package-wide definitions.
+Any directory with an :file:`__init__.py` file is considered a Python package.
+The different modules in the package are imported in a similar manner as plain
+modules, but with a special behavior for the :file:`__init__.py` file, which is
+used to gather all package-wide definitions.
-A file modu.py in the directory pack/ is imported with the statement `import
-pack.modu`. This statement will look for an __init__.py file in `pack`, execute
-all of its top-level statements. Then it will look for a file `pack/modu.py` and
+A file :file:`modu.py` in the directory :file:`pack/` is imported with the
+statement ``import pack.modu``. This statement will look for
+:file:`__init__.py` file in :file:`pack` and execute all of its top-level
+statements. Then it will look for a file named :file:`pack/modu.py` and
execute all of its top-level statements. After these operations, any variable,
-function, or class defined in modu.py is available in the pack.modu namespace.
+function, or class defined in :file:`modu.py` is available in the pack.modu
+namespace.
-A commonly seen issue is to add too much code to __init__.py
+A commonly seen issue is adding too much code to :file:`__init__.py`
files. When the project complexity grows, there may be sub-packages and
-sub-sub-packages in a deep directory structure, and then, importing a single item
-from a sub-sub-package will require executing all __init__.py files met while
-traversing the tree.
+sub-sub-packages in a deep directory structure. In this case, importing a
+single item from a sub-sub-package will require executing all
+:file:`__init__.py` files met while traversing the tree.
-Leaving an __init__.py file empty is considered normal and even a good practice,
-if the package's modules and sub-packages do not need to share any code.
+Leaving an :file:`__init__.py` file empty is considered normal and even good
+practice, if the package's modules and sub-packages do not need to share any
+code.
Lastly, a convenient syntax is available for importing deeply nested packages:
-`import very.deep.module as mod`. This allows you to use `mod` in place of the verbose
-repetition of `very.deep.module`.
+``import very.deep.module as mod``. This allows you to use `mod` in place of the
+verbose repetition of ``very.deep.module``.
+
+***************************
Object-oriented programming
----------------------------
+***************************
Python is sometimes described as an object-oriented programming language. This
-can be somewhat misleading and needs to be clarified.
+can be somewhat misleading and requires further clarifications.
In Python, everything is an object, and can be handled as such. This is what is
-meant when we say that, for example, functions are first-class objects.
+meant when we say, for example, that functions are first-class objects.
Functions, classes, strings, and even types are objects in Python: like any
-objects, they have a type, they can be passed as function arguments, they may
-have methods and properties. In this understanding, Python is an
-object-oriented language.
+object, they have a type, they can be passed as function arguments, and they
+may have methods and properties. In this understanding, Python can be considered
+as an object-oriented language.
However, unlike Java, Python does not impose object-oriented programming as the
main programming paradigm. It is perfectly viable for a Python project to not
be object-oriented, i.e. to use no or very few class definitions, class
inheritance, or any other mechanisms that are specific to object-oriented
-programming.
+programming languages.
Moreover, as seen in the modules_ section, the way Python handles modules and
namespaces gives the developer a natural way to ensure the
encapsulation and separation of abstraction layers, both being the most common
reasons to use object-orientation. Therefore, Python programmers have more
-latitude to not use object-orientation, when it is not required by the business
+latitude as to not use object-orientation, when it is not required by the business
model.
There are some reasons to avoid unnecessary object-orientation. Defining
-custom classes is useful when we want to glue together some state and some
-functionality. The problem, as pointed out by the discussions about functional
+custom classes is useful when we want to glue some state and some
+functionality together. The problem, as pointed out by the discussions about functional
programming, comes from the "state" part of the equation.
In some architectures, typically web applications, multiple instances of Python
-processes are spawned to respond to external requests that can
-happen at the same time. In this case, holding some state into instantiated
-objects, which means keeping some static information about the world, is prone
-to concurrency problems or race-conditions. Sometimes, between the initialization of
-the state of an object (usually done with the __init__() method) and the actual use
+processes are spawned as a response to external requests that happen simultaneously.
+In this case, holding some state in instantiated objects, which
+means keeping some static information about the world, is prone to concurrency
+problems or race conditions. Sometimes, between the initialization of the state
+of an object (usually done with the ``__init__()`` method) and the actual use
of the object state through one of its methods, the world may have changed, and
the retained state may be outdated. For example, a request may load an item in
memory and mark it as read by a user. If another request requires the deletion
-of this item at the same time, it may happen that the deletion actually occurs after
-the first process loaded the item, and then we have to mark as read a deleted
-object.
+of this item at the same time, the deletion may actually occur after the first
+process loaded the item, and then we have to mark a deleted object as read.
This and other issues led to the idea that using stateless functions is a
better programming paradigm.
Another way to say the same thing is to suggest using functions and procedures
with as few implicit contexts and side-effects as possible. A function's
-implicit context is made up of any of the global variables or items in the persistence layer
-that are accessed from within the function. Side-effects are the changes that a function makes
-to its implicit context. If a function saves or deletes data in a global variable or
-in the persistence layer, it is said to have a side-effect.
+implicit context is made up of any of the global variables or items in the
+persistence layer that are accessed from within the function. Side-effects are
+the changes that a function makes to its implicit context. If a function saves
+or deletes data in a global variable or in the persistence layer, it is said to
+have a side-effect.
Carefully isolating functions with context and side-effects from functions with
-logic (called pure functions) allow the following benefits:
+logic (called pure functions) allows the following benefits:
- Pure functions are deterministic: given a fixed input,
the output will always be the same.
@@ -263,13 +578,13 @@ logic (called pure functions) allow the following benefits:
- Pure functions are much easier to change or replace if they need to
be refactored or optimized.
-- Pure functions are easier to test with unit-tests: There is less
+- Pure functions are easier to test with unit tests: There is less
need for complex context setup and data cleaning afterwards.
-- Pure functions are easier to manipulate, decorate, and pass-around.
+- Pure functions are easier to manipulate, decorate, and pass around.
-In summary, pure functions, without any context or side-effects, are more
-efficient building blocks than classes and objects for some architectures.
+In summary, pure functions are more efficient building blocks than classes
+and objects for some architectures because they have no context or side-effects.
Obviously, object-orientation is useful and even necessary in many cases, for
example when developing graphical desktop applications or games, where the
@@ -277,14 +592,15 @@ things that are manipulated (windows, buttons, avatars, vehicles) have a
relatively long life of their own in the computer's memory.
+**********
Decorators
-----------
+**********
The Python language provides a simple yet powerful syntax called 'decorators'.
A decorator is a function or a class that wraps (or decorates) a function
or a method. The 'decorated' function or method will replace the original
'undecorated' function or method. Because functions are first-class objects
-in Python, it can be done 'manually', but using the @decorator syntax is
+in Python, this can be done 'manually', but using the @decorator syntax is
clearer and thus preferred.
.. code-block:: python
@@ -304,29 +620,105 @@ clearer and thus preferred.
# bar() is decorated
This mechanism is useful for separating concerns and avoiding
-external un-related logic 'polluting' the core logic of the function
+external unrelated logic 'polluting' the core logic of the function
or method. A good example of a piece of functionality that is better handled
-with decoration is memoization or caching: you want to store the results of an
+with decoration is `memoization `__ or caching: you want to store the results of an
expensive function in a table and use them directly instead of recomputing
them when they have already been computed. This is clearly not part
of the function logic.
+
+****************
+Context Managers
+****************
+
+A context manager is a Python object that provides extra contextual information
+to an action. This extra information takes the form of running a callable upon
+initiating the context using the ``with`` statement, as well as running a callable
+upon completing all the code inside the ``with`` block. The most well known
+example of using a context manager is shown here, opening on a file:
+
+.. code-block:: python
+
+ with open('file.txt') as f:
+ contents = f.read()
+
+Anyone familiar with this pattern knows that invoking ``open`` in this fashion
+ensures that ``f``'s ``close`` method will be called at some point. This reduces
+a developer's cognitive load and makes the code easier to read.
+
+There are two easy ways to implement this functionality yourself: using a class
+or using a generator. Let's implement the above functionality ourselves, starting
+with the class approach:
+
+.. code-block:: python
+
+ class CustomOpen(object):
+ def __init__(self, filename):
+ self.file = open(filename)
+
+ def __enter__(self):
+ return self.file
+
+ def __exit__(self, ctx_type, ctx_value, ctx_traceback):
+ self.file.close()
+
+ with CustomOpen('file') as f:
+ contents = f.read()
+
+This is just a regular Python object with two extra methods that are used
+by the ``with`` statement. CustomOpen is first instantiated and then its
+``__enter__`` method is called and whatever ``__enter__`` returns is assigned to
+``f`` in the ``as f`` part of the statement. When the contents of the ``with`` block
+is finished executing, the ``__exit__`` method is then called.
+
+And now the generator approach using Python's own
+`contextlib `_:
+
+.. code-block:: python
+
+ from contextlib import contextmanager
+
+ @contextmanager
+ def custom_open(filename):
+ f = open(filename)
+ try:
+ yield f
+ finally:
+ f.close()
+
+ with custom_open('file') as f:
+ contents = f.read()
+
+This works in exactly the same way as the class example above, albeit it's
+more terse. The ``custom_open`` function executes until it reaches the ``yield``
+statement. It then gives control back to the ``with`` statement, which assigns
+whatever was ``yield``'ed to `f` in the ``as f`` portion. The ``finally`` clause
+ensures that ``close()`` is called whether or not there was an exception inside
+the ``with``.
+
+Since the two approaches appear the same, we should follow the Zen of Python
+to decide when to use which. The class approach might be better if there's
+a considerable amount of logic to encapsulate. The function approach
+might be better for situations where we're dealing with a simple action.
+
+
+**************
Dynamic typing
---------------
+**************
-Python is said to be dynamically typed, which means that variables
-do not have a fixed type. In fact, in Python, variables are very
-different from what they are in many other languages, specifically
-strongly-typed languages. Variables are not a segment of the computer's
-memory where some value is written, they are 'tags' or 'names' pointing
-to objects. It is therefore possible for the variable 'a' to be set to
-the value 1, then to the value 'a string', then to a function.
+Python is dynamically typed, which means that variables do not have a fixed
+type. In fact, in Python, variables are very different from what they are in
+many other languages, specifically statically-typed languages. Variables are not
+a segment of the computer's memory where some value is written, they are 'tags'
+or 'names' pointing to objects. It is therefore possible for the variable 'a' to
+be set to the value 1, then the value 'a string', to a function.
The dynamic typing of Python is often considered to be a weakness, and indeed
-it can lead to complexities and hard-to-debug code. Something
-named 'a' can be set to many different things, and the developer or the
-maintainer needs to track this name in the code to make sure it has not
-been set to a completely unrelated object.
+it can lead to complexities and hard-to-debug code. Something named 'a' can be
+set to many different things, and the developer or the maintainer needs to track
+this name in the code to make sure it has not been set to a completely unrelated
+object.
Some guidelines help to avoid this issue:
@@ -350,7 +742,7 @@ Some guidelines help to avoid this issue:
def func():
pass # Do something
-Using short functions or methods helps reduce the risk
+Using short functions or methods helps to reduce the risk
of using the same name for two unrelated things.
It is better to use different names even for things that are related,
@@ -370,32 +762,33 @@ grows and each assignment is separated by other lines of code, including
'if' branches and loops, it becomes harder to ascertain what a given
variable's type is.
-Some coding practices, like functional programming, recommend never reassigning a variable.
-In Java this is done with the `final` keyword. Python does not have a `final` keyword
-and it would be against its philosophy anyway. However, it may be a good
-discipline to avoid assigning to a variable more than once, and it helps
-in grasping the concept of mutable and immutable types.
+Some coding practices, like functional programming, recommend never reassigning
+a variable. In Java this is done with the `final` keyword. Python does not have
+a `final` keyword and it would be against its philosophy anyway. However, it may
+be a good discipline to avoid assigning to a variable more than once, and it
+helps in grasping the concept of mutable and immutable types.
+
+***************************
Mutable and immutable types
----------------------------
+***************************
Python has two kinds of built-in or user-defined types.
-Mutable types are those that allow in-place modification
-of the content. Typical mutables are lists and dictionaries:
-All lists have mutating methods, like append() or pop(), and
-can be modified in place. The same goes for dictionaries.
+Mutable types are those that allow in-place modification of the content. Typical
+mutables are lists and dictionaries: All lists have mutating methods, like
+:py:meth:`list.append` or :py:meth:`list.pop`, and can be modified in place.
+The same goes for dictionaries.
-Immutable types provide no method for changing their content.
-For instance, the variable x set to the integer 6 has no "increment" method. If you
-want to compute x + 1, you have to create another integer and give it
-a name.
+Immutable types provide no method for changing their content. For instance, the
+variable x set to the integer 6 has no "increment" method. If you want to
+compute x + 1, you have to create another integer and give it a name.
.. code-block:: python
my_list = [1, 2, 3]
my_list[0] = 4
- print my_list # [4, 2, 3] <- The same list as changed
+ print(my_list) # [4, 2, 3] <- The same list has changed
x = 6
x = x + 1 # The new x is another object
@@ -414,11 +807,12 @@ and can be used as a key for a dictionary.
One peculiarity of Python that can surprise beginners is that
strings are immutable. This means that when constructing a string from
-its parts, it is much more efficient to accumulate the parts in a list,
-which is mutable, and then glue ('join') the parts together when the
-full string is needed. One thing to notice, however, is that list
-comprehensions are better and faster than constructing a list in a loop
-with calls to append().
+its parts, appending each part to the string is inefficient because
+the entirety of the string is copied on each append.
+Instead, it is much more efficient to accumulate the parts in a list,
+which is mutable, and then glue (``join``) the parts together when the
+full string is needed. List comprehensions are usually the fastest and
+most idiomatic way to do this.
**Bad**
@@ -427,18 +821,18 @@ with calls to append().
# create a concatenated string from 0 to 19 (e.g. "012..1819")
nums = ""
for n in range(20):
- nums += str(n) # slow and inefficient
- print nums
+ nums += str(n) # slow and inefficient
+ print(nums)
-**Good**
+**Better**
.. code-block:: python
# create a concatenated string from 0 to 19 (e.g. "012..1819")
nums = []
for n in range(20):
- nums.append(str(n))
- print "".join(nums) # much more efficient
+ nums.append(str(n))
+ print("".join(nums)) # much more efficient
**Best**
@@ -446,13 +840,13 @@ with calls to append().
# create a concatenated string from 0 to 19 (e.g. "012..1819")
nums = [str(n) for n in range(20)]
- print "".join(nums)
+ print("".join(nums))
-One final thing to mention about strings is that using join() is not always
+One final thing to mention about strings is that using ``join()`` is not always
best. In the instances where you are creating a new string from a pre-determined
-number of strings, using the addition operator is actually faster, but in cases
-like above or in cases where you are adding to an existing string, using join()
-should be your preferred method.
+number of strings, using the addition operator is actually faster. But in cases
+like above or in cases where you are adding to an existing string, using
+``join()`` should be your preferred method.
.. code-block:: python
@@ -464,11 +858,10 @@ should be your preferred method.
foo = ''.join([foo, 'ooo'])
.. note::
- You can also use the **%** formatting operator to concatenate the
- pre-determined number of strings besides **join()** and **+**. However,
- according to `PEP 3101 `_,
- **%** operator became deprecated in Python 3.1 and will be replaced by the
- **format()** method in the later versions.
+ You can also use the :ref:`% ` formatting operator
+ to concatenate a pre-determined number of strings besides :py:meth:`str.join`
+ and ``+``. However, :pep:`3101` discourages the usage of the ``%`` operator
+ in favor of the :py:meth:`str.format` method.
.. code-block:: python
@@ -480,16 +873,20 @@ should be your preferred method.
foobar = '{foo}{bar}'.format(foo=foo, bar=bar) # It is best
+************************
Vendorizing Dependencies
-------------------------
+************************
+
+*******
Runners
--------
+*******
+***************
Further Reading
----------------
+***************
-- http://docs.python.org/2/library/
-- http://www.diveintopython.net/toc/index.html
+- http://docs.python.org/3/library/
+- https://diveintopython3.net/
diff --git a/docs/writing/style.rst b/docs/writing/style.rst
index 8863b0c16..d8c096a05 100644
--- a/docs/writing/style.rst
+++ b/docs/writing/style.rst
@@ -1,26 +1,32 @@
.. _code_style:
+
+##########
Code Style
-==========
+##########
+
+.. image:: /_static/photos/33907150054_5ee79e8940_k_d.jpg
-If you ask Python programmers what they like most in Python, they will
-often say its high readability. Indeed, a high level of readability
+If you ask Python programmers what they like most about Python, they will
+often cite its high readability. Indeed, a high level of readability
is at the heart of the design of the Python language, following the
-recognised fact that code is read much more often than it is written.
+recognized fact that code is read much more often than it is written.
-One reason for Python code to be easily read and understood is its relatively
+One reason for the high readability of Python code is its relatively
complete set of Code Style guidelines and "Pythonic" idioms.
-Moreover, when a veteran Python developer (a Pythonista) points to portions of code
-and says they are not "Pythonic", it usually means that these lines
-of code do not follow the common guidelines and fail to express the intent in
+When a veteran Python developer (a Pythonista) calls portions of
+code not "Pythonic", they usually mean that these lines
+of code do not follow the common guidelines and fail to express its intent in
what is considered the best (hear: most readable) way.
On some border cases, no best way has been agreed upon on how to express
an intent in Python code, but these cases are rare.
+
+****************
General concepts
-----------------
+****************
Explicit code
~~~~~~~~~~~~~
@@ -53,15 +59,15 @@ One statement per line
While some compound statements such as list comprehensions are
allowed and appreciated for their brevity and their expressiveness,
-it is bad practice to have two disjoint statements on the same line of code.
+it is bad practice to have two disjointed statements on the same line of code.
**Bad**
.. code-block:: python
- print 'one'; print 'two'
+ print('one'); print('two')
- if x == 1: print 'one'
+ if x == 1: print('one')
if and :
# do something
@@ -70,11 +76,11 @@ it is bad practice to have two disjoint statements on the same line of code.
.. code-block:: python
- print 'one'
- print 'two'
+ print('one')
+ print('two')
if x == 1:
- print 'one'
+ print('one')
cond1 =
cond2 =
@@ -86,49 +92,51 @@ Function arguments
Arguments can be passed to functions in four different ways.
-1. **Positional arguments** are mandatory and have no default values. They are the
-simplest form of arguments and they can be used for the few function arguments
-that are fully part of the functions meaning and their order is natural. For
-instance, in ``send(message, recipient)`` or ``point(x, y)`` the user of the
-function has no difficulty remembering that those two functions require two
-arguments, and in which order.
-
-In those two cases, it is possible to use argument names when calling the functions
-and, doing so, it is possible to switch the order of arguments, calling for instance
-``send(recipient='World', message='Hello')`` and ``point(y=2, x=1)`` but this
-reduces readability and is unnecessarily verbose, compared to the more straightforward
-calls to ``send('Hello', 'World')`` and ``point(1, 2)``.
-
-2. **Keyword arguments** are not mandatory and have default values. They are often
-used for optional parameters sent to the function. When a function has more than
-two or three positional parameters, its signature will be more difficult to remember
-and using keyword argument with default values is helpful. For instance, a more
-complete ``send`` function could be defined as ``send(message, to, cc=None, bcc=None)``.
-Here ``cc`` and ``bcc`` are optional, and evaluate to ``None`` when they are not
-passed another value.
-
-Calling a function with keyword arguments can be done in multiple ways in Python,
-for example it is possible to follow the order of arguments in the definition without
-explicitly naming the arguments, like in ``send('Hello', 'World', 'Cthulhu', 'God')``,
-sending a blind carbon copy to God. It would also be possible to name arguments in
-another order, like in ``send('Hello again', 'World', bcc='God', cc='Cthulhu')``.
-Those two possibilities are better avoided without any strong reason to not
-follow the syntax that is the closest to the function definition: ``send('Hello',
-'World', cc='Cthulhu', bcc='God')``.
-
-As a side note, following `YAGNI `_
-principle, it is often harder to remove an optional argument (and its logic inside the
-function) that was added "just in case" and is seemingly never used, than to add a
-new optional argument and its logic when needed.
+1. **Positional arguments** are mandatory and have no default values. They are
+ the simplest form of arguments and they can be used for the few function
+ arguments that are fully part of the function's meaning and their order is
+ natural. For instance, in ``send(message, recipient)`` or ``point(x, y)``
+ the user of the function has no difficulty remembering that those two
+ functions require two arguments, and in which order.
+
+In those two cases, it is possible to use argument names when calling the
+functions and, doing so, it is possible to switch the order of arguments,
+calling for instance ``send(recipient='World', message='Hello')`` and
+``point(y=2, x=1)`` but this reduces readability and is unnecessarily verbose,
+compared to the more straightforward calls to ``send('Hello', 'World')`` and
+``point(1, 2)``.
+
+2. **Keyword arguments** are not mandatory and have default values. They are
+ often used for optional parameters sent to the function. When a function has
+ more than two or three positional parameters, its signature is more difficult
+ to remember and using keyword arguments with default values is helpful. For
+ instance, a more complete ``send`` function could be defined as
+ ``send(message, to, cc=None, bcc=None)``. Here ``cc`` and ``bcc`` are
+ optional, and evaluate to ``None`` when they are not passed another value.
+
+Calling a function with keyword arguments can be done in multiple ways in
+Python; for example, it is possible to follow the order of arguments in the
+definition without explicitly naming the arguments, like in
+``send('Hello', 'World', 'Cthulhu', 'God')``, sending a blind carbon copy to
+God. It would also be possible to name arguments in another order, like in
+``send('Hello again', 'World', bcc='God', cc='Cthulhu')``. Those two
+possibilities are better avoided without any strong reason to not follow the
+syntax that is the closest to the function definition:
+``send('Hello', 'World', cc='Cthulhu', bcc='God')``.
+
+As a side note, following the `YAGNI `_
+principle, it is often harder to remove an optional argument (and its logic
+inside the function) that was added "just in case" and is seemingly never used,
+than to add a new optional argument and its logic when needed.
3. The **arbitrary argument list** is the third way to pass arguments to a
-function. If the function intention is better expressed by a signature with an
-extensible number of positional arguments, it can be defined with the ``*args``
-constructs. In the function body, ``args`` will be a tuple of all the
-remaining positional arguments. For example, ``send(message, *args)`` can be
-called with each recipient as an argument: ``send('Hello', 'God', 'Mom',
-'Cthulhu')``, and in the function body ``args`` will be equal to ``('God',
-'Mom', 'Cthulhu')``.
+ function. If the function intention is better expressed by a signature with
+ an extensible number of positional arguments, it can be defined with the
+ ``*args`` constructs. In the function body, ``args`` will be a tuple of all
+ the remaining positional arguments. For example, ``send(message, *args)``
+ can be called with each recipient as an argument: ``send('Hello', 'God',
+ 'Mom', 'Cthulhu')``, and in the function body ``args`` will be equal to
+ ``('God', 'Mom', 'Cthulhu')``.
However, this construct has some drawbacks and should be used with caution. If a
function receives a list of arguments of the same nature, it is often more
@@ -139,11 +147,12 @@ it explicitly: ``send(message, recipients)`` and call it with ``send('Hello',
the recipient list as a list beforehand, and it opens the possibility to pass
any sequence, including iterators, that cannot be unpacked as other sequences.
-4. The **arbitrary keyword argument dictionary** is the last way to pass arguments
-to functions. If the function requires an undetermined series of named
-arguments, it is possible to use the ``**kwargs`` construct. In the function
-body, ``kwargs`` will be a dictionary of all the passed named arguments that
-have not been caught by other keyword arguments in the function signature.
+4. The **arbitrary keyword argument dictionary** is the last way to pass
+ arguments to functions. If the function requires an undetermined series of
+ named arguments, it is possible to use the ``**kwargs`` construct. In the
+ function body, ``kwargs`` will be a dictionary of all the passed named
+ arguments that have not been caught by other keyword arguments in the
+ function signature.
The same caution as in the case of *arbitrary argument list* is necessary, for
similar reasons: these powerful techniques are to be used when there is a
@@ -158,36 +167,36 @@ Python functions that are:
* easy to read (the name and arguments need no explanations)
-* easy to change (adding a new keyword argument does not break other parts of the
- code)
+* easy to change (adding a new keyword argument does not break other parts of
+ the code)
Avoid the magical wand
~~~~~~~~~~~~~~~~~~~~~~
A powerful tool for hackers, Python comes with a very rich set of hooks and
-tools allowing to do almost any kind of tricky tricks. For instance, it is
+tools allowing you to do almost any kind of tricky tricks. For instance, it is
possible to do each of the following:
* change how objects are created and instantiated
* change how the Python interpreter imports modules
-* it is even possible (and recommended if needed) to embed C routines in Python.
+* It is even possible (and recommended if needed) to embed C routines in Python.
However, all these options have many drawbacks and it is always better to use
the most straightforward way to achieve your goal. The main drawback is that
-readability suffers deeply from them. Many code analysis tools, such as pylint
-or pyflakes, will be unable to parse this "magic" code.
+readability suffers greatly when using these constructs. Many code analysis
+tools, such as pylint or pyflakes, will be unable to parse this "magic" code.
We consider that a Python developer should know about these nearly infinite
-possibilities, because it grows the confidence that no hard-wall will be on the
-way. However, knowing how to use them and particularly when **not** to use
-them is the most important.
+possibilities, because it instills confidence that no impassable problem will
+be on the way. However, knowing how and particularly when **not** to use
+them is very important.
-Like a Kungfu master, a Pythonista knows how to kill with a single finger, and
+Like a kung fu master, a Pythonista knows how to kill with a single finger, and
never to actually do it.
-We are all consenting adults
+We are all responsible users
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As seen above, Python allows many tricks, and some of them are potentially
@@ -195,11 +204,11 @@ dangerous. A good example is that any client code can override an object's
properties and methods: there is no "private" keyword in Python. This
philosophy, very different from highly defensive languages like Java, which
give a lot of mechanisms to prevent any misuse, is expressed by the saying: "We
-are consenting adults".
+are all responsible users".
This doesn't mean that, for example, no properties are considered private, and
-that no proper encapsulation is possible in Python. But, instead of relying on
-concrete walls erected by the developers between their code and other's, the
+that no proper encapsulation is possible in Python. Rather, instead of relying
+on concrete walls erected by the developers between their code and others', the
Python community prefers to rely on a set of conventions indicating that these
elements should not be accessed directly.
@@ -210,34 +219,36 @@ the code is modified is the responsibility of the client code.
Using this convention generously is encouraged: any method or property that is
not intended to be used by client code should be prefixed with an underscore.
-This will guarantee a better separation of duties and easier modifications of
-existing code, and it will always be possible to publicize a private property,
-while privatising a public property might be a much harder operation.
+This will guarantee a better separation of duties and easier modification of
+existing code; it will always be possible to publicize a private property,
+but making a public property private might be a much harder operation.
Returning values
~~~~~~~~~~~~~~~~
-When a function grows in complexity is not uncommon to use multiple return statements
-inside the function's body. However, in order to keep a clear intent and a sustainable
-readability level, it is preferable to avoid returning meaningful values from many
-output points in the body.
-
-There are two main cases for returning values in a function: The result of the function
-return when it has been processed normally, and the error cases that indicate a wrong
-input parameter or any other reason for the function to not be able to complete its
-computation or task.
-
-If you do not wish to raise exceptions for the second case, then returning a value, such
-as None or False, indicating that the function could not perform correctly might be needed. In this
-case, it is better to return as early as the incorrect context has been detected. It will
-help to flatten the structure of the function: all the code after the return-because-of-error
-statement can assume the condition is met to further compute the function's main result.
+When a function grows in complexity, it is not uncommon to use multiple return
+statements inside the function's body. However, in order to keep a clear intent
+and a sustainable readability level, it is preferable to avoid returning
+meaningful values from many output points in the body.
+
+There are two main cases for returning values in a function: the result of the
+function return when it has been processed normally, and the error cases that
+indicate a wrong input parameter or any other reason for the function to not be
+able to complete its computation or task.
+
+If you do not wish to raise exceptions for the second case, then returning a
+value, such as None or False, indicating that the function could not perform
+correctly might be needed. In this case, it is better to return as early as the
+incorrect context has been detected. It will help to flatten the structure of
+the function: all the code after the return-because-of-error statement can
+assume the condition is met to further compute the function's main result.
Having multiple such return statements is often necessary.
-However, when a function has multiple main exit points for its normal course, it becomes
-difficult to debug the returned result, and it may be preferable to keep a single exit
-point. This will also help factoring out some code paths, and the multiple exit points
-are a probable indication that such a refactoring is needed.
+However, when a function has multiple main exit points for its normal course,
+it becomes difficult to debug the returned result, so it may be preferable to
+keep a single exit point. This will also help factoring out some code paths,
+and the multiple exit points are a probable indication that such a refactoring
+is needed.
.. code-block:: python
@@ -253,17 +264,20 @@ are a probable indication that such a refactoring is needed.
return x # One single exit point for the returned value x will help
# when maintaining the code.
+
+******
Idioms
-------
+******
-A programming Idiom, put simply, is a *way* to write code. The notion of programming Idioms
-is discussed amply at `c2 `_ and at `Stack Overflow `_.
+A programming idiom, put simply, is a *way* to write code. The notion of
+programming idioms is discussed amply at `c2 `_
+and at `Stack Overflow `_.
-Idiomatic Python code is often referred to as being *Pythonic*.
+Idiomatic Python code is often referred to as being *Pythonic*.
-Although there usually is one-- and preferably only one --obvious way to do it;
-*the* way to write idiomatic Python code can be non-obvious to Python beginners. So,
-good idioms must be consciously acquired.
+Although there usually is one --- and preferably only one --- obvious way to do
+it; *the* way to write idiomatic Python code can be non-obvious to Python
+beginners. So, good idioms must be consciously acquired.
Some common Python idioms follow:
@@ -273,14 +287,15 @@ Unpacking
~~~~~~~~~
If you know the length of a list or tuple, you can assign names to its
-elements with unpacking:
+elements with unpacking. For example, since ``enumerate()`` will provide
+a tuple of two elements for each item in list:
.. code-block:: python
for index, item in enumerate(some_list):
# do something with index and item
-You can use this to swap variables, as well:
+You can use this to swap variables as well:
.. code-block:: python
@@ -292,6 +307,16 @@ Nested unpacking works too:
a, (b, c) = 1, (2, 3)
+In Python 3, a new method of extended unpacking was introduced by
+:pep:`3132`:
+
+.. code-block:: python
+
+ a, *rest = [1, 2, 3]
+ # a = 1, rest = [2, 3]
+ a, *middle, c = [1, 2, 3, 4]
+ # a = 1, middle = [2, 3], c = 4
+
Create an ignored variable
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -332,39 +357,77 @@ Instead, use a list comprehension:
.. code-block:: python
- four_lists = [[] for __ in xrange(4)]
+ four_lists = [[] for __ in range(4)]
+
+Create a string from a list
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A common idiom for creating strings is to use :py:meth:`str.join` on an empty
+string.
-A common idiom for creating strings is to use `join `_ on an empty string.::
+.. code-block:: python
letters = ['s', 'p', 'a', 'm']
word = ''.join(letters)
-This will set the value of the variable *word* to 'spam'. This idiom can be applied to lists and tuples.
+This will set the value of the variable *word* to 'spam'. This idiom can be
+applied to lists and tuples.
-Sometimes we need to search through a collection of things. Let's look at two options: lists and dictionaries.
+Searching for an item in a collection
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Take the following code for example::
+Sometimes we need to search through a collection of things. Let's look at two
+options: lists and sets.
+
+Take the following code for example:
+
+.. code-block:: python
- d = {'s': [], 'p': [], 'a': [], 'm': []}
+ s = set(['s', 'p', 'a', 'm'])
l = ['s', 'p', 'a', 'm']
- def lookup_dict(d):
- return 's' in d
+ def lookup_set(s):
+ return 's' in s
def lookup_list(l):
return 's' in l
-Even though both functions look identical, because *lookup_dict* is utilizing the fact that dictionaries in python are hashtables, the lookup performance between the two is very different.
-Python will have to go through each item in the list to find a matching case, which is time consuming. By analysing the hash of the dictionary, finding keys in the dict can be done very quickly.
-For more information see this `StackOverflow `_ page.
+Even though both functions look identical, because *lookup_set* is utilizing
+the fact that sets in Python are hashtables, the lookup performance
+between the two is very different. To determine whether an item is in a list,
+Python will have to go through each item until it finds a matching item.
+This is time consuming, especially for long lists. In a set, on the other
+hand, the hash of the item will tell Python where in the set to look for
+a matching item. As a result, the search can be done quickly, even if the
+set is large. Searching in dictionaries works the same way. For
+more information see this
+`StackOverflow `_
+page. For detailed information on the amount of time various common operations
+take on each of these data structures, see
+`this page `_.
+Because of these differences in performance, it is often a good idea to use
+sets or dictionaries instead of lists in cases where:
+
+* The collection will contain a large number of items
+
+* You will be repeatedly searching for items in the collection
+
+* You do not have duplicate items.
+
+For small collections, or collections which you will not frequently be
+searching through, the additional time and memory required to set up the
+hashtable will often be greater than the time saved by the improved search
+speed.
+
+
+*************
Zen of Python
--------------
+*************
-Also known as PEP 20, the guiding principles for Python's design.
+Also known as :pep:`20`, the guiding principles for Python's design.
-::
+.. code-block:: pycon
>>> import this
The Zen of Python, by Tim Peters
@@ -389,34 +452,38 @@ Also known as PEP 20, the guiding principles for Python's design.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
-For some examples of good Python style, see `this Stack Overflow question
-`_ or `these
-slides from a Python user group
-`_.
+For some examples of good Python style, see `these slides from a Python user
+group `_.
+
+*****
PEP 8
------
+*****
-PEP 8 is the de-facto code style guide for Python.
+:pep:`8` is the de facto code style guide for Python. A high quality,
+easy-to-read version of PEP 8 is also available at `pep8.org `_.
- `PEP 8 `_
+This is highly recommended reading. The entire Python community does their
+best to adhere to the guidelines laid out within this document. Some project
+may sway from it from time to time, while others may amend its recommendations.
-Conforming your Python code to PEP 8 is generally a good idea and helps make
-code more consistent when working on projects with other developers. There
-exists a command-line program, `pep8 `_,
-that can check your code for conformance. Install it by running the following
-command in your Terminal:
+That being said, conforming your Python code to PEP 8 is generally a good idea
+and helps make code more consistent when working on projects with other
+developers. There is a command-line program, `pycodestyle `_
+(previously known as ``pep8``), that can check your code for conformance.
+Install it by running the following command in your terminal:
-::
- $ pip install pep8
+.. code-block:: console
+
+ $ pip install pycodestyle
Then run it on a file or series of files to get a report of any violations.
-::
+.. code-block:: console
- $ pep8 optparse.py
+ $ pycodestyle optparse.py
optparse.py:69:11: E401 multiple imports on one line
optparse.py:77:1: E302 expected 2 blank lines, found 1
optparse.py:88:5: E301 expected 1 blank line, found 0
@@ -426,15 +493,85 @@ Then run it on a file or series of files to get a report of any violations.
optparse.py:472:29: E221 multiple spaces before operator
optparse.py:544:21: W601 .has_key() is deprecated, use 'in'
+Auto-Formatting
+~~~~~~~~~~~~~~~
+
+There are several auto-formatting tools that can reformat your code,
+in order to comply with PEP 8.
+
+**autopep8**
+
+The program `autopep8 `_ can be used to
+automatically reformat code in the PEP 8 style. Install the program with:
+
+.. code-block:: console
+
+ $ pip install autopep8
+
+Use it to format a file in-place with:
+
+.. code-block:: console
+
+ $ autopep8 --in-place optparse.py
+
+Excluding the ``--in-place`` flag will cause the program to output the modified
+code directly to the console for review. The ``--aggressive`` flag will perform
+more substantial changes and can be applied multiple times for greater effect.
+
+**yapf**
+
+While autopep8 focuses on solving the PEP 8 violations, `yapf `_
+tries to improve the format of your code aside from complying with PEP 8.
+This formatter aims at providing as good looking code as a programmer who
+writes PEP 8 compliant code.
+It gets installed with:
+
+.. code-block:: console
+
+ $ pip install yapf
+
+Run the auto-formatting of a file with:
+
+.. code-block:: console
+
+ $ yapf --in-place optparse.py
+
+Similar to autopep8, running the command without the ``--in-place`` flag will
+output the diff for review before applying the changes.
+
+**black**
+
+The auto-formatter `black `_ offers an
+opinionated and deterministic reformatting of your code base.
+Its main focus lies in providing a uniform code style without the need of
+configuration throughout its users. Hence, users of black are able to forget
+about formatting altogether. Also, due to the deterministic approach minimal
+git diffs with only the relevant changes are guaranteed. You can install the
+tool as follows:
+
+.. code-block:: console
+
+ $ pip install black
+
+A python file can be formatted with:
+
+.. code-block:: console
+
+ $ black optparse.py
+
+Adding the ``--diff`` flag provides the code modification for review without
+direct application.
+
+***********
Conventions
-----------------
+***********
Here are some conventions you should follow to make your code easier to read.
-Check if variable equals a constant
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Check if a variable equals a constant
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-You don't need to explicitly compare a value to True, or None, or 0 - you can
+You don't need to explicitly compare a value to True, or None, or 0 -- you can
just add it to the if statement. See `Truth Value Testing
`_ for a
list of what is considered false.
@@ -444,10 +581,10 @@ list of what is considered false.
.. code-block:: python
if attr == True:
- print 'True!'
+ print('True!')
if attr == None:
- print 'attr is None!'
+ print('attr is None!')
**Good**:
@@ -455,21 +592,21 @@ list of what is considered false.
# Just check the value
if attr:
- print 'attr is truthy!'
+ print('attr is truthy!')
# or check for the opposite
if not attr:
- print 'attr is falsey!'
+ print('attr is falsey!')
# or, since None is considered false, explicitly check for it
if attr is None:
- print 'attr is None!'
+ print('attr is None!')
Access a Dictionary Element
~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Don't use the ``has_key`` function. Instead use ``x in d`` syntax, or pass
-a default argument to ``get``.
+Don't use the :py:meth:`dict.has_key` method. Instead, use ``x in d`` syntax,
+or pass a default argument to :py:meth:`dict.get`.
**Bad**:
@@ -477,9 +614,9 @@ a default argument to ``get``.
d = {'hello': 'world'}
if d.has_key('hello'):
- print d['hello'] # prints 'world'
+ print(d['hello']) # prints 'world'
else:
- print 'default_value'
+ print('default_value')
**Good**:
@@ -487,76 +624,170 @@ a default argument to ``get``.
d = {'hello': 'world'}
- print d.get('hello', 'default_value') # prints 'world'
- print d.get('thingy', 'default_value') # prints 'default_value'
+ print(d.get('hello', 'default_value')) # prints 'world'
+ print(d.get('thingy', 'default_value')) # prints 'default_value'
# Or:
if 'hello' in d:
- print d['hello']
+ print(d['hello'])
Short Ways to Manipulate Lists
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
`List comprehensions
`_
-provide a powerful, concise way to work with lists. Also, the `map
-`_ and `filter
-`_ functions can perform
-operations on lists using a different concise syntax.
+provides a powerful, concise way to work with lists.
+
+`Generator expressions
+`_
+follows almost the same syntax as list comprehensions but return a generator
+instead of a list.
+
+Creating a new list requires more work and uses more memory. If you are just going
+to loop through the new list, prefer using an iterator instead.
+
+**Bad**:
+
+.. code-block:: python
+
+ # needlessly allocates a list of all (gpa, name) entires in memory
+ valedictorian = max([(student.gpa, student.name) for student in graduates])
+
+**Good**:
+
+.. code-block:: python
+
+ valedictorian = max((student.gpa, student.name) for student in graduates)
+
+
+Use list comprehensions when you really need to create a second list, for
+example if you need to use the result multiple times.
+
+
+If your logic is too complicated for a short list comprehension or generator
+expression, consider using a generator function instead of returning a list.
+
+**Good**:
+
+.. code-block:: python
+
+ def make_batches(items, batch_size):
+ """
+ >>> list(make_batches([1, 2, 3, 4, 5], batch_size=3))
+ [[1, 2, 3], [4, 5]]
+ """
+ current_batch = []
+ for item in items:
+ current_batch.append(item)
+ if len(current_batch) == batch_size:
+ yield current_batch
+ current_batch = []
+ yield current_batch
+
+
+Never use a list comprehension just for its side effects.
**Bad**:
+.. code-block:: python
+
+ [print(x) for x in sequence]
+
+**Good**:
+
+.. code-block:: python
+
+ for x in sequence:
+ print(x)
+
+
+Filtering a list
+~~~~~~~~~~~~~~~~
+
+**Bad**:
+
+Never remove items from a list while you are iterating through it.
+
.. code-block:: python
# Filter elements greater than 4
a = [3, 4, 5]
- b = []
for i in a:
if i > 4:
- b.append(i)
+ a.remove(i)
+
+Don't make multiple passes through the list.
+
+.. code-block:: python
+
+ while i in a:
+ a.remove(i)
**Good**:
+Use a list comprehension or generator expression.
+
+.. code-block:: python
+
+ # comprehensions create a new list object
+ filtered_values = [value for value in sequence if value != x]
+
+ # generators don't create another list
+ filtered_values = (value for value in sequence if value != x)
+
+
+Possible side effects of modifying the original list
+::::::::::::::::::::::::::::::::::::::::::::::::::::
+
+Modifying the original list can be risky if there are other variables referencing it. But you can use *slice assignment* if you really want to do that.
+
.. code-block:: python
- b = [i for i in a if i > 4]
- b = filter(lambda x: x > 4, a)
+ # replace the contents of the original list
+ sequence[::] = [value for value in sequence if value != x]
+
+Modifying the values in a list
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
**Bad**:
+Remember that assignment never creates a new object. If two or more variables refer to the same list, changing one of them changes them all.
+
.. code-block:: python
# Add three to all list members.
a = [3, 4, 5]
- count = 0
- for i in a:
- a[count] = i + 3
- count = count + 1
+ b = a # a and b refer to the same list object
+
+ for i in range(len(a)):
+ a[i] += 3 # b[i] also changes
**Good**:
+It's safer to create a new list object and leave the original alone.
+
.. code-block:: python
a = [3, 4, 5]
+ b = a
+
+ # assign the variable "a" to a new list without changing "b"
a = [i + 3 for i in a]
- # Or:
- a = map(lambda i: i + 3, a)
-Use `enumerate `_ to
-keep a count of your place in the list.
+Use :py:func:`enumerate` keep a count of your place in the list.
.. code-block:: python
+ a = [3, 4, 5]
for i, item in enumerate(a):
- print i + ", " + item
+ print(i, item)
# prints
- # 0, 3
- # 1, 4
- # 2, 5
+ # 0 3
+ # 1 4
+ # 2 5
-The ``enumerate`` function has better readability than handling a counter
-manually. Moreover,
-it is better optimized for iterators.
+The :py:func:`enumerate` function has better readability than handling a
+counter manually. Moreover, it is better optimized for iterators.
Read From a File
~~~~~~~~~~~~~~~~
@@ -570,7 +801,7 @@ files for you.
f = open('file.txt')
a = f.read()
- print a
+ print(a)
f.close()
**Good**:
@@ -579,23 +810,23 @@ files for you.
with open('file.txt') as f:
for line in f:
- print line
+ print(line)
The ``with`` statement is better because it will ensure you always close the
-file, even if an exception is raised.
+file, even if an exception is raised inside the ``with`` block.
Line Continuations
~~~~~~~~~~~~~~~~~~
When a logical line of code is longer than the accepted limit, you need to
-split it over multiple physical lines. Python interpreter will join consecutive
-lines if the last character of the line is a backslash. This is helpful
-sometimes but is preferably avoided, because of its fragility: a white space
-added to the end of the line, after the backslash, will break the code and may
-have unexpected results.
-
-A preferred solution is to use parentheses around your elements. Left with an
-unclosed parenthesis on an end-of-line the Python interpreter will join the
+split it over multiple physical lines. The Python interpreter will join
+consecutive lines if the last character of the line is a backslash. This is
+helpful in some cases, but should usually be avoided because of its fragility:
+a white space added to the end of the line, after the backslash, will break the
+code and may have unexpected results.
+
+A better solution is to use parentheses around your elements. Left with an
+unclosed parenthesis on an end-of-line, the Python interpreter will join the
next line until the parentheses are closed. The same behavior holds for curly
and square braces.
@@ -623,6 +854,6 @@ and square braces.
from some.deep.module.inside.a.module import (
a_nice_function, another_nice_function, yet_another_nice_function)
-However, more often than not having to split long logical line is a sign that
+However, more often than not, having to split a long logical line is a sign that
you are trying to do too many things at the same time, which may hinder
readability.
diff --git a/docs/writing/tests.rst b/docs/writing/tests.rst
index 70c08f577..28eb3260b 100644
--- a/docs/writing/tests.rst
+++ b/docs/writing/tests.rst
@@ -1,87 +1,96 @@
+
+
+#################
Testing Your Code
-=================
+#################
+
+.. image:: /_static/photos/34435687940_8f73fc1fa6_k_d.jpg
Testing your code is very important.
-Getting used to writing the testing code and the running code in parallel is
-now considered a good habit. Used wisely, this method helps you define more
-precisely your code's intent and have a more decoupled architecture.
+Getting used to writing testing code and running this code in parallel is now
+considered a good habit. Used wisely, this method helps to define your
+code's intent more precisely and have a more decoupled architecture.
Some general rules of testing:
- A testing unit should focus on one tiny bit of functionality and prove it
correct.
-- Each test unit must be fully independent. Each of them must be able to run
- alone, and also within the test suite, regardless of the order they are called.
- The implication of this rule is that each test must be loaded with a fresh
- dataset and may have to do some cleanup afterwards. This is usually
- handled by setUp() and tearDown() methods.
+- Each test unit must be fully independent. Each test must be able to run
+ alone, and also within the test suite, regardless of the order that they are
+ called. The implication of this rule is that each test must be loaded with
+ a fresh dataset and may have to do some cleanup afterwards. This is
+ usually handled by :meth:`setUp()` and :meth:`tearDown()` methods.
- Try hard to make tests that run fast. If one single test needs more than a
- few millisecond to run, development will be slowed down or the tests will not
- be run as often as desirable. In some cases, tests can't be fast because they
- need a complex data structure to work on, and this data structure must be
- loaded every time the test runs. Keep these heavier tests in a separate test
- suite that is run by some scheduled task, and run all other tests as often
- as needed.
+ few milliseconds to run, development will be slowed down or the tests will
+ not be run as often as is desirable. In some cases, tests can't be fast
+ because they need a complex data structure to work on, and this data structure
+ must be loaded every time the test runs. Keep these heavier tests in a
+ separate test suite that is run by some scheduled task, and run all other
+ tests as often as needed.
- Learn your tools and learn how to run a single test or a test case. Then,
- when developing a function inside a module, run this function's tests very
- often, ideally automatically when you save the code.
+ when developing a function inside a module, run this function's tests
+ frequently, ideally automatically when you save the code.
- Always run the full test suite before a coding session, and run it again
- after. This will give you more confidence that you did not break anything in
- the rest of the code.
+ after. This will give you more confidence that you did not break anything
+ in the rest of the code.
-- It is a good idea to implement a hook that runs all tests before pushing code
- to a shared repository.
+- It is a good idea to implement a hook that runs all tests before pushing
+ code to a shared repository.
-- If you are in the middle of a development session and have to interrupt your work, it
- is a good idea to write a broken unit test about what you want to develop next.
- When coming back to work, you will have a pointer to where you were and get
- faster on tracks.
+- If you are in the middle of a development session and have to interrupt
+ your work, it is a good idea to write a broken unit test about what you
+ want to develop next. When coming back to work, you will have a pointer
+ to where you were and get back on track faster.
- The first step when you are debugging your code is to write a new test
pinpointing the bug. While it is not always possible to do, those bug
- catching test are among the most valuable piece of code in your project.
+ catching tests are among the most valuable pieces of code in your project.
-- Use long and descriptive names for testing functions. The style guide here is
- slightly different than that of running code, where short names are often
- preferred. The reason is testing functions are never called explicitly.
+- Use long and descriptive names for testing functions. The style guide here
+ is slightly different than that of running code, where short names are
+ often preferred. The reason is testing functions are never called explicitly.
``square()`` or even ``sqr()`` is ok in running code, but in testing code you
- would has names such as ``test_square_of_number_2()``,
- ``test_square_negative_number()``. These function names are displayed when a
- test fail, and should be as descriptive as possible.
+ would have names such as ``test_square_of_number_2()``,
+ ``test_square_negative_number()``. These function names are displayed when
+ a test fails, and should be as descriptive as possible.
-- When something goes wrong or has to be changed, and if your code has a good
- set of tests, you or other maintainers will rely largely on the testing suite
- to fix the problem or modify a given behavior. Therefore the testing code will
- be read as much as or even more than the running code. A unit test whose
- purpose is unclear is not very helpful is this case.
+- When something goes wrong or has to be changed, and if your code has a
+ good set of tests, you or other maintainers will rely largely on the
+ testing suite to fix the problem or modify a given behavior. Therefore
+ the testing code will be read as much as or even more than the running
+ code. A unit test whose purpose is unclear is not very helpful in this
+ case.
- Another use of the testing code is as an introduction to new developers. When
someone will have to work on the code base, running and reading the related
- testing code is often the best they can do. They will or should discover the
- hot spots, where most difficulties arise, and the corner cases. If they have
- to add some functionality, the first step should be to add a test and, by this
- mean, ensure the new functionality is not already a working path that has not
- been plugged in the interface.
+ testing code is often the best thing that they can do to start. They will
+ or should discover the hot spots, where most difficulties arise, and the
+ corner cases. If they have to add some functionality, the first step should
+ be to add a test to ensure that the new functionality is not already a
+ working path that has not been plugged into the interface.
+
+
+**********
The Basics
-::::::::::
+**********
-Unittest
+unittest
--------
-Unittest is the batteries-included test module in the Python standard library.
-Its API will be familiar to anyone who has used any of the JUnit/nUnit/CppUnit
-series of tools.
+:mod:`unittest` is the batteries-included test module in the Python standard
+library. Its API will be familiar to anyone who has used any of the
+JUnit/nUnit/CppUnit series of tools.
-Creating testcases is accomplished by subclassing a TestCase base class
+Creating test cases is accomplished by subclassing :class:`unittest.TestCase`.
-::
+.. code-block:: python
import unittest
@@ -100,7 +109,7 @@ As of Python 2.7 unittest also includes its own test discovery mechanisms.
Doctest
-------
-The doctest module searches for pieces of text that look like interactive
+The :mod:`doctest` module searches for pieces of text that look like interactive
Python sessions in docstrings, and then executes those sessions to verify that
they work exactly as shown.
@@ -112,10 +121,10 @@ test suite runs.
A simple doctest in a function:
-::
+.. code-block:: python
def square(x):
- """Squares x.
+ """Return the square of x.
>>> square(2)
4
@@ -133,8 +142,10 @@ When running this module from the command line as in ``python module.py``, the
doctests will run and complain if anything is not behaving as described in the
docstrings.
+
+*****
Tools
-:::::
+*****
py.test
@@ -142,15 +153,15 @@ py.test
py.test is a no-boilerplate alternative to Python's standard unittest module.
-::
+.. code-block:: console
$ pip install pytest
-Despite being a fully-featured and extensible test tool it boasts a simple
+Despite being a fully-featured and extensible test tool, it boasts a simple
syntax. Creating a test suite is as easy as writing a module with a couple of
-functions
+functions:
-::
+.. code-block:: python
# content of test_sample.py
def func(x):
@@ -159,9 +170,9 @@ functions
def test_answer():
assert func(3) == 5
-and then running the `py.test` command
+and then running the `py.test` command:
-::
+.. code-block:: console
$ py.test
=========================== test session starts ============================
@@ -181,88 +192,81 @@ and then running the `py.test` command
test_sample.py:5: AssertionError
========================= 1 failed in 0.02 seconds =========================
-far less work than would be required for the equivalent functionality with the
-unittest module!
+is far less work than would be required for the equivalent functionality with
+the unittest module!
- `py.test `_
+ `py.test `_
-Nose
-----
+Hypothesis
+----------
-nose extends unittest to make testing easier.
+Hypothesis is a library which lets you write tests that are parameterized by
+a source of examples. It then generates simple and comprehensible examples
+that make your tests fail, letting you find more bugs with less work.
+.. code-block:: console
-::
+ $ pip install hypothesis
- $ pip install nose
+For example, testing lists of floats will try many examples, but report the
+minimal example of each bug (distinguished exception type and location):
-nose provides automatic test discovery to save you the hassle of manually
-creating test suites. It also provides numerous plugins for features such as
-xUnit-compatible test output, coverage reporting, and test selection.
+.. code-block:: python
- `nose `_
+ @given(lists(floats(allow_nan=False, allow_infinity=False), min_size=1))
+ def test_mean(xs):
+ mean = sum(xs) / len(xs)
+ assert min(xs) <= mean(xs) <= max(xs)
+
+.. code-block:: none
+
+ Falsifying example: test_mean(
+ xs=[1.7976321109618856e+308, 6.102390043022755e+303]
+ )
+
+Hypothesis is practical as well as very powerful and will often find bugs
+that escaped all other forms of testing. It integrates well with py.test,
+and has a strong focus on usability in both simple and advanced scenarios.
+
+ `hypothesis `_
tox
---
tox is a tool for automating test environment management and testing against
-multiple interpreter configurations
+multiple interpreter configurations.
-::
+.. code-block:: console
$ pip install tox
tox allows you to configure complicated multi-parameter test matrices via a
-simple ini-style configuration file.
-
- `tox `_
-
-Unittest2
----------
+simple INI-style configuration file.
-unittest2 is a backport of Python 2.7's unittest module which has an improved
-API and better assertions over the one available in previous versions of Python.
-
-If you're using Python 2.6 or below, you can install it with pip
-
-::
-
- $ pip install unittest2
-
-You may want to import the module under the name unittest to make porting code
-to newer versions of the module easier in the future
-
-::
-
- import unittest2 as unittest
-
- class MyTest(unittest.TestCase):
- ...
-
-This way if you ever switch to a newer python version and no longer need the
-unittest2 module, you can simply change the import in your test module without
-the need to change any other code.
-
- `unittest2 `_
+ `tox `_
mock
----
-mock is a library for testing in Python.
+:mod:`unittest.mock` is a library for testing in Python. As of Python 3.3, it is
+available in the
+`standard library `_.
+
+For older versions of Python:
-::
+.. code-block:: console
$ pip install mock
It allows you to replace parts of your system under test with mock objects and
make assertions about how they have been used.
-For example, you can monkey patch a method
+For example, you can monkey-patch a method:
-::
+.. code-block:: python
from mock import MagicMock
thing = ProductionClass()
@@ -275,7 +279,7 @@ To mock classes or objects in a module under test, use the ``patch`` decorator.
In the example below, an external search system is replaced with a mock that
always returns the same result (but only for the duration of the test).
-::
+.. code-block:: python
def mock_search(self):
class MockSearchQuerySet(SearchQuerySet):
@@ -287,10 +291,9 @@ always returns the same result (but only for the duration of the test).
# not where the SearchForm class itself is imported from
@mock.patch('myapp.SearchForm.search', mock_search)
def test_new_watchlist_activities(self):
- # get_search_results runs a search and iterates over the result
+ # get_search_results runs a search and iterates over the result
self.assertEqual(len(myapp.get_search_results(q="fish")), 3)
-Mock has many other ways you can configure it and control its behavior.
+Mock has many other ways with which you can configure and control its behaviour.
`mock `_
-
diff --git a/ext/python-guide-logo.ai b/ext/python-guide-logo.ai
new file mode 100644
index 000000000..4399e98df
--- /dev/null
+++ b/ext/python-guide-logo.ai
@@ -0,0 +1,1493 @@
+%PDF-1.5
%����
+1 0 obj
<>/OCGs[5 0 R]>>/Pages 3 0 R/Type/Catalog>>
endobj
2 0 obj
<>stream
+
+
+
+
+ application/pdf
+
+
+ python-guide-logo
+
+
+ Adobe Illustrator CC 2015 (Macintosh)
+ 2016-02-21T18:27:51-05:00
+ 2016-02-21T18:27:51-05:00
+ 2016-02-21T18:27:51-05:00
+
+
+
+ 256
+ 212
+ JPEG
+ /9j/4AAQSkZJRgABAgEASABIAAD/7QAsUGhvdG9zaG9wIDMuMAA4QklNA+0AAAAAABAASAAAAAEA
AQBIAAAAAQAB/+4ADkFkb2JlAGTAAAAAAf/bAIQABgQEBAUEBgUFBgkGBQYJCwgGBggLDAoKCwoK
DBAMDAwMDAwQDA4PEA8ODBMTFBQTExwbGxscHx8fHx8fHx8fHwEHBwcNDA0YEBAYGhURFRofHx8f
Hx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8f/8AAEQgA1AEAAwER
AAIRAQMRAf/EAaIAAAAHAQEBAQEAAAAAAAAAAAQFAwIGAQAHCAkKCwEAAgIDAQEBAQEAAAAAAAAA
AQACAwQFBgcICQoLEAACAQMDAgQCBgcDBAIGAnMBAgMRBAAFIRIxQVEGE2EicYEUMpGhBxWxQiPB
UtHhMxZi8CRygvElQzRTkqKyY3PCNUQnk6OzNhdUZHTD0uIIJoMJChgZhJRFRqS0VtNVKBry4/PE
1OT0ZXWFlaW1xdXl9WZ2hpamtsbW5vY3R1dnd4eXp7fH1+f3OEhYaHiImKi4yNjo+Ck5SVlpeYmZ
qbnJ2en5KjpKWmp6ipqqusra6voRAAICAQIDBQUEBQYECAMDbQEAAhEDBCESMUEFURNhIgZxgZEy
obHwFMHR4SNCFVJicvEzJDRDghaSUyWiY7LCB3PSNeJEgxdUkwgJChgZJjZFGidkdFU38qOzwygp
0+PzhJSktMTU5PRldYWVpbXF1eX1RlZmdoaWprbG1ub2R1dnd4eXp7fH1+f3OEhYaHiImKi4yNjo
+DlJWWl5iZmpucnZ6fkqOkpaanqKmqq6ytrq+v/aAAwDAQACEQMRAD8A9U4q7FXYq7FXYq7FXYq7
FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7F
XYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FX
Yq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXY
q7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq
7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7
FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7F
XYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FX
Yq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXY
q7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq
7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7
FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7F
XYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FX
Yq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXY
q7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq
7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7
FXYq7FXYq7FXYq7FXYq7FULqWq6XpdsbvU7yCxtQQpnuZEhjDN0HJyoqcVQel+bvK+rXpstK1W1v
7kRtMUtZUmAjVlViWjLLsXXatdxiqbYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXY
q7FXYq7FXYq7FXYq7FXYq7FXYq7FUu8wwPLo156Vl9fukgmNra/uwzytEyBVaUqilg5WpI2JxV5R
feTfPMdhoU+j6beWGsaZ5bWB5La5s4g+po9oiwzkTfvR6Vqa1qnHiK12CrLvy9sPPcOp6lc+aJLk
GVpVSF2he0YfWHaGSHhcTOhEDKhT0oxtvyb4iqzrFXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7F
XYq7FXYq7FXYq7FXYq7FXYq7FXYq87/OLU9W0qLyxdWGqXmnxXOuWlhqEdlGszy21wHMgWP0ppC/
wDjwFfbFUHqvmq9uvzU8sWel3mrRadeWmpXF9pc1pJZJM9ksDQrGb2CAnkZWD8Xp0qVxVQ0H80NX
nXyNFaQ3mtWvmBNRe+vLi3toLwi0cxpRIZUt04OfjpX4QKbnFUF+an5inVvyv1270S01W2sZbCa5
0nzNBIkEEv1eRUJVoZxcor8vg9SMBxXt1VXfmr+YI1DyB5z02C21Xy9ruk6bFqMHrOtrLJBPIyRz
RPaTyNwJjZWR6EdGXtiqdeYvzWt49F8zwvY6xoOqaJpyaixeCxkufqkxdUu7eN55IZAhjbkkhDDo
VrtiqhbeeDpX5h65ZXV3e6qbuw0ibQtBj9Frh5bk3huPQjJhQKqwo0ju/FB1bpiqZ3n5y+VbTXDp
UsdxWK+g0q8ugbYxwXtyFKROnreuwUyIrvHGyKT12biqlOp/mpe69oPmKHy7p2qaLcaP9aivdevb
e3Ntatp86pehOMlwJpo4uTRoFKtTrsaKsw8l+brfzLYfWba0uIbZY4XhupntpknSVSQVktZZ05rx
/eKaFSfAg4qyLFXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYqxjzv5NuvM
zaOYtTOnro9/FqcYWFZvUngr6Qcsy/AORqBufHFVut+Tb7UfOWkeZotUFu+iw3MFpZm3EiEXgjEx
duaMa+itKUpiqT6L+VE2j2nleGz1yRJPLEt36c31eM+vb37c5opFZmCtX7Lr0HY4qhpfyduT5Lv/
ACRD5hlXytcxPb2lrJbRPPbxSSeoUE4Kc1TolVqB1LbYqivOf5T/AOKbnUri41U20msaNDoeoCOA
MPThuHuPVirJ8DM0pFG5CmKqnmz8rm8x3eqXc2qfV5dZ0NPL9+EgDL6azSTNNEDJ8DEzsKMWA98V
QWq/kvb6rq8us3mqtHq621nBpmoWkHoXFlNYep6U0LmSSvP13EsbVVxQU2xVG6R+WV3pGvXmp2Wt
fudUuY9Q1S0lsoJOd2qIkskMh+KET+kOa/F/kFTviqO0DyJd6RoXmDTI9WYy67c318t5HAqPbXGo
s7yNGrNIrBHeqBvDeuKr/JP5fad5XvdU1CAWyXusCAXiWFstjaf6MHCMlurSUdvVYuxYk+wGKsrx
V2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KoTVbm9trCWWxtPrt58
KW9sZFhRndggMkrV4Rry5OVVm4g8UdqKVWN3fmnX9Lu9TvNT0e7ksLHTbeVrTTxJfyTXhJ5Jp8cV
ojTK7SemzyzIy+mGMMcZ9V1WL+YPzP8AM1rr0/lby/bwXOtpPLWG+LXuoLH9YjcSix0pPQitjBcI
lu99d2paoMrAKzsqgri886jTkl1ubUdPuY9Sms7Ge9ae+vHintXdkk03y+dFhuP3caSRvbSXjREv
Xh6cjRqsy8hQ+drAyaTr2qafrkFm1xEbu0EqXFoySJJaW9z681zJM8tncI/JyHTiObTl/VxVmWKu
xV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KvPZ/zbXUvMFz5f8ja
S/mm/wBObhqt4J1tNOtHrT05Ltkl5yVH2Ykbv4HFXav5w/NXQbCfUtS8oWGo2NujTXA0jVHknjjQ
VY+ldWlsJKDf4XxVmHlnXrbzD5d0zXbWKSC21W1hvIIpwokWOdBInIKWWvFuxxVEfonSv0V+iPqc
H6J9D6p+j/ST6v8AV+Hp+j6VOHp8Ph40pTbFUq80XGkeXfJ1zdnSReadokCXFvpNrCjUFkVkgWCK
nFTE0aslB8PEEdMVV/KHmNPMvlnTteSznsF1CETCzul4zRgkijD6KjxGKqvmXUNX07Rri90mwj1O
8gUutpLcfVVZVBLfvfTmodv5cVYv+S/5jXv5heTP8SXdlHp5lupoIraJ2k4xxcQOTsFq1a9AMVZ3
irsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirCvzq8wX/l/8qvMurWDt
FewWbJbyr9pHmZYQ6+6epyGKpH/zjPo1jpn5M6A1qFMmoLLe3copV5pZWBLEdSqqqf7HFXp80Uc0
TwyqHikUo6HcFWFCD8xiryr87/O3mv8ALPyTDqPlay039E2iw2KJc+s0kRascYiiTihVFUfaf6MV
Tj8x/N/nHyz+XLeZtJgsLu4sbRbnURemZVPwr/dJEN6sxNC64qlr+fPPv/KjrPzzYQaddawNMbVt
RjufVjh9FYWncRJGSSwAFAXHzxVX0H82FsvyRs/zE82lPUe1+s3EVonEO8spSGGJGY7mqru3uTTF
Uv1Of8+tU8sP5gsZNF0wzWzTp5XuIJpn9F05BJr0SJ++4fyxqoOxPfFWN/8AONfmPTvLX/OPFxr+
pMVstNnvrian2mCEEIv+Uxoq+5xVkP8AiH85NZ8gr580m70vTFktW1Ox8tzWkk/qWYT1Y0mvPVjP
qSRjkOEaruB/lYqzX8svPdp578k6b5ntoGtReq4mtWPIxyxO0cihtuS8kPE9xTp0xVlGKuxV2Kux
V2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KpT5u8t2fmbyxqnl+8YpbapbS2zyKKsnqK
QrqNt0ajD5Yq8G/KXzzf/lDHJ+Xf5mxSabY280jaB5hCPJYyxyuWaP1VB4guS616VIbjTFUf+f3m
r8lvMPku/ntNR0vWPN5tkh0KSykS5vVkModFjaEs6bkkg0xVKfP/AJRvYf8AnEPSrKz054Lizt7G
+v7QRMsoZmDXEjpTlXlIXevQVxVF/mJ+d/kvzT+Sep6b5bNxqep3GmIL21jglQWKAL6sl1K6rEoQ
gqOLHm1AtRviqMtPPXlRP+cUngj1mx+tjy6+ltbNcRCQXb2jR/V+HIN6pFSqdT1pTFUhXTH8/f8A
OJdlovlmeK/1rR4YZrvTIZEkuAbeZyY2jUlld0UvGrCrU264qzHRP+cj/KnmDyaLWzhurrzvLaNA
/luO3l9X60sfFyZePoxwK3xNIzjivUcvhxVgH5P2tt5v/wCcZPMHkjSLyCXzMTczLpnqKJyFljmj
PBiDxkKhA/2anriqK/KfzP8A849p5Pt9I85aTYaL5p02MWWsWt/aFJJ3hHD1SQnxtIPtKfi5V27l
V9DeSo/LqeWrJvLunDStGkVpLSzFsbOisx+MwMqMvqfb+IVNanFU7xV2KuxV2KuxV2KuxV2KuxV2
KuxV2KuxV2KuxV2KuxV2KuxV2KuxVD6hqOn6dZTX2oXMVnZW6857q4dYoo1/md3IVR8zirGpvzS/
KaaNopvN2gyxN9pH1GzZT33BkxVMPK2o+Rb+OWTyrdaXdxxmk76XJbyBS3ZzATSvHv4Yqxzzr5l8
z6d+ZXlHTdP9E6Te2mqXF/DLM0QkNqsHxNwgnb90kpKAH4yd6cQcVQXl38zriaHyzqs2i29j5d85
XTWulzQylrmKRo5ZbY3UQjCEzrCa8GPA7Et1xV6Y7BUZiCQoJoBU7eAGKsF8p/nBofmbzvqHk+00
3UrPUtLtfrl62oQC2ChmjCJwZzLyZZgwqgFMVZwkECSPKkarJJT1HAAZqdKnviq/FVF7OzecTvBG
06/ZlKKXFPBqVxVWxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KtOiOjI
6hkYEMpFQQdiCDir56/InQdLl/OT83JJLSB4rbUUiijaJWC+vPcs3Go2B9PemKoK903TtL/5yptb
jyRblLWw0qW5842umxlogwjm/dtDFsZZP3NEA+3xPWuKvQtK84flx+ZHm6yOlyaqdc8rSTK8scF3
Zrb+vtLBdMyIqiU23Hg25K0HfFUv8tz/AJUaV5vj8uWy6rHdeXTcX1rpt/DdLp+mI4cyXSvKixRx
cWcJI7lQD8HXFWM/lv8AnroT/mV59bzB5pe60d7m1h8rqFmkthAGnD+kkKMopWNS53bxOKqo816B
5V/5yW88arrNwLe2/QtlGiqrSSzTMLYJFFGgLySNTYKPwGKvR/I351eRPOerXGi6ZcT22tWys8mm
X8D21xxU0LKr7GlakA1A6jFU18x/mFoGh6pBorLc6lr1zE1xDo+nQtc3PoKaGVwtEiTlsGkZQTsM
VQflP82PKnmTW7jy/ELrS/MVqvqS6Lqlu9pden/OivVXXv8ACx236Yqlcn/OQX5WJqN7p66nI9zY
QCedBbTqxZmRY4Yo3RJJZX9QFVjU7VJIxVF+Q/zp8iedtUudH0q4nt9YtFLy6bfQvbT8FIBZVbY0
qKitR3GKs7xV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV87flDo2ia7+an5
yaPrFml7aTapbzelIPhBhnuqGoIINX2pir3Ty75V8teW7I2WgaZbaZbMeTx20ax82/mcgVdvdiTi
rxbVprP8sf8AnI+LV7h1tPLP5gWjx3cr/DFFqFvQlz/rNx3PeRsVegflHp01xY6p52vo2j1Lzlc/
pBUkFHi09F9LT4D/AKtuA5/ynOKsS/IcJ/ysr83iAOZ1uME96Brin8cVQvl23spf+cvfNMk0Ye5t
9AhktWYA8CRaRuyk7huL8ajsTiqzzlBDF/zlz5CliQJJcaTdi4dRQycLe9C86daUHXFW/wAmtQ9T
8/fzXt9SJbV3mtzZ+p9sWUDPHRa/s8Hg/DFVL87bGRfz5/KW80kU1me6mivGSvM2MMkTPy478fSl
n/HFUZ5atLWX/nLPzdcSxK81todqbeRgCYy626sVr0JXavhUd8VUfN6R2/8Azlx5GkgRY5L3SLtb
p1ABkCQXhXke5HAb+AxV7tirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVQ+oWj
3dlNbJcy2byrxFzblBKnuhdZFr81OKsC8p/khoHlbzHd+YdL1nWDqGoy+tqnr3EUiXTcy59ZTDvV
mPQg77HFXouKvKfzy8uaH51n8peTbhRNfX2prfgoRyisLRGN5IxH7LqyxL4swP7OKvVI444o1jjU
JGgCoigBVUCgAA6AYq89f8jfLA883fnCy1TWdMu9Rljn1PT7C9a2s7p4zX98kaiVgx3ZfUpufHFV
2mfkto2nec5/OcOuaw/mC7jSG9upJrYiaFPT/dOgtwvE+ggPGh264qjvNP5U6H5h84ab5vkv9Q0/
XNJga2sbixliQIj+pyJWSKWpYTMprtTFUN5s/JzQte8xW3miz1C+0HzTaoIv0zpkiRyzRgceFwjp
JHIOO32fwFMVTLy9+XWl6VrkvmO9vLrXPMs0Itm1fUTEZI4QamKCKCOGGFCTUhEBPcnFUq0r8mtI
03zpL5zi1zWJdeuljivZ5ZrdlnhjMZEMiCBV4EQqDxoffFUX5o/KjQ/MHnLTvOEmoahp+u6Vbm1s
Z7KWJFRG9TkeMkUoJYTsp9sVZrirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVd
irsVUbzh9Tn9Tn6fpvz9Ll6lOJrw4/Fy8Kb4q8e/5x9/5Upzvf8AAv1r9L+gnr/pX1Prn1Ll+79L
1Ph9DlT+72rTlvTFXs+KuxV2KuxV2KuxV2KuxV//2Q==
+
+
+
+ proof:pdf
+ uuid:65E6390686CF11DBA6E2D887CEACB407
+ xmp.did:12e70db7-1325-44c8-8488-d9e000f2ee97
+ uuid:1b5ef070-f5d2-0d46-a8ca-073615f2bf19
+
+ uuid:c5688439-e292-b24c-adf4-afab40481673
+ xmp.did:03801174072068118C14DE8EAF56EBAC
+ uuid:65E6390686CF11DBA6E2D887CEACB407
+ proof:pdf
+
+
+
+
+ saved
+ xmp.iid:03801174072068118C14DE8EAF56EBAC
+ 2011-06-23T03:13:45-04:00
+ Adobe Illustrator CS5.1
+ /
+
+
+ saved
+ xmp.iid:12e70db7-1325-44c8-8488-d9e000f2ee97
+ 2016-02-21T18:27:46-05:00
+ Adobe Illustrator CC 2015 (Macintosh)
+ /
+
+
+
+
+
+
+ EmbedByReference
+
+ /Users/kreitz/Library/Caches/TemporaryItems/page3f.gif
+
+
+
+
+
+
+
+ /Users/kreitz/Library/Caches/TemporaryItems/page3f.gif
+
+
+
+ Web
+ Document
+ 1
+ True
+ False
+
+ 300.000000
+ 300.000000
+ Pixels
+
+
+
+ Cyan
+ Magenta
+ Yellow
+ Black
+
+
+
+
+
+ Default Swatch Group
+ 0
+
+
+
+ White
+ RGB
+ PROCESS
+ 255
+ 255
+ 255
+
+
+ Black
+ RGB
+ PROCESS
+ 0
+ 0
+ 0
+
+
+ RGB Red
+ RGB
+ PROCESS
+ 255
+ 0
+ 0
+
+
+ RGB Yellow
+ RGB
+ PROCESS
+ 255
+ 255
+ 0
+
+
+ RGB Green
+ RGB
+ PROCESS
+ 0
+ 255
+ 0
+
+
+ RGB Cyan
+ RGB
+ PROCESS
+ 0
+ 255
+ 255
+
+
+ RGB Blue
+ RGB
+ PROCESS
+ 0
+ 0
+ 255
+
+
+ RGB Magenta
+ RGB
+ PROCESS
+ 255
+ 0
+ 255
+
+
+ R=193 G=39 B=45
+ RGB
+ PROCESS
+ 193
+ 39
+ 45
+
+
+ R=237 G=28 B=36
+ RGB
+ PROCESS
+ 237
+ 28
+ 36
+
+
+ R=241 G=90 B=36
+ RGB
+ PROCESS
+ 241
+ 90
+ 36
+
+
+ R=247 G=147 B=30
+ RGB
+ PROCESS
+ 247
+ 147
+ 30
+
+
+ R=251 G=176 B=59
+ RGB
+ PROCESS
+ 251
+ 176
+ 59
+
+
+ R=252 G=238 B=33
+ RGB
+ PROCESS
+ 252
+ 238
+ 33
+
+
+ R=217 G=224 B=33
+ RGB
+ PROCESS
+ 217
+ 224
+ 33
+
+
+ R=140 G=198 B=63
+ RGB
+ PROCESS
+ 140
+ 198
+ 63
+
+
+ R=57 G=181 B=74
+ RGB
+ PROCESS
+ 57
+ 181
+ 74
+
+
+ R=0 G=146 B=69
+ RGB
+ PROCESS
+ 0
+ 146
+ 69
+
+
+ R=0 G=104 B=55
+ RGB
+ PROCESS
+ 0
+ 104
+ 55
+
+
+ R=34 G=181 B=115
+ RGB
+ PROCESS
+ 34
+ 181
+ 115
+
+
+ R=0 G=169 B=157
+ RGB
+ PROCESS
+ 0
+ 169
+ 157
+
+
+ R=41 G=171 B=226
+ RGB
+ PROCESS
+ 41
+ 171
+ 226
+
+
+ R=0 G=113 B=188
+ RGB
+ PROCESS
+ 0
+ 113
+ 188
+
+
+ R=46 G=49 B=146
+ RGB
+ PROCESS
+ 46
+ 49
+ 146
+
+
+ R=27 G=20 B=100
+ RGB
+ PROCESS
+ 27
+ 20
+ 100
+
+
+ R=102 G=45 B=145
+ RGB
+ PROCESS
+ 102
+ 45
+ 145
+
+
+ R=147 G=39 B=143
+ RGB
+ PROCESS
+ 147
+ 39
+ 143
+
+
+ R=158 G=0 B=93
+ RGB
+ PROCESS
+ 158
+ 0
+ 93
+
+
+ R=212 G=20 B=90
+ RGB
+ PROCESS
+ 212
+ 20
+ 90
+
+
+ R=237 G=30 B=121
+ RGB
+ PROCESS
+ 237
+ 30
+ 121
+
+
+ R=199 G=178 B=153
+ RGB
+ PROCESS
+ 199
+ 178
+ 153
+
+
+ R=153 G=134 B=117
+ RGB
+ PROCESS
+ 153
+ 134
+ 117
+
+
+ R=115 G=99 B=87
+ RGB
+ PROCESS
+ 115
+ 99
+ 87
+
+
+ R=83 G=71 B=65
+ RGB
+ PROCESS
+ 83
+ 71
+ 65
+
+
+ R=198 G=156 B=109
+ RGB
+ PROCESS
+ 198
+ 156
+ 109
+
+
+ R=166 G=124 B=82
+ RGB
+ PROCESS
+ 166
+ 124
+ 82
+
+
+ R=140 G=98 B=57
+ RGB
+ PROCESS
+ 140
+ 98
+ 57
+
+
+ R=117 G=76 B=36
+ RGB
+ PROCESS
+ 117
+ 76
+ 36
+
+
+ R=96 G=56 B=19
+ RGB
+ PROCESS
+ 96
+ 56
+ 19
+
+
+ R=66 G=33 B=11
+ RGB
+ PROCESS
+ 66
+ 33
+ 11
+
+
+
+
+
+ Grays
+ 1
+
+
+
+ R=0 G=0 B=0
+ RGB
+ PROCESS
+ 0
+ 0
+ 0
+
+
+ R=26 G=26 B=26
+ RGB
+ PROCESS
+ 26
+ 26
+ 26
+
+
+ R=51 G=51 B=51
+ RGB
+ PROCESS
+ 51
+ 51
+ 51
+
+
+ R=77 G=77 B=77
+ RGB
+ PROCESS
+ 77
+ 77
+ 77
+
+
+ R=102 G=102 B=102
+ RGB
+ PROCESS
+ 102
+ 102
+ 102
+
+
+ R=128 G=128 B=128
+ RGB
+ PROCESS
+ 128
+ 128
+ 128
+
+
+ R=153 G=153 B=153
+ RGB
+ PROCESS
+ 153
+ 153
+ 153
+
+
+ R=179 G=179 B=179
+ RGB
+ PROCESS
+ 179
+ 179
+ 179
+
+
+ R=204 G=204 B=204
+ RGB
+ PROCESS
+ 204
+ 204
+ 204
+
+
+ R=230 G=230 B=230
+ RGB
+ PROCESS
+ 230
+ 230
+ 230
+
+
+ R=242 G=242 B=242
+ RGB
+ PROCESS
+ 242
+ 242
+ 242
+
+
+
+
+
+ Web Color Group
+ 1
+
+
+
+ R=63 G=169 B=245
+ RGB
+ PROCESS
+ 63
+ 169
+ 245
+
+
+ R=122 G=201 B=67
+ RGB
+ PROCESS
+ 122
+ 201
+ 67
+
+
+ R=255 G=147 B=30
+ RGB
+ PROCESS
+ 255
+ 147
+ 30
+
+
+ R=255 G=29 B=37
+ RGB
+ PROCESS
+ 255
+ 29
+ 37
+
+
+ R=255 G=123 B=172
+ RGB
+ PROCESS
+ 255
+ 123
+ 172
+
+
+ R=189 G=204 B=212
+ RGB
+ PROCESS
+ 189
+ 204
+ 212
+
+
+
+
+
+
+ Adobe PDF library 15.00
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
endstream
endobj
3 0 obj
<>
endobj
7 0 obj
<>/Resources<>/ExtGState<>/ProcSet[/PDF/ImageC/ImageI]/Properties<>/XObject<>>>/Thumb 15 0 R/TrimBox[0.0 0.0 300.0 300.0]/Type/Page>>
endobj
8 0 obj
<>stream
+H�lWK�d�
��)�2[�(�ں
xe�>@��E� �?���^V�&[Qғ(~��o�~}���v��߯�o�v�V�?��<�~�����o���q5�w������?������G���s�˧��v}����/�g{e�~jhv��M�F\��������>��Z��^��a[Z:0��E�e>���/��!Ι�5�<֫G��մ��e8�k"a������1�#�k�q���uf�iÐnɹ00��m�X�_����a[\���{�'n��p����(k6����\�
���sኳp� �����N0�B��Z�Ӓ�����o�0�]��b��_��{ǵR���x�~�t�^��'Pjs˼0��!B���7�����~��\
+g