<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Per Wagner Nielsen on Medium]]></title>
        <description><![CDATA[Stories by Per Wagner Nielsen on Medium]]></description>
        <link>https://medium.com/@perwagnernielsen?source=rss-413a7e290f04------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*jCX2ux-QWUTGtqMlE6Qnmg.jpeg</url>
            <title>Stories by Per Wagner Nielsen on Medium</title>
            <link>https://medium.com/@perwagnernielsen?source=rss-413a7e290f04------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 20:14:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@perwagnernielsen/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Vagrant, Ansible and Postgresql 10]]></title>
            <link>https://medium.com/@perwagnernielsen/vagrant-ansible-and-postgresql-10-398e1e472349?source=rss-413a7e290f04------2</link>
            <guid isPermaLink="false">https://medium.com/p/398e1e472349</guid>
            <category><![CDATA[ansible]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[vagrant]]></category>
            <dc:creator><![CDATA[Per Wagner Nielsen]]></dc:creator>
            <pubDate>Sun, 14 Apr 2019 13:17:07 GMT</pubDate>
            <atom:updated>2019-04-14T13:17:07.935Z</atom:updated>
            <content:encoded><![CDATA[<p>In this story I’ll be setting up Vagrant with a Postgresql server provisioned by Ansible. The setup will be pretty bare bone, so don’t use it as a blueprint for a production setup.</p><h4>Setup Vagrant</h4><p>Create a file named ‘Vagrantfile’ in the root directory of your project. This file holds all the information about the Vagrant box we are going to create.</p><pre>(Vagrantfile)</pre><pre># -*- mode: ruby -*-<br># vi: set ft=ruby :</pre><pre>Vagrant.configure(2) do |config|<br>  config.vm.box = &quot;bento/ubuntu-18.04&quot;<br>  config.vm.network &quot;forwarded_port&quot;, guest: 5432, host: 5433, host_ip: &quot;127.0.0.1&quot;<br>  config.vm.network &quot;private_network&quot;, ip: &quot;192.168.60.70&quot;<br>  config.vm.provider &quot;virtualbox&quot; do |vb|<br>     vb.memory = &quot;2048&quot;<br>  end<br>end</pre><p><strong>config.vm.box = “bento/ubuntu-18.04”</strong><br>-&gt; Tells Vagrant to base itself on a ubuntu 18.04 installation</p><p><strong>config.vm.network “forwarded_port”, guest: 5432, host: 5433, host_ip: “127.0.0.1”</strong><br>-&gt; Forwards the port 5432 on the Vagrant box to port 5433 on the host machine (the machine running Vagrant). We forward port 5432, which is the default port for postgresql to a different port on our host machine to avoid any collision in case we have a local installation of postgresql running (on the host machine). This is relevant when you access the Vagrant box as localhost (127.0.0.1), not when you use the “private_network”.</p><p><strong>config.vm.network “private_network”, ip: “192.168.60.70”</strong><br>-&gt; setup a private network so we can access the Vagrant box through the specified IP on your host machine (instead of using localhost).</p><p>You can now start up the Vagrant box by typing:</p><pre>$ vagrant up</pre><p>The first time we power up Vagrant it will take a bit of time, but subsequent start ups will be much faster as the image (ubuntu) will already be downloaded.</p><p>You can also log in to the Vagrant box by typing:</p><pre>$ vagrant ssh</pre><h4>Setup Ansible</h4><p>We want to do several things when setting up Ansible. <br>1. create a inventory file (hosts_vagrant) which specifies the server that Ansible should work on.<br>2. create a playbook (playbook.yml) with all the tasks that needs to be run on the Vagrant box.<br>3. create a task for updating the Vagrant box with basic packages<br>4. create a task for installing postgresql<br>5. create specific tasks to setup postgresql configuration</p><p><strong>1 Create the inventory file “hosts_vagrant”:</strong></p><pre>(hosts_vagrant)</pre><pre>[all:vars]<br>env=vagrant</pre><pre>[postgresdb]<br>192.168.60.70</pre><pre>[postgresdb:vars]<br>  ansible_user=vagrant <br>  ansible_private_key_file=&quot;./.vagrant/machines/default/virtualbox/private_key&quot;</pre><p>We are telling Ansible the IP address of our box as well as some login information specific for Vagrant.</p><p><strong>2 Create playbook “playbook.yml”</strong></p><pre>(playbook.yml)</pre><pre>---<br>- name: postgresdb<br>  hosts: all<br>  become: yes<br>  gather_facts: False<br>  pre_tasks:<br>    - name: Install python for Ansible<br>      raw: test -e /usr/bin/python || (apt -y update &amp;&amp; apt install -y python-minimal)<br>      changed_when: False<br>    - setup: # aka gather_facts</pre><p>We setup a few basic things like creating python for Ansible.</p><p>Run the below commands to ensure that you can actually connect to the Vagrant box.</p><pre>$ ansible all -m ping -i hosts_vagrant<br>$ ansible -i ./hosts_vagrant postgresdb -m ping<br>$ ansible-playbook -i ./hosts_vagrant playbook.yml</pre><h3>3, 4 Install base packages and Postgresql</h3><p>Update the playbook to contain the 2 roles below.</p><pre>(playbook.yml)</pre><pre>---<br>- name: postgresdb<br>  hosts: all<br>  become: yes<br>  gather_facts: False<br>  pre_tasks:<br>    - name: Install python for Ansible<br>      raw: test -e /usr/bin/python || (apt -y update &amp;&amp; apt install -y python-minimal)<br>      changed_when: False<br>    - setup: # aka gather_facts</pre><pre>roles:<br>    - base<br>    - postgresql</pre><p>Now, create the file ‘main.yml’ in the subfolder specified.</p><pre>(roles/base/tasks/main.yml)</pre><pre>- name: Install base dependencies<br>  apt:<br>    name: &quot;{{ packages }}&quot;<br>  vars:<br>    packages:<br>      - locales<br>      - build-essential<br>      - acl<br>      - ntp<br>      - htop<br>      - git<br>      - python-pip<br>      - python3-pip<br>      - libpq-dev<br>    state: present<br>  tags:<br>    - packages</pre><pre>- name: Upgrade pip<br>  pip: name=pip state=latest<br>  tags:<br>    - packages</pre><p>Also create the task to setup the postgresql.</p><pre>(roles/postgresql/tasks/main.yml)</pre><pre>- name: Install postgresql<br>  apt:<br>    name: &quot;{{ packages }}&quot;<br>  vars:<br>    packages:<br>      - postgresql<br>      - postgresql-contrib<br>      - postgresql-server-dev-10<br>      - python-psycopg2<br>    state: present<br>  tags:<br>    - postgres</pre><pre>- name: Enable passwordless sudo<br>  lineinfile: dest=/etc/sudoers regexp=^vagrant line=&quot;vagrant ALL=(postgres) NOPASSWD:/bin/sh&quot;</pre><pre>- name: Create a PostgreSQL database user<br>  postgresql_user: name=project password=project role_attr_flags=CREATEDB state=present<br>  become: yes<br>  become_user: postgres<br>  become_method: sudo</pre><pre>- name: Create acme database<br>  become: yes<br>  become_user: postgres<br>  postgresql_db:<br>    name: acme<br>    encoding: UTF-8</pre><p><strong>5 Need to change these files in Vagrant</strong></p><pre>(pg_hba.conf), add:</pre><pre>host all all 0.0.0.0/0 trust</pre><pre>(postgresql.conf) update:</pre><pre>listen_addresses = &#39;*&#39;</pre><p>Restart server:</p><pre>sudo service postgresql restart</pre><p>Now, to put this into Ansible we will create 2 templates and have Ansible copy them to the Vagrant box.</p><p>Create the file “pg_hba.conf.j2” in ‘roles/postgresql/templates’</p><pre># Database administrative login by Unix domain socket<br>local   all             postgres                                peer</pre><pre># TYPE  DATABASE        USER            ADDRESS                 METHOD</pre><pre># &quot;local&quot; is for Unix domain socket connections only<br>local   all             all                                     peer<br>host    all             all             0.0.0.0/0               trust<br>host    all             all             all                     trust<br># IPv4 local connections:<br>host    all             all             127.0.0.1/32            trust<br>host    all             all             192.168.60.70/24        trust<br># IPv6 local connections:<br>host    all             all             ::1/128                 ident<br># Allow replication connections from localhost, by a user with the<br># replication privilege.<br>local   replication     all                                     peer<br>host    replication     all             127.0.0.1/32            md5<br>host    replication     all             ::1/128                 md5</pre><p>And the file “postgresql.conf.j2” in ‘roles/postgresql/templates’</p><pre># The default values of these variables are driven from the -D command-line<br># option or PGDATA environment variable, represented here as ConfigDir.</pre><pre>data_directory = &#39;/var/lib/postgresql/10/main&#39;  # use data in another directory<br>     # (change requires restart)<br>hba_file = &#39;/etc/postgresql/10/main/pg_hba.conf&#39; # host-based authentication file<br>     # (change requires restart)<br>ident_file = &#39;/etc/postgresql/10/main/pg_ident.conf&#39; # ident configuration file<br>     # (change requires restart)</pre><pre># If external_pid_file is not explicitly set, no extra PID file is written.<br>external_pid_file = &#39;/var/run/postgresql/10-main.pid&#39;   # write an extra PID file<br>     # (change requires restart)</pre><pre>#------------------------------------------------------------------------------<br># CONNECTIONS AND AUTHENTICATION<br>#------------------------------------------------------------------------------</pre><pre># - Connection Settings -</pre><pre>listen_addresses = &#39;*&#39;  # what IP address(es) to listen on;<br>     # comma-separated list of addresses;<br>     # defaults to &#39;localhost&#39;; use &#39;*&#39; for all<br>     # (change requires restart)<br>port = 5432    # (change requires restart)<br>max_connections = 100   # (change requires restart)<br>#superuser_reserved_connections = 3 # (change requires restart)<br>unix_socket_directories = &#39;/var/run/postgresql&#39; # comma-separated list of directories<br>     # (change requires restart)<br>#unix_socket_group = &#39;&#39;   # (change requires restart)<br>#unix_socket_permissions = 0777  # begin with 0 to use octal notation<br>     # (change requires restart)<br>#bonjour = off    # advertise server via Bonjour<br>     # (change requires restart)<br>#bonjour_name = &#39;&#39;   # defaults to the computer name<br>     # (change requires restart)</pre><pre># - Security and Authentication -</pre><pre>#authentication_timeout = 1min  # 1s-600s<br>ssl = on<br>#ssl_ciphers = &#39;HIGH:MEDIUM:+3DES:!aNULL&#39; # allowed SSL ciphers<br>#ssl_prefer_server_ciphers = on<br>#ssl_ecdh_curve = &#39;prime256v1&#39;<br>#ssl_dh_params_file = &#39;&#39;<br>ssl_cert_file = &#39;/etc/ssl/certs/ssl-cert-snakeoil.pem&#39;<br>ssl_key_file = &#39;/etc/ssl/private/ssl-cert-snakeoil.key&#39;<br>#ssl_ca_file = &#39;&#39;<br>#ssl_crl_file = &#39;&#39;<br>#password_encryption = md5  # md5 or scram-sha-256<br>#db_user_namespace = off<br>#row_security = on</pre><pre># GSSAPI using Kerberos<br>#krb_server_keyfile = &#39;&#39;<br>#krb_caseins_users = off</pre><pre># - TCP Keepalives -<br># see &quot;man 7 tcp&quot; for details</pre><pre>#tcp_keepalives_idle = 0  # TCP_KEEPIDLE, in seconds;<br>     # 0 selects the system default<br>#tcp_keepalives_interval = 0  # TCP_KEEPINTVL, in seconds;<br>     # 0 selects the system default<br>#tcp_keepalives_count = 0  # TCP_KEEPCNT;<br>     # 0 selects the system default</pre><pre>#------------------------------------------------------------------------------<br># RESOURCE USAGE (except WAL)<br>#------------------------------------------------------------------------------</pre><pre># - Memory -</pre><pre>shared_buffers = 128MB   # min 128kB<br>     # (change requires restart)<br>#huge_pages = try   # on, off, or try<br>     # (change requires restart)<br>#temp_buffers = 8MB   # min 800kB<br>#max_prepared_transactions = 0  # zero disables the feature<br>     # (change requires restart)<br># Caution: it is not advisable to set max_prepared_transactions nonzero unless<br># you actively intend to use prepared transactions.<br>#work_mem = 4MB    # min 64kB<br>#maintenance_work_mem = 64MB  # min 1MB<br>#replacement_sort_tuples = 150000 # limits use of replacement selection sort<br>#autovacuum_work_mem = -1  # min 1MB, or -1 to use maintenance_work_mem<br>#max_stack_depth = 2MB   # min 100kB<br>dynamic_shared_memory_type = posix # the default is the first option<br>     # supported by the operating system:<br>     #   posix<br>     #   sysv<br>     #   windows<br>     #   mmap<br>     # use none to disable dynamic shared memory<br>     # (change requires restart)</pre><pre># - Disk -</pre><pre>#temp_file_limit = -1   # limits per-process temp file space<br>     # in kB, or -1 for no limit</pre><pre># - Kernel Resource Usage -</pre><pre>#max_files_per_process = 1000  # min 25<br>     # (change requires restart)<br>#shared_preload_libraries = &#39;&#39;  # (change requires restart)</pre><pre># - Cost-Based Vacuum Delay -</pre><pre>#vacuum_cost_delay = 0   # 0-100 milliseconds<br>#vacuum_cost_page_hit = 1  # 0-10000 credits<br>#vacuum_cost_page_miss = 10  # 0-10000 credits<br>#vacuum_cost_page_dirty = 20  # 0-10000 credits<br>#vacuum_cost_limit = 200  # 1-10000 credits</pre><pre># - Background Writer -</pre><pre>#bgwriter_delay = 200ms   # 10-10000ms between rounds<br>#bgwriter_lru_maxpages = 100  # 0-1000 max buffers written/round<br>#bgwriter_lru_multiplier = 2.0  # 0-10.0 multiplier on buffers scanned/round<br>#bgwriter_flush_after = 512kB  # measured in pages, 0 disables</pre><pre># - Asynchronous Behavior -</pre><pre>#effective_io_concurrency = 1  # 1-1000; 0 disables prefetching<br>#max_worker_processes = 8  # (change requires restart)<br>#max_parallel_workers_per_gather = 2 # taken from max_parallel_workers<br>#max_parallel_workers = 8  # maximum number of max_worker_processes that<br>     # can be used in parallel queries<br>#old_snapshot_threshold = -1  # 1min-60d; -1 disables; 0 is immediate<br>     # (change requires restart)<br>#backend_flush_after = 0  # measured in pages, 0 disables</pre><pre>#------------------------------------------------------------------------------<br># WRITE AHEAD LOG<br>#------------------------------------------------------------------------------</pre><pre># - Settings -</pre><pre>#wal_level = replica   # minimal, replica, or logical<br>     # (change requires restart)<br>#fsync = on    # flush data to disk for crash safety<br>     # (turning this off can cause<br>     # unrecoverable data corruption)<br>#synchronous_commit = on  # synchronization level;<br>     # off, local, remote_write, remote_apply, or on<br>#wal_sync_method = fsync  # the default is the first option<br>     # supported by the operating system:<br>     #   open_datasync<br>     #   fdatasync (default on Linux)<br>     #   fsync<br>     #   fsync_writethrough<br>     #   open_sync<br>#full_page_writes = on   # recover from partial page writes<br>#wal_compression = off   # enable compression of full-page writes<br>#wal_log_hints = off   # also do full page writes of non-critical updates<br>     # (change requires restart)<br>#wal_buffers = -1   # min 32kB, -1 sets based on shared_buffers<br>     # (change requires restart)<br>#wal_writer_delay = 200ms  # 1-10000 milliseconds<br>#wal_writer_flush_after = 1MB  # measured in pages, 0 disables</pre><pre>#commit_delay = 0   # range 0-100000, in microseconds<br>#commit_siblings = 5   # range 1-1000</pre><pre># - Checkpoints -</pre><pre>#checkpoint_timeout = 5min  # range 30s-1d<br>#max_wal_size = 1GB<br>#min_wal_size = 80MB<br>#checkpoint_completion_target = 0.5 # checkpoint target duration, 0.0 - 1.0<br>#checkpoint_flush_after = 256kB  # measured in pages, 0 disables<br>#checkpoint_warning = 30s  # 0 disables</pre><pre># - Archiving -</pre><pre>#archive_mode = off  # enables archiving; off, on, or always<br>    # (change requires restart)<br>#archive_command = &#39;&#39;  # command to use to archive a logfile segment<br>    # placeholders: %p = path of file to archive<br>    #               %f = file name only<br>    # e.g. &#39;test ! -f /mnt/server/archivedir/%f &amp;&amp; cp %p /mnt/server/archivedir/%f&#39;<br>#archive_timeout = 0  # force a logfile segment switch after this<br>    # number of seconds; 0 disables</pre><pre>#------------------------------------------------------------------------------<br># REPLICATION<br>#------------------------------------------------------------------------------</pre><pre># - Sending Server(s) -</pre><pre># Set these on the master and on any standby that will send replication data.</pre><pre>#max_wal_senders = 10  # max number of walsender processes<br>    # (change requires restart)<br>#wal_keep_segments = 0  # in logfile segments, 16MB each; 0 disables<br>#wal_sender_timeout = 60s # in milliseconds; 0 disables</pre><pre>#max_replication_slots = 10 # max number of replication slots<br>    # (change requires restart)<br>#track_commit_timestamp = off # collect timestamp of transaction commit<br>    # (change requires restart)</pre><pre># - Master Server -</pre><pre># These settings are ignored on a standby server.</pre><pre>#synchronous_standby_names = &#39;&#39; # standby servers that provide sync rep<br>    # method to choose sync standbys, number of sync standbys,<br>    # and comma-separated list of application_name<br>    # from standby(s); &#39;*&#39; = all<br>#vacuum_defer_cleanup_age = 0 # number of xacts by which cleanup is delayed</pre><pre># - Standby Servers -</pre><pre># These settings are ignored on a master server.</pre><pre>#hot_standby = on   # &quot;off&quot; disallows queries during recovery<br>     # (change requires restart)<br>#max_standby_archive_delay = 30s # max delay before canceling queries<br>     # when reading WAL from archive;<br>     # -1 allows indefinite delay<br>#max_standby_streaming_delay = 30s # max delay before canceling queries<br>     # when reading streaming WAL;<br>     # -1 allows indefinite delay<br>#wal_receiver_status_interval = 10s # send replies at least this often<br>     # 0 disables<br>#hot_standby_feedback = off  # send info from standby to prevent<br>     # query conflicts<br>#wal_receiver_timeout = 60s  # time that receiver waits for<br>     # communication from master<br>     # in milliseconds; 0 disables<br>#wal_retrieve_retry_interval = 5s # time to wait before retrying to<br>     # retrieve WAL after a failed attempt</pre><pre># - Subscribers -</pre><pre># These settings are ignored on a publisher.</pre><pre>#max_logical_replication_workers = 4 # taken from max_worker_processes<br>     # (change requires restart)<br>#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers</pre><pre>#------------------------------------------------------------------------------<br># QUERY TUNING<br>#------------------------------------------------------------------------------</pre><pre># - Planner Method Configuration -</pre><pre>#enable_bitmapscan = on<br>#enable_hashagg = on<br>#enable_hashjoin = on<br>#enable_indexscan = on<br>#enable_indexonlyscan = on<br>#enable_material = on<br>#enable_mergejoin = on<br>#enable_nestloop = on<br>#enable_seqscan = on<br>#enable_sort = on<br>#enable_tidscan = on</pre><pre># - Planner Cost Constants -</pre><pre>#seq_page_cost = 1.0   # measured on an arbitrary scale<br>#random_page_cost = 4.0   # same scale as above<br>#cpu_tuple_cost = 0.01   # same scale as above<br>#cpu_index_tuple_cost = 0.005  # same scale as above<br>#cpu_operator_cost = 0.0025  # same scale as above<br>#parallel_tuple_cost = 0.1  # same scale as above<br>#parallel_setup_cost = 1000.0 # same scale as above<br>#min_parallel_table_scan_size = 8MB<br>#min_parallel_index_scan_size = 512kB<br>#effective_cache_size = 4GB</pre><pre># - Genetic Query Optimizer -</pre><pre>#geqo = on<br>#geqo_threshold = 12<br>#geqo_effort = 5   # range 1-10<br>#geqo_pool_size = 0   # selects default based on effort<br>#geqo_generations = 0   # selects default based on effort<br>#geqo_selection_bias = 2.0  # range 1.5-2.0<br>#geqo_seed = 0.0   # range 0.0-1.0</pre><pre># - Other Planner Options -</pre><pre>#default_statistics_target = 100 # range 1-10000<br>#constraint_exclusion = partition # on, off, or partition<br>#cursor_tuple_fraction = 0.1  # range 0.0-1.0<br>#from_collapse_limit = 8<br>#join_collapse_limit = 8  # 1 disables collapsing of explicit<br>     # JOIN clauses<br>#force_parallel_mode = off</pre><pre>#------------------------------------------------------------------------------<br># ERROR REPORTING AND LOGGING<br>#------------------------------------------------------------------------------</pre><pre># - Where to Log -</pre><pre>#log_destination = &#39;stderr&#39;  # Valid values are combinations of<br>     # stderr, csvlog, syslog, and eventlog,<br>     # depending on platform.  csvlog<br>     # requires logging_collector to be on.</pre><pre># This is used when logging to stderr:<br>#logging_collector = off  # Enable capturing of stderr and csvlog<br>     # into log files. Required to be on for<br>     # csvlogs.<br>     # (change requires restart)</pre><pre># These are only used if logging_collector is on:<br>#log_directory = &#39;log&#39;   # directory where log files are written,<br>     # can be absolute or relative to PGDATA<br>#log_filename = &#39;postgresql-%Y-%m-%d_%H%M%S.log&#39; # log file name pattern,<br>     # can include strftime() escapes<br>#log_file_mode = 0600   # creation mode for log files,<br>     # begin with 0 to use octal notation<br>#log_truncate_on_rotation = off  # If on, an existing log file with the<br>     # same name as the new log file will be<br>     # truncated rather than appended to.<br>     # But such truncation only occurs on<br>     # time-driven rotation, not on restarts<br>     # or size-driven rotation.  Default is<br>     # off, meaning append to existing files<br>     # in all cases.<br>#log_rotation_age = 1d   # Automatic rotation of logfiles will<br>     # happen after that time.  0 disables.<br>#log_rotation_size = 10MB  # Automatic rotation of logfiles will<br>     # happen after that much log output.<br>     # 0 disables.</pre><pre># These are relevant when logging to syslog:<br>#syslog_facility = &#39;LOCAL0&#39;<br>#syslog_ident = &#39;postgres&#39;<br>#syslog_sequence_numbers = on<br>#syslog_split_messages = on</pre><pre># This is only relevant when logging to eventlog (win32):<br># (change requires restart)<br>#event_source = &#39;PostgreSQL&#39;</pre><pre># - When to Log -</pre><pre>#client_min_messages = notice  # values in order of decreasing detail:<br>     #   debug5<br>     #   debug4<br>     #   debug3<br>     #   debug2<br>     #   debug1<br>     #   log<br>     #   notice<br>     #   warning<br>     #   error</pre><pre>#log_min_messages = warning  # values in order of decreasing detail:<br>     #   debug5<br>     #   debug4<br>     #   debug3<br>     #   debug2<br>     #   debug1<br>     #   info<br>     #   notice<br>     #   warning<br>     #   error<br>     #   log<br>     #   fatal<br>     #   panic</pre><pre>#log_min_error_statement = error # values in order of decreasing detail:<br>     #   debug5<br>     #   debug4<br>     #   debug3<br>     #   debug2<br>     #   debug1<br>     #   info<br>     #   notice<br>     #   warning<br>     #   error<br>     #   log<br>     #   fatal<br>     #   panic (effectively off)</pre><pre>#log_min_duration_statement = -1 # -1 is disabled, 0 logs all statements<br>     # and their durations, &gt; 0 logs only<br>     # statements running at least this number<br>     # of milliseconds</pre><pre># - What to Log -</pre><pre>#debug_print_parse = off<br>#debug_print_rewritten = off<br>#debug_print_plan = off<br>#debug_pretty_print = on<br>#log_checkpoints = off<br>#log_connections = off<br>#log_disconnections = off<br>#log_duration = off<br>#log_error_verbosity = default  # terse, default, or verbose messages<br>#log_hostname = off<br>log_line_prefix = &#39;%m [%p] %q%u@%d &#39;  # special values:<br>     #   %a = application name<br>     #   %u = user name<br>     #   %d = database name<br>     #   %r = remote host and port<br>     #   %h = remote host<br>     #   %p = process ID<br>     #   %t = timestamp without milliseconds<br>     #   %m = timestamp with milliseconds<br>     #   %n = timestamp with milliseconds (as a Unix epoch)<br>     #   %i = command tag<br>     #   %e = SQL state<br>     #   %c = session ID<br>     #   %l = session line number<br>     #   %s = session start timestamp<br>     #   %v = virtual transaction ID<br>     #   %x = transaction ID (0 if none)<br>     #   %q = stop here in non-session<br>     #        processes<br>     #   %% = &#39;%&#39;<br>     # e.g. &#39;&lt;%u%%%d&gt; &#39;<br>#log_lock_waits = off   # log lock waits &gt;= deadlock_timeout<br>#log_statement = &#39;none&#39;   # none, ddl, mod, all<br>#log_replication_commands = off<br>#log_temp_files = -1   # log temporary files equal or larger<br>     # than the specified size in kilobytes;<br>     # -1 disables, 0 logs all temp files<br>log_timezone = &#39;UTC&#39;</pre><pre># - Process Title -</pre><pre>cluster_name = &#39;10/main&#39;   # added to process titles if nonempty<br>     # (change requires restart)<br>#update_process_title = on</pre><pre>#------------------------------------------------------------------------------<br># RUNTIME STATISTICS<br>#------------------------------------------------------------------------------</pre><pre># - Query/Index Statistics Collector -</pre><pre>#track_activities = on<br>#track_counts = on<br>#track_io_timing = off<br>#track_functions = none   # none, pl, all<br>#track_activity_query_size = 1024 # (change requires restart)<br>stats_temp_directory = &#39;/var/run/postgresql/10-main.pg_stat_tmp&#39;</pre><pre># - Statistics Monitoring -</pre><pre>#log_parser_stats = off<br>#log_planner_stats = off<br>#log_executor_stats = off<br>#log_statement_stats = off</pre><pre>#------------------------------------------------------------------------------<br># AUTOVACUUM PARAMETERS<br>#------------------------------------------------------------------------------</pre><pre>#autovacuum = on   # Enable autovacuum subprocess?  &#39;on&#39;<br>     # requires track_counts to also be on.<br>#log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and<br>     # their durations, &gt; 0 logs only<br>     # actions running at least this number<br>     # of milliseconds.<br>#autovacuum_max_workers = 3  # max number of autovacuum subprocesses<br>     # (change requires restart)<br>#autovacuum_naptime = 1min  # time between autovacuum runs<br>#autovacuum_vacuum_threshold = 50 # min number of row updates before<br>     # vacuum<br>#autovacuum_analyze_threshold = 50 # min number of row updates before<br>     # analyze<br>#autovacuum_vacuum_scale_factor = 0.2 # fraction of table size before vacuum<br>#autovacuum_analyze_scale_factor = 0.1 # fraction of table size before analyze<br>#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced vacuum<br>     # (change requires restart)<br>#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age<br>     # before forced vacuum<br>     # (change requires restart)<br>#autovacuum_vacuum_cost_delay = 20ms # default vacuum cost delay for<br>     # autovacuum, in milliseconds;<br>     # -1 means use vacuum_cost_delay<br>#autovacuum_vacuum_cost_limit = -1 # default vacuum cost limit for<br>     # autovacuum, -1 means use<br>     # vacuum_cost_limit</pre><pre>#------------------------------------------------------------------------------<br># CLIENT CONNECTION DEFAULTS<br>#------------------------------------------------------------------------------</pre><pre># - Statement Behavior -</pre><pre>#search_path = &#39;&quot;$user&quot;, public&#39; # schema names<br>#default_tablespace = &#39;&#39;  # a tablespace name, &#39;&#39; uses the default<br>#temp_tablespaces = &#39;&#39;   # a list of tablespace names, &#39;&#39; uses<br>     # only default tablespace<br>#check_function_bodies = on<br>#default_transaction_isolation = &#39;read committed&#39;<br>#default_transaction_read_only = off<br>#default_transaction_deferrable = off<br>#session_replication_role = &#39;origin&#39;<br>#statement_timeout = 0   # in milliseconds, 0 is disabled<br>#lock_timeout = 0   # in milliseconds, 0 is disabled<br>#idle_in_transaction_session_timeout = 0 # in milliseconds, 0 is disabled<br>#vacuum_freeze_min_age = 50000000<br>#vacuum_freeze_table_age = 150000000<br>#vacuum_multixact_freeze_min_age = 5000000<br>#vacuum_multixact_freeze_table_age = 150000000<br>#bytea_output = &#39;hex&#39;   # hex, escape<br>#xmlbinary = &#39;base64&#39;<br>#xmloption = &#39;content&#39;<br>#gin_fuzzy_search_limit = 0<br>#gin_pending_list_limit = 4MB</pre><pre># - Locale and Formatting -</pre><pre>datestyle = &#39;iso, mdy&#39;<br>#intervalstyle = &#39;postgres&#39;<br>timezone = &#39;UTC&#39;<br>#timezone_abbreviations = &#39;Default&#39;     # Select the set of available time zone<br>     # abbreviations.  Currently, there are<br>     #   Default<br>     #   Australia (historical usage)<br>     #   India<br>     # You can create your own file in<br>     # share/timezonesets/.<br>#extra_float_digits = 0   # min -15, max 3<br>#client_encoding = sql_ascii  # actually, defaults to database<br>     # encoding</pre><pre># These settings are initialized by initdb, but they can be changed.<br>lc_messages = &#39;en_US.UTF-8&#39;   # locale for system error message<br>     # strings<br>lc_monetary = &#39;en_US.UTF-8&#39;   # locale for monetary formatting<br>lc_numeric = &#39;en_US.UTF-8&#39;   # locale for number formatting<br>lc_time = &#39;en_US.UTF-8&#39;    # locale for time formatting</pre><pre># default configuration for text search<br>default_text_search_config = &#39;pg_catalog.english&#39;</pre><pre># - Other Defaults -</pre><pre>#dynamic_library_path = &#39;$libdir&#39;<br>#local_preload_libraries = &#39;&#39;<br>#session_preload_libraries = &#39;&#39;</pre><pre>#------------------------------------------------------------------------------<br># LOCK MANAGEMENT<br>#------------------------------------------------------------------------------</pre><pre>#deadlock_timeout = 1s<br>#max_locks_per_transaction = 64  # min 10<br>     # (change requires restart)<br>#max_pred_locks_per_transaction = 64 # min 10<br>     # (change requires restart)<br>#max_pred_locks_per_relation = -2 # negative values mean<br>     # (max_pred_locks_per_transaction<br>     #  / -max_pred_locks_per_relation) - 1<br>#max_pred_locks_per_page = 2            # min 0</pre><pre>#------------------------------------------------------------------------------<br># VERSION/PLATFORM COMPATIBILITY<br>#------------------------------------------------------------------------------</pre><pre># - Previous PostgreSQL Versions -</pre><pre>#array_nulls = on<br>#backslash_quote = safe_encoding # on, off, or safe_encoding<br>#default_with_oids = off<br>#escape_string_warning = on<br>#lo_compat_privileges = off<br>#operator_precedence_warning = off<br>#quote_all_identifiers = off<br>#standard_conforming_strings = on<br>#synchronize_seqscans = on</pre><pre># - Other Platforms and Clients -</pre><pre>#transform_null_equals = off</pre><pre>#------------------------------------------------------------------------------<br># ERROR HANDLING<br>#------------------------------------------------------------------------------</pre><pre>#exit_on_error = off   # terminate session on any error?<br>#restart_after_crash = on  # reinitialize after backend crash?<br>#data_sync_retry = off   # retry or panic on failure to fsync data?</pre><pre>#------------------------------------------------------------------------------<br># CONFIG FILE INCLUDES<br>#------------------------------------------------------------------------------</pre><pre># These options allow settings to be loaded from files other than the<br># default postgresql.conf.</pre><pre>include_dir = &#39;conf.d&#39;   # include files ending in &#39;.conf&#39; from<br>     # directory &#39;conf.d&#39;<br>#include_if_exists = &#39;exists.conf&#39; # include file only if it exists<br>#include = &#39;special.conf&#39;  # include file</pre><pre>#------------------------------------------------------------------------------<br># CUSTOMIZED OPTIONS<br>#------------------------------------------------------------------------------</pre><pre># Add settings for extensions here</pre><p>Actually, compared to the default config file, we are only changing one thing, the line 59 to: “listen_addresses = ‘*’”.</p><p>We also want to create a handler that restarts the postgresql server after we have written the two config files to the Vagrant box.</p><p>Create the file ‘main.yml’ in a folder “roles/postgresql/handlers”:</p><pre>- name: restart postgresql<br>  service:<br>    name: postgresql<br>    state: restarted</pre><p>Now, you can run:</p><pre>$ ansible-playbook -i ./hosts_vagrant playbook.yml</pre><p>and you should see something like:</p><pre>PLAY [postgresdb] ***********************************************************************************************************************</pre><pre>TASK [Install python for Ansible] *******************************************************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [setup] ****************************************************************************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [base : Install base dependencies] *************************************************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [base : Upgrade pip] ***************************************************************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [postgresql : Install postgresql] **************************************************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [postgresql : Create pg_hba.conf file in /etc/postgresql/10/main] ******************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [postgresql : Create posgtresql.conf file in /etc/postgresql/10/main] **************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [postgresql : Enable passwordless sudo] ********************************************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [postgresql : Create a PostgreSQL database user] ***********************************************************************************<br>ok: [192.168.60.70]</pre><pre>TASK [postgresql : Create acme database] ************************************************************************************************<br>ok: [192.168.60.70]</pre><pre>PLAY RECAP ******************************************************************************************************************************<br>192.168.60.70              : ok=10   changed=0    unreachable=0    failed=0</pre><p>(instead of the green ‘ok’ you might see a different message if it the first time you run the script).</p><p>If you want to test that you can access the database on the Vagrant box, you can create this little Python3 script (call the file test_db.py):</p><pre>import psycopg2<br>try:<br>    connection = psycopg2.connect(<br>        user = &quot;project&quot;,<br>        password = &quot;project&quot;,<br>        host = &quot;192.168.60.70&quot;,<br>        # host = &quot;127.0.0.1&quot;,<br>        port = &quot;5432&quot;,  # use 5433 if you use localhost<br>        database = &quot;acme&quot;,<br>)<br>    cursor = connection.cursor()<br>    # Print PostgreSQL Connection properties<br>    print ( connection.get_dsn_parameters(),&quot;\n&quot;)<br>    # Print PostgreSQL version<br>    cursor.execute(&quot;SELECT version();&quot;)<br>    record = cursor.fetchone()<br>    print(&quot;You are connected to - &quot;, record,&quot;\n&quot;)<br>except (Exception, psycopg2.Error) as error :<br>    print (&quot;Error while connecting to PostgreSQL&quot;, error)<br>    connection = None<br>finally:<br>    #closing database connection.<br>        if(connection):<br>            cursor.close()<br>            connection.close()<br>            print(&quot;PostgreSQL connection is closed&quot;)</pre><p>and run it with:</p><pre>$ python3 test_db.py</pre><p>and you should see something like:</p><pre>{&#39;user&#39;: &#39;project&#39;, &#39;dbname&#39;: &#39;acme&#39;, &#39;host&#39;: &#39;192.168.60.70&#39;, &#39;port&#39;: &#39;5432&#39;, &#39;tty&#39;: &#39;&#39;, &#39;options&#39;: &#39;&#39;, &#39;sslmode&#39;: &#39;prefer&#39;, &#39;sslcompression&#39;: &#39;1&#39;, &#39;krbsrvname&#39;: &#39;postgres&#39;, &#39;target_session_attrs&#39;: &#39;any&#39;}</pre><pre>You are connected to -  (&#39;PostgreSQL 10.7 (Ubuntu 10.7-0ubuntu0.18.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0, 64-bit&#39;,)</pre><pre>PostgreSQL connection is closed</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=398e1e472349" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ansible tutorial part 2: Installing packages]]></title>
            <link>https://medium.com/@perwagnernielsen/ansible-tutorial-part-2-installing-packages-41d3ab28337d?source=rss-413a7e290f04------2</link>
            <guid isPermaLink="false">https://medium.com/p/41d3ab28337d</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[ansible]]></category>
            <dc:creator><![CDATA[Per Wagner Nielsen]]></dc:creator>
            <pubDate>Sat, 27 Jan 2018 01:15:50 GMT</pubDate>
            <atom:updated>2018-01-27T01:15:50.361Z</atom:updated>
            <content:encoded><![CDATA[<p>In this part we will install some base packages as well as Python 3 on our server. We will organize our playbook into different roles to make it easy to read and extend.</p><p><a href="https://medium.com/@perwagnernielsen/ansible-tutorial-part-1-inventory-files-vagrant-and-remote-hosts-33a15b0185c0">Part 1 can be found here.</a></p><h3>The deployment setup</h3><p>Our setup will be quite simple. We are developing on our local machine and test it on our local Vagrant box. Ones we are happy, we want to be able to deploy our app to our remote server. All these ‘movement’ of files will be done by Ansible as well as the installation and configuration of the servers and tools like Nginx and Gunicorn.</p><p>One of the benefits of using Ansible for these tasks is that we can ensure that the environment we develop on locally (via Vagrant) is the same as what we have on our production environment, on our remote host. This way, we avoid or at least significantly diminish the dreaded ‘but it worked on my machine…’ problem.</p><p>Another benefit of using Ansible is that our infrastructure gets documented and we will be able to commit these to source control. This is what is meant by ‘<a href="https://en.wikipedia.org/wiki/Infrastructure_as_Code">infrastructure as code</a>’.</p><h3>Roles in our playbook</h3><p>In the root of our project, create a new folder called <em>roles</em>, and within this create another folder called <em>base</em>, and again, within <em>base</em> create a folder called <em>tasks</em>.</p><pre>$ mkdir roles<br>$ mkdir roles/base<br>$ mkdir roles/base/tasks</pre><p>Within the roles/base/tasks folder, we need to create a file called <em>main.yml</em> which will hold the tasks to run for this particular role.</p><pre>touch roles/base/tasks/main.yml</pre><p>You can think of roles in Ansible as collections of tasks. For example, our base role will hold tasks related to the basic setup of our server, like installing needed dependencies and updates.</p><p>In our newly created <em>main.yml</em> add the following:</p><pre>- name: Install base packages<br>  apt: name={{ item }} state=installed<br>  with_items:<br>    - locales<br>    - build-essential<br>    - acl<br>    - ntp<br>    - htop<br>    - git<br>    - supervisor<br>    - python-pip<br>    - python3-pip<br>    - nginx<br>  tags:<br>    - packages</pre><pre>- name: Upgrade pip<br>  pip: name=pip state=latest<br>  tags:<br>    - packages</pre><pre>- name: Create user<br>  user:<br>    name: &quot;deploy_user&quot;<br>    comment: &quot;Used to deploy Django&quot;<br>    state: present</pre><pre>- name: Create .ssh folder<br>  file:<br>    path: &quot;/home/deploy_user/.ssh/&quot;<br>    state: directory<br>    mode: 0755</pre><p>This is the list of specific tasks we want this <em>base</em> role to perform. We now need to hook up this to our playbook, so Ansible knows to execute these tasks when we run our playbook. Change the <em>playbook.yml</em> to:</p><pre>---<br>- name: Web01<br>  hosts: all<br>  become: yes<br>  gather_facts: False<br>  pre_tasks:<br>    - name: Install python for Ansible<br>      raw: test -e /usr/bin/python || (apt -y update &amp;&amp; apt install -y python-minimal)<br>      changed_when: False<br>    - setup: # aka gather_facts</pre><pre>roles:<br>  - base</pre><p>Now try and run the playbook again on the vagrant host</p><pre>$ ansible-playbook -i ./hosts_vagrant playbook.yml</pre><p>It might take some time, but you should see feedback from Anaible as Ansible progresses through the playbook.</p><pre>PLAY [Web01] ********************************************************************</pre><pre>TASK [Install python for Ansible] ********************************************************************<br>ok: [127.0.0.1]</pre><pre>TASK [setup] ********************************************************************<br>ok: [127.0.0.1]</pre><pre>TASK [base : Install base packages] ********************************************************************<br>ok: [127.0.0.1] =&gt; (item=[&#39;locales&#39;, &#39;build-essential&#39;, &#39;acl&#39;, &#39;ntp&#39;, &#39;htop&#39;, &#39;git&#39;, &#39;supervisor&#39;, &#39;python-pip&#39;, &#39;python3-pip&#39;, &#39;nginx&#39;])</pre><pre>TASK [base : Upgrade pip] ********************************************************************<br>ok: [127.0.0.1]</pre><pre>TASK [base : Create user] ********************************************************************<br>changed: [127.0.0.1]</pre><pre>TASK [base : Create .ssh folder] ********************************************************************<br>changed: [127.0.0.1]</pre><pre>PLAY RECAP ********************************************************************<br>127.0.0.1                  : ok=6    changed=2    unreachable=0    failed=0</pre><p>You can now ssh into the Vagrant box with vagrant ssh if you want and check that everything is as expected.</p><h3>Installing Python 3</h3><p>Lets create a role that installs a Python 3 version from source.</p><pre>$ mkdir roles/python<br>$ mkdir roles/python/tasks<br>$ touch roles/python/tasks/main.yml</pre><p>In main.yml add this:</p><pre>---<br>- name: Check that the /usr/local/bin/python3.6 exists<br>  stat:<br>    path: /usr/local/bin/python3.6<br>  tags: python<br>  register: python_result</pre><pre>- name: Update apt-cache<br>  apt: update_cache=yes<br>  tags: packages, python<br>  when: python_result.stat.exists == False</pre><pre>- name: Install packages needed for installing Python<br>  apt: pkg={{ item }}<br>  with_items:<br>    - wget<br>    - xz-utils<br>    - build-essential<br>    - libsqlite3-dev<br>    - libreadline-dev<br>    - libssl-dev<br>    - openssl<br>  tags: packages, python<br>  when: python_result.stat.exists == False</pre><pre>- name: Extract python 3.6.4 into /tmp<br>  unarchive:<br>    src: <a href="https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tar.xz">https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tar.xz</a><br>    dest: /tmp/<br>    remote_src: yes<br>  tags: packages, python</pre><pre>- name: Configure python 3.6.4<br>  shell: ./configure chdir=/tmp/Python-3.6.4<br>  tags: packages, python<br>  when: python_result.stat.exists == False</pre><pre>- name: Make<br>  shell: make chdir=/tmp/Python-3.6.4<br>  tags: packages, python<br>  when: python_result.stat.exists == False</pre><pre>- name: Install Python 3.6.4<br>  shell: make install chdir=/tmp/Python-3.6.4<br>  become: yes<br>  become_user: root<br>  tags: packages, python<br>  when: python_result.stat.exists == False</pre><pre>- name: Remove tmp files used for Python 3.6.4 installation<br>  file: path={{ item }} state=absent<br>  with_items:<br>    - /tmp/Python.tar.xz<br>    - /tmp/Python-3.6.4<br>  tags: packages, python<br>  when: python_result.stat.exists == False</pre><p>Update the playbook.yml to include the python role:</p><pre>---<br>- name: Web01<br>  hosts: all<br>  become: yes<br>  gather_facts: False<br>  pre_tasks:<br>    - name: Install python for Ansible<br>      raw: test -e /usr/bin/python || (apt -y update &amp;&amp; apt install -y python-minimal)<br>      changed_when: False<br>    - setup: # aka gather_facts</pre><pre>roles:<br>  - base<br>  - python</pre><p>And rerun the entire playbook</p><pre>$ ansible-playbook -i ./hosts_vagrant playbook.yml</pre><p>If all goes well, you should be able to ssh into the Vagrant box and use Python 3.6.4:</p><pre>$ vagrant ssh<br>Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-87-generic x86_64)</pre><pre>* Documentation:  <a href="https://help.ubuntu.com/">https://help.ubuntu.com</a><br> * Management:     <a href="https://landscape.canonical.com/">https://landscape.canonical.com</a><br> * Support:        <a href="https://ubuntu.com/advantage">https://ubuntu.com/advantage</a></pre><pre>35 packages can be updated.<br>27 updates are security updates.</pre><pre>Last login: Sat Jan 27 00:35:51 2018 from 10.0.2.2<br>vagrant@web01:~$ python3<br>Python 3.6.4 (default, Jan 27 2018, 00:34:49) <br>[GCC 5.4.0 20160609] on linux<br>Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.<br>&gt;&gt;&gt;</pre><pre>* Documentation:  <a href="https://help.ubuntu.com/">https://help.ubuntu.com</a><br> * Management:     <a href="https://landscape.canonical.com/">https://landscape.canonical.com</a><br> * Support:        <a href="https://ubuntu.com/advantage">https://ubuntu.com/advantage</a></pre><pre>35 packages can be updated.<br>27 updates are security updates.</pre><pre>Last login: Sat Jan 27 00:35:51 2018 from 10.0.2.2<br>vagrant@web01:~$ python3<br>Python 3.6.4 (default, Jan 27 2018, 00:34:49) <br>[GCC 5.4.0 20160609] on linux<br>Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.<br>&gt;&gt;&gt;</pre><h3>Conslusion</h3><p>We installed a new version of Python from source and refactored our playbook. Next up is getting git to play nice. Stay tuned.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=41d3ab28337d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ansible tutorial part 1: Inventory files , vagrant and remote hosts]]></title>
            <link>https://medium.com/@perwagnernielsen/ansible-tutorial-part-1-inventory-files-vagrant-and-remote-hosts-33a15b0185c0?source=rss-413a7e290f04------2</link>
            <guid isPermaLink="false">https://medium.com/p/33a15b0185c0</guid>
            <category><![CDATA[vagrant]]></category>
            <category><![CDATA[ansible]]></category>
            <dc:creator><![CDATA[Per Wagner Nielsen]]></dc:creator>
            <pubDate>Fri, 05 Jan 2018 22:40:43 GMT</pubDate>
            <atom:updated>2018-01-26T23:58:56.896Z</atom:updated>
            <content:encoded><![CDATA[<p>In this tutorial series we will use Ansible to setup a Flask website with a database on an Ubuntu 16.04 server. We will go step by step from getting the Ubuntu server setup, installing and running our little Flask app on a Vagrant machine and deploying the same Flask app server on the internet (on Linode).</p><p>The local setup I will be using is a Linux Mint 18.1 machine with Vagrant 2.0.1 installed.</p><p>This is not a tutorial for complete beginners, so you might need to do some googling on some of the topics that I don’t cover.</p><p>Part 1 will cover how to deploy a simple playbook to both a local Vagrant VM and a remote server.</p><h4>Additional resources</h4><p><a href="https://blacksaildivision.com/ansible-tutorial-part-1">Ansible tutorial - part 1 - intro to Ansible</a></p><h3>Run locally on Vagrant</h3><h4>installing Ansible</h4><p>We will install Ansible in a virtualenv with pip. Create a folder to hold all our code:</p><pre>$ mkdir ansible_tutorial<br>$ cd ansible_tutorial<br>$ virtualenv venv -p python3.6<br>$ source venv/bin/activate<br>$ pip install ansible flask</pre><p>here is a one liner for it all:</p><pre>mkdir ansible_tutorial;cd ansible_tutorial;virtualenv venv -p python3.6;source venv/bin/activate;pip install ansible flask</pre><p>If you don’t have python3.6 installed, you can just do $ virtualenv venv -p python3 instead. I use Python 3.6 for most of my own stuff, but we won’t be using any specific 3.6 features in this tutorial.</p><h4>Setting up Vagrant</h4><p>Create the Vagrant file:</p><pre>$ touch Vagrantfile</pre><p>Add the following to the file</p><pre># -*- mode: ruby -*-<br># vi: set ft=ruby :<br>Vagrant.configure(&quot;2&quot;) do |config|<br>  config.vm.define &quot;web01&quot; do |web01|<br>    web01.vm.box = &quot;bento/ubuntu-16.04&quot;<br>    web01.vm.network &quot;private_network&quot;, ip: &quot;192.168.33.11&quot;<br>    web01.vm.hostname = &quot;web01&quot;<br>    web01.ssh.forward_agent = true<br>    web01.ssh.port = 2222    <br>    web01.vm.synced_folder &quot;.&quot;, &quot;/vagrant&quot;<br>    web01.vm.provider &quot;virtualbox&quot; do |vb|<br>      vb.memory = &quot;1024&quot;<br>    end<br>    web01.vm.provision &quot;ansible&quot; do |ansible|<br>      ansible.playbook = &quot;playbook.yml&quot;<br>    end<br>  end<br>end</pre><p>We are doing a few things here. First, we base ourselves on a Ubuntu 16.04 and sets the hostname to ‘web01’. We are also telling Vagrant that once we provision this box we want it to run the ‘playbook.yml’ playbook, which should be located in the same directory as the Vagrantfile.</p><h4>Setting up an initial playbook</h4><p>We need to create our playbook.yml file:</p><pre>$ touch playbook.yml</pre><p>In the playbook.yml add this</p><pre>---<br>- name: Web01<br>  hosts: all<br>  become: yes</pre><pre>  gather_facts: False<br>  pre_tasks:<br>    - name: Install python for Ansible<br>      raw: test -e /usr/bin/python || (apt -y update &amp;&amp; apt install -y python-minimal)<br>      changed_when: False<br>    - setup: # aka gather_facts</pre><p>This small playbook will only do one thing, install python on the box in case it is not already installed.</p><p>Activate Vagrant by typing:</p><pre>$ vagrant up</pre><p>It will take a bit of time (some minutes) for Vagrant to be ready, when it is, you should see something like:</p><pre>web01: Running ansible-playbook...<br> ______________<br>&lt; PLAY [Web01] &gt;<br> --------------<br>        \   ^__^<br>         \  (oo)\_______<br>            (__)\       )\/\<br>                ||----w |<br>                ||     ||</pre><pre>___________________________________<br>&lt; TASK [Install python for Ansible] &gt;<br> -----------------------------------<br>        \   ^__^<br>         \  (oo)\_______<br>            (__)\       )\/\<br>                ||----w |<br>                ||     ||</pre><pre>ok: [web01]<br> ______________<br>&lt; TASK [setup] &gt;<br> --------------<br>        \   ^__^<br>         \  (oo)\_______<br>            (__)\       )\/\<br>                ||----w |<br>                ||     ||</pre><pre>ok: [web01]<br> ____________<br>&lt; PLAY RECAP &gt;<br> ------------<br>        \   ^__^<br>         \  (oo)\_______<br>            (__)\       )\/\<br>                ||----w |<br>                ||     ||</pre><pre>web01                      : ok=2    changed=0    unreachable=0    failed=0</pre><p>So now we can see Ansible is actually working, the playbook was successfully run on the VM box.</p><p>Note: If you want to get rid of the ascii cow, create a file called ‘ansible.cfg’ and add this to it:</p><pre>[defaults]</pre><pre>nocows = 1</pre><h3>Deploy to a host on the internet</h3><h4>Add ssh key to remote host</h4><p>In order to configure a host on the internet, we first need to have a place to host it. For this tutorial, I’ll go for the smallest Linode instance. First we need to add our public key file to the remote server as Ansible requires SSH in order to work.</p><p>Make sure you have a ssh key in ~/.ssh and run:</p><pre>$ ssh-copy-id -i ~/.ssh/id_rsa.pub root@123.123.123.123</pre><p>where you insert the correct IP address of your remote server. This will copy your local public key to the remote server.</p><h4>Create Inventory File</h4><p>Inventory files hold information about how you connect to your servers and in our case will look like this:</p><pre>---</pre><pre>[web01]<br>123.123.123.123</pre><pre>[web01:vars]<br>ansible_ssh_user=root<br>ansible_ssh_private_key_file=/home/&lt;USERNAME&gt;/.ssh/id_rsa</pre><p>We will be creating 2 different hosts files, one for our Vagrant box and one for our staging (Linode) server. Make two files called hosts_vagrant and hosts_staging:</p><pre>touch hosts_vagrant hosts_staging</pre><p>Add the following to hosts_vagrant:</p><pre>[all:vars]<br>env=vagrant</pre><pre>[web]<br>127.0.0.1</pre><pre>[web:vars]<br> ansible_port=2222 <br> ansible_user=vagrant <br> ansible_private_key_file=&quot;./.vagrant/machines/web01/virtualbox/private_key&quot;</pre><p>And this to hosts_staging:</p><pre>[all:vars]<br>env=staging</pre><pre>[web]<br>123.123.123.123</pre><pre>[web:vars]<br>ansible_ssh_user=root<br>ansible_ssh_private_key_file=/home/perwagner/.ssh/id_rsa</pre><p>Put the files in the same folder as the Vagrantfile. (substitute the IP of your remote host and the &lt;USERNAME&gt; with your username and insert whatever file path that points to your private key). It is important that the group name is the same in the hosts file and in the playbook.</p><p>The first line with [web] is the name of the group of servers which are specified below. In our case we have just one. If you wanted to, you can have several different groups in the inventory file, like servers for your web apps and servers for your databases. The [web:vars] specifies additional settings for the group.</p><p>Lets test our connection to the remote host:</p><pre>$ ansible -i ./hosts_staging web -m ping</pre><p>If all goes well, you should see this message:</p><pre>123.123.123.123 | SUCCESS =&gt; {<br>    &quot;changed&quot;: false,<br>    &quot;ping&quot;: &quot;pong&quot;<br>}</pre><p>Which shows that our Ansible successfully connected to our remote host.</p><p>The -i option tells ansible that the next argument (./hosts) is the inventory file and the web argument specifies which group in the inventory file Ansible should communicate with. The final -m ping is the module we want to execute (in this case a simple ping).</p><h4>Run the playbook on the remote host</h4><p>In order to have Ansible run the playbook on our remote host, we need to invoke a different Ansible command, the ansible-playbook command:</p><pre>$ ansible-playbook -i ./hosts_staging playbook.yml</pre><p>Again, the -i option specifies the inventory file while the last argument is the playbook we want to execute.</p><p>If all goes well, you should receive a similar success message as before when we had our Vagrant box run the playbook. Thus, we have successfully run the same playbook on both the local VM and the remote server.</p><h3>Conclusion</h3><p>I hope all went well and that you successfully managed to setup both the local Vagrant and remote host.</p><p>If you have any questions or suggestions on how to improve this tutorial, please reach out to me and leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33a15b0185c0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[React Native simple navigation 1]]></title>
            <link>https://medium.com/@perwagnernielsen/react-native-simple-navigation-1-4eea47b86046?source=rss-413a7e290f04------2</link>
            <guid isPermaLink="false">https://medium.com/p/4eea47b86046</guid>
            <category><![CDATA[navigator]]></category>
            <category><![CDATA[react-native]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Per Wagner Nielsen]]></dc:creator>
            <pubDate>Sat, 03 Sep 2016 10:29:37 GMT</pubDate>
            <atom:updated>2016-09-03T10:29:37.220Z</atom:updated>
            <content:encoded><![CDATA[<p>I’m starting to get into react-native, but had some trouble getting my head around the navigation system. This post will demonstrate a simple way to navigate between two pages in a small to-do style application.</p><p>Here are a few good learning resources:</p><ul><li><a href="https://playcode.org/navigation-in-react-native/">https://playcode.org/navigation-in-react-native/</a></li><li><a href="http://blog.paracode.com/2016/01/05/routing-and-navigation-in-react-native/">http://blog.paracode.com/2016/01/05/routing-and-navigation-in-react-native/</a></li></ul><p>I’ll assume that you have a virtual device running and have a developer environment setup already. This tutorial is tested on Android with react-native 0.32.</p><h3>Setup</h3><p>Start a new react-native project:</p><pre>&gt;&gt; react-native init NavProject<br>...<br>...<br>&gt;&gt; cd NavProject<br>&gt;&gt; react-native start</pre><p>The last command (react-native start) will start the server so you can use ‘react-native run-android’ to start your app. In a new tap (and from within your project’s directory) run:</p><pre>&gt;&gt; react-native run-android<br>...<br>...<br>&gt;&gt;&gt; react-native log-android</pre><p>‘react-native log-android’ prints out the log (console.log, etc.) which can come in quite handy.</p><p>Your virtual device should now be displaying the default ‘Welcome to React Native!’ style app and you can open up your code editor and open the file <em>index.android.js</em>. This will be the main entry point for our application.</p><p>In index.android.js you need to delete most of the content there to begin with and end up with:</p><pre>import React, { Component } from &#39;react&#39;;<br>import {<br>  AppRegistry,<br>  StyleSheet,<br>  Text,<br>  View<br>} from &#39;react-native&#39;;</pre><pre>class SimpleTodoApp extends Component {<br>  render() {<br>    return (<br>      &lt;View&gt;<br>        &lt;Text&gt;Hello world&lt;/Text&gt;<br>      &lt;/View&gt;<br>    );<br>  }<br>}</pre><pre>const styles = StyleSheet.create({</pre><pre>});</pre><pre>AppRegistry.registerComponent(&#39;SimpleTodoApp&#39;, () =&gt; SimpleTodoApp);</pre><h3>navigator</h3><p>We will be using the <a href="https://facebook.github.io/react-native/docs/navigator.html">navigator component</a> to handle our navigation. There are two concepts that you need to become familiar with when working with the navigator: routes and scenes.</p><h4>routes and scenes</h4><p>A route is <a href="https://facebook.github.io/react-native/docs/using-navigators.html">an object that contains information about a scene</a>. This object is supposed to hold the information needed to render your scene (basically a screen or view). A scene is most easily understood a a screen and is made up of components like &lt;View&gt;, &lt;Text&gt; etc. and is typically rendered full-screen.</p><h4>Creating our screens</h4><p>Lets start by importing the Navigator component:</p><pre>import React, { Component } from &#39;react&#39;;<br>import {<br>  AppRegistry,<br>  StyleSheet,<br><strong>  </strong><a href="https://facebook.github.io/react-native/docs/navigator.html"><strong>Navigator</strong></a><strong>,<br>  </strong><a href="https://facebook.github.io/react-native/docs/touchableopacity.html"><strong>TouchableOpacity</strong></a><strong>,</strong><br>  Text,<br>  View<br>} from &#39;react-native&#39;;</pre><p>We will create two screens, one for our todo list and one with some about-us information.</p><p>First out TodoScreen:</p><pre>var TodoScreen = React.createClass({<br>  _handlePress() {<br>    console.log(&quot;nothing yet...&quot;);<br>  },</pre><pre>render() {<br>    return (<br>      &lt;View&gt;<br>        &lt;Text&gt;TO DO&lt;/Text&gt;<br>        &lt;TouchableOpacity onPress={this._handlePress}&gt;<br>          &lt;View&gt;<br>            &lt;Text&gt;To AboutScreen&lt;/Text&gt;<br>          &lt;/View&gt;<br>        &lt;/TouchableOpacity&gt;<br>      &lt;/View&gt;<br>    )<br>  },<br>});</pre><p>and then the About Screen:</p><pre>var AboutScreen = React.createClass({<br>  _handlePress() {<br>    console.log(&quot;nothing yet...&quot;);<br>  },</pre><pre>render() {<br>    return (<br>      &lt;View&gt;<br>        &lt;Text&gt;ABOUT US&lt;/Text&gt;<br>        &lt;TouchableOpacity onPress={this._handlePress}&gt;<br>          &lt;View&gt;<br>            &lt;Text&gt;To TodoScreen&lt;/Text&gt;<br>          &lt;/View&gt;<br>        &lt;/TouchableOpacity&gt;<br>      &lt;/View&gt;<br>    )<br>  },<br>});</pre><h4>Add &lt;Navigator&gt;</h4><p>We need to update the main SimpleTodoApp class to render the &lt;Navigator&gt; component:</p><pre>class SimpleTodoApp extends Component {<br>  _renderScene(route, navigator) {<br>    if(route.id === 1) {<br>      return &lt;TodoScreen navigator={navigator} /&gt;<br>    } else if(route.id === 2) {<br>      return &lt;AboutScreen navigator={navigator} /&gt;<br>    }<br>  }</pre><pre>render() {<br>    return (<br>      &lt;Navigator<br>        initialRoute={{id: 1, }}<br>        renderScene={this._renderScene} /&gt;<br>    );<br>  }<br>}</pre><p>We have 2 props in the &lt;Navigator&gt; component:</p><ul><li>initialRoute</li><li>renderScene</li></ul><p>initialRoute holds the initial route for navigation, eg. your starting page. Here we pass in a route object with just one attribute, id: 1. The renderScene prop is required and renders the scene for a given route. The prop is invoked with the <em>route</em> and <em>navigator</em> objects. In our case above, we define the function <em>_renderScene(route, navigator)</em> to handle this, in this way when our application loads, the _renderScene function is invoked with the initialRoute object (with id: 1) and the navigator object. In our _renderScene function, we then have a conditional structure (if/elseif) that renders our TodoScreen if id: 1 and our AboutScreen if id: 2. Since the initialRoute is set to {id: 1}, then the first screen to be rendered is our TodoScreen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*4NnrFQi9tRpV2oWSf5nqvw.png" /></figure><p>Now, nothing happens yet when you click the “To AboutScreen”. This is because we haven’t implemented the functionality on the TodoScreen yet, we have only made the placeholder function <em>_handlePress(),</em> lets change that now.</p><h4>Push and pop</h4><p>When the user presses the “To AboutScreen”, we want the AboutScreen to be displayed. The way to do this is to use the navigator component’s push method which takes a route object. In the &lt;Navigator&gt; component we passed down the <em>navigator</em> as a prop through the _renderScene() function to the scene to be rendered, thus navigator will be available as a prob in any scene rendered by the &lt;Navigator&gt;.</p><pre>var TodoScreen = React.createClass({<br>  _handlePress() {<br>   <strong> this.props.navigator.push({id: 2});</strong><br>  },</pre><pre>render() {<br>    return (<br>      &lt;View&gt;<br>        &lt;Text&gt;TO DO&lt;/Text&gt;<br>        &lt;TouchableOpacity onPress={this._handlePress}&gt;<br>          &lt;View&gt;<br>            &lt;Text&gt;To AboutScreen&lt;/Text&gt;<br>          &lt;/View&gt;<br>        &lt;/TouchableOpacity&gt;<br>      &lt;/View&gt;<br>    )<br>  },<br>});</pre><p>Reload your app on the virtual device and try again. We can now navigate to the AboutScreen, but we still can’t navigate back to the TodoScreen. We can fix this by updating the <em>_handlePress()</em> function in the AboutScreen class.</p><pre>var AboutScreen = React.createClass({<br>  _handlePress() {<br>   <strong> this.props.navigator.pop();</strong><br>  },</pre><p>Reload and try again. Now you can navigate back and forth between the TodoScreen and the AboutScreen.</p><p>The full source is:</p><pre>import React, { Component } from &#39;react&#39;;<br>import {<br>  AppRegistry,<br>  StyleSheet,<br>  Navigator,<br>  TouchableOpacity,<br>  Text,<br>  View<br>} from &#39;react-native&#39;;</pre><pre>var TodoScreen = React.createClass({<br>  _handlePress() {<br>    this.props.navigator.push({id: 2});<br>  },</pre><pre>render() {<br>    return (<br>      &lt;View&gt;<br>        &lt;Text&gt;TO DO&lt;/Text&gt;<br>        &lt;TouchableOpacity onPress={this._handlePress}&gt;<br>          &lt;View&gt;<br>            &lt;Text&gt;To AboutScreen&lt;/Text&gt;<br>          &lt;/View&gt;<br>        &lt;/TouchableOpacity&gt;<br>      &lt;/View&gt;<br>    )<br>  },<br>});</pre><pre>var AboutScreen = React.createClass({<br>  _handlePress() {<br>    this.props.navigator.pop();<br>  },<br>  render() {<br>    return (<br>      &lt;View&gt;<br>        &lt;Text&gt;ABOUT US&lt;/Text&gt;<br>        &lt;TouchableOpacity onPress={this._handlePress}&gt;<br>          &lt;View&gt;<br>            &lt;Text&gt;To TodoScreen&lt;/Text&gt;<br>          &lt;/View&gt;<br>        &lt;/TouchableOpacity&gt;<br>      &lt;/View&gt;<br>    )<br>  },<br>});</pre><pre>class SimpleTodoApp extends Component {<br>  _renderScene(route, navigator) {<br>    if(route.id === 1) {<br>      return &lt;TodoScreen navigator={navigator} /&gt;<br>    } else if(route.id === 2) {<br>      return &lt;AboutScreen navigator={navigator} /&gt;<br>    }<br>  }</pre><pre>render() {<br>    return (<br>      &lt;Navigator<br>        initialRoute={{id: 1, }}<br>        renderScene={this._renderScene} /&gt;<br>    );<br>  }<br>}</pre><pre>const styles = StyleSheet.create({</pre><pre>});</pre><pre>AppRegistry.registerComponent(&#39;SimpleTodoApp&#39;, () =&gt; SimpleTodoApp);</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4eea47b86046" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[(26th November, 2017: Minor updates.]]></title>
            <link>https://medium.com/@perwagnernielsen/getting-started-with-flask-login-can-be-a-bit-daunting-in-this-tutorial-i-will-use-d68791e9b5b5?source=rss-413a7e290f04------2</link>
            <guid isPermaLink="false">https://medium.com/p/d68791e9b5b5</guid>
            <category><![CDATA[flask-login]]></category>
            <category><![CDATA[login]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[flask]]></category>
            <dc:creator><![CDATA[Per Wagner Nielsen]]></dc:creator>
            <pubDate>Tue, 23 Feb 2016 12:53:57 GMT</pubDate>
            <atom:updated>2017-11-26T00:08:12.926Z</atom:updated>
            <content:encoded><![CDATA[<p>(26th November, 2017: Minor updates. Updated to use Python 3.6 and fixed minor typos, formatting and bugs)</p><p>Getting started with flask-login can be a bit daunting. In this tutorial I will use:</p><ul><li>flask</li><li>flask-login</li><li>flask-wtf</li><li>flask-sqlalchemy</li></ul><p>and Python 3.6 to make the simplest possible flask-login application.<br>(On a machine running Linux Mint 17.3).</p><p>The purpose of this small tutorial is not to create a working production ready prototype, implement all best practices or show the ‘right’ way to use flask-login, but rather to show a very simple way to get flask-login to work. If you want to use flask-login in a real app, you will probably want to do some reorganization of the code and implement testing (this is not covered here).<br>My basic idea is, that once you have a system where the ‘happy-path’ works, it is a lot easier to focus on getting all the other pieces together.</p><h3>1 Make folder and setup virtualenv</h3><p>Create a folder called flasklogin_tutorial:</p><p>&gt;&gt; mkdir flasklogin_tutorial<br>&gt;&gt; cd flasklogin_tutorial</p><p>The <em>flasklogin_tutorial</em> folder will hold all our files, including the virtualenvironment we will be using.</p><p>Create the virtualenv for python 3 within the <em>flasklogin_tutorial</em> directory:</p><p>&gt;&gt; virtualenv -p python3 venv<br>(if you want to use Python 3.6, make sure this is install on your computer and type “virtualenv -p python3.6 venv” instead)</p><p>This creates a virtual environment in the ‘venv’ folder using python 3 as its base (make sure you have both Python 3 installed and virtual environment installed). The virtual environment can be activated by running the command ‘&gt;&gt; source venv/bin/activate’ and deactivated again by running ‘deactivate’</p><p>Install flask, flask-login, flask-wtf and flask-sqlalchemy</p><p>&gt;&gt; source venv/bin/activate<br>&gt;&gt; pip install flask flask-login flask-wtf flask-sqlalchemy</p><p>We run <em>pip install</em> after we have activated the virtual environment. This way, the modules are installed locally in the virtual env rather than globally on your computer.</p><p><a href="http://flask.pocoo.org/">flask</a>: The flask web app<br><a href="https://flask-login.readthedocs.org/en/latest/">flask-login</a>: Login functionality for Flask<br><a href="https://flask-wtf.readthedocs.org/en/latest/index.html">flask-wtf</a>: Makes working with forms easy<br><a href="http://flask-sqlalchemy.pocoo.org/">flask-sqlalchemy</a>: ORM for working with databases</p><h3>2 Create app and config files</h3><p>We need to create some files to hold our application, but first create a folder called ‘app’ inside the root folder. This ‘app’ folder will hold all of the below:</p><p><strong>app.py</strong>: Contains the application together with routes and login facilities created by flask-login.<br><strong>forms.py</strong>: Holds our form definitions created by flask-wtf.<br><strong>models.py</strong>: Holds our database model(s) created by flask-sqlalchemy.<br><strong>/db folder</strong>: To hold our sqlite3 database. The database will be automatically created by the application.<br><strong>/templates folder</strong>: To hold all our templates (.html files)</p><p>Go ahead and create these files and the /db folder within the <em>flasklogin_tutorial</em> folder.</p><h3>3 Create flask app</h3><p>Open the app.py file and insert:</p><pre>from flask import Flask</pre><pre>app = Flask(__name__)<br>app.secret_key = <strong>&#39;A0Zr98j/3yX R~XHH!jmN]LWX/,?RT&#39;<br></strong>app.config<strong>[&#39;SQLALCHEMY_DATABASE_URI&#39;] = &#39;sqlite:////database/database.sqlite&#39;</strong></pre><pre>@app.route(&#39;/&#39;)<br>def index():<br>    return &quot;Welcome to Flask&quot;<br>if __name__ == &#39;__main__&#39;:<br>    app.run(port=5000, host=&#39;localhost&#39;)</pre><p>We begin by importing the Flask application from flask. We then create our first route with @app.route(‘/’) which tells Flask that the function defined here should be run when you hit the ‘localhost:5000/’ link. The function ‘def index()’ just returns a string (‘Welcome to Flask’) which lets us know that the application is working. Finally, we are running the app with app.run and sets the port and host values (you can change these as you see fit).</p><p>The secret_key is used later for the forms we create.</p><p>Now, in the Terminal run:<br>&gt;&gt; python app.py</p><p>Open a browser and go to “localhost:5000” and you should see the “Welcome to Flask” in the browser window.</p><h3>4 Setting up sign-up form</h3><p>Now we will create a sign-up form where the user can enter sign up for our app by providing an email address (as username) and a password. We will use flask-WTF to create the form and later hook it up to a database.</p><h3>4.1 Creating the SignupForm class</h3><p>We will need three elements on our form: a field for entering the email address, a field for the password and a submit button. In the forms.py file enter this:</p><pre>from flask_wtf import FlaskForm<br>from wtforms import StringField, PasswordField, SubmitField</pre><pre>class SignupForm(FlaskForm):<br>    email = StringField(&#39;email&#39;)<br>    password = PasswordField(&#39;password&#39;)<br>    submit = SubmitField(&quot;Sign In&quot;)</pre><p>We start by importing Form from flask_wtf and 3 field types from wtforms. Each field (<a href="https://wtforms.readthedocs.org/en/latest/fields.html#basic-fields">StringField, PasswordField and SubmitField</a>) have slightly different properties and wtforms contains other field types as well.</p><h3>4.2 Creating the signup view</h3><p>In our app.py we need to create a route to the url where we want the signup to take place and then create the template to be rendered to the user.</p><p>In app.py, add/edit this:</p><pre>from flask import Flask, request, render_template, redirect, url_for</pre><pre>from .forms import SignupForm</pre><p>at the top to import the form from the forms.py file and to import some flask classes we will be needing as well.</p><p>After the first ‘/’ route we need to define a route for our signup page. Lets make this ‘/signup’:</p><pre>@app.route(&#39;/signup&#39;, methods=[&#39;GET&#39;, &#39;POST&#39;])<br>def register():<br>    return &quot;Signup&quot;</pre><p>We have added ‘methods=[‘GET’, ‘POST’]) to the ‘@app.route’ decorator. This tells Flask, that only GET and POST methods should be accepted. We will need both GET and POST requests to the /signup url to allow both to render the signup template (GET) and to process any input the user enters in the form (POST).</p><p>We will need to handle the following eventualities for the /signup url:</p><p>1. Render the signup template (not created yet)<br>2. Handle new user signup<br>3. Handle signups where the username is already taken<br>4. Handle signups where the input is not valid (eg. like empty input)</p><p>We will do it like this:</p><pre>@app.route(&#39;/signup&#39;, methods=[&#39;GET&#39;, &#39;POST&#39;])<br>def signup():<br>    form = SignupForm()</pre><pre>    if request.method == &#39;GET&#39;:<br>        return render_template(&#39;signup.html&#39;, form = form)<br>    elif request.method == &#39;POST&#39;:<br>        if form.validate_on_submit():<br>            if &#39;user already exist in database&#39;:<br>                return &quot;Email address already exists&quot;<br>            else:<br>                return &quot;will create user here&quot;<br>        else:<br>            return &quot;Form didn&#39;t validate&quot;</pre><p>We have just entered some text to display the different eventualities.</p><p>We start by assigning the SignupForm to a local variable ‘form’, which we later pass into the render_template class. This way, the form elements becomes available to the template which we will create next.</p><p>The ‘form.validate_on_submit()’ validates the input in the form against the validators we add in the forms.py file. We didn’t add any yet, so for now it servers no purpose, but later we will add so validation to the form.</p><h3>4.3 Creating the signup template</h3><p>In the /templates folder create a file ‘signup.html’ and enter this:</p><pre>&lt;!DOCTYPE html&gt;<br>&lt;html lang=&quot;en&quot;&gt;<br>&lt;head&gt;<br>    &lt;meta charset=&quot;UTF-8&quot;&gt;<br>    &lt;title&gt;Sign-up&lt;/title&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>    &lt;h1&gt;Sign-up&lt;/h1&gt;</pre><pre>    &lt;form method=&quot;POST&quot; action=&quot;/signup&quot;&gt;<br>        {{ form.hidden_tag() }}<br>        {{ form.email.label }} {{ form.email(size=20) }}<br>        {{ form.password.label }} {{ form.password(size=20) }}<br>        &lt;input type=&quot;submit&quot; value=&quot;Signup&quot;&gt;<br>    &lt;/form&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><p>The stuff in the {{ }} are the important parts.</p><p>{{ form.hidden_tag() }} enters a hidden field that prevents certain kind of exploits. As you can see, we are referencing the ‘form’ variable that we passed into the render_template in our view. We need to add a secret key to our app before the form will work. In the top of the app.py file, after creating the app, add this:</p><pre>app.secret_key = &#39;secretkeyhereplease&#39;</pre><p>Next, we add two fields with labels and input fields from the form class as well (email and password). Before testing the app, we should add some validation to the form. We want to ensure that the fields are not empty and that the email field contains a valid email. We add these validations in the forms.py file. Update forms.py to this:</p><pre>from flask_wtf import Form<br>from wtforms import StringField, PasswordField, SubmitField<br>from wtforms.validators import Email, DataRequired</pre><pre>class SignupForm(Form):<br>    email = StringField(&#39;email&#39;, <br>                validators=[DataRequired(),Email()])<br>    password = PasswordField(<br>                &#39;password&#39;, <br>                validators=[DataRequired()])<br>    submit = SubmitField(&quot;Sign In&quot;)</pre><p>We import validators from <a href="https://wtforms.readthedocs.org/en/latest/validators.html#built-in-validators">wtforms.validators</a> and add them to the fields as arguments.</p><p>Try restarting the application and go to localhost:5000/signup and test the application. We now have a way to sign users but not yet a way to store it. We will handle this next.</p><h3>5 Setting up the database</h3><p>We will use sqlite3 as our database and use flask-sqlalchemy as our ORM layer. SQLAlchemy makes it easy to work with databases and has lots of advanced features that are useful for more advanced applications. For our purpose here, using SQLAlchemy is a bit of an overkill, but it is easy to use and makes for clean coding.</p><p>We start by defining our database model in the models.py file. We just need a table with username (email) and password.</p><p>In models.py add this:</p><pre>from app import app<br>from flask_sqlalchemy import SQLAlchemy</pre><pre>db = SQLAlchemy(app)<br></pre><pre>class User(db.Model):<br>    email = db.Column(db.String(80), primary_key=True, unique=True)<br>    password = db.Column(db.String(80))</pre><pre>    def __init__(self, email, password):<br>        self.email = email<br>        self.password = password</pre><pre>    def __repr__(self):<br>        return &#39;&lt;User %r&gt;&#39; % self.email</pre><p>We start by importing the app from our app.py and import the SQLAlchemy from flask_sqlalchemy. With db=SQLAlchemy(app) we instantiate the SQLAlchemy class and pass it our app as the argument, this ‘attaches’ the instance to our application.</p><p>Our table (class) is defined as a subclass of the db.Model class; It will only contain two columns, email and password, which we set to strings and the email also as the primary key and we set its unique property to be true. The __init__ is the constructor is straightforward and we save the password as plain text, which is NOT what you want to do in production (and we can easily fix this later).</p><p>In order to actually create the database, we can do several things:</p><p>1. Enter an interactive python session, import the db instance and run the create_all() method (db.create_all()).<br>2. We create a function in the app.py file that runs create_all().</p><p>Here, we will use option 2.</p><p>First, import db into app.py by adding:</p><pre>from .models import db</pre><p>The define a function that calls create_all():</p><pre>def init_db():<br>    db.init_app(app)<br>    db.app = app<br>    db.create_all()</pre><p>It is a bit difficult to find good information on this (the init_db() structure) online, and several sources seems to disagree with each other. This <a href="http://piotr.banaszkiewicz.org/blog/2012/06/29/flask-sqlalchemy-init_app/">blog post</a> provides some insight.</p><p>And finally, we add a call to this function before the app is run:</p><pre>if __name__ == &#39;__main__&#39;:<br>    app.init_db()<br>    app.run(port=5000, host=&#39;localhost&#39;, debug=True)</pre><p>This will create a sqlite3 database in the /db folder. If a database already exist, then a new one will NOT be created, and thus this construction will not work if you want to adjust your database later in development; For this, you would need to construct migrations, but this is beyond this article.</p><h3>6 Save user to database</h3><p>We need to hook up our form on /signup to the database, so when a user fills out the form, the information is saved in the database.</p><p>First, we need to check that the email the user enters doesn’t already exist in the database, and secondly, if the email doesn’t exist then create the user in the database.</p><h3>6.1 Check if email already exists</h3><p>This is easy, we simply query the database to see if the email address exist, and if it does, we DON’T create the user but returns an error message. If the email doesn’t exist, we create it. We add this in the app.py file in the signup() function like this: (if User.query.filter_by(email = form.email.data).first():)</p><pre>@app.route(&#39;/signup&#39;, methods=[&#39;GET&#39;, &#39;POST&#39;])<br>def signup():<br>    form = SignupForm()</pre><pre>    if request.method == &#39;GET&#39;:<br>        return render_template(&#39;signup.html&#39;, form = form)<br>    elif request.method == &#39;POST&#39;:<br>        if form.validate_on_submit():<br>            if User.query.filter_by(email=form.email.data).first():<br>                return &quot;Email address already exists&quot;<br>            else:<br>                return &quot;will create user here&quot;<br>    else:<br>        return &quot;Form didn&#39;t validate&quot;</pre><h3>6.2 Add user to database</h3><p>If the form validates and the email address doesn’t exist, we add the email address by adding this:</p><pre>newuser = User(form.email.data, form.password.data)<br>db.session.add(newuser)<br>db.session.commit()</pre><p>Like this:</p><pre>@app.route(&#39;/signup&#39;, methods=[&#39;GET&#39;, &#39;POST&#39;])<br>def signup():<br>    form = SignupForm()</pre><pre>    if request.method == &#39;GET&#39;:<br>        return render_template(&#39;signup.html&#39;, form = form)<br>    elif request.method == &#39;POST&#39;:<br>        if form.validate_on_submit():<br>            if User.query.filter_by(email=form.email.data).first():<br>                return &quot;Email address already exists&quot;<br>            else:<br>                newuser = User(form.email.data, form.password.data)<br>                db.session.add(newuser)<br>                db.session.commit()</pre><pre>                return &quot;User created!!!&quot;        <br>        else:<br>            return &quot;Form didn&#39;t validate&quot;</pre><h3>7 Flask-login</h3><p>Flask-login requires a few things to work:</p><p>1. Import of flask-login.<br>2. User class to fulfil certain criteria.<br>3. Certain functions must exist.</p><h3>7.1 Import flask-login</h3><p>In the app.py we must import some stuff from flask-login:</p><pre>from flask_login import LoginManager, login_user, login_required, logout_user</pre><p>We then need to initialize the LoginManager:</p><pre>login_manager = LoginManager()<br>login_manager.init_app(app)</pre><p>The ‘login_user’, ’login_required’, ’logout_user’ will be used later.</p><h3>7.2 Amend the User class (model)</h3><p>Flask-login requires our User class to contain certain methods:</p><p>is_authenticated, is_active, is_anonymous and get_id.</p><p>Flask-login does contain a UserMixin class we can subclass that contains implementations of these methods, but I’ll implement them manually in models.py:</p><pre>class User(db.Model):<br>    email = db.Column(db.String(80), primary_key=True, unique=True)<br>    password = db.Column(db.String(80))</pre><pre>    def __init__(self, email, password):<br>        self.email = email<br>        self.password = password</pre><pre>    def __repr__(self):<br>        return &#39;&lt;User %r&gt;&#39; % self.email</pre><pre>    def is_authenticated(self):<br>        return True</pre><pre>    def is_active(self):<br>        return True</pre><pre>    def is_anonymous(self):<br>        return False</pre><pre>    def get_id(self):<br>        return str(self.email)</pre><p>The ‘get_id()’ method must return a unicode identified for our user; In this case the primary key is email, so it returns this. If you unique id had been an integer, you would have to return the string value of this (eg. str(self.id)).</p><h3>7.3 Create default functions</h3><p>In our app.py, we need to create a required function with decorator ‘user_loader’, which is a callback that takes the unicode id of the user and returns the user object. Add this to app.py:</p><pre>@login_manager.user_loader<br>def load_user(email):<br>    return User.query.filter_by(email = email).first()</pre><p>This is straightforward, given the email the function returns the user with this given email address.</p><h3>7.4 Add login functionality</h3><p>We want to add login to the system in two places: 1. in a separate login page and 2. after a user successfully signs up.</p><p>First however, we will create a route that requires a user to be logged in, which is easily done with the @login_required decorator:</p><pre>@app.route(&#39;/protected&#39;)<br>@login_required<br>def protected():<br>    return &quot;protected area&quot;</pre><p>Lets add login when a user successfully signs up. This is easily done in our signup() function in app.py by simply adding login_user(newuser) to the function:</p><pre>@app.route(&#39;/signup&#39;, methods=[&#39;GET&#39;, &#39;POST&#39;])<br>def signup():<br>    form = SignupForm()</pre><pre>    if request.method == &#39;GET&#39;:<br>        return render_template(&#39;signup.html&#39;, form = form)<br>    elif request.method == &#39;POST&#39;:<br>        if form.validate_on_submit():<br>            if User.query.filter_by(email=form.email.data).first():<br>                return &quot;Email address already exists&quot; <br>            else:<br>                newuser = User(form.email.data, form.password.data)<br>                db.session.add(newuser)<br>                db.session.commit()</pre><pre>                login_user(newuser)</pre><pre>                return &quot;User created!!!&quot;        <br>         else:<br>             return &quot;Form didn&#39;t validate&quot;</pre><p>Now, lets create a separate login template. We can take the signup template and save it as login.html and modify it slightly:</p><pre>&lt;!DOCTYPE html&gt;<br>&lt;html lang=&quot;en&quot;&gt;<br>&lt;head&gt;<br>    &lt;meta charset=&quot;UTF-8&quot;&gt;<br>    &lt;title&gt;Login&lt;/title&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>    &lt;h1&gt;Login&lt;/h1&gt;</pre><pre>    &lt;form method=&quot;POST&quot; action=&quot;/login&quot;&gt;<br>        {{ form.hidden_tag() }}<br>        {{ form.email.label }} {{ form.email(size=20) }}<br>        {{ form.password.label }} {{ form.password(size=20) }}<br>        &lt;input type=&quot;submit&quot; value=&quot;login&quot;&gt;<br>    &lt;/form&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><p>Next, we need to create a /login route in app.py:</p><pre>@app.route(&#39;/login&#39;, methods=[&#39;GET&#39;,&#39;POST&#39;])<br>def login():<br>    form = SignupForm()</pre><pre>    if request.method == &#39;GET&#39;:<br>        return render_template(&#39;login.html&#39;, form=form)<br>    elif request.method == &#39;POST&#39;:<br>        if form.validate_on_submit():<br>            user=User.query.filter_by(email=form.email.data).first()<br>            if user:<br>                if user.password == form.password.data:<br>                    login_user(user)<br>                    return &quot;User logged in&quot;                <br>                else:<br>                    return &quot;Wrong password&quot;            <br>            else:<br>                return &quot;user doesn&#39;t exist&quot;        <br>     else:<br>            return &quot;form not validated&quot;</pre><p>Here we are reusing the SignupForm because it contains the fields we need for logging in as well. If the route /login receives a GET request, the template is returned. If a POST request is received the form is validated, and it checks if the user exists (by checking if a user with the email entered exists in the database), and if a user exist, then goes on to check the password. If the password matches, then the system logs in the user.</p><p>Try it, create a user or log in, and go to localhost:5000/protected, you should see the logged in return message.</p><h3>7.5 Logout</h3><p>We can easily add a logout route as well:</p><pre>@app.route(&quot;/logout&quot;)<br>@login_required<br>def logout():<br>    logout_user()<br>    return &quot;Logged out&quot;</pre><p>If you navigate to localhost:5000/logout as a logged in user, you will be logged out. Try it, and see what happens when you then navigate to localhost:5000/protected.</p><h3>8 Conclusions</h3><p>That is it for this tutorial. I hope it is clear how you can implement flask-login with a simple database. Remember, you should not store passwords in plain-text as is done in this example, and neither is the code here particular clean, but I hope that it is easy to understand and that it helps in understanding the mechanics of getting flask-login to work.</p><p>References:<br><a href="http://alextechrants.blogspot.dk/2013/11/10-common-stumbling-blocks-for.html">10 common stumbling blocks for SQLAlchemy newbies</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d68791e9b5b5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>