<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Martín Lamas on Medium]]></title>
        <description><![CDATA[Stories by Martín Lamas on Medium]]></description>
        <link>https://medium.com/@martin.lamas?source=rss-69a70c39b5e5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*-xlDzlHQNEXJmNLYcZ1dmw.jpeg</url>
            <title>Stories by Martín Lamas on Medium</title>
            <link>https://medium.com/@martin.lamas?source=rss-69a70c39b5e5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 18:10:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@martin.lamas/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a GUI application with Python and Qt]]></title>
            <link>https://medium.com/trabe/building-a-gui-application-with-python-and-qt-147579ae8e0a?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/147579ae8e0a</guid>
            <category><![CDATA[qt]]></category>
            <category><![CDATA[gui]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Tue, 14 Sep 2021 08:19:42 GMT</pubDate>
            <atom:updated>2021-09-15T09:03:35.132Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a GUI Application with Python and Qt</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*8ozA6p4Xs_XCMGZ7" /><figcaption>Photo by <a href="https://unsplash.com/@maxcodes">Maxwell Nelson</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>I recently started developing a desktop application in which I needed to implement a <a href="https://en.wikipedia.org/wiki/Graphical_user_interface">GUI</a>. Of course, I used <a href="https://www.python.org/">Python</a> to develop this project :)</p><p>The first thing I did was look for a suitable framework or library to develop GUIs in Python. I found the following:</p><ul><li><a href="https://www.qt.io/qt-for-python">Qt for Python</a> / <a href="https://riverbankcomputing.com/software/pyqt/intro">PyQt</a>: these frameworks provide a set of Python bindings for the <a href="https://www.qt.io/product/framework">Qt</a> application framework.</li><li><a href="https://www.wxpython.org/">wxPython</a>: a cross-platform GUI toolkit used to create native user interfaces for Python applications. It runs with little or no modifications on Windows, MacOS and Linux or other unix-like systems.</li><li><a href="https://docs.python.org/3/library/tkinter.html">tkinter</a>: the standard Python interface to the <a href="https://www.tcl.tk/">Tk GUI toolkit</a>. Both Tk and tkinter are available on most Unix platforms, as well as on Windows systems. (Tk itself is not part of Python; it is maintained at <a href="https://www.activestate.com/">ActiveState</a>).</li><li><a href="https://pygobject.readthedocs.io/en/latest/">PyGObject</a>: a Python package which provides bindings for <a href="https://docs.gtk.org/gobject/">GObject</a> based libraries such as <a href="https://www.gtk.org/">GTK</a>, <a href="https://gstreamer.freedesktop.org/">GStreamer</a>, <a href="https://webkitgtk.org/">WebKitGTK</a>, <a href="https://developer.gnome.org/glib/">GLib</a>, <a href="https://developer.gnome.org/gio/stable/">GIO</a> and many more. It is the way to go for developing a Python application for GNOME or GTK.</li><li><a href="https://github.com/PySimpleGUI/PySimpleGUI">PySimpleGUI</a>: a GUI library that transforms tkinter, Qt, Remi and WxPython into portable people-friendly Pythonic interfaces.</li></ul><p>I found in the Qt for Python the best alternative to develop my desktop application: the framework supports a lot of widgets and features and is available for all desktop platforms.</p><h3>About Qt for Python</h3><p>Qt for Python provide a set of Python bindings for the Qt application framework and a collection of tools for building GUI applications. These bindings and tools are available for all platforms, so we can develop GUI applications for Windows, MacOS and GNU/Linux.</p><p>One of the most interesting tools that I found is the <a href="https://www.qt.io/product/development-tools">Qt Creator</a> IDE. We can fully develop our projects using this IDE, but for me, the most powerful feature that I found is the GUI design editor. The GUI design editor simplifies the creation of GUIs by allowing us to arrange graphical control elements using a <a href="https://en.wikipedia.org/wiki/Drag-and-drop">drag-and-drop</a> <a href="https://en.wikipedia.org/wiki/WYSIWYG">WYSIWYG</a> editor:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kyyIBgRS3AnHm9tV2CSfJg.png" /><figcaption>Qt Creator GUI design editor</figcaption></figure><h3>Building our first Qt application</h3><p>Now we are going to learn how to create a very simple Todo list application. The source code of this example is available in <a href="https://github.com/martinlamas/pyside6-todo">this GitHub repository</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*4Ges6UIFeDLqCZSMpUC0aA.png" /><figcaption>A very simple Todo list application</figcaption></figure><p>The first thing will be to create the entry point of the application:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e92346280e5ed32465ed1ee0ace7457a/href">https://medium.com/media/e92346280e5ed32465ed1ee0ace7457a/href</a></iframe><p>We start by creating a QApplication instance. This instance manages the GUI application’s control flow and main settings. This is a mandatory step when developing a Qt application. Then, we create the main window (more on this later) and we call the show method to open it. Finally, at line 13, we call app.exec_() to start up the event loop. The event loop handles the user interactions with the application and it runs until the user closes the application.</p><p>The UI components in the Qt Framework are grouped hierarchically. All of them extend the QWidget class. Therefore, our MainWindow class must extend the QWidget class:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/04a92c2c48e4c450a86a2be39b8bd1a0/href">https://medium.com/media/04a92c2c48e4c450a86a2be39b8bd1a0/href</a></iframe><p>In our MainWindow constructor we must define the window settings and we must create the widgets, placing them in a layout. For example, setting the minimun size of the window (setMinimumSize call at line 7).</p><p>Layout-wise, you can choose between:</p><ul><li><a href="https://doc.qt.io/qt-5/qhboxlayout.html">QHBoxLayout</a>: widgets are lined up horizontally.</li><li><a href="https://doc.qt.io/qt-5/qvboxlayout.html">QVBoxLayout</a>: widgets are lined up vertically.</li><li><a href="https://doc.qt.io/qt-5/qformlayout.html">QFormLayout</a>: manages forms of input widgets and their associated labels.</li><li><a href="https://doc.qt.io/qt-5/qgridlayout.html">QGridLayout</a>: widgets are placed in a grid.</li><li><a href="https://doc.qt.io/qt-5/qstackedlayout.html">QStackedLayout</a>: provides a stack of widgets where only one widget is visible at a time.</li></ul><p>This example uses a grid layout. At 9–10 lines we are creating the layout and setting it in the window. Later, in the buildWindowContent method, we place the widgets by calling the addWidget method of the layout instance. We are passing three arguments to this method: the widget, the grid row and the column in which the component should be placed.</p><p>The framework provides <a href="https://doc.qt.io/qtforpython/PySide6/QtWidgets/index.html#module-PySide6.QtWidgets">many widgets</a> that we can use in our projects. In this sample application, we are using some of them:</p><ul><li><a href="https://doc.qt.io/qtforpython/PySide6/QtWidgets/QLabel.html">QLabel</a>: text or image display.</li><li><a href="https://doc.qt.io/qtforpython/PySide6/QtWidgets/QTextEdit.html">QTextEdit</a>: edit and display both plain and rich text.</li><li><a href="https://doc.qt.io/qtforpython/PySide6/QtWidgets/QListWidget.html">QListWidget</a>: an item-based list.</li><li>QListWidgetItem: an item for use with the <a href="https://doc.qt.io/qtforpython/PySide6/QtWidgets/QListWidget.html#PySide6.QtWidgets.PySide6.QtWidgets.QListWidget">QListWidget</a>.</li><li><a href="https://doc.qt.io/qtforpython/PySide6/QtWidgets/QPushButton.html">QPushButton</a>: a command button.</li></ul><p>Also, in the buildMenuBar method, we are building a menu bar using the following widgets: <a href="https://doc.qt.io/qtforpython/PySide6/QtWidgets/QMenuBar.html">QMenuBar</a>, a horizontal menu bar, and <a href="https://doc.qt.io/qtforpython/PySide6/QtWidgets/QMenu.html">QMenu</a>, a menu widget for use in menu bars, context menus, and other pop-up menus. We are using an ampersand in the menu labels to define some keyboard shortcuts.</p><h3>UI interaction: signals and slots</h3><p>An aspect not covered yet is how to bind the UI user interactions with the application logic. For example, how to add a task to the list widget when user inputs a task and pushes the <em>Add</em> task button. For this purpose the framework provides the <a href="https://doc.qt.io/qt-5/signalsandslots.html">signals and slots</a>. A signal is emitted when a particular event occurs, and a slot is a function that is called in response to a particular signal. We can connect a signal to one or more slots that will be invoked when the signal is emitted.</p><p>At line 44 of the code we are connecting the clicked signal of the addTaskButton to the addTask slot (that is defined as a method of the MainWindow class). Here, we are doing the following:</p><ol><li>Read the task description that the user inputs in the taskEdit widget.</li><li>Create a checkable QListWidgetItem with the task and add it to taskList widget.</li><li>Clear the task’s input widget in order that the user can input a new task.</li></ol><p>At line 53 we connect the clicked signal of the deleteAllTasksButton to the deleteAllTasks method. In this method we are using the following ConfirmDialog to confirm the deletion of all tasks:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/00b0ccc3022689b7c577231404ec59b4/href">https://medium.com/media/00b0ccc3022689b7c577231404ec59b4/href</a></iframe><p>This confirm dialog class extends the QMessageBox, a class that provides a modal dialog for informing the user or for asking the user a question and receiving an answer. If the operation is confirmed then all tasks are deleted (line 86 of the MainWindow class code).</p><p>There are more samples of signals and slots in the example. Feel free to explore them and play with the code :)</p><h3>Summing up</h3><p>Qt for Python is one of the best GUI framework available for Python. We can build standard desktop GUI applications but the framework also comes with a lot of features such as data visualization or OpenGL support.</p><p>The developer tools available are another powerful feature of the framework. For example, with the Qt Creator IDE we can quickly design GUIs using drag and drop.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=147579ae8e0a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/building-a-gui-application-with-python-and-qt-147579ae8e0a">Building a GUI application with Python and Qt</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Rich Console Interfaces in Python]]></title>
            <link>https://medium.com/trabe/building-rich-console-interfaces-in-python-16338cc30eaa?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/16338cc30eaa</guid>
            <category><![CDATA[rich]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[console]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 28 Sep 2020 08:03:14 GMT</pubDate>
            <atom:updated>2020-09-28T08:03:14.370Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*nn1IvQqO_AvSPZcW" /><figcaption>Photo by <a href="https://unsplash.com/@cookiethepom">Cookie the Pom</a><strong> </strong>on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>Lately, I have been working on a project in which I needed to pretty print some data on the console. That was how I discovered <a href="https://github.com/willmcgugan/rich">Rich</a>, a Python library for rich text and beautiful formatting in the terminal.</p><p>In this story I will show you some of the features of this library, and how you can use it in your projects.</p><h3>Installing the library</h3><p>First, we need to install the library in our environment using <a href="https://pypi.org/project/pip/">pip</a>.</p><pre>pip install rich</pre><p>Once installed we can try some of the features of the library.</p><h3>Rich text printing</h3><p>One of the features that I use the most is to print rich text.</p><p>To do so, we must first import and build a <a href="https://rich.readthedocs.io/en/latest/reference/console.html#rich.console.Console">Console</a> object:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8cb3b1e1b141eb55f94583716ca915fb/href">https://medium.com/media/8cb3b1e1b141eb55f94583716ca915fb/href</a></iframe><p>Now, we can style and print text on the console using the print method:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bc9b30eec2dd3c8be82a77df51706c8a/href">https://medium.com/media/bc9b30eec2dd3c8be82a77df51706c8a/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*I6wlZUQmk9CGIUTw9iBDUQ.png" /><figcaption>Console output of the previous example</figcaption></figure><p>In this example we can see that:</p><ul><li>We can either style text fragments inline or style the whole text by passing the style keyword argument.</li><li>Inline styles are defined using strings. These strings contain keywords to set the colors and other attributes for the foreground styles (like bold white) and for the background color (on blue).</li><li>If we pass a hash to print it will be automatically styled.</li><li>We can print emojis <a href="https://www.webfx.com/tools/emoji-cheat-sheet/">using colon notation</a>.</li></ul><p>This is a list with the available attributes to define styles:</p><ul><li>bold or b for bold text.</li><li>blink for text that flashes (use this one sparingly).</li><li>blink2 for text that flashes rapidly (not supported by most terminals).</li><li>conceal for concealed text (not supported by most terminals).</li><li>italic or i for italic text (not supported on Windows).</li><li>reverse or r for text with reversed foreground and background colors.</li><li>strike or s for text with a line through it.</li><li>underline or u for underlined text.</li></ul><p>It also supports the following attributes, which are not universally supported and may not work on your terminal:</p><ul><li>underline2 or uu for doubly underlined text.</li><li>frame for framed text.</li><li>encircle for encircled text.</li><li>overline or o for overlined text.</li></ul><p>We can choose colors between one of the 256 <a href="https://rich.readthedocs.io/en/latest/appendix/colors.html#appendix-colors">Standard Colors</a>. But we can also set a custom color using <a href="https://en.wikipedia.org/wiki/RGB_color_model">RGB</a> codes. In the following example we print the same color using both its standard name and its RGB code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79d2b17b2ee57db576d70f73ad84fdb3/href">https://medium.com/media/79d2b17b2ee57db576d70f73ad84fdb3/href</a></iframe><p>Another useful feature of the library is the logger. We use this feature calling the log method on the Console instance. This method prints an output with three columns: the current time, the data to print and the file and line from which the call to log was made. As we have seen before, the library does pretty format Python structures and <a href="https://docs.python.org/3.4/library/functions.html?highlight=repr#repr">repr</a> strings. If we log a collection (i.e. a dict or a list) the library pretty prints it so that it fits in the available terminal space.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/564496f312e29faa9429d20ae43350f2/href">https://medium.com/media/564496f312e29faa9429d20ae43350f2/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*W6U4g817conWucXK4pUg_Q.png" /><figcaption>Console output of the previous example</figcaption></figure><h3>Tables</h3><p>Another useful feature of the library is the ability to render tables on the terminal.</p><p>In the following example, we use the <a href="https://rich.readthedocs.io/en/latest/tables.html">Table</a> class to create a new table instance and then we call the add_column and add_row methods to define the table columns and add the data rows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf11db3a37d74479e735a5699647c1b3/href">https://medium.com/media/cf11db3a37d74479e735a5699647c1b3/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*5eA7viGf7K76UmIwpQqrJA.png" /><figcaption>Console output of the table example</figcaption></figure><p>We define the table header style and the title passing the show_header, header_style and title keyword arguments in the constructor. For a headless table we have to set the keyword argument show_header to False and skip the others.</p><p>The column definition is performed by calling the add_column method. We simply call this method as many times as columns we want to create. We have to provide the title of the column and, optionally, the justification of the data cells (left, center or right).</p><p>Once the table structure is defined, we can insert a data row by calling the add_row method. We must pass the data as arguments in the same order the columns were defined. Each cell can be styled individually.</p><p>You can also change the default border using the <a href="https://rich.readthedocs.io/en/latest/appendix/box.html#appendix-box">box</a> keyword in the Table constructor:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/afbc6354dde2a54de36f463ad547d85f/href">https://medium.com/media/afbc6354dde2a54de36f463ad547d85f/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*NuXXhSFeJ5GJ10EUKoi15g.png" /><figcaption>Console output of the table without borders</figcaption></figure><h3>Progress bars</h3><p>To track long-running tasks we can use a progress bar to improve the user experience. We need to import the track function to use the basic progress bar API. Then, we can wrap any sequence in this function and iterate over the result.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3d5f8d0414655742377e43f1c52dcc95/href">https://medium.com/media/3d5f8d0414655742377e43f1c52dcc95/href</a></iframe><p>In this example, we iterate over a list of remote files in order to download them. Each iteration a file is downloaded and the progress bar is updated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*lUd0bdZ4hp5IL3P8PscK3w.png" /><figcaption>A progress bar. As expected</figcaption></figure><p>We can use the advanced API if we need to further customize our progress bar. In the following example we have a customized version of the example above:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/235ea862d7f6c44d3aabc6dac518521e/href">https://medium.com/media/235ea862d7f6c44d3aabc6dac518521e/href</a></iframe><p>In this example we first instantiate a <a href="https://rich.readthedocs.io/en/latest/reference/progress.html">Progress</a> object to define the different columns that compose the progress bar: the task description, the bar, a live progress showing how many files have been downloaded and the remaining time.</p><p>Then, we add a task to the Progress instance using the add_task method passing the description and the total steps (the number of the files).</p><p>Finally, we iterate the file list and for each iteration we update the progress status by calling the update method.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/1*IsnW-0ihG4lcEYrWrQqfOA.png" /><figcaption>An improved progress bar.</figcaption></figure><h3>Markdown rendering</h3><p>Another awesome<strong> </strong>Rich<strong> </strong>feature is markdown rendering.</p><p>Take the following markdown:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/89b8d1cd73800f9e02987fb92fc81df6/href">https://medium.com/media/89b8d1cd73800f9e02987fb92fc81df6/href</a></iframe><p>We can render it in the terminal with ease:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fbee65b7c1004117e41bf5a0d6eb0c87/href">https://medium.com/media/fbee65b7c1004117e41bf5a0d6eb0c87/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*PLf10LIXQmaY9uD7iRSjhg.png" /><figcaption>The formatted markdown output</figcaption></figure><h3>Other useful features</h3><p>There are other interesting functionalities of the library.</p><h4><strong>Columns</strong></h4><p>When we need to tabulate the data into columns we can create a <a href="https://rich.readthedocs.io/en/latest/reference/columns.html">Columns</a> instance and pass the list of elements to tabulate in the constructor.</p><p>The following example shows how to render a directory content using columns:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b82bfa1d107e30a9fd9e16fd88e59ede/href">https://medium.com/media/b82bfa1d107e30a9fd9e16fd88e59ede/href</a></iframe><p>We are using a helper function file_info to get the file info (name, type and file extension). We list the directory passed as command line argument and we wrap each found file into a <a href="https://rich.readthedocs.io/en/latest/reference/panel.html">Panel</a> object to show the info inside a box. And, in turn, we put each panel inside a Columns object.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*vSvr0AnmK4ci1BnUUhj0vA.png" /><figcaption>Files shown in columns</figcaption></figure><h4><strong>Syntax highlighting</strong></h4><p>Rich uses the <a href="https://pygments.org/">pygments</a> library under the hood to implement <a href="https://rich.readthedocs.io/en/latest/syntax.html">syntax highlighting</a>. We can use this feature directly building a <a href="https://rich.readthedocs.io/en/latest/reference/syntax.html">Syntax</a> object:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a2e418e2d9b12ff3789514f7bbc9279/href">https://medium.com/media/2a2e418e2d9b12ff3789514f7bbc9279/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/261/1*YRaypwUO07gprzy8WYQTeQ.png" /><figcaption>Syntax highlighted Python code</figcaption></figure><h4><strong>Tracebacks</strong></h4><p>We can handle and print <a href="https://rich.readthedocs.io/en/latest/traceback.html">beautiful tracebacks</a> in our programs. These tracebacks are easier to read and show more code than standard Python tracebacks.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c1cfe8eaa42508404530ff73d582a944/href">https://medium.com/media/c1cfe8eaa42508404530ff73d582a944/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*JXY7anwxUw9koF9mVTNxEA.png" /><figcaption>Beautiful tracebacks</figcaption></figure><h3>Summing up</h3><p>Rich is a good choice to improve the user experience of our console applications. In this story we have learned how to use the most interesting features and how to apply them with some simple examples.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=16338cc30eaa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/building-rich-console-interfaces-in-python-16338cc30eaa">Building Rich Console Interfaces in Python</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Static type checking in Python]]></title>
            <link>https://medium.com/trabe/static-type-checking-in-python-58a46cf94f8a?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/58a46cf94f8a</guid>
            <category><![CDATA[static-type-checking]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[mypy]]></category>
            <category><![CDATA[duck-typing]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 31 Aug 2020 08:01:03 GMT</pubDate>
            <atom:updated>2020-08-31T08:01:03.475Z</atom:updated>
            <content:encoded><![CDATA[<h3>Static Type Checking in Python</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*qf7Wp0dM5mBFTK7y" /><figcaption>Photo by <a href="https://unsplash.com/@sytha">Sylvain Thrd</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>Python 3.5 introduced <a href="https://www.python.org/dev/peps/pep-0484">Type Hints</a>, a new feature that allows developers to use static type checking in python.</p><h3>Type checking in Python</h3><p>Python is a <a href="https://realpython.com/lessons/duck-typing/">duck typed</a> programming language. Duck typing in computer programming is an application of the <a href="https://en.wikipedia.org/wiki/Duck_test">duck test</a>:</p><blockquote>If it walks like a duck and it quacks like a duck, then it must be a duck.</blockquote><p>Duck typing is a concept related to <a href="https://en.wikipedia.org/wiki/Type_system#Dynamic_type_checking_and_runtime_type_information">dynamic typing</a>, where the type or the class of an object is less important than the methods it defines. When you use duck typing, you do not check types at all. Instead, you check for the presence of a given method or attribute.</p><p>In the following gist you can see an example that demonstrates how any object may be used in any context, up until it is used in a way that it is not supported:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6c21a6c6cf55298fad08176536e073ce/href">https://medium.com/media/6c21a6c6cf55298fad08176536e073ce/href</a></iframe><p>Running the code above would result in the following output:</p><pre>Duck flying<br>Sparrow flying<br>AttributeError: &#39;Whale&#39; object has no attribute &#39;fly&#39;</pre><p>Duck and Sparrow are Flying animals because they know how to fly. Whale isn’t because it doesn’t :)</p><h3>Static type checking in Python</h3><p>Since Python 3.5 we can use the Type Hints to check the types in our python code. Now, we can write something similar like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8249ddd60deee7b1197aa021d60daea6/href">https://medium.com/media/8249ddd60deee7b1197aa021d60daea6/href</a></iframe><p>In the example above we have defined a function that takes a string type parameter and returns a string value. Then, if we call the greeting function we must pass a string value as parameter. We can assume that the returned value of the function will also be a string.</p><p>As its name suggests, the Type Hints are just that, hints, so we can write this code and run it without errors:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f5260800693fab7169c3a6cfde599ff5/href">https://medium.com/media/f5260800693fab7169c3a6cfde599ff5/href</a></iframe><p>So, how we can check these hints? We must use a static type checker for this purpose. <a href="http://mypy-lang.org/">mypy</a> is an optional static type checker for Python that aims to combine the benefits of duck typing and static typing. We can install mypy by running this command:</p><pre>$ python -m pip install mypy</pre><p>Once installed, we can check the last example:</p><pre>$ mypy sample.py<br>sample.py:4: error: Argument 1 to &quot;greeting&quot; has incompatible type &quot;int&quot;; expected &quot;str&quot;<br>Found 1 error in 1 file (checked 1 source file)</pre><p>As we can see an error is found because we are passing an integer value to the greeting function, which expected a string.</p><p>We can integrate this tool in our favorite editor and check these errors while coding. In Visual Studio Code, we can integrate it by installing the <a href="https://marketplace.visualstudio.com/items?itemName=ms-python.python">python extension</a> and adding the &quot;python.linting.mypyEnabled&quot;: true line to the .vscode/settings.json project file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/51ae9212502fce3bef32aed2434e7c6d/href">https://medium.com/media/51ae9212502fce3bef32aed2434e7c6d/href</a></iframe><h3>Type basics</h3><p>The most basic thing that we can do with the Type Hints is to assign types to variables using the built-in types:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/af6a4849c78e8215b75f3ff9c74886da/href">https://medium.com/media/af6a4849c78e8215b75f3ff9c74886da/href</a></iframe><p>Also, we can use the type hints for our functions, declaring the argument types but also the returned type:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/17fd6cfaf2a6faf91b2fc33821aa75be/href">https://medium.com/media/17fd6cfaf2a6faf91b2fc33821aa75be/href</a></iframe><p>Type hints can also be used with classes:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0abf55912d75bde4e99917592aee995d/href">https://medium.com/media/0abf55912d75bde4e99917592aee995d/href</a></iframe><h3>Declaring our own types</h3><p>While we can use the built-in types in our projects, we can also declare our own.</p><p>With type aliases we can declare a type assigning an existing type to an alias. For example, we could declare a Point type reusing the Tuple[float, float] type:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f74a3b90cac2d4e35136e7c6b82ceedc/href">https://medium.com/media/f74a3b90cac2d4e35136e7c6b82ceedc/href</a></iframe><p>The typing module also includes the NewType helper function that we can use to create distinct types. This function accepts two parameters: the name of the new type and the underlying type to use. In the following gist we refactored the last example in order to use the NewType helper:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9f2211f1bb36b981190be4f45fe02e99/href">https://medium.com/media/9f2211f1bb36b981190be4f45fe02e99/href</a></iframe><p>There are some differences between using aliases or types defined with the NewType helper. The use of a type alias declares two types to be <em>equivalent</em> to one another. Doing Point = Tuple[float, float] will make the static type checker treat Point as being <em>exactly equivalent</em> to Tuple[float, float]. This is useful when you want to simplify complex type signatures. In contrast, NewType declares one type to be a <em>subtype</em> of another. Doing Point= NewType(&#39;Point&#39;, Tuple[float, float]) will make the static type checker treat Point as a <em>subclass</em> of Tuple[float, float], which means a value of type Tuple[float, float] cannot be used in places where a value of type Point is expected. This is useful when you want to prevent logic errors with minimal runtime cost.</p><h3>Summing up</h3><p>Type Hints is a powerful feature that give us support to do type checking in our projects. This can help us to avoid code mistakes and it’s also useful to better understand the intent of the code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58a46cf94f8a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/static-type-checking-in-python-58a46cf94f8a">Static type checking in Python</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Loading binary files in Python that were written using C structures]]></title>
            <link>https://medium.com/trabe/loading-binary-files-in-python-that-were-written-using-c-structures-71cb76b7da6b?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/71cb76b7da6b</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[geonaute-onmove]]></category>
            <category><![CDATA[struct]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 27 Jul 2020 08:40:26 GMT</pubDate>
            <atom:updated>2020-07-27T08:40:26.869Z</atom:updated>
            <content:encoded><![CDATA[<h3>Loading Binary Files in Python that Were Written Using C Structures</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*IXN-c79iWRDkwNyc" /><figcaption>Photo by <a href="https://unsplash.com/@asoggetti">asoggetti</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>In C/C++ applications it’s usual to use <a href="https://en.cppreference.com/w/c/language/struct">structures</a> to group related data registers. The data contained into these structures can be persisted/loaded into/from a file through a serialization/deserialization process.</p><p>During the development of one of my pet projects I needed to analyze my workout statistics registered in my <a href="https://customercare.geonaute.com/hc/en-gb/sections/202504565-ONmove-220-GPS-watch">Geoanute ONMove 220 GPS watch</a>. In a previous research, I discovered that this watch stores the workout statistics and the GPS track log into binary files. The content of these files is defined by C structures, so I had to do some more research. Fortunately, I discovered the <a href="https://sourceforge.net/projects/omx2gpx">omx2gpx</a> project and I was able to know the format of these structures.</p><h3>The workout format</h3><p>The watch registers the workout into two separate files: a OMH file containing the workout data, such as the distance, the time and the date, and a OMD file containing the GPS track data.</p><p>In this post I´ll show you how to load the workout data stored into the OMH file using Python. As I said before, this file holds the serialized version of a structure containing the data. In the following gist we can see the format of this structure stored into the OMH file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b54e6de0768122c401bcd7cd7498b30e/href">https://medium.com/media/b54e6de0768122c401bcd7cd7498b30e/href</a></iframe><h3>Loading the data by reading the raw bytes</h3><p>A first approach to load this data is just to open the file in binary mode and read the different fields in byte chunks, performing the needed conversions:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f0cf8b3b727cbf023666f38c5c7c6e2e/href">https://medium.com/media/f0cf8b3b727cbf023666f38c5c7c6e2e/href</a></iframe><p>Notice that, inside the file, the structure registers are stored sequentially. Also, we have to keep in mind the <a href="https://en.wikipedia.org/wiki/Endianness">byte ordering</a> of these registers.</p><p>This is a valid and simple approach but I thought that there may be a more elegant way to do this. That was how I discovered the <a href="https://docs.python.org/3/library/struct.html">struct</a> module from the python standard library.</p><h3>Loading the data using struct</h3><p>As we can see in the docs, the struct module performs conversions between Python values and C structs represented as Python <a href="https://docs.python.org/3/library/stdtypes.html#bytes">bytes</a> objects. We can use this module to handle binary data stored in files or network connections, among other sources.</p><p>Using this module is simple: we need to define a declarative format string to specify the type of data being packed/unpacked. Also, with some special characters we can control the <a href="https://docs.python.org/3/library/struct.html#struct-alignment">Byte Order, Size, and Alignment</a>.</p><p>In the following table we can see the available format characters with their type equivalences:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*bfs35AIF4DXOHbP8KCIEHg.png" /></figure><p>With these format characters we declare the format string that specifies the structure. Then, loading the workout data is pretty simple now:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fbf29b4c8d35bdad706857ac3f5ee14f/href">https://medium.com/media/fbf29b4c8d35bdad706857ac3f5ee14f/href</a></iframe><p>We only need to pass two arguments to the unpack function: the format string &lt;I4H8BH18B5H10B and the byte data. With the first character of the format string we specify the byte ordering: we’re using &lt; to tell the unpack function that the registers are stored using little-endian byte ordering. Next, we use the format characters to describe the format of the structure registers: one unsigned integer I for the distance (field 0); four unsigned short 4H for the time (1), average speed (2), max speed (3) and kilocalories (4); eight unsigned char 8B for the average heart rate (5), max heart rate (6), year in short format (7), month (8), day (9), hour (10), minute (11) and file number; one unsigned short H; eighteen unsigned char 18B; five unsigned short 5H and ten unsigned char 10B.</p><p>Notice that although we’re only getting some registers of the struct we must provide a format string for the whole structure.</p><p>And that’s it. Using the struct module we can read binary files in a simple way. I hope you found this post useful :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71cb76b7da6b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/loading-binary-files-in-python-that-were-written-using-c-structures-71cb76b7da6b">Loading binary files in Python that were written using C structures</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A brief introduction to WebGL]]></title>
            <link>https://medium.com/trabe/a-brief-introduction-to-webgl-5b584db3d6d6?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b584db3d6d6</guid>
            <category><![CDATA[webgl]]></category>
            <category><![CDATA[3d-graphics]]></category>
            <category><![CDATA[opengl]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 06 Jul 2020 08:01:01 GMT</pubDate>
            <atom:updated>2020-07-06T08:01:01.593Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*O9O6UIUtIBalthbt" /><figcaption>Photo by <a href="https://unsplash.com/@pursuit_of_shutter">Ranit Chakraborty</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>In this post I will cover a brief introduction to <a href="https://www.khronos.org/webgl">WebGL</a> for all those who want to get started with 3D web graphics. I will explain how the Rendering Pipeline works and we will render a simple graphic as an example.</p><h3>What is WebGL</h3><p>WebGL is a cross-platform, royalty-free API used to create 3D graphics in a Web browser. Based on <a href="https://www.khronos.org/opengles">OpenGL ES</a> 2.0, WebGL uses the OpenGL shading language, <a href="https://www.khronos.org/opengl/wiki/Core_Language_(GLSL)">GLSL</a>, and offers the familiarity of the standard OpenGL API.</p><p>The main features of WebGL are the following:</p><ul><li>Cross-platform and Cross-browser</li><li>GPU 3D acceleration</li><li>Native API with GLSL support</li><li>Works inside a <a href="https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API">canvas</a></li><li>Integration with <a href="https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model">DOM interfaces</a></li></ul><h3>How it works</h3><p>In order to render some graphics, we will need to learn what is the <a href="https://www.khronos.org/opengl/wiki/Rendering_Pipeline_Overview">Rendering Pipeline</a> and how it works.</p><p>The Rendering Pipeline is the sequence of stages that WebGL takes when rendering 3D graphics. This rendering model fits properly since the GPU is a highly parallel processor: pipeline stages run simultaneously in the GPU processing units.</p><p>In the following image we can see the stages of the rendering pipeline:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*_z7Vbb0msvXsUFq3sSMZAg.png" /><figcaption>Rendering Pipeline</figcaption></figure><p>The output from a stage is used as the input for the next stage, and so on. The <a href="https://www.khronos.org/opengl/wiki/Vertex_Shader">Vertex Shader</a> and <a href="https://www.khronos.org/opengl/wiki/Fragment_Shader">Fragment Shader</a> stages are programmable, so we can program our own.</p><p>The Vertex Shader handles the vertex processing of our graphic. At this stage we must provide a stream with the vertices. Then, the Vertex Shader receives a single vertex from this stream and generates a single vertex to the output vertex stream. There must be a 1:1 mapping from input vertices to output vertices. This is the stage in which we can apply vertex transformations, such as rotations, translations or scaling.</p><p>The Fragment Shader process a <a href="https://www.khronos.org/opengl/wiki/Fragment">Fragment</a> generated by the <a href="https://www.khronos.org/opengl/wiki/Rasterization">Rasterization</a> stage into a set of colors and a single depth value. This is the stage after a <a href="https://www.khronos.org/opengl/wiki/Primitive">primitive</a> is rasterized (at the rasterization stage the geometric primitives are converted to fragments related to the pixel area). For each sample of the pixels covered by a primitive, a “fragment” is generated. Each fragment has a <a href="https://www.khronos.org/opengl/wiki/Window_Space">Window Space</a> position, a few other values, and it contains all of the interpolated per-vertex output values from the last <a href="https://www.khronos.org/opengl/wiki/Vertex_Processing">Vertex Processing</a> stage. This is the stage in which we can apply colors or textures to the graphics.</p><p>In the next section we will see how to render a simple triangle as example. To do it, we will program a Vertex Shader and a Fragmend shader.</p><h3>Rendering a triangle in WebGL</h3><p>As we said before, we will need to define a canvas to render our graphics inside it, so we will make a simple HTML template like the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15288462813990a40c27990f03925a72/href">https://medium.com/media/15288462813990a40c27990f03925a72/href</a></iframe><p>The width and height attributes of the canvas define the viewport size, but there is a consideration that we have to take into account: WebGL expects all the vertices that we want to become visible, to be in Normalized Device Coordinates after each vertex shader run. That is, the x, y and z coordinates of each vertex should be between -1.0 and 1.0 values; coordinates outside this range will not be visible. These Normalized Device Coordinates and the size of the viewport are then given to the rasterizer to transform them to 2D coordinates/pixels on your screen, so we must worry about taking into account the viewport size and its aspect ratio.</p><p>We will use a square canvas to make this example simpler. We will need to define three vertices to make the triangle, as we can see in the following image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*PBe3af8A9lJElZ0G6eSZYA.png" /><figcaption>Triangle coordinates</figcaption></figure><p>In the script section we will declare the vertex array first:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97ba229ffb7462eb985b76e06933e280/href">https://medium.com/media/97ba229ffb7462eb985b76e06933e280/href</a></iframe><p>We must skip the z coordinate since we are drawing in 2D.</p><p>Then, we will initialize the context and we will define the vertex shader first:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3bff0db951efa5a34eabdc9f66759c7e/href">https://medium.com/media/3bff0db951efa5a34eabdc9f66759c7e/href</a></iframe><p>As we learned before, the vertex shader runs for each input vertex and generates a new one as output. In the example, we use the position attribute as input (attributes are inputs to a Vertex Shader that get their data from buffers, more on this later) and we place the output in the gl_Position variable (the built-in gl_Position variable holds the output vertex in the Vertex Shader). No conversions are made so the output vertex will be the same as the input one. We will see how to initialize the buffer with the input vertex array in a moment.</p><p>Now we will define the Fragment Shader:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/34f7f078c87b4c041744db1cb32e6fef/href">https://medium.com/media/34f7f078c87b4c041744db1cb32e6fef/href</a></iframe><p>Here we are using a uniform that holds the color of the fragment (uniforms are values that stay the same for all vertices of a single draw call). Just as the Vertex Shader, the built-in gl_FragColor variable holds the output color of the shader. As we can see no color conversions are made.</p><p>At last, we will compile these shaders so that the GPU can run them and we will define a WebGL program attaching these shaders to it:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0cf1379d8175a55b23314694531ac854/href">https://medium.com/media/0cf1379d8175a55b23314694531ac854/href</a></iframe><p>Once we have defined and compiled these shaders we must provide the input data of our triangle.</p><p>First, we will create a array buffer and fill it with the vertex coordinates. The Vertex Shader we defined before will use this buffer as input through the position attribute. To make this, we create the buffer with the gl.createBuffer method and bind it as a gl.ARRAY_BUFFER type. Then, we fill this buffer with data using the gl.bufferData method. Finally, we point the position attribute to the buffer and bind it calling the gl.vertexAttribPointer and gl.enableVertexAttribArray methods, respectively:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/081541055ee644eef9519abd31b458e7/href">https://medium.com/media/081541055ee644eef9519abd31b458e7/href</a></iframe><p>Now we will define the color using the gl.uniform4f method and pass it to the Fragment Shader through the color uniform:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/164db05820c2de8a754df6addf9cc860/href">https://medium.com/media/164db05820c2de8a754df6addf9cc860/href</a></iframe><p>Note that the color coordinates must be defined using Normalized Device Coordinates, so we compute each coordinate using the following formula: RGBA normalized coordinate = RGBA coordinate / 255.</p><p>At this point we have all we need to render the triangle. Then, we clear the canvas background and we render the triangle by calling the gl.drawArrays method:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/751d478413850b98158b4dff2b7d691b/href">https://medium.com/media/751d478413850b98158b4dff2b7d691b/href</a></iframe><p>The gl.drawArrays method uses the bounded array buffer as input data. In this example the three vertices are taken from the buffer and the gl.TRIANGLES mode is selected to render the shape. In WebGL the following shapes are available:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/0*HQHB5lCGqlOUiysy.jpg" /><figcaption>Available WebGL shapes</figcaption></figure><p>As you can see we could render a wireframe of our triangle simply replacing the mode with the gl.LINE_LOOP value.</p><p>Finally, we can see the triangle opening the template in the web browser. Something like the following :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*i8R70Cd_KznxT83fKVvlxw.png" /><figcaption>Rendered WebGL triangle</figcaption></figure><h3>Summing up</h3><p>Currently, the WebGL API is available on most web browsers. We can use it to render accelerated 2D and 3D graphics in our web applications. Some high-level libraries are built on top of WebGL API making the programmer’s job easier. However, it is interesting to learn the low-level WebGL API and how the rendering pipeline works.</p><p>In future posts we will see how to render 3D shapes and how to apply some transformations like rotation or translation. Also, we will see how to apply textures and we will take a look at some interesting libraries that work on top of WebGL.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b584db3d6d6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/a-brief-introduction-to-webgl-5b584db3d6d6">A brief introduction to WebGL</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The danger of storing third-party objects in a React component state]]></title>
            <link>https://medium.com/trabe/the-danger-of-storing-third-party-objects-in-a-react-component-state-68b7777f173c?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/68b7777f173c</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[react-admin]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 03 Feb 2020 09:01:01 GMT</pubDate>
            <atom:updated>2020-02-03T09:01:01.367Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*exQvtl_Jr5hBk7Hd" /><figcaption>Photo by <a href="https://unsplash.com/@opticonor">Conor Luddy</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>During the development of one of my <a href="https://reactjs.org/">React</a> projects I made a mistake using the <a href="https://reactjs.org/docs/hooks-state.html">useState</a> hook by saving a third-party object in the component state. It took me some time to fix it because the error trace in the browser console was confusing.</p><h3>The context of the problem</h3><p>In this project I used the <a href="https://marmelab.com/react-admin/">react-admin</a> library, a web framework to build B2B applications. The library provides <a href="https://marmelab.com/react-admin/Reference.html">user interface components</a> and <a href="https://marmelab.com/react-admin/DataProviders.html">data providers</a> to communicate with backend APIs. Thanks to these data providers we don’t need to implement the API clients ourselves: we only need to code our API using a standard format and set the endpoint in the frontend. The library supports several data providers, and we can add our own if needed. The code that I was using in the project to connect with a <a href="https://graphql.org/learn/">GraphQL</a> API was the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6a54863c523f829cf951b162582d9eb0/href">https://medium.com/media/6a54863c523f829cf951b162582d9eb0/href</a></iframe><p>As we can see, the buildGraphQLProvider factory is called once the component is mounted. This factory returns a promise that, when it’s resolved, returns the graphql provider. This provider is stored in the component state and passed via props to theAdmin component below.</p><p>At first glance, this code may seem correct, but if we run the application an error will appear in the browser console and the app will crash. If we look at the browser console, the problem is not clear:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/869/1*T22OxAwVcMyxyoemBFj4Mw.png" /></figure><h3>What’s happening here?</h3><p>Initially I didn’t realize the problem but finally I found it after debugging: the graphql provider returned by the promise is a function. The setState function accepts two kinds of parameters: an object or a function. If we pass a function as parameter, it’s called to make a <a href="https://reactjs.org/docs/hooks-reference.html#functional-updates">functional update</a>. This is why the setState function used the provider function as the setState callback, which wasn’t what I expected.</p><p>To fix the problem we change the setDataProvider to use a functional update:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d001e2bef9ee04a2e459b22b2dc06c15/href">https://medium.com/media/d001e2bef9ee04a2e459b22b2dc06c15/href</a></iframe><p>With this change, the function passed to the setDataProvider is called and its returned value (the data provider function) is stored in the component state.</p><h3>Lessons learned</h3><p>We must be careful when we store third-party objects in the component state. We should always check the type of these objects. If we have a function object we could use a functional update or a wrapper to store it in the state.</p><p>React Hooks are a powerful tool that we can use in functional components, but we are still starting to use and understand them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68b7777f173c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/the-danger-of-storing-third-party-objects-in-a-react-component-state-68b7777f173c">The danger of storing third-party objects in a React component state</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a Flask application with Gunicorn and Docker]]></title>
            <link>https://medium.com/trabe/deploying-a-flask-application-with-gunicorn-and-docker-2bc7c4c10dd4?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/2bc7c4c10dd4</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[gunicorn]]></category>
            <category><![CDATA[ssl]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[flask]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 23 Dec 2019 09:01:01 GMT</pubDate>
            <atom:updated>2019-12-23T09:01:01.232Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*KT02qjrlzdnV9xnZ" /><figcaption>Photo by <a href="https://unsplash.com/@deadqueenines">Inês Pimentel</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>In this story, we will learn how to deploy a <a href="https://www.palletsprojects.com/p/flask/">Flask</a> application with <a href="https://gunicorn.org/">Gunicorn</a> and <a href="https://www.docker.com/">Docker</a>. Also, we will discover how we can use HTTPS with free SSL/TLS <a href="https://letsencrypt.org/">Let’s Encrypt</a> certificates.</p><p>Let’s go!</p><h3>Coding our flask application</h3><p>First of all, we will write a very simple flask application that we will deploy later:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/744e0406459ebcc8afc4f8cd4bc728ea/href">https://medium.com/media/744e0406459ebcc8afc4f8cd4bc728ea/href</a></iframe><p>The <a href="https://github.com/pypa/pipfile">Pipfile</a> should look like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/faffe1c6989662246008523ab529471b/href">https://medium.com/media/faffe1c6989662246008523ab529471b/href</a></iframe><p>Now, you can test the application running the following commands:</p><pre>$ pipenv install<br>$ env FLASK_APP=hello.py pipenv run flask run</pre><h3>Getting the SSL/TLS certificates</h3><p>Security is essential when we build web applications. <a href="https://en.wikipedia.org/wiki/Transport_Layer_Security">SSL/TLS</a> certificates are a very important part of web security, allowing secure connections over HTTPS. These certificates must be signed by a WebTrust-certified certificate authority so they can be validated by the web browsers.</p><p>Let’s Encrypt is a free, automated, and open certificate authority, run for the public’s benefit. It is a service provided by the <a href="https://www.abetterinternet.org/">Internet Security Research Group (ISRG)</a>. Most web browsers include this certificate authority so the certificates signed by Let’s Encrypt are validated successfully.</p><p>To get a free HTTPS certificate with Let’s encrypt we can use the <a href="https://certbot.eff.org/">certbot</a> tool. Certbot is a free, open source software tool for automatically using Let’s Encrypt certificates on manually-administrated websites to enable HTTPS. Instructions on how to obtain a new certificate are provided on the website but, in a Debian-Based GNU/Linux distribution, we can install the tool running the following command:</p><pre>$ sudo apt-get install certbot</pre><p>Once installed, we can get the certificate. With the -d parameter we must set the certified domain name:</p><pre>$ sudo certbot certonly -d my-server-domain-name.com -n --standalone</pre><p>The tool validates the domain name <a href="https://letsencrypt.org/docs/challenge-types/">spinning up a web server in the machine</a>. That is why the machine must be the server of the web application and must be accessible through this domain name.</p><p>The certificate is saved at /etc/letsencrypt/live/my-server-domain-name.com/fullchain.pem and the key file at /etc/letsencrypt/live/my-server-domain-name.com/privkey.pem. Now, we must create a certs directory and copy into it the certificate and the key files. Be careful setting the permissions so our system user can access to them:</p><pre>$ mkdir /home/martin/certs<br>$ sudo cp /etc/letsencrypt/live/my-server-domain-name.com/fullchain.pem /home/martin/certs/<br>$ sudo cp /etc/letsencrypt/live/my-server-domain-name.com/privkey.pem /home/martin/certs/<br>$ sudo chown martin:martin /home/martin/certs/*</pre><h3>Writing the Dockerfile</h3><p>To write the <a href="https://docs.docker.com/engine/reference/builder/">Dockerfile</a> we will choose the python:3.8 as base image:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9d374f7af53bf71a917b604621fbba5f/href">https://medium.com/media/9d374f7af53bf71a917b604621fbba5f/href</a></iframe><p>The application is deployed into the image using the gunicorn WSGI HTTP Server. With the -b parameter we can setup the binding port and with the -w the number of worker processes. Also, we must setup the certificate (--certfile) and the key (--keyfile).</p><p>We execute the following command to build the docker image:</p><pre>$ docker build -t sample-app .</pre><h3>Running the application</h3><p>Once the docker image has been created, we will run a container:</p><pre>$ docker run -dit --name my-sample-app -p 443:443 --mount type=bind,source=/home/martin/certs,target=/certs sample-app</pre><p>Don’t forget to set the <a href="https://docs.docker.com/storage/bind-mounts/">bind mount</a> to map the certificates directory.</p><p>And that’s all! To check if the application is running we will access to the following URL in our favourite web browser: <a href="https://my-server-domain-name.com/?name=Martin">https://my-server-domain-name.com?name=Martin</a></p><h3>Renewing the certificates</h3><p>Let’s Encrypt certificates are valid for 90 days. This forces us to renew the certificates from time to time.</p><p>A quick workaround to solve this problem is to use a script and a cron task. The script could be similar to this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/886c9f8db24e020832f264fb92bba141/href">https://medium.com/media/886c9f8db24e020832f264fb92bba141/href</a></iframe><p>Then, we could setup a cron task to run this script every 75 days, for example. Don’t forget to run this script with <a href="https://en.wikipedia.org/wiki/Sudo">sudo</a> and <a href="https://wiki.archlinux.org/index.php/Sudo#Configuration">configure it</a> with the NOPASSWD option.</p><h3>Summing up</h3><p>With Docker and Gunicorn we can deploy our flask application in an easy and quick way. We can also generate SSL/TLS certificates for free thanks to Let’s Encrypt and automate the renewal process to make our web application secure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2bc7c4c10dd4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/deploying-a-flask-application-with-gunicorn-and-docker-2bc7c4c10dd4">Deploying a Flask application with Gunicorn and Docker</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[User authentication with Flask-JWT]]></title>
            <link>https://medium.com/trabe/user-authentication-with-flask-jwt-be0e6f457dff?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/be0e6f457dff</guid>
            <category><![CDATA[jwt]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[flask]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 16 Sep 2019 08:01:00 GMT</pubDate>
            <atom:updated>2019-09-16T08:01:00.998Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*dmJsGdo5TVVwrjKe" /><figcaption>Photo by <a href="https://unsplash.com/@flik185">Simone Viani</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>In this story we’ll learn how to implement a <a href="https://jwt.io/">JWT</a> authentication in <a href="https://palletsprojects.com/p/flask/">flask</a> with the <a href="https://pythonhosted.org/Flask-JWT">Flask-JWT</a> library. All the code implemented is available at <a href="https://github.com/martinlamas/flask-jwt-auth-test">this github repository</a>.</p><h3>Starting the project</h3><p>We’ll start the project from scratch, so the first step will be to initialize it and install some required dependencies using <a href="https://github.com/pypa/pipenv">pipenv</a>:</p><pre>$ mkdir jwt-auth-test-project<br>$ cd jwt-auth-test-project<br>$ pipenv --three<br>$ pipenv install flask flask-jwt flask-sqlalchemy</pre><p>We’ll use the <a href="https://flask-sqlalchemy.palletsprojects.com/en/2.x">flask-sqlalchemy</a> <a href="https://en.wikipedia.org/wiki/Object-relational_mapping">ORM</a> library to manage the user data from the database.</p><p>Before continuing it’s recommended that we spawn a shell within the virtualenv running the following command:</p><pre>$ pipenv shell</pre><p>Next, we’ll create a <a href="https://docs.python.org/3/tutorial/modules.html#packages">python package</a> that will contain the application source files:</p><pre>mkdir jwtauthtest<br>touch jwtauthtest/__init__.py</pre><p>Then we’ll place the following code in the jwtauthtest/__init__.py file in order to initialize the flask application:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/53e0a98b341a8e8299e408291abafc10/href">https://medium.com/media/53e0a98b341a8e8299e408291abafc10/href</a></iframe><p>Now, we’ll add a sample view function in the jwtauthtest/views.py file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5831a816d3822e84f06862ddcad03f4c/href">https://medium.com/media/5831a816d3822e84f06862ddcad03f4c/href</a></iframe><p>At this point we’ll have a basic flask application that will serve us as a starting point to continue. To test the application we must run the following commands:</p><pre>$ export FLASK_APP=jwtauthtest<br>$ flask run</pre><p>Next, open the <a href="http://localhost:5000/hellop">http://localhost:5000/hello</a> URL in your favourite browser to check that everything is ok :)</p><h3>Configuring the database</h3><p>Now we are going to configure the database. First, we’ll create the jwtauthtest/database.py file with the following code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/acd1509989b4f1764842c65232532cc7/href">https://medium.com/media/acd1509989b4f1764842c65232532cc7/href</a></iframe><p>As we can see, the jwtauthtest.models module is imported. This module must contain the user model that holds the user data:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb01dba4334b7c0123130dc9656938a5/href">https://medium.com/media/fb01dba4334b7c0123130dc9656938a5/href</a></iframe><p>Finally, we’ll add the database initialization code in the jwtauthtest/__init__.py file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/979035a46efe5ebe9f3285b3a32fe36e/href">https://medium.com/media/979035a46efe5ebe9f3285b3a32fe36e/href</a></iframe><p>The app.teardown_appcontext decorator registers a function to be called when the flask application context ends. We’ll use this decorator to close the database session.</p><h3>Adding users</h3><p>At this point we have a basic flask application that supports the users management in the database. Before implementing the JWT authentication, we should code a script to add a new user to the database. Let’s go!</p><p>First, install the <a href="https://click.palletsprojects.com/en/7.x">click</a> and the <a href="https://passlib.readthedocs.io/en/stable/">passlib</a> libraries as dependencies:</p><pre>$ pipenv install click passlib</pre><p>We’ll use these libraries to implement the script as a command line utility and to store the passwords as secure hashes, respectively. Now, create the useradd.py file and add the following code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ebadb85312bed141e8d7a1b9e8a9a4a0/href">https://medium.com/media/ebadb85312bed141e8d7a1b9e8a9a4a0/href</a></iframe><p>Then, to add a user we can run this script manually:</p><pre>python useradd.py</pre><p>We’ll be prompted for the username and the password, and then a new user will be created into the database.</p><h3>Implementing the JWT authentication</h3><p>Now, we are going to implement the JWT authentication. The Flask-JWT library handles the process of token generation, decoding and signing for us.</p><p>First, we’ll create the jwtauthtest/jwt.py file with the following code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/27a5830b21126eadc8871379c8435cca/href">https://medium.com/media/27a5830b21126eadc8871379c8435cca/href</a></iframe><p>As we can see, an instance of JWT class is created. We pass the flask application instance and the authentication and identity handlers as constructor parameters. These handlers specify the identity handler functions.</p><p>The authentication handler receives two positional arguments: the username and the password. If the authentication is successful, then an authenticated entity object must be returned.</p><p>The identity handler receives the JWT payload. Just like the authentication handler, an authenticated entity object must also be returned by this function.</p><p>Choosing a strong secret key is important because this key is used to sign the JWT tokens.</p><p>Finally, this module must be imported in the jwtauthtest/__init__.py file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d9acc1cf81c9bbbf7557c9194cfe173/href">https://medium.com/media/6d9acc1cf81c9bbbf7557c9194cfe173/href</a></iframe><h3>Authenticating the view functions</h3><p>As the last step we are going to authenticate the view functions. Just import the jwt_required decorator and annotate the view functions with it:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8027f9bc0689fb96513ce1ef95481cae/href">https://medium.com/media/8027f9bc0689fb96513ce1ef95481cae/href</a></iframe><h3>Testing the application</h3><p>At this point, we’ll have an authenticated flask application with JWT. Now, we are going to make an authenticated request to test the application. First, we’ll need to get a new JWT token. It’s easy with <a href="https://curl.haxx.se/">curl</a>:</p><pre>$ curl -X POST -H &quot;Content-Type: application/json&quot; <a href="http://localhost:5000/auth">http://localhost:5000/auth</a> -d &#39;{ &quot;username&quot;: &quot;admin&quot;, &quot;password&quot;: &quot;mysupersecretpassword&quot; }&#39;</pre><p>Then we can make the authenticated request with the returned JWT token:</p><pre>$ curl -X GET -H &quot;Authorization: JWT eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1Njc0OTcyNjEsImlhdCI6MTU2NzQ5Njk2MSwibmJmIjoxNTY3NDk2OTYxLCJpZGVudGl0eSI6MX0.GncHdTHADlkeXYggn03nnmaS4tCYGNryHP6MQD7NywE&quot; <a href="http://localhost:5000/hello">http://localhost:5000/hello</a></pre><p>If everything is ok, we should see a Hello world! in our terminal :)</p><h3>Summing up</h3><p>Flask is a great web application framework designed to get you started quickly and easily. We can use it to develop small and lightweight projects, but also complex ones.</p><p>The Flask-JWT library provides a way to manage the authentication process when we need to use JWT tokens in our flask applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be0e6f457dff" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/user-authentication-with-flask-jwt-be0e6f457dff">User authentication with Flask-JWT</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Docker Engine API securely]]></title>
            <link>https://medium.com/trabe/using-docker-engine-api-securely-584e0882158e?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/584e0882158e</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[ssl]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 19 Aug 2019 08:01:01 GMT</pubDate>
            <atom:updated>2019-08-19T08:01:01.376Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*I8HdS1TWFb5Msx-M" /><figcaption>Photo by <a href="https://unsplash.com/@florianklauer">Florian Klauer</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>In this story we’ll learn how to use the <a href="https://docs.docker.com/engine/api/v1.40/">Docker Engine API</a> through the network in a secure way. The Engine API is an HTTP API served by the Docker Engine. It’s the API the Docker client uses to communicate with the Engine, so everything the Docker client can do can also be done with the API.</p><h3>Enabling the API</h3><p>In order to use the Docker Engine API, a TCP socket must be enabled when the engine daemon starts. By default, a unix domain socket (or IPC socket) is created at /var/run/docker.sock. However, we can configure the daemon to listen to multiple sockets at the same time using multiple -H options:</p><pre>$ dockerd -H unix:///var/run/docker.sock -H tcp://192.168.1.100:2375</pre><p>In this example, the daemon will open two sockets: a unix domain socket, at /var/run/docker.sock, and a TCP socket listening on the 2375 TCP port for the 192.168.1.100 interface. By convention, 2375 TCP and 2376 TCP ports should be used for un-encrypted and encrypted connections, respectively.</p><p>On <a href="https://www.freedesktop.org/wiki/Software/systemd">systemd</a> based <a href="https://www.getgnulinux.org/en/linux">GNU/Linux</a> distributions we can enable the engine API by creating the /etc/systemd/system/docker.service.d/override.conf file with the following contents:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c25884c34850c26862774857c3a7f5bf/href">https://medium.com/media/c25884c34850c26862774857c3a7f5bf/href</a></iframe><p>With -H fd:// we enable the support to establish the communication with the daemon via <a href="http://0pointer.de/blog/projects/socket-activation.html">Systemd socket activation</a>. Also, with -H tcp://0.0.0.0:2375, the daemon will open a TCP socket listening on 2375 TCP port for all system interfaces. Moreover, we had to explicitly clear ExecStart before setting it again, as it is an additive setting and can have multiple entries.</p><p>Then, we must execute the following command to reload the configuration of the <a href="https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files">systemd units</a>:</p><pre>$ sudo systemctl daemon-reload</pre><p>As the last step, the service must be restarted:</p><pre>$ sudo systemctl restart docker.service</pre><p>And that’s all. Now we can retrieve the containers list using any http client:</p><pre>$ curl -X GET <a href="http://localhost:2375/containers/json?all=1">http://localhost:2375/containers/json?all=1</a></pre><h3>Securing the API</h3><p>At this point the API is public and available to anyone. This is not a good idea, especially if we want to expose this API to the internet. Fortunately, we can configure a secured <a href="https://en.wikipedia.org/wiki/Transport_Layer_Security">TLS</a> connection using self-signed certificates. Let’s go!</p><p>First, you must check that the <a href="https://www.openssl.org/">openssl</a> package is installed in your system.</p><p>Then, on the Docker daemon’s host machine, we need to generate a CA certificate:</p><pre>$ openssl genrsa -aes256 -out ca-key.pem 4096<br>Generating RSA private key, 4096 bit long modulus (2 primes)<br>.........++++<br>...............................................................................++++<br>e is 65537 (0x010001)<br>Enter pass phrase for ca-key.pem:<br>Verifying - Enter pass phrase for ca-key.pem:</pre><pre>$ openssl req -new -x509 -days 365 -key ca-key.pem -sha256 \<br>-out ca.pem<br>Enter pass phrase for ca-key.pem:<br>You are about to be asked to enter information that will be incorporated<br>into your certificate request.<br>What you are about to enter is what is called a Distinguished Name or a DN.<br>There are quite a few fields but you can leave some blank<br>For some fields there will be a default value,<br>If you enter &#39;.&#39;, the field will be left blank.<br>-----<br>Country Name (2 letter code) [AU]:ES<br>State or Province Name (full name) [Some-State]:A Coruna<br>Locality Name (eg, city) []:A Coruna<br>Organization Name (eg, company) [Internet Widgits Pty Ltd]:Trabe<br>Organizational Unit Name (eg, section) []:<br>Common Name (e.g. server FQDN or YOUR name) []:docker.acme.com<br>Email Address []:</pre><p>We must fill the Common Name field with the <a href="https://en.wikipedia.org/wiki/Fully_qualified_domain_name">FQDN</a> of the docker host machine.</p><p>Now that we have a CA, we must create a server key and a certificate signing request (CSR):</p><pre>$ openssl genrsa -out server-key.pem 4096<br>Generating RSA private key, 4096 bit long modulus (2 primes)<br>.............................................................................................................++++<br>....................++++<br>e is 65537 (0x010001)</pre><pre>$ openssl req -subj &quot;/CN=docker.acme.com&quot; -sha256 -new \<br>-key server-key.pem -out server.csr</pre><p>Next, we’ll sign the server CSR with the CA:</p><pre>$ openssl x509 -req -days 365 -sha256 -in server.csr -CA ca.pem \<br>-CAkey ca-key.pem -CAcreateserial -out server-cert.pem<br>Signature ok<br>subject=CN = docker.acme.com<br>Getting CA Private Key<br>Enter pass phrase for ca-key.pem:</pre><p>If we need to establish the SSL connection through the IP address of the server then we must create a certificate extensions file before generating the server certificate:</p><pre>$ echo subjectAltName = \<br>DNS:docker.acme.com,IP:10.0.0.100,IP:127.0.0.1 &gt;&gt; extfile.cnf</pre><pre>$ echo extendedKeyUsage = serverAuth &gt;&gt; extfile.cnf</pre><pre>$ openssl x509 -req -days 365 -sha256 -in server.csr -CA ca.pem \<br>-CAkey ca-key.pem -CAcreateserial -out server-cert.pem \<br>-extfile extfile.cnf<br>Signature ok<br>subject=CN = docker.acme.com<br>Getting CA Private Key<br>Enter pass phrase for ca-key.pem:</pre><p>Now, we must generate a client certificate:</p><pre>$ openssl genrsa -out key.pem 4096<br>Generating RSA private key, 4096 bit long modulus (2 primes)<br>............++++<br>...................................................................................................++++<br>e is 65537 (0x010001)</pre><pre>$ openssl req -subj &#39;/CN=client&#39; -new -key key.pem -out client.csr</pre><pre>$ echo extendedKeyUsage = clientAuth &gt; extfile-client.cnf</pre><pre>$ openssl x509 -req -days 365 -sha256 -in client.csr -CA ca.pem \<br>-CAkey ca-key.pem -CAcreateserial -out cert.pem \<br>-extfile extfile-client.cnf<br>Signature ok<br>subject=CN = client<br>Getting CA Private Key<br>Enter pass phrase for ca-key.pem:</pre><p>As the last step, we must configure the daemon to enable TLS using these certificates. Place them under /etc/ssl and update the /etc/systemd/system/docker.service.d/override.conf file content:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2d46ad129e2574ce447c441fe941d8e/href">https://medium.com/media/e2d46ad129e2574ce447c441fe941d8e/href</a></iframe><p>Then, reload the configuration of the systemd units and restart the service as we did before.</p><h3>Configuring the client to consume the API</h3><p>With everything in place, we can start making secure calls to consume the API, using the generated certificates to set the authentication in the applications.</p><p>For example, you may want to consume the API using Python and the <a href="https://2.python-requests.org/en/master/">requests library</a>. In the next snippet we’ll make a request to fetch the list of containers available in the docker host. We must include the certificates in the library calls:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d1fbbe77e2d927e3037094e44653e49e/href">https://medium.com/media/d1fbbe77e2d927e3037094e44653e49e/href</a></iframe><p>As we said at the beginning, you can execute any docker command through this API. You can read the <a href="https://docs.docker.com/engine/api/v1.40/">documentation</a> to get more info about the available API methods.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=584e0882158e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/using-docker-engine-api-securely-584e0882158e">Using Docker Engine API securely</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sync your data with Unison]]></title>
            <link>https://medium.com/trabe/sync-your-data-with-unison-3f28e96b05ff?source=rss-69a70c39b5e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f28e96b05ff</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[network]]></category>
            <category><![CDATA[unison]]></category>
            <category><![CDATA[sync]]></category>
            <dc:creator><![CDATA[Martín Lamas]]></dc:creator>
            <pubDate>Mon, 06 May 2019 08:01:01 GMT</pubDate>
            <atom:updated>2019-05-06T08:01:01.128Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*8hLdn9NoUDxFTAaf" /><figcaption>Photo by <a href="https://www.pexels.com/@freeimages9">icon0.com</a> on <a href="https://www.pexels.com/">Pexels</a></figcaption></figure><p>Currently I am managing two little servers and I need to keep some data in sync. Until now, I was performing this task using the <a href="https://rsync.samba.org/">rsync</a> tool and a <a href="https://en.wikipedia.org/wiki/Cron">cron</a> task. This approach is simple and functional, but there is a problem: the sync must be bidirectional if we need to keep two identical replicas and avoid integrity issues. Rsync does not support this.</p><p>There are several options to carry out this task, such as <a href="https://en.wikipedia.org/wiki/Clustered_file_system">clustered file systems</a>, but I have chosen a simple approach using <a href="https://www.cis.upenn.edu/~bcpierce/unison/">Unison</a>, a user-level tool that can be used to sync two replicas.</p><h3>Introduction to Unison</h3><p>Unison is a file-sync tool for OSX, Unix, and Windows. It allows two replicas of a collection of files to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.</p><p>This tool works like rsync but it keeps stateful data supporting the bidirectional sync. Also, security is guaranteed because it can work over SSH tunnels.</p><h3>Installation</h3><p>To install Unison we can download the precompiled binaries from <a href="http://unison-binaries.inria.fr/">here</a>. In order to use the binary from a terminal, we must place it in a directory included in the system path.</p><p>Alternatively, in a debian-like distribution, we can install the tool using <a href="https://www.debian.org/doc/manuals/debian-faq/ch-pkgtools.en.html">apt</a>:</p><pre>sudo apt-get -y install unison</pre><p>In OSX, we must use <a href="https://brew.sh/">homebrew</a> to perform the installation:</p><pre>brew install unison</pre><p>Once installed, we can sync two directories running the following command:</p><pre>unison -batch dir1 dir2</pre><p>We must use the -batch CLI option to launch the tool in batch mode to skip user intervention.</p><h3>Synching remote hosts through SSH</h3><p>Once installed we can sync two servers connected over the internet. To keep things safe we will use SSH tunnels to transfer the data.</p><p>In this scenario, one of the servers acts as the master and the other as the slave. Unison must be installed on both master and slave. Configuration must be set in the master, which is responsible for launching the sync process.</p><p>Unison provides an extensive set of CLI options that we can use to establish the sync configuration. Also, we can set those options using a configuration file.</p><p>The configuration file must be named .unison/default.prf and placed under the user’s home directory. The following gist shows the configuration of my master server:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d32004f44b38981818f7640cfba1a844/href">https://medium.com/media/d32004f44b38981818f7640cfba1a844/href</a></iframe><p>The most important option to setup is the root of the paths to sync. There must be exactly two: the local server path and the remote server path. Because we are using the SSH protocol in the remote root, we need to <a href="https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2">exchange ssh keys</a> between slave and master.</p><p>Other relevant options are:</p><ul><li>The path option designate the paths inside the root that must be synched.</li><li>With the ignore options we can define path exclusions using globs or exact matches.</li><li>The auto and batch options must be used to avoid the user intervention when the sync process is performed.</li><li>The purpose of the fastcheck option is to define which method must be used to check if a file should be updated. When set to true, the size and the last modification time of the file are used to check if the file has changed. When set to false a full-content comparison is used.</li></ul><p>The sample file is self-descriptive but we can query the full documentation running man unison.</p><h3>Scheduling the sync</h3><p>Once configured, we can run the unison command in the master server to perform the sync.</p><p>A simple approach to avoid manual synching is to use a cron task to schedule it every minute. Since we want to avoid multiple unison processes running at the same time, we must create a simple script to launch unison only if it is not currently running:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/81003d469e9a1c3e8afa4624dd91f421/href">https://medium.com/media/81003d469e9a1c3e8afa4624dd91f421/href</a></iframe><p>The last step is adding our script to the crontab:</p><pre>*/1 * * * * /usr/local/bin/sync.sh &gt; /dev/null</pre><h3>Summing up</h3><p>Unison and cron simplify the process of periodically synching a pair of local or remote folders. Unison is similar to rsync but adds support for bidirectional synching.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f28e96b05ff" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trabe/sync-your-data-with-unison-3f28e96b05ff">Sync your data with Unison</a> was originally published in <a href="https://medium.com/trabe">Trabe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>