<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vadim Pushtaev on Medium]]></title>
        <description><![CDATA[Stories by Vadim Pushtaev on Medium]]></description>
        <link>https://medium.com/@vadimpushtaev?source=rss-78921613d74d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*6oPty7P94n3T7AOaWuagZQ.jpeg</url>
            <title>Stories by Vadim Pushtaev on Medium</title>
            <link>https://medium.com/@vadimpushtaev?source=rss-78921613d74d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 02:32:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vadimpushtaev/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Python stories, October 2018]]></title>
            <link>https://medium.com/hackernoon/python-stories-october-2018-1842af5173ba?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1842af5173ba</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Mon, 12 Nov 2018 14:26:55 GMT</pubDate>
            <atom:updated>2018-11-12T14:26:55.950Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QzZYvV8cpen7QvV9.jpeg" /></figure><p>I’m running <a href="https://t.me/pythonetc">@pythonetc</a>, a Telegram channel about Python and programming in general. Here are the best posts of October 2018.</p><h3>The -- separator</h3><p>Every well-behaved command line utility should accept arguments in the form of options (e. g. -h or --help), options with parameters (--log-level 2) or positional parameters (cp file1 file2).</p><p>Options are distinguished from positional parameters by having a leading dash (or two dashes). Problems start when positional arguments have to start with a dash, e. g. you want to remove a file with the name of -rf: rm -rf doesn&#39;t work this way.</p><p>The conventional way to solve this problem is to support -- delimiter. Arguments after -- are never interpreted as options:</p><pre>$ echo test &gt; -rf<br>$ cat -rf<br>cat: invalid option -- &#39;r&#39;<br>Try &#39;cat --help&#39; for more information.<br>$ cat -- -rf<br>test<br>$ rm -- -rf<br>$ cat -- -rf<br>cat: -rf: No such file or directory</pre><p>The argparse module automatically handles -- for you.</p><h3>Stable sot</h3><p>The default Python sorting is stable meaning it preserves the order of equal objects:</p><pre>In : a = [2, -1, 0, 1, -2]</pre><pre>In : sorted(a, key=lambda x: x**2)<br>Out: [0, -1, 1, 2, -2]</pre><p>max and min functions also tries to be consistent with sorted. max works as sorted(a, reverse=True)[0] while min does sorted(a)[0]. That means that both max and min returns the leftmost possible answer:</p><pre>In : max([2, -2], key=lambda x: x**2)<br>Out: 2</pre><pre>In : max([-2, 2], key=lambda x: x**2)<br>Out: -2</pre><pre>In : min([2, -2], key=lambda x: x**2)<br>Out: 2</pre><pre>In : min([-2, 2], key=lambda x: x**2)<br>Out: -2</pre><h3>Default argument cache</h3><p>Probably the most common newbie mistake with Python it providing a mutable object as a default function argument. That object is shared between all function calls that can lead to bizarre results:</p><pre>def append_length(lst=[]):<br>    lst.append(len(lst))<br>    return lst</pre><pre>print(append_length([1, 2])) # [1, 2, 2]<br>print(append_length())       # [0]<br>print(append_length())       # [0, 1]</pre><p>However, for various caches sharing may be a good thing:</p><pre>def fact(x, cache={0: 1}):<br>    if x not in cache:<br>        cache[x] = x * fact(x - 1)</pre><pre>    return cache[x]</pre><pre>print(fact(5))</pre><p>In this example, we store calculated factorial values inside the default function value. It can even be extracted:</p><pre>&gt;&gt;&gt; fact.__defaults__<br>({0: 1, 1: 1, 2: 2, 3: 6, 4: 24, 5: 120},)</pre><h3>pathlib</h3><p>Python allows you to work with filesystem paths with the os.path module. The module contains a lot of functions that treat strings as paths and perform useful operations like concating paths and stuff like that:</p><pre>&gt;&gt;&gt; import os.path<br>&gt;&gt;&gt; os.path.join(&#39;/usr&#39;, &#39;local&#39;)<br>&#39;/usr/local&#39;<br>&gt;&gt;&gt; os.path.dirname(&#39;/var/log&#39;)<br>&#39;/var&#39;</pre><p>However, since Python 3.4 the new pathlib module is available which offers an object-oriented approach:</p><pre>&gt;&gt;&gt; from pathlib import Path<br>&gt;&gt;&gt; Path(&#39;/usr&#39;) / Path(&#39;local&#39;)<br>PosixPath(&#39;/usr/local&#39;)<br>&gt;&gt;&gt; Path(&#39;/usr&#39;) / &#39;local&#39;<br>PosixPath(&#39;/usr/local&#39;)<br>&gt;&gt;&gt; Path(&#39;/var/log&#39;).parent<br>PosixPath(&#39;/var&#39;)<br>&gt;&gt;&gt; Path(&#39;/var/log&#39;).parent.name<br>&#39;var&#39;</pre><h3>Callable objects</h3><p>In Python, you can create a callable object not only by creating functions (with def or lambda). An object is also callable if it has the __call__ magic method:</p><pre>class truncater:<br>    def __init__(self, length):<br>        self._length = length</pre><pre>    def __call__(self, s):<br>        return s[0:self._length]<br></pre><pre>print(truncater(4)(&#39;abcdabcd&#39;)) # abcd</pre><p>Since a decorator is basically a higher-order function, it can also be expressed with a callable object instead of a function:</p><pre>class cached:<br>    def __init__(self, func):<br>        self._func = func<br>        self._cache = {}</pre><pre>    def __call__(self, arg):<br>        if arg not in self._cache:<br>            self._cache[arg] = self._func(arg)</pre><pre>        return self._cache[arg]<br></pre><pre>@cached<br>def sqr(x):<br>    return x * x</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1842af5173ba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/python-stories-october-2018-1842af5173ba">Python stories, October 2018</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python stories, September 2018]]></title>
            <link>https://medium.com/hackernoon/python-stories-september-2018-d34e526edde2?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/d34e526edde2</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[asyncio]]></category>
            <category><![CDATA[coroutine]]></category>
            <category><![CDATA[oop]]></category>
            <category><![CDATA[metaprogramming]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Wed, 03 Oct 2018 12:59:47 GMT</pubDate>
            <atom:updated>2019-05-21T08:32:58.737Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jct3DcUSr6FdyTCF.png" /></figure><p>I’m running <a href="https://t.me/pythonetc">@pythonetc</a>, a Telegram channel about <a href="https://hackernoon.com/tagged/python">Python</a> and <a href="https://hackernoon.com/tagged/programming">programming</a> in general. Here are the best posts of September 2018.</p><h3>Overriding vs overloading</h3><p>There are two concepts with similar names that can be easily confused: overriding and overloading.</p><p>Overriding happens when a child class defines a method that is already provided by its parents effectively replacing it. In some languages you have to explicitly mark the overriding method (C# requires the override modifier), in some languages it&#39;s optional (the @Override annotation in Java). Python doesn&#39;t require any special modifier nor does it have a standard way to mark such methods (some people like to use a custom @override decorator that does virtually nothing, just for the sake of readability).</p><p>Overloading is another story. Overloading is having multiple functions with the same name but different signatures. It’s supported by languages like Java and C++ and is often used as a way to provide default arguments:</p><pre>class Foo {<br>    public static void main(String[] args) {<br>        System.out.println(Hello());<br>    }</pre><pre>public static String Hello() {<br>        return Hello(&quot;world&quot;);<br>    }</pre><pre>public static String Hello(String name) {<br>        return &quot;Hello, &quot; + name;<br>    }<br>}</pre><p>Python doesn’t support finding functions by their signatures, only be their names. You can write code that analyzes the types and number of arguments explicitly. That usually looks clumsy and generally is not a nice thing to do:</p><pre>def quadrilateral_area(*args):<br>    if len(args) == 4:<br>        quadrilateral = Quadrilateral(*args)<br>    elif len(args) == 1:<br>        quadrilateral = args[0]<br>    else:<br>        raise TypeError()</pre><pre>return quadrilateral.area()</pre><p>If you need type hints for this, the typing module can help you with the @overload decorator:</p><pre>from typing import overload</pre><pre>@overload<br>def quadrilateral_area(<br>    q: Quadrilateral<br>) -&gt; float: ...</pre><pre>@overload<br>def quadrilateral_area(<br>    p1: Point, p2: Point,<br>    p3: Point, p4: Point<br>) -&gt; float: ...</pre><h3>Autovivification</h3><p>collections.defaultdict allows you to create a dictionary that returns the default value if the requested key is missing (instead of raising KeyError). To create a defaultdict you should provide not a default value but a factory of such values.</p><p>That allows you to create a dictionary that virtually contains infinite levels of nested dicts, allowing you to do something like d[a][b][c]...[z].</p><pre>&gt;&gt;&gt; def infinite_dict():<br>...     return defaultdict(infinite_dict)<br>...<br>&gt;&gt;&gt; d = infinite_dict()<br>&gt;&gt;&gt; d[1][2][3][4] = 10<br>&gt;&gt;&gt; dict(d[1][2][3][5])<br>{}</pre><p>Such behavior is called “autovivification”, the term came from the Perl language.</p><h3>Object instantiation</h3><p>An object instantiation includes two significant steps. First, the __new__ method of a class is called. It creates and returns a brand new object. Second, Python calls the __init__ method of that object. Its work is to set up the initial state of the object.</p><p>However, __init__ isn&#39;t called if __new__ returns an object that is not an instance of the original class. The reason for this is that it was probably created by another class, hence __init__ was already called for that object:</p><pre>class Foo:<br>    def __new__(cls, x):<br>        return dict(x=x)</pre><pre>    def __init__(self, x):<br>        print(x)  # Never called</pre><pre>print(Foo(0))</pre><p>That also means that you should not ever create instances of the same class in __new__ with a regular constructor (Foo(...)). It could lead to the double __init__ execution or even infinite recursion.</p><p>Infinite recursion:</p><pre>class Foo:<br>    def __new__(cls, x):<br>        return Foo(-x)  # Recursion</pre><p>Double __init__:</p><pre>class Foo:<br>    def __new__(cls, x):<br>        if x &lt; 0:<br>            return Foo(-x)<br>        return super().__new__(cls)</pre><pre>    def __init__(self, x):<br>        print(x)<br>        self._x = x</pre><p>The proper way:</p><pre>class Foo:<br>    def __new__(cls, x):<br>        if x &lt; 0:<br>            return cls.__new__(cls, -x)<br>        return super().__new__(cls)</pre><pre>    def __init__(self, x):<br>        print(x)<br>        self._x = x</pre><h3>[]</h3><p>In Python, you can override square brackets operator ([]) by defining __getitem__ magic method. This is how you create an object that virtually contains an infinite number of repeated elements:</p><pre>class Cycle:<br>    def __init__(self, lst):<br>        self._lst = lst</pre><pre>    def __getitem__(self, index):<br>        return self._lst[<br>            index % len(self._lst)<br>        ]</pre><pre>print(Cycle([&#39;a&#39;, &#39;b&#39;, &#39;c&#39;])[100])  # &#39;b&#39;</pre><p>The unusual thing here is that the [] operator supports a unique syntax. It can be used not only like this — [2], but also like this — [2:10], or [2:10:2], or [2::2], or even [:]. The semantic is [start:stop:step], but you can use it any way you want for your custom objects.</p><p>But what __getitem__ gets as an index parameter if you call it using that syntax? The slice objects exist precisely for that.</p><pre>In : class Inspector:<br>...:     def __getitem__(self, index):<br>...:         print(index)<br>...:<br>In : Inspector()[1]<br>1<br>In : Inspector()[1:2]<br>slice(1, 2, None)<br>In : Inspector()[1:2:3]<br>slice(1, 2, 3)<br>In : Inspector()[:]<br>slice(None, None, None)</pre><p>You can even combine tuple and slice syntaxes:</p><pre>In : Inspector()[:, 0, :]<br>(slice(None, None, None), 0, slice(None, None, None))</pre><p>slice is not doing anything for you except simply storing start, stop and step attributes.</p><pre>In : s = slice(1, 2, 3)<br>In : s.start<br>Out: 1<br>In : s.stop<br>Out: 2<br>In : s.step<br>Out: 3</pre><h3>Coroutine cancellation</h3><p>Any running asyncio coroutine can be cancelled via the cancel() method. CancelledError will be thrown into the coroutine that will lead for it and all wrapping coroutines to be terminated, unless the error is caught and suppressed.</p><p>CancelledError is a subclass of Exception that means that it can be accidentally caught by try ... except Exception that is meant to catch “any error”. To safely do this within a coroutine, you stuck with something like this:</p><pre>try:<br>    await action()<br>except asyncio.CancelledError:<br>    raise<br>except Exception:<br>    logging.exception(&#39;action failed&#39;)</pre><p>In asyncio, the common practice to schedule execution of some code at a later time is to spawn a task that does await asyncio.sleep(x):</p><pre>import asyncio</pre><pre>async def do(n=0):<br>    print(n)<br>    await asyncio.sleep(1)<br>    loop.create_task(do(n + 1))<br>    loop.create_task(do(n + 1))</pre><pre>loop = asyncio.get_event_loop()<br>loop.create_task(do())<br>loop.run_forever()</pre><p>However, creating a new task may be expensive and is not necessary if you aren’t planning to any asynchronous operations (like the do function in the example). Another way to do this is to use loop.call_later and loop.call_at functions that schedule an asynchronous callback to be called:</p><pre>import asyncio                     <br>                                   <br>def do(n=0):                       <br>    print(n)                       <br>    loop = asyncio.get_event_loop()<br>    loop.call_later(1, do, n+1)    <br>    loop.call_later(1, do, n+1)    <br>                                   <br>loop = asyncio.get_event_loop()    <br>do()                               <br>loop.run_forever()</pre><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d34e526edde2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/python-stories-september-2018-d34e526edde2">Python stories, September 2018</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python stories, August 2018]]></title>
            <link>https://medium.com/hackernoon/python-stories-august-2018-5af3036ff225?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5af3036ff225</guid>
            <category><![CDATA[metaprogramming]]></category>
            <category><![CDATA[oop]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Tue, 11 Sep 2018 10:39:48 GMT</pubDate>
            <atom:updated>2019-05-21T08:32:59.083Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*ZKnFQMq_AopA4wYaelWPsg.jpeg" /></figure><p>I’m running <a href="https://t.me/pythonetc">@pythonetc</a>, a Telegram channel about <a href="https://hackernoon.com/tagged/python">Python</a> and <a href="https://hackernoon.com/tagged/programming">programming</a> in general. Here are the best posts of August 2018.</p><h3>Factory method</h3><p>If you create new objects inside your __init__, it may be better to pass them as arguments and have a factory method instead. It separates business logic from technical details on how objects are created.</p><p>In this example __init__ accepts host and port to construct a database connection:</p><pre>class Query:<br>    def __init__(self, host, port):<br>        self._connection = Connection(host, port)</pre><p>The possible refactoring is:</p><pre>class Query:<br>    def __init__(self, connection):<br>        self._connection = connection<br>        <br>    @classmethod<br>    def create(cls, host, port):<br>        return cls(Connection(host, port))</pre><p>This approach has at least these advantages:</p><ul><li>It makes dependency injection easy. You can do Query(FakeConnection()) in your tests.</li><li>The class can have as many factory methods as needed; the connection may be constructed not only by host and port but also by cloning another connection, reading a config file or object, using the default, etc.</li><li>Such factory methods can be turned into asynchronous functions; this is completely impossible for __init__.</li></ul><h3>super VS next</h3><p>The super() function allows referring to the base class. This can be extremely helpful in cases when a derived class wants to <em>add</em> something to the method implementation instead of overriding it completely:</p><pre>class BaseTestCase(TestCase):<br>    def setUp(self):<br>        self._db = create_db()</pre><pre>class UserTestCase(BaseTestCase):<br>    def setUp(self):<br>        super().setUp()<br>        self._user = create_user()</pre><p>The function’s name doesn’t mean <em>excellent</em> or <em>very good</em>. The word <em>super</em> implies <em>above</em> in this context (like in <em>superintendent</em>). Despite what I said earlier, super() doesn&#39;t always refer to the base class, it can easily return a sibling. The proper name could be next() since the next class according to MRO is returned.</p><pre>class Top:<br>    def foo(self):<br>        return &#39;top&#39;</pre><pre>class Left(Top):<br>    def foo(self):<br>        return super().foo()</pre><pre>class Right(Top):<br>    def foo(self):<br>        return &#39;right&#39;</pre><pre>class Bottom(Left, Right):<br>    pass</pre><pre># prints &#39;right&#39;<br>print(Bottom().foo())</pre><p>Mind that super() may produce different results since they depend on the MRO of the original call.</p><pre>&gt;&gt;&gt; Bottom().foo()<br>&#39;right&#39;<br>&gt;&gt;&gt; Left().foo()<br>&#39;top&#39;</pre><h3>Custom namespace for class creation</h3><p>The creation of a class consists of two big steps. First, the class body is evaluated, just like any function body. Second, the resulted namespace (the one that is returned by locals()) is used by a metaclass (type by default) to construct an actual class object.</p><pre>class Meta(type):<br>    def __new__(meta, name, bases, ns):<br>        print(ns)<br>        return super().__new__(<br>            meta, name,<br>            bases, ns<br>        )</pre><pre>class Foo(metaclass=Meta):<br>    B = 2</pre><p>The above code prints {&#39;__module__&#39;: &#39;__main__&#39;, &#39;__qualname__&#39;: &#39;Foo&#39;, &#39;B&#39;: 3}.</p><p>Obviously, if you do something like B = 2; B = 3, then the metaclass only knows about B = 3, since only that value is in ns. This limitation is based on the fact, that a metaclass works <em>after</em> the body evaluation.</p><p>However, you can interfere in the evaluation by providing <em>custom namespace</em>. By default, a simple dictionary is used but you can provide a custom dictionary-like object using the metaclass __prepare__ method.</p><pre>class CustomNamespace(dict):<br>    def __setitem__(self, key, value):<br>        print(f&#39;{key} -&gt; {value}&#39;)<br>        return super().__setitem__(key, value)</pre><pre>class Meta(type):<br>    def __new__(meta, name, bases, ns):<br>        return super().__new__(<br>            meta, name,<br>            bases, ns<br>        )</pre><pre>@classmethod<br>    def __prepare__(metacls, cls, bases):<br>        return CustomNamespace()</pre><pre>class Foo(metaclass=Meta):<br>    B = 2<br>    B = 3</pre><p>The output is the following:</p><pre>__module__ -&gt; __main__<br>__qualname__ -&gt; Foo<br>B -&gt; 2<br>B -&gt; 3</pre><p>And this is how enum.Enum is <a href="https://docs.python.org/3/library/enum.html#duplicating-enum-members-and-values">protected from duplicates</a>.</p><h3>matplotlib</h3><p>matplotlib is a complex and flexible Python plotting library. It&#39;s supported by a wide range of products, Jupyter and Pycharm including.</p><p>This is how you draw a simple fractal figure with matplotlib: <a href="https://repl.it/@VadimPushtaev/myplotlib">https://repl.it/@VadimPushtaev/myplotlib</a>, see the image at the beginning of the article.</p><h3>timezones support</h3><p>Python provides the powerful library to work with date and time: datetime. The interesting part is, datetime objects have the special interface for timezone support (namely the tzinfo attribute), but this module only has limited support of its interface, leaving the rest of the job to different modules.</p><p>The most popular module for this job is pytz. The tricky part is, pytz doesn&#39;t fully satisfy tzinfo interface. The pytz documentation states this at one of the first lines: “This library differs from the documented Python API for tzinfo implementations.”</p><p>You can’t use pytz timezone objects as the tzinfo attribute. If you try, you may get the absolute insane results:</p><pre>In : paris = pytz.timezone(&#39;Europe/Paris&#39;)<br>In : str(datetime(2017, 1, 1, tzinfo=paris))<br>Out: &#39;2017-01-01 00:00:00+00:09&#39;</pre><p>Look at that +00:09 offset. The proper use of pytz is following:</p><pre>In : str(paris.localize(datetime(2017, 1, 1)))<br>Out: &#39;2017-01-01 00:00:00+01:00&#39;</pre><p>Also, after any arithmetic operations, you should normalize your datetime object in case of offset changes (on the edge of the DST period for instance).</p><pre>In : new_time = time + timedelta(days=2)<br>In : str(new_time)<br>Out: &#39;2018-03-27 00:00:00+01:00&#39;<br>In : str(paris.normalize(new_time))<br>Out: &#39;2018-03-27 01:00:00+02:00&#39;</pre><p>Since Python 3.6, it’s recommended to use dateutil.tz instead of pytz. It&#39;s fully compatible with tzinfo, can be passed as an attribute, doesn&#39;t require normalize, though works a bit slower.</p><p>If you are interested why pytz doesn&#39;t support datetime API, or you wish to see more examples, consider reading the decent <a href="https://blog.ganssle.io/articles/2018/03/pytz-fastest-footgun.html">article</a> on the topic.</p><h3>StopIteration magic</h3><p>Every call to next(x) returns the new value from the x iterator unless an exception is raised. If this is StopIteration, it means the iterator is exhausted and can supply no more values. If a generator is iterated, it automatically raises StopIteration upon the end of the body:</p><pre>&gt;&gt;&gt; def one_two():<br>...     yield 1<br>...     yield 2<br>...<br>&gt;&gt;&gt; i = one_two()<br>&gt;&gt;&gt; next(i)<br>1<br>&gt;&gt;&gt; next(i)<br>2<br>&gt;&gt;&gt; next(i)<br>Traceback (most recent call last):<br>  File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;<br>StopIteration</pre><p>StopIteration is automatically handled by tools that calls next for you:</p><pre>&gt;&gt;&gt; list(one_two())<br>[1, 2]</pre><p>The problem is, any unexpected StopIteration that is raised within a generator causes it to stop silently instead of actually raising an exception:</p><pre>def one_two():<br>    yield 1<br>    yield 2</pre><pre>def one_two_repeat(n):<br>    for _ in range(n):<br>        i = one_two()<br>        yield next(i)<br>        yield next(i)<br>        yield next(i)</pre><pre>print(list(one_two_repeat(3)))</pre><p>The last yield here is a mistake: StopIteration is raised and makes list(...) to stop the iteration. The result is [1, 2], surprisingly.</p><p>However, that was changed in Python 3.7. Such foreign StopIteration is now replaced with RuntimeError:</p><pre>Traceback (most recent call last):<br>  File &quot;test.py&quot;, line 10, in one_two_repeat<br>    yield next(i)<br>StopIteration</pre><pre>The above exception was the direct cause of the following exception:</pre><pre>Traceback (most recent call last):<br>  File &quot;test.py&quot;, line 12, in &lt;module&gt;<br>    print(list(one_two_repeat(3)))<br>RuntimeError: generator raised StopIteration</pre><p>You can enable the same behavior since Python 3.5 by from __future__ import generator_stop.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5af3036ff225" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/python-stories-august-2018-5af3036ff225">Python stories, August 2018</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python stories, July 2018]]></title>
            <link>https://medium.com/hackernoon/python-stories-july-2018-c0b5443faff?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0b5443faff</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[regex]]></category>
            <category><![CDATA[debugging]]></category>
            <category><![CDATA[blog]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Mon, 06 Aug 2018 09:11:20 GMT</pubDate>
            <atom:updated>2019-05-21T08:32:59.390Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EdeQ8QvBdLMMnTSp.png" /></figure><p>I’m running <a href="https://t.me/pythonetc">@pythonetc</a>, a Telegram channel about Python and <a href="https://hackernoon.com/tagged/programming">programming</a> in general. Here are the best posts of July 2018.</p><h3>Regular languages</h3><p>A regular language is a formal language that can be recognized by a finite-state machine (<a href="/service/tg://unsafe_url?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FFinite-state_machine">FSM</a>). Simply put, that means that to process text character by character, you only need to remember the current state, and the number of such states is finite.</p><p>The beautiful and simple example is a machine that checks whether an input is a simple number like -3, 2.2 or 001. The picture at the beginnig of the article is an FSM diagram. Double circles mean <em>accept states</em>, they identify where the machine can stop.</p><p>The machine starts at ①, possibly matches minus sign, then processes as many digits as required at ③. After that, it may match a dot (③ → ④) which must be followed by one digit (④ → ⑤), but maybe more (⑤ → ⑤).</p><p>The classic example of a non-regular language is a family of strings like:</p><pre>a-b<br>aaa-bbb<br>aaaaa-bbbbb</pre><p>Formally, we need a line that contains N occurrences of a, then -, then N occurrences of b. N is an integer greater than zero. You can&#39;t do it with a finite machine, because you have to remember the number of a chars you encountered which leads you to the infinite number of states.</p><p>Regular expressions can match only regular languages. Remember to check whether the line you are trying to process can be handled by FSM at all. JSON, XML or even mere arithmetic expression with nested brackets cannot be.</p><p>The funny thing is, a lot of modern regular expression engines <em>are not regular</em>. For example, <a href="https://hackernoon.com/tagged/python">Python</a> regex module supports recursion (<a href="/service/tg://unsafe_url?url=https%3A%2F%2Fregex101.com%2Fr%2FJzH9vZ%2F1">which will help</a> with that aaa-bbb problem).</p><h3>Dynamic dispatch</h3><p>When Python executes a method call, say a.f(b, c, d), it should first select the right f function. Due to polymorphism, what is selected depends on the type of a. The process of choosing the method is usually called <em>dynamic dispatch</em>.</p><p>Python supports only single-dispatch polymorphism, that means a single object alone (a in the example) affects the method selection. Some other languages, however, may also consider types of b, c and d. This mechanism is called <em>multiple dispatch</em>. C# is a notable example of languages that support that technique.</p><p>However, multiple dispatch can be emulated via single-dispatch. The <em>visitor</em> design pattern is created exactly for this. <em>visitor</em> essentially just uses single-dispatch twice to imitate double-dispatch.</p><p>Mind, that the ability to overload methods (like in Java and C++) is not the same as multiple dispatch. Dynamic dispatch works in runtime while overloading solely affects compile time.</p><p>These are some code examples to understand the topic better: <a href="/service/tg://unsafe_url?url=https%3A%2F%2Frepl.it%2F%40VadimPushtaev%2Fvisitor">Python visitor example</a>, <a href="/service/tg://unsafe_url?url=https%3A%2F%2Frepl.it%2F%40VadimPushtaev%2Fsingledispatch">Java overloading doesn’t work as multiple dispatch</a>, <a href="/service/tg://unsafe_url?url=https%3A%2F%2Frepl.it%2F%40VadimPushtaev%2Fmultipledispatch">C# multiple dispatch</a>.</p><h3>Built-in names</h3><p>In Python, you can easily modify all standard variables that are available in the global namespace:</p><pre>&gt;&gt;&gt; print = 42<br>&gt;&gt;&gt; print(42)<br>Traceback (most recent call last):<br>  File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;<br>TypeError: &#39;int&#39; object is not callable</pre><p>That may be helpful if your module defines some functions that have the same name as built-in ones. That also happens if you practice metaprogramming and you accept an arbitrary string as an identifier.</p><p>However, even if you shadow some built-in names, you still may want to have access to things they initially referred to. The builtins module exists exactly for that:</p><pre>&gt;&gt;&gt; import builtins<br>&gt;&gt;&gt; print = 42<br>&gt;&gt;&gt; builtins.print(1)<br>1</pre><p>The __builtins__ variable is also available in most modules. There is a catch though. First, this is a cpython implementation detail and usually should not be used at all. Second, __builtins__ might refer to either builtins or builtins.__dict__, depending on how exactly the current module was loaded.</p><h3>strace</h3><p>Sometimes software starts to behave weirdly in the production. Instead of simply restarting it, you probably wish to understand what exactly is happening so you can fix it later.</p><p>The obvious way to do it is to analyze what a program does and try to guess which piece of code is executing. Surely proper logging makes that task easier, but your application’s logs may be not verbose enough, either by design or because the high level of logging is set in the configuration.</p><p>In that case, strace may be quite beneficial. It&#39;s a Unix utility which traces system calls for you. You can run it in advance — strace python script.py — but usually connecting to the already executing application is more suitable: strace -p PID.</p><pre>$ cat test.py<br>with open(&#39;/tmp/test&#39;, &#39;w&#39;) as f:<br>    f.write(&#39;test&#39;)<br>$ strace python test.py 2&gt;&amp;1 | grep open | tail -n 1<br>open(&quot;/tmp/test&quot;, O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3</pre><p>Each line in the trace contains the system call name, followed by its arguments in parentheses and its return value. Since some arguments are used for returning a result from the system call, not for passing data into it, line outputting may be interrupted until system call is finished.</p><p>In this example, the output is interrupted until someone writes to STDIN:</p><pre>$ strace python -c &#39;input()&#39;<br>read(0,</pre><h3>Tuple literals</h3><p>One of the most inconsistent part of the Python syntax is tuple literals.</p><p>Basically, to create a tuple you just write values separated by commas: 1, 2, 3. OK, so far, so good. What about tuple containing only one element? You just add trailing comma to the only value: 1,. Well, that’s somewhat ugly and error prone, but makes sense.</p><p>What about empty tuple? Is it a bare ,? No, it’s (). Do parentheses create tuple as well as commas? No, they don’t, (4) is not a tuple, it’s just 4.</p><pre>In : a = [<br>...:     (1, 2, 3),<br>...:     (1, 2),<br>...:     (1),<br>...:     (),<br>...: ]</pre><pre>In : [type(x) for x in a]<br>Out: [tuple, tuple, int, tuple]</pre><p>To make things more obscure, tuple literals often require additional parentheses. If you want a tuple to be the only argument of a function, that f(1, 2, 3) doesn’t work for an obvious reason, you need f((1, 2, 3)) instead.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0b5443faff" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/python-stories-july-2018-c0b5443faff">Python stories, July 2018</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python: choosing subclass]]></title>
            <link>https://medium.com/@vadimpushtaev/python-choosing-subclass-cf5b1b67c696?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf5b1b67c696</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[metaprogramming]]></category>
            <category><![CDATA[python3]]></category>
            <category><![CDATA[telegram]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Mon, 02 Apr 2018 20:37:06 GMT</pubDate>
            <atom:updated>2018-08-19T20:03:48.879Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is the new version of my old article. It was amended and supplemented for Python etc. —</em><a href="https://t.me/pythonetc"><em> telegram channel about Python and more</em></a><em>.</em></p><p>Let’s delve into a pretty typical situation. You have some subclasses, and you have to pick one according to some parameter provided by a user (usually called type or something like this).<br>Here is an example: your user sends a message via your service specifying a type of the message and some extra parameters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/35acf14fb94f488265766eb0a832324d/href">https://medium.com/media/35acf14fb94f488265766eb0a832324d/href</a></iframe><p>This code works quite well, but there are some techniques you can use to improve it.</p><h3>Factory</h3><p>First of all, I want a separate entity to care about message objects creation. There are two conventional types of that entity: factory or factory method.</p><p>Factory:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2ee54ec2cd574356889f66f7013f5d6/href">https://medium.com/media/e2ee54ec2cd574356889f66f7013f5d6/href</a></iframe><p>Factory method:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d9c91a2e1f00afff6b9df31eb0582f60/href">https://medium.com/media/d9c91a2e1f00afff6b9df31eb0582f60/href</a></iframe><p>For me, the factory is somewhat redundant here, so I stick to the factory method approach in the following examples.</p><h3>Choose class, not object</h3><p>Now we can improve the code of factory method itself. For instance, since classes are <a href="https://en.wikipedia.org/wiki/First-class_citizen">first class objects</a> in Python, you don’t really need to choose a message object, just choose a class:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f2033a3b393e3b9d3d4f5048c903bec/href">https://medium.com/media/6f2033a3b393e3b9d3d4f5048c903bec/href</a></iframe><h3>Use mapping</h3><p>Now it’s obvious that we deal with a dull mapping of types to classes. So it may be a good idea to transform our if statements to an actual <em>map</em>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8ff8c2cff52173179104da5795d6531a/href">https://medium.com/media/8ff8c2cff52173179104da5795d6531a/href</a></iframe><p>Note, that we can’t make MESSAGE_TYPE_TO_CLASS_MAP a class attribute since descendants of BaseMessage should be declared <em>after</em> BaseMessage and therefore not yet exist in the time of BaseMessage creation.</p><h3>Generate class name (don’t)</h3><p>You may notice that the name of every class almost matches the name of the type. We can use that to generate class name by type name and don’t store an actual mapping:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b50d3607a8080fa3a8164be09c86cfed/href">https://medium.com/media/b50d3607a8080fa3a8164be09c86cfed/href</a></iframe><p>But you actually <strong>shouldn’t do this</strong>. There are many problems with this approach. First of all, you can’t guarantee that some class called SomethingMessage will be found despite the fact you don’t want it to. Second, it’s not as obvious as it can be. Third, your IDE probably won’t understand a thing in this mess.</p><p>If for some reason you decide to stick to this method, you should at least check whether the result class is a subclass of BaseMessage.</p><h3>Register subclasses</h3><p>The plain mapping we use is still not perfect. If you create a new message subclass, you have to both define it <em>and </em>add to the mapping. It would be a good idea to mark the subclass somehow, so it will appear in the mapping automatically. And this is where python decorators shine.</p><p>Remember I said we can’t make the mapping a BaseMessage attribute because descendants are not yet defined? Now we can make them <em>add themselves</em> to the mapping upon declaration.</p><p>For this, we need subclasses class attribute and register_subclass decorator to add a subclass to that attribute. Note, that register_subclass is a parameterized decorator, so it returns the function that accepts a subclass as an argument.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2ca8b66a9b91ff71ef91e52902f0f447/href">https://medium.com/media/2ca8b66a9b91ff71ef91e52902f0f447/href</a></iframe><p>This approach has been working for me pretty well for a long time. The best thing about it is that every time a new programmer wants to create a new message subclass, he merely copies an existing one as a scaffold and never forgets to add the decorator.</p><h3>Metaclass</h3><p>Alternatively, you can take advantage of the fact that metaclass of some class also works for all of its descendants. We may write metaclass for BaseMessage that automatically registers all of the subclasses.</p><p>Subclass still needs to store a type, let’s put it in _MESSAGE_TYPE attribute. Also, metaclass needs some way to detect BaseMessage. To do that, we use the fact that it’s the first class created via metaclass.</p><p>Look at the result:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6b6caf3160ae7cb126862cae2a7bafd2/href">https://medium.com/media/6b6caf3160ae7cb126862cae2a7bafd2/href</a></iframe><p>If you can’t quite grasp what is going on here, there is no surprise. Solutions that use metaclasses are notorious for being overcomplicated and hard to read and understand. And there are a little we gained this way: subclasses need to specify _MESSAGE_TYPE anyway, that is not any simpler than @register decorator.</p><p>Also, classes with custom metaclasses may be hard to inherit from. If you want to inherit from two classes, their metaclasses should be equal or one them should be a descendant of another to avoid so-called metaclass conflicts.</p><h3>Init subclass hook</h3><p>Since Python 3.6 it’s possible to hook subclass creation without using a metaclass, you can do it via __init_subclass__ magic method:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/05b8f7631ce6ee786e1e888b0e749743/href">https://medium.com/media/05b8f7631ce6ee786e1e888b0e749743/href</a></iframe><p>You may or may not find it more elegant than decorators solution.</p><h3>Importing</h3><p>All above solutions are suitable only for subclasses that present in the same file that base class or at least at any other that are already imported for some reason. Without it, they wouldn’t have a chance to be registered.</p><p>To fix that you may import them manually, one by one (which is tedious) or organize some auto-discovering (which may be insecure). You also may use mapping solution and import modules dynamically upon objects creation, but it’s even uglier. Sadly, I still don’t have any elegant solution for this problem.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf5b1b67c696" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decorator inside Python class]]></title>
            <link>https://medium.com/@vadimpushtaev/decorator-inside-python-class-1e74d23107f6?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e74d23107f6</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[decorators]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[metaprogramming]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Thu, 06 Apr 2017 14:25:18 GMT</pubDate>
            <atom:updated>2017-04-07T09:25:58.440Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s suppose we need a decorator that we gonna use inside one specific class only. Something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3747e6a7013b815cf1ee59915a0a7b0c/href">https://medium.com/media/3747e6a7013b815cf1ee59915a0a7b0c/href</a></iframe><p>Where to define this decorator?</p><h3>Function inside class</h3><p>The immediate urge is to put atomic_rating_change into the class itself:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/badd432d19f9dd7ca3f0acc2308cee65/href">https://medium.com/media/badd432d19f9dd7ca3f0acc2308cee65/href</a></iframe><p>That would actually work, but there is major flaw in this approach: atomic_rating_change becomes <em>an instance method</em> of theUser class. That doesn’t make any sense. More than this, it doesn’t even work as a method: if you call it, the decorated parameter will be used as self.</p><p>Note, that within the class body atomic_rating_change is still a plain function. It becomes a method only <em>after </em>the class is actually created, and at this time the decorator is already successfully applied.</p><p>Here is a small example that may help you understand this concept:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bded4840dc43b31539c82dee77bfcec9/href">https://medium.com/media/bded4840dc43b31539c82dee77bfcec9/href</a></iframe><p>You can see indeed that func is a function within the class body, but it’s a method outside of the class. Moreover, A.func and A().func values differ too, thanks to Python <a href="https://docs.python.org/2/howto/descriptor.html">descriptor </a>magic (if you are unfamiliar with descriptors in Python, you should check the link). In Python 3 A.func is just a plain function though.</p><p>What can we do to make this work properly? Here are some ideas.</p><h3>Make it static (don’t)</h3><p>The simple thing you do every time you want to store just a plain function in a class is just to make it a <em>static method.</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/02eb726d7fc297edcd51a35f30829489/href">https://medium.com/media/02eb726d7fc297edcd51a35f30829489/href</a></iframe><p>You may think that it resolves all the problems completely, but sadly it doesn’t. staticmethod is another descriptor, and descriptors behave differently inside and outside of a class (yeah, again).</p><p>So, once you apply this decorator to some method you get TypeError: &#39;staticmethod&#39; object is not callable, because you see a raw descriptor inside the class.</p><h3>Declare outside</h3><p>The most simple solution you are probably already thinking of is to declare function outside of the class.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76b8187987e0beb467fd1c354c34b4df/href">https://medium.com/media/76b8187987e0beb467fd1c354c34b4df/href</a></iframe><p>The only downside is the function is not anyhow bound to the class anymore.</p><h3>Declare in another class</h3><p>To fix this we can declare a helper class that contains all the decorators for User:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/57d149fcae07ee4e4a36c55b1c0746b6/href">https://medium.com/media/57d149fcae07ee4e4a36c55b1c0746b6/href</a></iframe><p>UserDecorators isn’t bound to the original class either (save the name), but since it’s not a function anymore, we can put it in User!</p><h3>Declare in inner class</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0a914b80abed6dc12c4a99264e9b4ac0/href">https://medium.com/media/0a914b80abed6dc12c4a99264e9b4ac0/href</a></iframe><p>You also can make Decorators private (by renaming it to _Decorators) if you don’t like other classes to use your decorator.</p><p>This solution is pretty solid and recommended, but there is one appealing method I’m aware of.</p><h3>Make class, not method</h3><p>Thanks to Python __call__ magic method, you can declare any decorator as a class, not as a function. Recall, that our initial problem is <em>functions</em> behaving differently inside and outside of a class body, but that doesn’t apply at all to <em>classes</em> inside a class body (look at the Decorators in the previous examples: it works just fine). So converting the decorator from the first example to a class would work perfectly.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f87c901429e921bdd18bb43c0d8061e9/href">https://medium.com/media/f87c901429e921bdd18bb43c0d8061e9/href</a></iframe><p>Fun fact: both working solutions involve changing the inner function to the inner class (due to the specific behaviour of function attributes, duh).</p><p>Note that the described problem is not of a conceptual nature, it’s just a way to fight the fact that a function can’t be a simple class attribute in Python.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e74d23107f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to test private methods]]></title>
            <link>https://medium.com/@vadimpushtaev/how-to-test-private-methods-4bc57d4410ff?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/4bc57d4410ff</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[unit-testing]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Mon, 16 Jan 2017 21:53:02 GMT</pubDate>
            <atom:updated>2018-11-18T06:30:11.985Z</atom:updated>
            <content:encoded><![CDATA[<p>To test a method you need to execute it, but calling private methods directly can be hard or even impossible, depending on the programming language you use. In this article I’ll primarily speak about Python and Java, but the described techniques surely can be applied to many other languages (but I can’t be sure about <em>all of them</em>).</p><h3>What is a private method?</h3><p>Have you ever asked yourself, why do you even need private methods? You need private data to maintain consistency, but why private methods?</p><p>Strictly speaking, you don’t. But they still be helpful at least for two major reasons:</p><ul><li>You want to extract some code that works with private data;</li><li>You want to extract some code that doesn’t work with private data, but still doesn’t suit API (because user simple <em>doesn’t care</em>).</li></ul><p>While the first one prevents private data corruption, the second merely makes your API cleaner.</p><h3>Don’t test it</h3><p>Surprisingly, when I first came to Java programming and googled “How to test private method in Java”, the most popular answer was something like “Why do you want to test private methods? Users call public methods, test them instead”.</p><p>Well, that sounds crazy for me. I surely can just ignore the fact that a public methods call a private one, just imagine that private method code is inlined, but that means that code of the private method will be tested again and again with every public method that calls the private one.</p><p>In the following example you really want to test private method, not the two almost identical public ones.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8aacd0bd1297c355b8d17f7fc02ec991/href">https://medium.com/media/8aacd0bd1297c355b8d17f7fc02ec991/href</a></iframe><p>One can notice that some refactoring can allow me not to test the private method. That’s true and we will talk about it later. But testing _set_status without any changes is still clean and reasonable way to do. I don’t buy this “don’t test private methods”.</p><h3>Just call it</h3><p>The simplest and most straightforward way to call a private method is, you know, call it. And that’s exactly what we do in Python and other languages that has <em>private by convention</em> (e. g. Perl).</p><p>To “just call it” in Java we need to change visibility of the method.</p><p>The first way is to make method <em>package private</em> (no access modifier) and put tests into the same package. This is a fairly common practice, but you still might want (or already have) another code structure.</p><p>The second way is to make method public. To let people know you still don’t want to call this method you can use <a href="https://google.github.io/guava/releases/19.0/api/docs/com/google/common/annotations/VisibleForTesting.html">@VisibleForTesting</a>annotation from Guava or any other convention you like (that’s actually what Python and Perl do). By the way, IDEA totally understands that annotation and will warn you about using such <em>public </em>method outside of tests.</p><h3>Nested class</h3><p>You can also put a test class inside a tested one (at least in Java), but that doesn’t look great. You have to use the same file for both classes, and your production binaries will actually contain test code.</p><h3>Reflection</h3><p>In some languages reflections will do fine, in Ruby it’s that simple:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/62627631940ce2d81984b9e02a3598e8/href">https://medium.com/media/62627631940ce2d81984b9e02a3598e8/href</a></iframe><p>But in Java such code is so heavy I gave up on idea of providing an example here. More than this, you have to abandon all cool stuff your favourite IDE does for you.</p><h3>Eliminate private</h3><p>Personally I prefer the “just call it” method, and I like to use @VisibleForTesting in Java to make it happen.</p><p>But let us talk again about refactoring that allows me avoiding testing private methods (by eliminating them).</p><p>The point is to merge all private data and private methods into an object of <em>another </em>class that contains no private methods or data, but put the instance of that class into the only private attribute of the original class. Doesn’t sound simple, but it is, consider examples:</p><p>Before:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a8bca02fd7a6fc5799701c92de03a1c5/href">https://medium.com/media/a8bca02fd7a6fc5799701c92de03a1c5/href</a></iframe><p>After:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e67a21527de32679e5991d0365922ab6/href">https://medium.com/media/e67a21527de32679e5991d0365922ab6/href</a></iframe><p>So now you can freely test EntityPrivateData, the only thing that remains private is data attribute. Like I said before, you actually don’t need private methods, only private data.</p><p>The described method can be useful not only for testing private methods, but also for more expressive design of your software. You can use any number of such private data classes so they have more sense semantically, not only technically.</p><p>For me, the most important thing about this pattern is that it proves that you technically can eliminate all private methods. But I still doubt that it’s reasonable to do it every time, your code can bloat without any significant benefit.</p><p><em>This article was written with the help of </em><a href="https://github.com/NikolayRys"><em>Nikolay Rys</em></a>.</p><h3>See also</h3><ul><li><a href="https://axelfontaine.com/blog/private-methods.html">Testing private methods: easier than you think! - Axel Fontaine - Entrepreneur, Architect, Developer, Trainer, Public Speaker, Author</a></li><li><a href="http://stackoverflow.com/questions/34571/how-do-i-test-a-class-that-has-private-methods-fields-or-inner-classes">How do I test a class that has private methods, fields or inner classes?</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4bc57d4410ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Name of a Python function]]></title>
            <link>https://medium.com/@vadimpushtaev/name-of-python-function-e6d650806c4?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6d650806c4</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[metaprogramming]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[function]]></category>
            <category><![CDATA[stackoverflow]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Wed, 11 Jan 2017 20:16:05 GMT</pubDate>
            <atom:updated>2017-01-12T07:29:44.120Z</atom:updated>
            <content:encoded><![CDATA[<p>In Python there are three things that can be considered being <em>name</em> of the function:</p><h4>The original name of the code block</h4><p>It’s stored in the f.__code__.co_name (where f is the function object). If you use def orig_name to create function, orig_name is that name. For lambas it&#39;s &lt;lambda&gt;.</p><p>This attribute is readonly and can’t be changed. So the only way to create function with the custom name in runtime I’m aware of is exec:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba8ebb07b0d83e38bd098a703fffbf57/href">https://medium.com/media/ba8ebb07b0d83e38bd098a703fffbf57/href</a></iframe><p>(There is also more low-level <a href="http://stackoverflow.com/questions/16064409/how-to-create-a-code-object-in-python">way to do this</a> that.)</p><p>The immutability of co_name actually makes sense: with that you can be sure that the name you see in the debugger (or just stack trace) is exactly the same you see in the source code (along with the filename and line number).</p><h4>The __name__ attribute of the function object</h4><p>It’s also aliased to func_name.</p><p>You can modify it (orig_name.__name__ = &#39;updated name&#39;) and you surely do on a daily basis: <a href="https://docs.python.org/2/library/functools.html#functools.wraps">@functools.wraps</a> copies the __name__ of the decorated function to the new one.</p><p>__name__ is used by tools like pydoc, that&#39;s why you need @functools.wraps: so you don&#39;t see the technical details of every decorator in your documentation. Look at the example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5baa3e2c4477e4a708d1c44a8158ac15/href">https://medium.com/media/5baa3e2c4477e4a708d1c44a8158ac15/href</a></iframe><p>Here is the pydoc output:</p><pre>FUNCTIONS<br>    decorator1(f)</pre><pre>    decorator2(f)</pre><pre>    test1 = decorated(*args, **kwargs)</pre><pre>    test2(*args, **kwargs)</pre><p>With wraps there is no sign of decorated in the documentation.</p><h3>Name of the reference</h3><p>One more thing that can be called <em>function name</em> (though it hardly is) is the name of a variable or an attribute where reference to that function is stored.</p><p>If you create function with def name, the name attribute will be added to the current scope. lambda, on the other hand, just returns a new value (which also can be assigned to attribute or variable: name = lambda: None`).</p><p>Obviously you can create more than one reference to the same function and all that references can have different names.</p><p>The only way all that three things are connected to each other is the<br>def foo statement that creates function object with both __name__ and __code__.co_name equal to foo and assign it to the fooattribute of the current scope. But they are not bound in any way and can be different from each other:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d23265ceb9c7ead5d668e40ac7da869/href">https://medium.com/media/6d23265ceb9c7ead5d668e40ac7da869/href</a></iframe><p>Output:</p><pre>File &quot;my.py&quot;, line 13, in &lt;module&gt;<br>    name_in_module()<br>  File &quot;my.py&quot;, line 7, in orig_name<br>    traceback.print_stack()</pre><p>Pydoc:</p><pre>FUNCTIONS<br>    make_function()</pre><pre>    name_in_module = updated name()<br>        Docstring here</pre><p><em>This article was originally posted as an answer to my own </em><a href="http://stackoverflow.com/questions/40661758/name-of-a-python-function-in-a-stack-trace/40663943"><em>Stack Overflow question</em></a><em>. I thank other people for comments and answers, they helped me to organize my thoughts and knowledge</em>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6d650806c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unit tests cases table]]></title>
            <link>https://medium.com/@vadimpushtaev/unit-tests-cases-table-924b3e55300c?source=rss-78921613d74d------2</link>
            <guid isPermaLink="false">https://medium.com/p/924b3e55300c</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[metaprogramming]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[unit-testing]]></category>
            <dc:creator><![CDATA[Vadim Pushtaev]]></dc:creator>
            <pubDate>Tue, 10 Jan 2017 13:22:32 GMT</pubDate>
            <atom:updated>2017-01-10T20:07:23.788Z</atom:updated>
            <content:encoded><![CDATA[<p>Quite often a unit test I write looks like a bunch of cases that do the same thing with different data. Consider this naive example where I check whether a <em>year </em>is prime or not:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc1dbaeba55de0e270eabec2a19c762b/href">https://medium.com/media/dc1dbaeba55de0e270eabec2a19c762b/href</a></iframe><p>There are two things I want to improve. First of all, I want my test cases look like an actual table, something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5553683b24167ce691b3d8f54a761187/href">https://medium.com/media/5553683b24167ce691b3d8f54a761187/href</a></iframe><p>I’ve been using this approach for quite a while, but the second thing I always wanted to improve is to have separate method for each case. These cases are quite independent and I might want to run setup/teardown for each of them. Moreover, test report for given code only tells about one test, but I would prefer to see it as three.</p><pre>test_is_prime (test_year.YearTestCase) ... ok</pre><p>So a straightforward solution may look like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f6892ba61e8a4cddb923cc840d42a21e/href">https://medium.com/media/f6892ba61e8a4cddb923cc840d42a21e/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/183/0*smDPlZkxWtKI1EN0.gif" /></figure><p>I use double underscore to separate a method name and a case name. I follow the suggestion of the best book about unit testing I’ve ever read: <a href="http://xunitpatterns.com/"><em>XUnit Test Patterns — Refactoring Test Code</em></a><em>.</em></p><p>It looks OK, especially in terms of test report:</p><pre>test_is_prime__composite (test_year.YearTestCase) ... ok<br>test_is_prime__prime (test_year.YearTestCase) ... ok<br>test_is_prime__undef (test_year.YearTestCase) ... ok</pre><p>But now again I have no table to store my cases, this methods look <em>too </em>independed, but I want them to be semantically connected.</p><p>So, how do I solve this problem? Thanks to Python I can merge this two solutions making <em>the table of methods</em>. And to make three methods out of one I apparently want to use decorators.</p><p>I decorate code which I want to become a test method with the cases table. The decorated method will be called for each row of the table, and the row will be passed into the method as extra parameters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d0c15711d57a5e9d2b36f6abcead2f0/href">https://medium.com/media/5d0c15711d57a5e9d2b36f6abcead2f0/href</a></iframe><p>The following code makes that decorator work. The @BaseTestCases.cases decorator itself just adds test_cases attribute to the method object and the metaclass will use this to turn the single decorated method to a number of actual test methods. The original method will be persisted in the class with the __test_is_prime name to be used from test_is_prime__* methods.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b3906efc2d15e686a02ee41e568b46c7/href">https://medium.com/media/b3906efc2d15e686a02ee41e568b46c7/href</a></iframe><p>So, now I have what I wanted: a test cases table <em>and </em>a pretty test run report.</p><p>I came up with that solution a couple of months ago and now I’m not quite sure it’s the best one. Let me know if you have more elegant solution for this exact problem. Thank you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=924b3e55300c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>