<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ben de Haan on Medium]]></title>
        <description><![CDATA[Stories by Ben de Haan on Medium]]></description>
        <link>https://medium.com/@bjf.dehaan?source=rss-f3bb16ddd042------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*CnV8M4m6adF4JDG8</url>
            <title>Stories by Ben de Haan on Medium</title>
            <link>https://medium.com/@bjf.dehaan?source=rss-f3bb16ddd042------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:15:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bjf.dehaan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[My selection of InfoSec podcasts]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@bjf.dehaan/my-selection-of-infosec-podcasts-cbd906e9d8b7?source=rss-f3bb16ddd042------2"><img src="https://cdn-images-1.medium.com/max/960/0*4mYBJfgzMHDdV740.jpg" width="960"></a></p><p class="medium-feed-snippet">Podcasts and audio books are my favorite way to keep me entertained during my daily commutes. To combine the entertaining with the useful&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/@bjf.dehaan/my-selection-of-infosec-podcasts-cbd906e9d8b7?source=rss-f3bb16ddd042------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@bjf.dehaan/my-selection-of-infosec-podcasts-cbd906e9d8b7?source=rss-f3bb16ddd042------2</link>
            <guid isPermaLink="false">https://medium.com/p/cbd906e9d8b7</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[podcast]]></category>
            <category><![CDATA[cybercrime]]></category>
            <category><![CDATA[infosec]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <dc:creator><![CDATA[Ben de Haan]]></dc:creator>
            <pubDate>Fri, 11 Oct 2019 13:58:48 GMT</pubDate>
            <atom:updated>2019-10-11T13:58:48.850Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Easy input validation in Python apps]]></title>
            <link>https://medium.com/@bjf.dehaan/easy-input-validation-in-python-apps-c5859b6a6f8d?source=rss-f3bb16ddd042------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5859b6a6f8d</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[input-validation]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Ben de Haan]]></dc:creator>
            <pubDate>Mon, 03 Dec 2018 15:55:52 GMT</pubDate>
            <atom:updated>2022-11-01T20:46:45.035Z</atom:updated>
            <content:encoded><![CDATA[<h4>All you need to know about Cerberus</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TiPJa25n_VLe0BVfSrLfgg.jpeg" /><figcaption><em>CERBERUS, n. The watch-dog of Hades, whose duty it was to guard the entrance; everybody, sooner or later, had to go there, and nobody wanted to carry off the entrance.</em><strong><em> </em></strong><em>- </em>Ambrose Bierce, The Devil’s Dictionary</figcaption></figure><p>In this article you’ll learn:</p><ul><li>Why we need data validation</li><li>How to implement validation in a sample app</li><li>How to test whether the validation works correctly</li></ul><h3>Why do we need this?</h3><p>Imagine an input dialog like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/243/1*suO7GLsVa__h9h3BhpaUiw.png" /><figcaption>Taking gender identity issues to a whole new level.</figcaption></figure><p>This might well be the GUI of an application not validating input. Input validation is among the top controls for mitigating security vulnerabilities. <a href="https://www.owasp.org/index.php/Top_10-2017_Top_10">It also features as a prevention method for multiple items on OWASP’s Top Ten.</a></p><p>It can be hard to manage validation rules for data flowing across different <a href="https://en.wikipedia.org/wiki/Trust_boundary">trust boundaries</a>. Client-side validation with JavaScript just doesn’t cut it as this is easily bypassed, and users are not bound to the options you provide in the GUI. This is why you’ll want to use a <strong>data validation framework</strong>, like Cerberus.</p><h3>What is Cerberus?</h3><p><a href="http://docs.python-cerberus.org/en/stable/">Cerberus</a> is a python library providing lightweight and easily extensible data validation. It is also the library behind the data validation in <a href="http://docs.python-eve.org/en/latest/">Python Eve</a>, an open source REST API framework.</p><p>Instead of implementing separate data validation in every method, you can just call the Cerberus library, where all of the fields of interest are defined in your schemas. Data validation rule management is thus consolidated to this one part of the application, decreasing the likelihood that you’ll miss anything.</p><p>Whatever language you use, there’s probably a data validation framework available, so make sure to check frameworks for your specific language.</p><p>Alright, let’s get cracking!</p><h3>The Basic App</h3><p>You can clone the source code at <a href="https://github.com/benno001/cerberus-demo">https://github.com/benno001/cerberus-demo</a> (note the app was made using <strong>Python 3.5</strong>).</p><p>It’s always a good idea to contain your development in a virtual environment to avoid <a href="https://medium.com/knerd/the-nine-circles-of-python-dependency-hell-481d53e3e025">dependency hell</a>. I’ll create one using Python 3 with <a href="https://virtualenv.pypa.io/en/latest/installation/">virtualenv</a>, then source the activation script to activate the virtual environment venv.</p><pre>virtualenv venv -p python3<br>source venv/bin/activate</pre><p>Firstly, we’ll install the requirements. If you’d like to check the requirements for security issues, by all means, do a <a href="https://pyup.io/safety/">safety check</a> :).</p><pre>pip install -r requirements</pre><p>Now, consider the following sample app (test_price_calculator.py):</p><pre><strong>import cerberus</strong></pre><pre>products = {<br>    &#39;book&#39;: {&#39;stock&#39;: 10, &#39;price&#39;: 10},<br>    &#39;microphone&#39;: {&#39;stock&#39;: 20, &#39;price&#39;: 20},<br>    &#39;laptop&#39;: {&#39;stock&#39;: 2, &#39;price&#39;: 1000}<br>}</pre><pre>def <strong>calculate_price</strong>(product_name, number):<br>    &quot;&quot;&quot;Calculate a price given a product name.&quot;&quot;&quot;<br>    price = products[product_name][&#39;price&#39;] * number<br>    return price</pre><pre>def <strong>test_calculate_price</strong>():<br>    &quot;&quot;&quot;Test ordering products.&quot;&quot;&quot;<br>    assert calculate_price(&#39;laptop&#39;, 1) == 1000</pre><p>There are a few products (book, microphone and laptop) and we can call a method to calculate prices. Running <em>pytest</em>, the method <em>test_calculate_price()</em> will run. And the test’s passing. Yay! Right?</p><p>We’re actually only doing happy path testing… What if we would call our method with a negative number?</p><pre>calculate_price(&#39;laptop&#39;, -10) <strong># Returns -10000. Whoops!</strong></pre><p>To prevent ourselves from running into debt quickly, we could extend the function to check for negative values. But what if you enter a string?</p><pre>calculate_price(&#39;laptop&#39;, &#39;asdf&#39;) <strong># Returns &#39;asdf&#39; 1000 times. Wait, what?</strong></pre><p>Alternatively, we could also calculate the price of 0.001 laptops. Or 20 компьютеры. Or one <a href="https://unicode.org/emoji/charts/full-emoji-list.html#1f4a3">\u1F4A3</a>.</p><p>The point is clear: if we want to catch all of this weirdness or straight-up maliciousness in if-else statements, we’ll end up with a forest of conditions. <em>For every input field</em>. If you have more methods and classes (and chances are, you have), maintenance of this will quickly spiral out of control. It’s better to use some sort of allowlisting…</p><p>Enter Cerberus.</p><h3>Implementing Cerberus</h3><p>So, how does Cerberus work? It comes down to three steps:</p><ol><li>Define a <a href="http://docs.python-cerberus.org/en/stable/schemas.html">schema</a>.</li><li>Initialize a <a href="http://docs.python-cerberus.org/en/stable/api.html#validator-class">Validator</a>.</li><li>Validate document(s).</li></ol><p>Let’s do this. First off, the schema:</p><pre><strong>import cerberus</strong></pre><pre># Create a schema for our method<strong><br>calculate_price_schema</strong> = {<br>    &#39;product_name&#39;: {&#39;type&#39;: &#39;string&#39;},<br>    &#39;number&#39;: {&#39;type&#39;: &#39;integer&#39;}<br>}</pre><p>The schema validates the product name is a string, and the number is a whole number. Let’s implement this.</p><pre>def <strong>calculate_price</strong>(product_name, number):<br>    &quot;&quot;&quot;Calculate a price given a product name.&quot;&quot;&quot;</pre><pre><strong>    d = {&#39;product_name&#39;: product_name, &#39;number&#39;: number}</strong><br>    <strong>v = cerberus.Validator(calculate_price_schema)</strong></pre><pre><strong>    </strong>if <strong>v.validate(d):</strong><br>        price = products[product_name][&#39;price&#39;] * number<br>        return price<br>    else:<strong><br>        return v.error </strong># Contains error if validation fails</pre><p>Notice that we’ve mitigated the ‘asdf’ and 0.001 input, but a negative number is still a problem. How do we mitigate that? Simple, we specify a <em>min</em>. We’ll also set a generous<em> max</em>.</p><pre>calculate_price_schema = {<br>    &#39;product_name&#39;: {<br>        &#39;type&#39;: &#39;string&#39;<br>    },<br>    &#39;number&#39;: {<br>        &#39;type&#39;: &#39;integer&#39;<strong>,</strong><br><strong>        &#39;min&#39;: 0,<br>        &#39;max&#39;: 10000</strong><br>    }<br>}</pre><p>Okay, so we’ve constrained the input for our number to the range 0–10000. But what about the product name? We only have a limited set of products available…</p><p>We can use <em>allowed </em>for this:</p><pre>calculate_price_schema = {<br>    &#39;product_name&#39;: {<br>        &#39;type&#39;: &#39;string&#39;<strong>,</strong><br><strong>        &#39;allowed&#39;: [<br>            &#39;book&#39;,<br>            &#39;microphone&#39;,<br>            &#39;laptop&#39;<br>        ]</strong><br>    },<br>    &#39;number&#39;: {<br>        &#39;type&#39;: &#39;integer&#39;,<br>        &#39;min&#39;: 0,<br>        &#39;max&#39;: 10000<br>    }<br>}</pre><p>I’ve just listed some values, but you can set this in other ways (like list comprehension) if you so desire ;). Cerberus supports various data types other than strings and integers. More on validation rules can be found <a href="http://docs.python-cerberus.org/en/stable/validation-rules.html">here</a>.</p><p>Let’s test this schema by adding some new cases:</p><pre>def test_calculate_price():<br>    &quot;&quot;&quot;Test ordering products.&quot;&quot;&quot;</pre><pre># Test laptop pricing - happy path<br>    assert calculate_price(&#39;laptop&#39;, 1) == 1000<br>    assert calculate_price(&#39;laptop&#39;, 2) == 2000<br>    assert calculate_price(&#39;laptop&#39;, 10) == 10000</pre><pre><strong># Test laptop pricing - sad/bad path</strong><br>    assert calculate_price(&#39;laptop&#39;, -10) == {&#39;number&#39;: [&#39;min value is 0&#39;]}</pre><pre>    assert calculate_price(&#39;laptop&#39;, &#39;asdf&#39;) == {&#39;number&#39;: [&#39;must be of integer type&#39;]}<br>    assert calculate_price(&#39;laptop&#39;, 0.0001) == {&#39;number&#39;: [&#39;must be of integer type&#39;]}</pre><pre>    assert calculate_price(&#39;laptop&#39;, sys.maxsize) == {&#39;number&#39;: [&#39;max value is 10000&#39;]}<br>    assert calculate_price(&#39;laptop&#39;, sys.maxsize**1000) == {&#39;number&#39;: [&#39;max value is 10000&#39;]}</pre><pre>    assert calculate_price(&#39;laptop&#39;, -sys.maxsize**1000) == {&#39;number&#39;: [&#39;min value is 0&#39;]}</pre><pre>    assert calculate_price(&#39;table&#39;, 1) == {&#39;product_name&#39;: [&#39;unallowed value table&#39;]}<br>    assert calculate_price(u&#39;\u1F4A3&#39;, 1) == {&#39;product_name&#39;: [&#39;unallowed value \u1f4a3&#39;]}</pre><p>Of course, this list is by no means comprehensive. You could try and fuzz the function with another tool, or make use of the <a href="https://github.com/minimaxir/big-list-of-naughty-strings">Big List of Naughty Strings</a>.</p><h4>No silver bullet</h4><p>What if you have a free text input box? How would you prevent scripts from being inserted? Consider another sample app (test_comment.py):</p><pre>import cerberus<br>import re</pre><pre>comment_schema = {<br>    &#39;email&#39;: {<br>        &#39;type&#39;: &#39;string&#39;,<br>        &#39;regex&#39;: &#39;^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$&#39;<br>    },<br>    &#39;comment&#39;: {<br>        &#39;type&#39;: &#39;string&#39;<br>    }<br>}</pre><pre>comments = []</pre><pre>def add_comment(email, comment):<br>    &quot;&quot;&quot;Add a comment.&quot;&quot;&quot;<br>    d = {&#39;email&#39;: email, &#39;comment&#39;: comment}<br>    v = cerberus.Validator(comment_schema)<br>    if v.validate(d):<br>        comments.append(d)<br>        return d<br>    else:<br>        return v.errors</pre><pre>def test_add_comment():<br>    value_regex = re.compile(&quot;^value does not match regex*&quot;)</pre><pre>    # Email - Happy path<br>    assert add_comment(&#39;ed@example.com&#39;, &quot;What&#39;s up?&quot;) == {&#39;email&#39;: &#39;ed@example.com&#39;, &#39;comment&#39;: &quot;What&#39;s up?&quot;}<br>    assert add_comment(&#39;ed+eddy@example.com&#39;, &quot;What&#39;s up?&quot;) == {&#39;email&#39;: &#39;ed+eddy@example.com&#39;, &#39;comment&#39;: &quot;What&#39;s up?&quot;}<br>    assert add_comment(&#39;ed+eddy@test.example.com&#39;, &quot;What&#39;s up?&quot;) == {&#39;email&#39;: &#39;ed+eddy@test.example.com&#39;, &#39;comment&#39;: &quot;What&amp;#x27;s up?&quot;}</pre><pre>    # Email - sad/bad path<br>    assert value_regex.match(add_comment(&#39;ed@&#39;, &quot;What&#39;s up?&quot;)[&#39;email&#39;][0])<br>    assert value_regex.match(add_comment(&#39;@.com&#39;, &quot;What&#39;s up?&quot;)[&#39;email&#39;][0])<br>    assert value_regex.match(add_comment(&#39;ed@@example.com&#39;, &quot;What&#39;s up?&quot;)[&#39;email&#39;][0])</pre><p>The email regex allows canonical email addresses but might not be fully RFC5322 compliant.</p><p>The comment would just allow <em>any</em> string. What if we’d enter something in the comment body like:</p><pre>&lt;script&gt;alert(&#39;hi!&#39;);&lt;/script&gt;</pre><p>We’ve just allowed a script into our database. Depending on where this input will be used, this might lead to (stored) XSS, XXE or a deserialization attack.</p><p>Of course, we could try and create a regex for the comment. We’ll quickly find ourselves playing regex whack-a-mole when applying anything but an ultra-restrictive [a-zA-Z0–9] filter: there are so many possibilities and alternative encodings that Cerberus is simply not a viable solution to that problem.</p><p>The solution here is to use proper <a href="https://code.google.com/archive/p/browsersec/wikis/Part1.wiki#HTML_entity_encoding">HTML entity encoding</a>. Python has an <a href="https://docs.python.org/3/library/html.html#html.escape">HTML library</a> to support this. Let’s modify our code slightly:</p><pre><strong>import html</strong></pre><pre># rest of code</pre><pre>def add_comment(email, comment):<br>    &quot;&quot;&quot;Add a comment.&quot;&quot;&quot;<br>    <strong>html_safe_comment</strong> = <strong>html.escape</strong>(comment)<br>    d = {&#39;email&#39;: email, &#39;comment&#39;: <strong>html_safe_comment</strong>}<br>    v = cerberus.Validator(comment_schema)<br>    if v.validate(d):<br>        comments.append(d)<br>        return d<br>    else:<br>        return v.errors</pre><pre>def test_add_comment():</pre><pre>    assert add_comment(&#39;ed@example.com&#39;, &quot;What&#39;s up?&quot;) == {&#39;email&#39;: &#39;ed@example.com&#39;, &#39;comment&#39;: &quot;<strong>What&amp;#x27;s up?</strong>&quot;}<br>    assert add_comment(&#39;ed+eddy@example.com&#39;, &quot;What&#39;s up?&quot;) == {&#39;email&#39;: &#39;ed+eddy@example.com&#39;, &#39;comment&#39;: &quot;<strong>What&amp;#x27;s up?</strong>&quot;}<br>    assert add_comment(&#39;ed+eddy@test.example.com&#39;, &quot;What&#39;s up?&quot;) == {&#39;email&#39;: &#39;ed+eddy@test.example.com&#39;, &#39;comment&#39;: &quot;<strong>What&amp;#x27;s up?</strong>&quot;}</pre><pre>    # Some unmodified code..</pre><pre><strong>    assert add_comment(&#39;ed@example.com&#39;, &#39;&lt;script&gt;alert(&quot;hi!&quot;)&lt;/script&gt;&#39;) == {&#39;email&#39;: &#39;ed@example.com&#39;, &#39;comment&#39;: &#39;&amp;lt;script&amp;gt;alert(&amp;quot;hi!&amp;quot;)&amp;lt;/script&amp;gt;&#39;}</strong></pre><p>This adds some security, although I’d encourage running more test cases. Don’t forget to check data coming from the database too! And take into account that this is also valid JavaScript…</p><pre>(+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[])[([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+{})[+!+[]]+([]+([]+[])[([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+{})[+!+[]]+([]+[][+!+[]])[+!+[]]+([]+![])[+!+[]+!+[]+!+[]]+([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+!![])[+!+[]]+([]+!![])[+!+[]+!+[]]+([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+{})[+!+[]]+([]+!![])[+!+[]]])[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+!![])[+!+[]]+([]+[][+[]])[+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+[][+!+[]])[+!+[]]+([]+([]+[])[([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+{})[+!+[]]+([]+[][+!+[]])[+!+[]]+([]+![])[+!+[]+!+[]+!+[]]+([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+!![])[+!+[]]+([]+!![])[+!+[]+!+[]]+([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+([]+{})[+!+[]]+([]+!![])[+!+[]]])[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]](+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[])+([]+!![])[+!+[]+!+[]]+([]+{})[+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]</pre><h3>Conclusion</h3><p>With a very simple application we demonstrated that a data validation framework can make your life easier:</p><ul><li>Allowlisting using schemas allows us to avoid lots of if-else statements</li><li>We can manage validation schemas in a convenient place</li><li>We mitigate <em>a lot</em> of potential (security) problems arising from missing input validation</li></ul><p>Now, go forth and implement a data validation framework in your projects! And don’t forget to do HTML entity encoding :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5859b6a6f8d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>