<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Patrick Favre on Medium]]></title>
        <description><![CDATA[Stories by Patrick Favre on Medium]]></description>
        <link>https://medium.com/@patrickfav?source=rss-f5582e224131------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ekOrZrHyQQ8NwHuP-K07LQ.jpeg</url>
            <title>Stories by Patrick Favre on Medium</title>
            <link>https://medium.com/@patrickfav?source=rss-f5582e224131------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 10:13:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@patrickfav/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Centralize your Checkstyle Configuration with Maven]]></title>
            <link>https://codeburst.io/how-to-centralize-your-checkstyle-configuration-with-maven-7575eacd7295?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/7575eacd7295</guid>
            <category><![CDATA[configuration-management]]></category>
            <category><![CDATA[checkstyle]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[maven]]></category>
            <category><![CDATA[static-code-analysis]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Tue, 14 Apr 2020 18:12:23 GMT</pubDate>
            <atom:updated>2023-02-26T11:37:20.867Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c8_tUoRsb2N2LJ4mZ-Y4LQ.jpeg" /></figure><p>Checkstyle, loved by those who set it up, hated by those who didn’t, is Java&#39;s most popular tool to force your code style flavor onto others. Whatever your feelings about this static analyzer may be, if you have to manage it in your project(s) this article is for you.</p><figure><img alt="Screenshot of the IntelliJ Checkstyle Plugin showing some issues" src="https://cdn-images-1.medium.com/max/1024/1*XdkFokFbY9QXsq0Kal_0Kw.png" /><figcaption>A familiar sight for Checkstyle users</figcaption></figure><p>The default setup is quite easy. Create your checkstyle.xml add the rules you prefer and reference it in your Checkstyle Maven plugin. This may be fine for a single project, managing more would require copying the configuration file over and manually syncing them. Since most developers nowadays can’t escape the ubiquitous microservices style architecture, where the myriads of services are managed either in a <a href="https://medium.com/@mattklein123/monorepos-please-dont-e9a279be011b">monorepo</a> or in multiple individual ones, the latter needs a better solution for this problem.</p><h3>Maven to the Rescue</h3><p>Of course there is a direct solution for this in your favorite build management system (which is, of course, Maven)! The required steps are as follows:</p><ol><li>Create a new Maven project only containing your checkstyle.xml</li><li>Reference it in the Checkstyle Maven plugin of your consumer project</li><li>Make your project deployable to publish it to your Maven repo</li></ol><p>Done!</p><h4>1. Create a Project for your Checkstyle Config</h4><p>Create a new project with your <a href="http://maven.apache.org/archetypes/maven-archetype-simple/">preferred method</a>. We name our configuration file checkstyle.xml and put it into /src/main/resource.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*Clg9ncxuFap1xMCg4gPJEA.png" /></figure><p>As test configuration I used this unbearable annoying line length restriction (which helps to see if it works later):</p><pre>&lt;?xml version=&quot;1.0&quot; ?&gt;&lt;!DOCTYPE module PUBLIC<br>        &quot;-//Puppy Crawl//DTD Check Configuration 1.2//EN&quot;<br>        &quot;http://www.puppycrawl.com/dtds/configuration_1_2.dtd&quot;&gt;<br><br>&lt;module name=&quot;Checker&quot;&gt;<br>    &lt;module name=&quot;LineLength&quot;&gt;<br>        &lt;property name=&quot;max&quot; value=&quot;25&quot;/&gt;<br>    &lt;/module&gt;<br>&lt;/module&gt;</pre><p>Now all you have to do is to install it to your local repo with:</p><pre>mvn clean install</pre><h4>2. Reference it in your Consumer Project</h4><p>Find the Checkstyle Plugin in your consumer POM and add your newly create config project as dependency. This may look like this:</p><pre>&lt;plugin&gt;<br>    &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;<br>    &lt;artifactId&gt;maven-checkstyle-plugin&lt;/artifactId&gt;<br>    &lt;dependencies&gt;<br>        &lt;dependency&gt;<br>            &lt;groupId&gt;com.puppycrawl.tools&lt;/groupId&gt;<br>            &lt;artifactId&gt;checkstyle&lt;/artifactId&gt;<br>            &lt;version&gt;8.31&lt;/version&gt;<br>        &lt;/dependency&gt;<br>        <strong>&lt;dependency&gt;<br>            &lt;groupId&gt;com.company.projectname&lt;/groupId&gt;<br>            &lt;artifactId&gt;checkstyle-config&lt;/artifactId&gt;<br>            &lt;version&gt;1&lt;/version&gt;<br>        &lt;/dependency&gt;</strong><br>    &lt;/dependencies&gt;<br>    &lt;configuration&gt;<br>        &lt;configLocation&gt;<strong>checkstyle.xml</strong>&lt;/configLocation&gt;<br>    &lt;/configuration&gt;<br>&lt;/plugin&gt;</pre><p>The filename used in configLocation must match the filename used in your <em>checkstyle-config</em> project. Don’t forget to delete your local Checkstyle configuration file. Now if you do:</p><pre>mvn checkstyle:check</pre><p>it should use the config from your config project.</p><h4>3. Make your Config Project Deployable</h4><p>Some Maven repositories, like <a href="https://search.maven.org/">Maven Central</a>, require a sources and javadoc -jar if you want to deploy them there. Since there is neither source code nor javadoc we have to create placeholders. In our <em>checkstyle-config</em> project add the plugin config for the source code:</p><pre>&lt;plugin&gt;<br>    &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;<br>    &lt;artifactId&gt;maven-source-plugin&lt;/artifactId&gt;<br>    &lt;executions&gt;<br>        &lt;execution&gt;<br>            &lt;id&gt;attach-sources&lt;/id&gt;<br>            &lt;goals&gt;<br>                 &lt;goal&gt;jar-no-fork&lt;/goal&gt;<br>            &lt;/goals&gt;<br>        &lt;/execution&gt;<br>    &lt;/executions&gt;<br>&lt;/plugin&gt;</pre><p>then for the javadoc</p><pre>&lt;plugin&gt;<br>    &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;<br>    &lt;artifactId&gt;maven-jar-plugin&lt;/artifactId&gt;<br>    &lt;executions&gt;<br>        &lt;execution&gt;<br>            &lt;id&gt;empty-javadoc-jar&lt;/id&gt;<br>            &lt;phase&gt;package&lt;/phase&gt;<br>            &lt;goals&gt;<br>                &lt;goal&gt;jar&lt;/goal&gt;<br>            &lt;/goals&gt;<br>            &lt;configuration&gt;<br>                &lt;classifier&gt;javadoc&lt;/classifier&gt;<br>                &lt;classesDirectory&gt;${basedir}/javadoc&lt;/classesDirectory&gt;<br>            &lt;/configuration&gt;<br>        &lt;/execution&gt;<br>    &lt;/executions&gt;<br>&lt;/plugin&gt;</pre><p>Now after setting the correct coordinates for the distribution management, you should be able to deploy and publish your configuration project with</p><pre>mvn deploy</pre><p>A full example, deployed to Maven Central, can be found here</p><p><a href="https://github.com/patrickfav/checkstyle-config/">patrickfav/checkstyle-config</a></p><h3>How to use Suppressions</h3><p>While it is possible to also package a checkstyle-suppression.xml in the same way as described above, I do not think a global suppression file makes a lot of sense.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M0RDHsE9W9keSGFPT2GeGQ.png" /></figure><p>It is, however, possible to set one locally, in basically the same way you would normally. Create your suppression config in your project and reference it in your Checkstyle plugin configuration. In this example we name the file checkstyle-suppression.xml and put it into the project’s root folder. Then we add the reference in the POM:</p><pre>&lt;plugin&gt;<br>    &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;<br>    &lt;artifactId&gt;maven-checkstyle-plugin&lt;/artifactId&gt;<br>    ...<br>    &lt;configuration&gt;<br>        &lt;configLocation&gt;checkstyle.xml&lt;/configLocation&gt;<br>        &lt;suppressionsLocation&gt;checkstyle-suppressions.xml&lt;/suppressionsLocation&gt;<br>    &lt;/configuration&gt;<br>&lt;/plugin&gt;</pre><p>An example using suppressions with a global <em>checkstyle-config</em> can be found <a href="https://github.com/patrickfav/density-converter">here</a> (Checkstyle config is found in parent POM).</p><h3>Keep using your IDE Checkstyle Plugin</h3><p>If you use the excellent <a href="https://plugins.jetbrains.com/plugin/1065-checkstyle-idea">Checkstyle IntelliJ plugin</a> (or a similar tool), you may wonder where the local configuration can be found. After you mvn install it will be located at /target/checkstyle-checker.xml.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c-kOGSr6Knb_gH6AJ1zQhg.png" /></figure><h3>Summary</h3><p>We’ve created a separate Maven project to put a global Checkstyle configuration in it for easier distribution among other projects. We adjusted this project to be able to deploy it to Maven repositories with stricter rules, like Maven Central. We then discussed how to handle suppressions and local Checkstyle plugins.</p><h4>Sources</h4><ul><li><a href="https://stackoverflow.com/a/53707024/774398">How to upload an artifact to Maven Central with an empty javadoc jar (or empty sources jar), because of no Java code nor resources?</a></li><li><a href="https://stackoverflow.com/questions/19682455/how-to-externalise-the-checkstyle-config-for-maven-checkstyle-plugin/19690484#19690484">how to externalise the checkstyle config for maven-checkstyle-plugin</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7575eacd7295" width="1" height="1" alt=""><hr><p><a href="https://codeburst.io/how-to-centralize-your-checkstyle-configuration-with-maven-7575eacd7295">How to Centralize your Checkstyle Configuration with Maven</a> was originally published in <a href="https://codeburst.io/">codeburst</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Better Way to Protect Your Database IDs]]></title>
            <link>https://medium.com/@patrickfav/a-better-way-to-protect-your-database-ids-a33fa9867552?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/a33fa9867552</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[id]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[hashids]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Fri, 26 Apr 2019 07:01:00 GMT</pubDate>
            <atom:updated>2023-04-01T06:35:46.733Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FmlVUU9ozKJhEXoVtrVZsg.png" /></figure><h3><strong>A Better Way to Protect Your IDs</strong></h3><p>Your web application has a strong way of authentication and every resource has checks if your user is actual authorized to access it. So why bother if the user actual knows the internal IDs for the models she is accessing?</p><h4>Issue #1: Leak of Business Intelligence Data</h4><p>Your IDs might expose more information than you might realize. For instance in a web shop when you make an order you will probably be redirected to a success site with your order_id found as a query parameter (or similar):</p><pre> https://www.myshop.com/account/orders?orderid=7865</pre><p>From this you can probably estimate how many orders they processed. But it gets worse. If you make another other one, let&#39;s say 14 days later, and it gets the ID 7921 you can deduce that they receive about 4 orders a day. This is business intelligence data you maybe don’t want your competition to know (see also this <a href="https://medium.com/lightrail/prevent-business-intelligence-leaks-by-using-uuids-instead-of-database-ids-on-urls-and-in-apis-17f15669fd2e">article</a> for a more thorough discussion on this issue).</p><h4>Issue #2: Brute Force Guessing of IDs</h4><p>Often resources are public, but are protected by the fact that nobody knows the correct identifier. Think about a naive photo sharing site where you can make a single photo accessible via a link:</p><pre>https://www.myphotos.com/p/share?id=10989927</pre><p>Now this ID might be totally random, or it might follow a sequence. An attacker could easily check a reasonable interval from e.g. 10989900 to 10990000 and see if any of these links work.</p><h4>Issue #3: <strong>Leak of Personal Information Through IDs</strong></h4><p>Sometimes you are required to share your user IDs with a statistics tool or other external service. If this ID is your primary key in the DB and used as ID in the shop it can probably be used on other parts of the site as an user_id parameter and might leak personal information. Think about a message board where users usually have a public profile.</p><h3>What can be done?</h3><h4>Solution #1: Using UUIDs</h4><p>A simple solution which tackles most of the issues is to not use <em>a </em>sequence in a <em>small numeric range</em> (i.e. a 64-bit integer) but to use a random value of a <em>big numeric range</em>. An implementation of such a concept is the universally unique identifier or <a href="https://en.wikipedia.org/wiki/Universally_unique_identifier">UUID</a>. Version 4 is based on random numbers and the one you want to use. Due to having some metadata it cannot use all the 128-bit data, but is limited to 122-bit (which does not make a big difference in real world applications). There is no standardized text representation, but it is usually formatted as something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CE1zWxkU0ITyjKAhkHM8Og.png" /><figcaption>Example UUID</figcaption></figure><p>The range of 122-bit is so huge, you can pick any such number randomly and have a nearly 100% chance of it being unique in your context. <em>In fact, you are probably the first person to every generate this exact number.</em> Note though that a UUID does not give guarantees whatsoever of it being truly random — most implementation are, however. See <a href="https://stackoverflow.com/a/44227131/774398">this post</a> for more info on the issue.</p><p>It is now absolutely <a href="https://security.stackexchange.com/questions/6141/amount-of-simple-operations-that-is-safely-out-of-reach-for-all-humanity/6149#6149">infeasible</a> for an attacker to guess your IDs (but guessing <em>is</em> theoretically still possible) and if the UUIDs are truly random, there is no observing a sequence (fixes <em>Issue #1 and #2</em>).</p><p>The downside of using UUIDs is that it is maybe slower or more expensive to index in your DB if you use it as primary key and might be a hassle to create a new column if you use it as correlation ID. <a href="https://tomharrisonjr.com/uuid-or-guid-as-primary-keys-be-careful-7b2aa3dcb439">Read here</a> for more in-depth discussion about UUIDs used in databases.</p><p>Also, you still expose an internal ID to the public, which, if used as primary key, cannot change. This requirement might not happen often, <a href="https://stackoverflow.com/q/3838414/774398">but it does</a>. You may also still be prone to <em>Issue #3</em>.</p><h4>Solution #2: Mask your IDs</h4><p>The idea is not new, in fact there is a very popular set of libraries called <a href="https://hashids.org/"><em>HashIds</em></a> which tries to tackle this exact issue. The basic principle is this:</p><blockquote>Before publishing your IDs, encode or encrypt them in a way that makes the underlying value incomprehensible for any client not knowing the secret key.</blockquote><p>In my opinion HashIds has, among others, two main issues:</p><ul><li>Only supports integer types and restricted by limitations set by the original Javascript implementation (e.g. only positive integers up to 2⁵³ bit).</li><li>No real security, more like a home-brew keyed encoding schema and no forgery protection which means an attacker can still easily brute force IDs without understanding them.</li></ul><h3>Improved ID Protection: ID-Mask</h3><p>Unsatisfied with these properties I tried to create an improved version which tackles these and other issues called <strong>ID-Mask</strong> with a <a href="https://github.com/patrickfav/id-mask">reference implementation in Java</a>. The basic features are:</p><ol><li>Support of all types usually used for IDs</li><li>Strong cryptography with forgery protection</li><li>Optional randomized IDs</li></ol><p>Note that with this (and HashIds) approach, <em>there is no possibility of collision </em>since no compression happens (like with a hash).</p><h4>Full Type-Support for IDs</h4><p>If you think about it, there are only a handful of common types used as identifier:</p><ul><li>64-bit integers (often called <a href="https://docs.oracle.com/javase/7/docs/api/java/lang/Long.html">long</a>)</li><li><a href="https://docs.oracle.com/javase/7/docs/api/java/util/UUID.html">UUIDs</a>(which are essentially 128-bit numbers)</li><li>Arbitrary precision integers (called <a href="https://docs.oracle.com/javase/7/docs/api/java/math/BigInteger.html">BigInteger</a> in Java)</li></ul><p>If we somewhat restrict the arbitrary precision part to around 128 bit, we can group into two basic ID types: 64-bit and 128-bit IDs. All of those data types (and <a href="https://github.com/patrickfav/id-mask#step-2-choosing-the-correct-type">some more exotic types</a> for specific uses cases) are supported by the library.</p><h4>Strong Cryptography with Forgery Protection</h4><p>Instead of encoding the ID with a shuffled alphabet, we can just use proven cryptography with the <a href="https://en.wikipedia.org/wiki/Advanced_Encryption_Standard">Advanced Encryption Standard</a> (AES) and a <a href="https://en.wikipedia.org/wiki/HMAC">hash-based message authentication code</a> (HMAC), the latter of which protects the IDs from being altered by an attacker. All the caller needs to provide is a strong secret key. Fortunately since the key-id is encoded in the ID, the secret key can be changed if it gets compromised. There is a slight difference in the scheme for 64-bit based and 128-bit IDs to optimize for output size. For the interested reader, here is the <a href="https://github.com/patrickfav/id-mask#encryption-schema">full explanation of the schema</a>. There is also a <a href="https://crypto.stackexchange.com/q/68415/44838">discussion on crypto.stackexchange.com</a></p><p>In addition, of solving the main <em>Issue #1</em>, these properties also protect from the attack described in <em>Issue #2: brute forcing. </em>With an so-called <em>authentication tag</em> (i.e. the HMAC) attached to the ID, it is now extremely unlikely for an attacker to generate a valid ID.</p><h4>Support for Randomized IDs</h4><p>With IDs for domain models it makes sense that these are deterministic (i.e. do not change for the same model over time) so that the client can check for equality for example for caching. However, some use cases benefit from IDs that generate randomly looking output to make it hard for an attacker to compare or replicate IDs.</p><p>An example would be shareable links. Using the same scenario as above, of the photo sharing app, instead of the actual value it would just look like this:</p><pre>https://www.myphotos.com/p/share?id=U1P72UtA6uS6ddMcTmzdKJg</pre><p>Using the same ID, generating 2 more masked IDs will result in unrelated looking output:</p><pre>https://www.myphotos.com/p/share?id=aGjTc5AQQlWl8REodDmAM1c<br>https://www.myphotos.com/p/share?id=bGx6LykZ0N_B2WpoT-1XbHg</pre><p>Using this method the problem described in <em>Issue #3: leak of personal information</em> can be solved by generating randomized IDs for e.g. your user_id which cannot be used to find context information in e.g. your main site since they do not match. You are however still able to map the ids back to the original users.</p><h4>Adapt the Encoding to Your Needs</h4><p>The reference implementation supports a wide array of encodings which may be chosen due to various requirements (<em>short IDs</em> vs. <em>readability</em> vs. <em>should not contain words</em>), all of them being <em>url-safe</em> of course.</p><p>Example using 64-bit IDs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9NIMYbG1_Q3Llq8loecM9A.png" /></figure><p>with optional formatting for better readability:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L1aSv55wdwbINe0a_rKcbA.png" /><figcaption>Formatted ID</figcaption></figure><p>To avoid the problem of <strong>randomly occurring (english) words</strong> in the masked IDs which could create embarrassing URLs like</p><pre>https://www.myportfolio.com/p?id=SH4RTM4N</pre><p>a Base32 dialect was added with a custom alphabet containing <strong>no vowels</strong> and <strong>other problematic letters</strong> and numbers. For example these could look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PXL4pFWKVngS1aGyiAH_WA.png" /><figcaption>Encoding optimized to not contain words</figcaption></figure><h4>And More</h4><p>Currently, the reference implementation is quite fast with around 2–8µs per ID encryption. There is a built-in cache to improve performance for frequently recurring IDs. Additionally, there are default implementations for Java Jackson JSON <a href="https://fasterxml.github.io/jackson-databind/javadoc/2.9/com/fasterxml/jackson/databind/JsonSerializer.html">Serializer</a> and JAX-RS <a href="https://docs.oracle.com/javaee/7/api/javax/ws/rs/ext/ParamConverter.html">ParamConverter</a>.</p><h4>Code Example</h4><p>Here is a simple example using a 64-bit ID:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2999846dec24fede05c7d05af52a674a/href">https://medium.com/media/2999846dec24fede05c7d05af52a674a/href</a></iframe><p>For more see the readme of the <a href="https://github.com/patrickfav/id-mask">GitHub project</a>.</p><h3>tl;dr</h3><p>There are many reasons why you may want to protect your internal IDs, either because they <strong>leak business intelligence</strong>, allow for<strong> brute forcing</strong> to find hidden content and may leak context of <strong>personal information. </strong>A possible solution is to use an additional ID column in your DB using <strong>UUIDs</strong> which solves many, but not all of the issues and <strong>may not be feasible</strong> with millions of data-sets. Another option is to <strong>encrypt your IDs</strong> to protect their value. There is a set of libraries called <strong>HashIds </strong>which tries to use this approach but currently has some major issues. I implemented a new approach, called <strong>ID-Mask</strong>, with a reference implementation in Java which supports <strong>wide variety of data types</strong> usually used as IDs, <strong>strong cryptography</strong> and also supports generation <strong>randomized IDs</strong>. The library is ready to use and can be <a href="https://github.com/patrickfav/id-mask">found on GitHub</a>.</p><p><a href="https://github.com/patrickfav/id-mask">patrickfav/id-mask</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a33fa9867552" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Security Best Practices: Symmetric Encryption with AES in Java and Android: Part 2]]></title>
            <link>https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-and-android-part-2-b3b80e99ad36?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3b80e99ad36</guid>
            <category><![CDATA[encryption]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[androiddev]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Mon, 12 Nov 2018 09:20:35 GMT</pubDate>
            <atom:updated>2023-04-01T06:32:36.646Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D-bhytqsx7isYbv7I9s_uQ.png" /></figure><h3>Security Best Practices: Symmetric Encryption with AES in Java and Android: Part 2: AES-CBC + HMAC</h3><p><em>This is the follow-up to my previous article: “</em><a href="https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-7616beaaade9"><em>Symmetric Encryption with AES in Java and Android</em></a><em>” where I summarize the most important facts about AES and show how to put it to use with AES-GCM. I highly recommend reading it before this one, because it explains the most important basics, before diving right into the next topic.</em></p><p>This article discusses the following scenario: what if you can’t use the <a href="https://en.wikipedia.org/wiki/Advanced_Encryption_Standard">Advanced Encryption Standard (AES)</a> with authenticated encryption mode like the <a href="https://en.wikipedia.org/wiki/Galois/Counter_Mode">Galois/Counter Mode (GCM)</a>? Either it is not supported on your currently used platform or you have to incorporate a legacy or third party protocol? <a href="https://security.stackexchange.com/questions/184305/why-would-i-ever-use-aes-256-cbc-if-aes-256-gcm-is-more-secure">Whatever reason</a> you may have ditching GCM, you should not ditch the security properties it comes with:</p><ul><li><strong>Confidentiality</strong>: Nobody without the key can read the message</li><li><strong>Integrity: </strong>Nobody has changed the content of the message</li><li><strong>Authenticity: </strong>The originator of the message can be verified</li></ul><p>Choosing a non-authenticated encryption, like the block mode <a href="https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Block_Chaining_(CBC)">cipher block chaining</a> (CBC) by itself, will unfortunately lack the latter two properties since it is <a href="https://crypto.stackexchange.com/questions/13264/why-is-cbc-based-on-aes-malleable-in-blocks-other-than-the-first-block">malleable</a>. How to fix this? As stated in my previous article, a possible solution would be to combine cryptographic primitives to include a <a href="https://en.wikipedia.org/wiki/Message_authentication_code">message authentication code (MAC)</a>.</p><h4>Message Authentication Code (MAC)</h4><p>So what is a MAC, and why do we need it? A MAC is similar to a hash function, meaning it takes a message as input and generates a short so-called <em>tag</em>. To make sure not everybody can create a tag for any arbitrary message, the MAC function requires a secret key for its calculation. In contrast to a signature used with asymmetric encryption, a MAC has the same key for both generation and authentication.</p><p>For example if two parties securely exchanged MAC keys, and every message has an authentication tag attached, they both can check if the message was created by the other party and that it wasn’t changed during transmission. An attacker would need the secret MAC key to forge the authentication tag.</p><p>One of the most widely used types of MAC is the <a href="https://en.wikipedia.org/wiki/HMAC">keyed-hash message authentication code (HMAC)</a> which incorporates a <a href="https://en.wikipedia.org/wiki/Cryptographic_hash_function">cryptographic hash function</a>, often SHA256. As I won’t get into detail of the algorithm, I’ll recommend reading the <a href="https://tools.ietf.org/html/rfc2104">RFC</a>. Other types are available which are based on symmetric ciphers, like <a href="https://en.wikipedia.org/wiki/CBC-MAC">CBC-MAC</a>. Nearly all crypto frameworks include at least a HMAC implementation, including the <a href="https://docs.oracle.com/javase/7/docs/technotes/guides/security/crypto/CryptoSpec.html#Mac">JCA/JCE</a> through its <a href="https://docs.oracle.com/javase/7/docs/api/javax/crypto/Mac.html">Mac</a> class.</p><h4>Using a MAC with Encryption: Scheme</h4><p>So what is the right way to apply this MAC? According to security researcher <a href="https://researcher.watson.ibm.com/researcher/view.php?person=us-hugokraw">Hugo Krawcyzk</a> there are <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.106.5488&amp;rep=rep1&amp;type=pdf">basically three options</a>:</p><ul><li><strong>MAC-then-Encrypt</strong>: MAC the plain-text and then append it before encrypting both (used in <a href="http://tools.ietf.org/html/rfc5246#section-6.2.3">SSL</a>)</li><li><strong>Encrypt-then-MAC</strong>: MAC the cipher-text and initial vector then append it to the cipher-text (used in <a href="http://tools.ietf.org/html/rfc4303#section-3.3.2.1">IPsec</a>)</li><li><strong>Encrypt-and-MAC</strong>: MAC the plain-text, encrypt and then append it to the cipher-text (used in <a href="http://tools.ietf.org/html/rfc4253#section-6.4">SSH</a>)</li></ul><p>Every option has its own properties, and I’ll encourage you to read a full argument of <a href="https://crypto.stackexchange.com/a/205/44838">either option in this post</a>. To summarize, <a href="https://crypto.stackexchange.com/a/224/44838">most</a> <a href="https://moxie.org/2011/12/13/the-cryptographic-doom-principle.html">researchers</a> <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.106.5488&amp;rep=rep1&amp;type=pdf">recommend</a> <a href="https://tools.ietf.org/html/rfc7366">Encrypt-then-MAC (EtM)</a>. It protects against chosen cipher-text attacks, since the MAC can prevent decryption of incorrect messages. Additionally, the MAC can’t leak information about the plain-text since it operates on the cipher-text. On the downside, it is slightly harder to implement since the IV and a possible protocol /algorithm version or type must be included in the tag. The important thing is to never do any cryptographic operation before verifying the MAC, otherwise you can be vulnerable to a <a href="https://en.wikipedia.org/wiki/Padding_oracle_attack">padding-oracle attack</a> (<a href="https://en.wikipedia.org/wiki/Moxie_Marlinspike">Moxie</a> calls this the <a href="https://moxie.org/2011/12/13/the-cryptographic-doom-principle.html">Doom Principle</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X3g7xKlAWuOfPwbsEhQkmQ.png" /><figcaption>Encrypt-then-Mac schema</figcaption></figure><p><strong><em>Addendum: GCM vs. Encrypt-then-Mac</em></strong> Generally the security strength of using either is probably similar. GCM has some advantages:</p><ul><li>Simple to use aka less to get wrong</li><li>Faster, as it only needs a single pass through the whole message</li></ul><p>On the downside it only allows 96 bit initial vector (vs. 128 bit) and HMAC is theoretically stronger than GCM’s internal MAC algorithm GHASH (128 bit tag size vs. 256 bit+). GCM also fails miserably on <a href="https://crypto.stackexchange.com/questions/26790/how-bad-it-is-using-the-same-iv-twice-with-aes-gcm">IV+Key reuse</a>. For more detailed discussion <a href="https://crypto.stackexchange.com/a/10808/44838">read here</a>.</p><h4>Using a MAC with Encryption: Authentication Key</h4><p>The last issue we have to solve: where do we get the secret key for the MAC calculation? There seems to be <a href="https://crypto.stackexchange.com/a/8086/44838">no known problem</a> when using the same key as for the encryption (when using HMAC) if the used secret key is strong (i.e. sufficiently random and securely exchanged). However, best practice is to use a key derivation function (KDF) to derive 2 sub-keys just to be on the “safe” side if any problems may be found in the future. This can be as simple as calculating a SHA256 on the main secret key and splitting it to two 16 byte blocks. However, I rather much prefer standardized protocols like the <a href="https://tools.ietf.org/html/rfc5869">HMAC-based Extract-and-Expand Key Derivation Function</a> (HKDF) which directly support such use cases without byte fiddling.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i1dsvrKy7SH8PE4l-eJJug.png" /><figcaption>Key derivation of the two sub-keys</figcaption></figure><h3>Implementing AES-CBC with EtM in Java and Android</h3><p>Enough theory, let’s code! In the following examples I will use AES-CBC, a seemingly conservative decision. The reason for this is, it should be <a href="https://docs.oracle.com/javase/7/docs/api/javax/crypto/Cipher.html">guaranteed to be available on nearly every JRE</a> and <a href="https://developer.android.com/reference/javax/crypto/Cipher">Android version</a>. As explained before we will be using the Encrypt-then-Mac scheme with HMAC. The only external dependency is <a href="https://github.com/patrickfav/hkdf">HKDF</a>. This code is basically a port of the example using GCM described in my <a href="https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-7616beaaade9">previous article</a>.</p><h4>Encryption</h4><p>To keep it simple, we use a randomly generated 128-bit key. Java will automatically choose the correct mode when you pass a key with 128, 192 or 256 bit length. Note however, 256-bit encryption usually requires the <a href="http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html">JCE Unlimited Strength Jurisdiction Policy</a> installed in your JRE (OpenJDK &amp; Android is fine). If you are not sure what key size to use, read the passage about this topic in my <a href="https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-7616beaaade9">previous article</a>.</p><pre>SecureRandom secureRandom = new SecureRandom();<br>byte[] key = new byte[<em>16</em>];<br>secureRandom.nextBytes(key);</pre><p>Then we have to create our initialization vector. For CBC a 16 byte long initial vector (IV) should be used. Be mindful to always use a strong <a href="https://en.wikipedia.org/wiki/Pseudorandom_number_generator">pseudorandom number generator (PRNG)</a> like SecureRandom.</p><pre>byte[] iv = new byte[<em>16</em>];<br>secureRandom.nextBytes(iv);</pre><p>Reusing an IV is not as catastrophic as with GCM, but should be avoided nevertheless. See a <a href="https://stackoverflow.com/a/3008544/774398">possible attack here</a>.</p><p>In the next step we will derive our 2 sub-keys needed for encryption and authentication. We will use <a href="https://tools.ietf.org/html/rfc5869">HKDF</a> in the configuration HMAC-SHA256 (<a href="https://github.com/patrickfav/hkdf">with this library</a>) since it is easy and straight forward to use. We generate two 16 byte sub keys using the info parameter in HKDF to differentiate between them.</p><pre>// import at.favre.lib.crypto.HKDF;</pre><pre>byte[] encKey = HKDF.fromHmacSha256().expand(key, &quot;encKey&quot;.getBytes(StandardCharsets.<em>UTF_8</em>), 16);<br>byte[] authKey = HKDF.fromHmacSha256().expand(key, &quot;authKey&quot;.getBytes(StandardCharsets.<em>UTF_8</em>), 32); //HMAC-SHA256 key is 32 byte</pre><p>Next we will initialize the cipher and encrypt our plain-text. Since CBC is and behaves like a block mode we need a padding mode for messages which do not exactly fit the 16 byte block size. Since there seems to be <a href="https://crypto.stackexchange.com/questions/1486/how-to-choose-a-padding-mode-with-aes/1488#1488">no security implication</a> regarding the used padding scheme, we chose the most widely supported: <a href="https://en.wikipedia.org/wiki/Padding_(cryptography)#PKCS%235_and_PKCS%237">PKCS#7</a>.</p><p><strong><em>Note: </em></strong>due to <a href="https://crypto.stackexchange.com/questions/9043/what-is-the-difference-between-pkcs5-padding-and-pkcs7-padding">legacy reasons</a>, we have to set our cipher suite to PKCS5. Both are practically the same but defined for different block sizes; normally PKCS#5 would not be compatible with AES, but since the definitions date back to 3DES where 8 byte blocks were used, we are stuck with it. If your JCE provider accepts AES/CBC/PKCS7Padding it&#39;s better to use this definition, so your code is easier to understand.</p><pre>final Cipher cipher = Cipher.<em>getInstance</em>(&quot;AES/CBC/PKCS5Padding&quot;); //actually uses PKCS#7<br>cipher.init(Cipher.<em>ENCRYPT_MODE</em>, new SecretKeySpec(encKey, &quot;AES&quot;), new IvParameterSpec(iv));<br>byte[] cipherText = cipher.doFinal(plainText);</pre><p>Next we have to prepare the MAC and add the main data to authenticate.</p><pre>SecretKey macKey = new SecretKeySpec(authKey, &quot;HmacSHA256<em>&quot;</em>);<br>Mac hmac = Mac.<em>getInstance</em>(&quot;HmacSHA256<em>&quot;</em>);<br>hmac.init(macKey);<br>hmac.update(iv);<br>hmac.update(cipherText);</pre><p>If you want to authenticate additional metadata, like the protocol version, you could also add it to the mac generation. This is the same concept as adding associated data to an <a href="https://en.wikipedia.org/wiki/Authenticated_encryption#Authenticated_Encryption_with_Associated_Data">authenticated encryption algorithm</a>.</p><pre>if (associatedData != null) {<br>    hmac.update(associatedData);<br>}</pre><p>Then calculate the mac.</p><pre>byte[] mac = hmac.doFinal();</pre><p>Finally, serialize all of it to a single message.</p><pre>ByteBuffer byteBuffer = ByteBuffer.<em>allocate</em>(1 + iv.length + 1 + mac.length + cipherText.length);<br>byteBuffer.put((byte) iv.length);<br>byteBuffer.put(iv);<br>byteBuffer.put((byte) mac.length);<br>byteBuffer.put(mac);<br>byteBuffer.put(cipherText);<br>byte[] cipherMessage = byteBuffer.array();</pre><p>And that’s basically it for encryption. For constructing the message, the length of the IV, the IV, the length of the mac, the mac and the encrypted data are appended to a single byte array.</p><p>Optionally encode it with e.g. <a href="https://en.wikipedia.org/wiki/Base64">Base64</a> if you require a string representation. <a href="https://developer.android.com/reference/android/util/Base64.html">Android does have a standard implementation</a> of this encoding, the JDK only from<a href="https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html"> version 8 on</a> (I would avoid <a href="https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/binary/Base64.html">Apache Commons Codec</a> if possible since it is slow and a messy implementation).</p><p>It is <a href="https://security.stackexchange.com/questions/74280/would-it-be-good-secure-programming-practice-to-overwrite-a-sensitive-variable">best practice to try to wipe sensible</a> data like a cryptographic key or IV from memory as fast as possible. Since Java is a language with <a href="https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)">automatic memory management</a>, we don’t have any guarantees that the following works as intended, but it should in most cases:</p><pre>Arrays.fill(authKey, (byte) 0);<br>Arrays.fill(encKey, (byte) 0);</pre><p>Be mindful to not overwrite data that is still used somewhere else.</p><h4>Decryption</h4><p>Decryption works similar to encryption in reverse: first deconstruct the message.</p><pre>ByteBuffer byteBuffer = ByteBuffer.<em>wrap</em>(cipherMessage);<br><br>int ivLength = (byteBuffer.get());<br>if (ivLength != 16) { // check input parameter<br>    throw new IllegalArgumentException(&quot;invalid iv length&quot;);<br>}</pre><pre>byte[] iv = new byte[ivLength];<br>byteBuffer.get(iv);<br><br>int macLength = (byteBuffer.get());<br>if (macLength != 32) { // check input parameter<br>    throw new IllegalArgumentException(&quot;invalid mac length&quot;);<br>}</pre><pre>byte[] mac = new byte[macLength];<br>byteBuffer.get(mac);<br><br>byte[] cipherText = new byte[byteBuffer.remaining()];<br>byteBuffer.get(cipherText);</pre><p>Be careful to <a href="https://cwe.mitre.org/data/definitions/789.html">validate input parameters</a> to prevent denial of service attacks, like the IV or mac length, as an attacker may change the value.</p><p>Then derive the keys needed for decryption and authentication.</p><pre>// import at.favre.lib.crypto.HKDF;</pre><pre>byte[] encKey = HKDF.fromHmacSha256().expand(key, &quot;encKey&quot;.getBytes(StandardCharsets.<em>UTF_8</em>), 16);<br>byte[] authKey = HKDF.fromHmacSha256().expand(key, &quot;authKey&quot;.getBytes(StandardCharsets.<em>UTF_8</em>), 32);</pre><p><strong>Before </strong>we decrypt anything, we will verify the MAC. First we calculate the MAC as before; don’t forget previously added associated data.</p><pre>SecretKey macKey = new SecretKeySpec(authKey, &quot;HmacSHA256<em>&quot;</em>);<br>Mac hmac = Mac.<em>getInstance</em>(&quot;HmacSHA256<em>&quot;</em>);<br>hmac.init(macKey);<br>hmac.update(iv);<br>hmac.update(cipherText);<br>if (associatedData != null) {<br>    hmac.update(associatedData);<br>}<br>byte[] refMac = hmac.doFinal();</pre><p>Now when comparing the mac, we need a constant time comparing function to avoid <a href="https://en.wikipedia.org/wiki/Side-channel_attack">side channel attacks</a>; <a href="https://codahale.com/a-lesson-in-timing-attacks/">read here why this is important</a>. Luckily we can use <a href="https://docs.oracle.com/javase/7/docs/api/java/security/MessageDigest.html#isEqual(byte[],%20byte[])">MessageDigest.isEquals()</a> (the old bug was fixed in <a href="https://www.oracle.com/technetwork/java/javase/6u17-141447.html">Java 6u17</a>):</p><pre>if (!MessageDigest.isEqual(refMac, mac)) {<br>    throw new SecurityException(&quot;could not authenticate&quot;);<br>}</pre><p>As the last step we can finally decrypt our message.</p><pre>final Cipher cipher = Cipher.<em>getInstance</em>(&quot;AES/CBC/PKCS5Padding&quot;);<br>cipher.init(Cipher.<em>DECRYPT_MODE</em>, new SecretKeySpec(encKey, &quot;AES&quot;), new IvParameterSpec(iv));<br>byte[] plainText = cipher.doFinal(cipherText);</pre><p>That’s it! If you like to see a full example check out my <a href="https://github.com/patrickfav/armadillo?utm_source=android-arsenal.com&amp;utm_medium=referral&amp;utm_campaign=6636">GitHub project Armadillo</a> where <a href="https://github.com/patrickfav/armadillo/blob/master/armadillo/src/main/java/at/favre/lib/armadillo/AesCbcEncryption.java">I use AES-CBC</a>. You can also find this exact <a href="https://gist.github.com/patrickfav/b323f0d9cbd81d5fa9cc4c971b732c77">example as Gist</a> if you have trouble following the snippets.</p><h3>Summary</h3><p>We showed that the usage of AES with cipher block chaining (CBC) and Encrypt-then-MAC schema using HMAC provides all desirable security properties we would like to see from our encryption protocol:</p><blockquote><strong>Confidentiality, integrity and authenticity</strong></blockquote><p>As can be seen, the protocol is a bit more involved as just using GCM. However, these primitives are generally available in all Java/Android environments, so it may be the only option you have. Just consider the following:</p><ul><li>Use a 16 byte random initialization vector (use a strong <a href="https://en.wikipedia.org/wiki/Pseudorandom_number_generator">PRNG</a>)</li><li>Use 128+ bit MAC length (HMAC-SHA256 outputs 256 bit)</li><li>Use Encrypt-then-Mac</li><li>Use a KDF to derive the 2 Sub-Keys</li><li>Verify before decryption (<a href="https://moxie.org/2011/12/13/the-cryptographic-doom-principle.html">Doom Principle</a>)</li><li>Prevent timing attacks by using constant time equals implementations</li><li>Use 128-bit encryption key length (you will be fine!)</li><li>Pack everything together into a single message</li></ul><h4>References</h4><ul><li><a href="https://github.com/patrickfav/hkdf">patrickfav/hkdf</a></li><li><a href="https://github.com/patrickfav/armadillo">patrickfav/armadillo</a></li></ul><h4>Further Reading</h4><p><a href="https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-7616beaaade9">Security Best Practices: Symmetric Encryption with AES in Java and Android</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3b80e99ad36" width="1" height="1" alt=""><hr><p><a href="https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-and-android-part-2-b3b80e99ad36">Security Best Practices: Symmetric Encryption with AES in Java and Android: Part 2</a> was originally published in <a href="https://proandroiddev.com/">ProAndroidDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Bcrypt Protocol… is kind of a mess]]></title>
            <link>https://medium.com/hackernoon/the-bcrypt-protocol-is-kind-of-a-mess-4aace5eb31bd?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/4aace5eb31bd</guid>
            <category><![CDATA[bcrypt-protocol]]></category>
            <category><![CDATA[bcrypt]]></category>
            <category><![CDATA[passwords]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Wed, 07 Nov 2018 16:16:01 GMT</pubDate>
            <atom:updated>2023-02-26T11:25:41.392Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0nnp9MK5uVf5e2ncH6aftw.jpeg" /></figure><p><em>While writing my own bcrypt library, being unsatisfied with the current Java de-facto standard implementation jBcrypt, I discovered a lot of odd things surrounding the bcrypt protocol (mind you: not the underlying cryptographic primitive ‘Eksblowfish’).</em></p><p>Bcrypt is a password hashing function <a href="http://www.usenix.org/events/usenix99/provos/provos_html/node1.html">designed by Niels Provos and David Mazières</a> in 1999 which became popular as the default password hash algorithm for OpenBSD<a href="https://en.wikipedia.org/wiki/Bcrypt">¹</a>. In comparison to simple <a href="https://en.wikipedia.org/wiki/Cryptographic_hash_function">cryptographic hash functions</a> (like SHA-256), the main benefit of using bcrypt is that a developer can set how expensive it is to calculate the hash. This is called <a href="https://en.wikipedia.org/wiki/Key_stretching">key stretching</a> and should be used with any (usually weak) user provided password to protect against brute force attacks (i.e. simple guessing).</p><h4>Introduction to Bcrypt</h4><p>So how does it work? First you need a password and set the iteration count as a logarithmic work factor between 4–31, doubling the required computation every increment. So for example it could look like this:</p><pre>bcrypt(&quot;secretPassword&quot;, 8)</pre><p>which may output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wFASmAwFFZot0rTrJv1W1Q.png" /><figcaption>$2a$08$0SN/h83Gt1jZMR6924.Kd.HaK3MyTDt/W8FCjUOtbY3Pmres5rsma</figcaption></figure><p>This is the <a href="https://passlib.readthedocs.io/en/stable/modular_crypt_format.html">modular crypt format</a> defined and used by OpenBSD. The first part $2a$ is the protocol version identifier. Historically bcrypt used to be $2$ but since nobody back then thought about Unicode handling, a new version had to be defined. The next part 08$ is the work factor as passed to the function. The next 22 ASCII characters (16 byte raw) 0SN/h83Gt1jZMR6924.Kd. represent the encoded salt. Most implementations automatically create a salt for the caller, which is nice. This measure protects against rainbow tables, i.e. using a list of pre-calculated hashes of commonly used passwords.</p><p>And finally the last 23 byte or encoded 31 ASCII characters are the actual bcrypt hash: HaK3MyTDt/W8FCjUOtbY3Pmres5rsma. This format is especially convenient for storing password hashes, since all the parameters apart from the actual password are included.</p><h4>Issue 1: Non-Standard encoding</h4><p>One oddity of bcrypt is that it uses a non-standard Base64 dialect for encoding the salt and hash. This dialect is listed in <a href="https://en.wikipedia.org/wiki/Base64#Radix-64_applications_not_compatible_with_Base64">Wikipedia</a>’s “Radix-64 Applications” section, and is attributed to <a href="https://man7.org/linux/man-pages/man3/crypt.3.html">crypt(3)</a>, with modern usage limited to bcrypt. Although it uses the same alphabet as standard Base64, it is permutated, making it incompatible with <a href="https://tools.ietf.org/html/rfc4648#section-5">RFC 4648</a>. This makes implementing it unnecessarily harder and more error-prone, since practically all programming languages have an RFC 4648 implementation of Base64 or support for it.</p><h4>Issue 2: Using 23 byte instead of the full 24 bytes hash</h4><p>As stated before, nearly all bcrypt implementations output a 23 byte long hash. The bcrypt algorithm however generates a 24 byte password hash by encrypting three 8 byte blocks using a password-derived blowfish key. The original reference implementation however choose truncate the hash output, it is rumored the reason is to cap it to a more manageable length of 60-character limit (a strange reason if you ask me). The <a href="https://news.ycombinator.com/item?id=2654586">consensus seems</a> to be that the issue of cutting a hash byte off is not a meaningful degradation in security, so it stays an oddity inherited from the reference implementation.</p><h4>Issues 3: Derivations of the White Paper vs Reference Implementation</h4><p>According to the <a href="http://javadox.com/org.bouncycastle/bcprov-jdk15on/1.53/org/bouncycastle/crypto/generators/BCrypt.html">BouncyCastle Javadoc</a> the OpenBSD reference implementation derivatives from the algorithm described in the <a href="http://www.openbsd.org/papers/bcrypt-paper.ps">whitepaper</a>:</p><blockquote>In contrast to the paper, the order of key setup and salt setup is reversed: state &lt;- ExpandKey(state, 0, key) state &lt;- ExpandKey(state, 0, salt). This corresponds to the OpenBSD reference implementation of Bcrypt.</blockquote><p>There is no reference to a potential security vulnerability because of that (and I wouldn’t expect one). Since basically everybody copied the behavior of the reference implementation, the specification seems to be superseded.</p><h4>Issue 4: No handling for passwords longer than 56/72 bytes</h4><p>According to the <a href="http://www.openbsd.org/papers/bcrypt-paper.ps">whitepaper</a>:</p><blockquote>(…) the key argument is a secret encryption key, which can be a user-chosen password of up to 56 bytes (including a terminating zero byte when the key is an ASCII string).</blockquote><p>Others <a href="https://security.stackexchange.com/questions/39849/does-bcrypt-have-a-maximum-password-length">have pointed out</a> that the algorithm, internally, manages things as 18 32-bit words, for a total of 72 bytes (including a null-terminator byte). Mind that a UTF-8 character can take up to 4 bytes of space, which would limit the password with the reference limitation to 14 characters in the worst case.</p><p>The confusion about the actual limitation and the lack of specification on what to do on long passwords can create a lot of issues with compatibility as well as <a href="https://security.stackexchange.com/questions/39849/does-bcrypt-have-a-maximum-password-length">possibly creating a lot of different dialects</a> of bcrypt.</p><h4>Issue 5: Many Non-Standard Versions</h4><p>As stated earlier, bcrypt started out with version $2$ which lacked definition on how to handle non-ASCII characters, so the most prevalent version is $2a$ which fixed that. But of course other implementations have had issues too, so there is a $2x$ and $2y$ for the PHP version and there is <a href="http://undeadly.org/cgi?action=article&amp;sid=20140224132743">talk to bump the version output</a> of the original to $2b$. Most of these version changes address bugs in specific implementation and may not apply to others. This however makes it harder to achieve interoperability between different systems (most often databases which are used by PHP and other systems, e.g. <a href="https://stackoverflow.com/questions/49878948/hashing-password-with-2y-identifier">this</a>)</p><h4>Issue 6: Slightly Inefficient Format</h4><p>The output format</p><pre>$2a$08$0SN/h83Gt1jZMR6924.Kd.HaK3MyTDt/W8FCjUOtbY3Pmres5rsma</pre><p>is clearly optimized to be user readable. It is also slightly inefficient to parse: first the whole string has to be read in as ASCII, then it has to be parsed character to character until the last $. After that the next 22 and 32 characters are decoded separately. Using a more compact message format and encoded only once, for example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vUs9KREMBXKziDZFXL0H2w.png" /></figure><p>the storage demand is reduced from 60 bytes to ~56 byte (Base64 encoded). This is irrelevant in most use cases, but in the grant scale when storing millions or billions of password hashes this can make a difference slightly reducing storage demand and parsing computational time.</p><h4>Issue 7: No Official Test Vectors</h4><p>When trying to implement bcrypt, the developer is faced with the lack of official test vectors (aka test cases) to verify the algorithm. Neither the white paper, nor a Google search reveals a lot of useful examples other than some random test cases. This makes it hard to verify compatibility with other implementations. That’s actually the reason I posted some of my own, trying to capture most of the edge cases; <a href="https://github.com/patrickfav/bcrypt/wiki/Published-Test-Vectors">they can be found here</a>.</p><h4>Issue 8: Not well suited as Key Derivation Function</h4><p>Bcrypt on OpenBSD was designed for password storage. Many times however it is necessary to create a secret key from a user password requiring a key derivation function (KDF). The two lacking properties are the ability to set arbitrary out length to satisfy different key types and to just output the raw hash without the whole message format. Currently, one would need to parse the last 31 characters from the hash message and then expand it with a proper KDF like. <a href="https://en.wikipedia.org/wiki/HKDF">HKDF</a>.</p><p><strong>Note</strong>: Bcrypt not being a KDF is not necessarily an issue with bcrypt (it never claimed to be). However, often times developers misuse it as such without really knowing about the potential problems. Giving an option to use as KDF would help.</p><h3>Conclusion</h3><p>The lack of a strong specification by an authority, age and various quirks and bugs in many implementations make it hard to properly implement this time tested password hashing function.</p><p>Security is hard to get right. So even if some of the issues seem nitpicky (and maybe they are), there is no reason to strive for the most simple, straight forward implementation and API resulting in a reviewed specification which can be used as a base for an implementation. Trust me, <a href="https://github.com/patrickfav/armadillo/issues/16">I’ve been there</a>.</p><p><strong><em>Just to reiterate</em></strong>: I do not challenge the security strength of the underlying “Eksblowfish” (“expensive key schedule Blowfish”), those <a href="https://security.stackexchange.com/questions/4781/do-any-security-experts-recommend-bcrypt-for-password-storage">analyses</a> should be left to the cryptographic experts. I would summarize however that bcrypt is still in the realm of recommended password hashing functions.</p><p>Stay tuned for part 2 where I propose a KDF based on bcrypt and an improved password hashing protocol.</p><p>A small plug: most of the issues explained can be overcome with my Java implementation of bcrypt (using the <em>Eksblowfish</em> algorithm from jBcrypt). The main goal of this lib is to be as straight forward as possible to make it hard for people not familiar with bcrypt to get it wrong, but still allowing as much flexibility as possible. Check it out, you maybe find it useful (it’s Apache v2):</p><p><a href="https://github.com/patrickfav/bcrypt">patrickfav/bcrypt</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4aace5eb31bd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/the-bcrypt-protocol-is-kind-of-a-mess-4aace5eb31bd">The Bcrypt Protocol… is kind of a mess</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Concise Interface Implementation Pattern]]></title>
            <link>https://medium.com/@patrickfav/the-concise-interface-implementation-pattern-9b15f35a806b?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/9b15f35a806b</guid>
            <category><![CDATA[interfaces]]></category>
            <category><![CDATA[patterns]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[dependency-injection]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Wed, 11 Apr 2018 17:07:12 GMT</pubDate>
            <atom:updated>2023-02-26T11:32:32.036Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lKIhln05G2WR5Lpi8trfvw.jpeg" /></figure><h4>A convenient pattern to always use interfaces where possible but to keep the code-footprint and complexity of relations small</h4><p><em>This article recaps why using abstraction is better than concrete implementation and how to minimize the burden of overhead for using it. This is aimed at Java, but is valid in many statically typed languages which support OOP and the concept of contracts and inner classes.</em></p><p>Interfaces are great. They expose a clear API, enhance encapsulation and make clean polymorphism possible. Interfaces encourage you to think about responsibility and therefore the required methods and their signature, as well as nearly invite you to properly document them. Interfaces create the basic blocks of abstraction for a clean architecture.</p><p>Now in daily life it can get annoying to use interfaces everywhere. Especially in those instances were you are sure there will only be one implementation or usage in the foreseeable future. In which package should the interface and implementation go? What would be appropriate names? Is the overhead really worth it, considering the interface will not be exposed to other modules? For these instance I use the following pattern:</p><h4>The Concise Interface Implementation</h4><p>Use this pattern if</p><ul><li>…you keep the interface module-private and do not plan the expose it as an API; i.e. it is implementation detail of the module</li><li>…expect only a single implementation (apart from creating mocks during testing)</li><li>…it is OK that interface and implementation have the same visibility</li></ul><p>The goal is to keep the implementation concise, that means the interface and default implementation can be at the exact same location creating a kind of “mini” module. The implementation doesn’t even need its own unique name.</p><p>The template looks like this (Java 7+):</p><pre>public interface MyInterface {</pre><pre>    void interfaceMethod1();<br><br>    void interfaceMethod2();</pre><pre>    final class Default implements MyInterface{</pre><pre>        @Override<br>        public void interfaceMethod1() {<br>            //impl<br>        }</pre><pre>        @Override<br>        public void interfaceMethod2() {<br>            //impl<br>        }</pre><pre>    }<br>}</pre><p>You would define your interface, like any other. Then you add an inner final static (implied) class called Default representing your default implementation. Static inner classes do not have any reference to their outer class, so it behaves just like a normal class defined as top-level class.</p><p>And you would use it like this:</p><pre>MyInterface m = new MyInterface.Default();</pre><p>As you can see, the intent and usage is very clear: you instantiate the default implementation of MyInterface.</p><p>I believe seeing such a construct also conveys the exact properties described above: single implementation and module private; so developers can treat it accordingly.</p><h4>Future-proof</h4><p>If in any point in time you wish to have multiple implementation or expose this interface as an API you just move the Default implementation to its own class and give it a proper name (<a href="https://www.jetbrains.com/help/idea/move-inner-to-upper-level-dialog-for-java.html">IntelliJ can do this for you</a>). The interface and implementation are not tied to each other, they are just defined in the same location in the source code.</p><h4>Avoid unnecessary Naming</h4><p><a href="https://martinfowler.com/bliki/TwoHardThings.html">Naming is hard</a>. With this concept we avoid having to invent a synthetic name for the default implementation of a simple interface. Often times if developers get the naming wrong, it can get very confusing and hard to read, like this example:</p><pre>TextTransformer t = new StringManager();</pre><h4>Avoid Codebase Clutter</h4><p>No need to create new packages structures that do not match the intended one. No need to search for the implementation (although IDEs like <a href="https://www.jetbrains.com/help/idea/navigating-to-super-method-or-implementation.html">IntelliJ make it very easy</a>).</p><h4>Testable</h4><p>One of the main benefits of using interfaces is the better testability. By using well-defined contracts mocking becomes easy. You can use either a mocking library like <a href="http://site.mockito.org/">Mockito</a> or you just implement the interface in your test package. Either way you are able to exactly test the behavior you are interested in and mock the rest.</p><h3>Conclusion</h3><p>By abstracting and creating contracts with interfaces we can create better software. Unfortunately in some cases using interfaces can be unnecessary overhead. The proposed concise interface implementation pattern mitigates this issue by defining the name and location of the implementation without removing the flexibility of later using multiple implementations or as a standalone API.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b15f35a806b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Improving ProGuard Name Obfuscation]]></title>
            <link>https://proandroiddev.com/improving-proguard-name-obfuscation-83b27b34c52a?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/83b27b34c52a</guid>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[androiddev]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[proguard]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Sun, 01 Apr 2018 17:16:01 GMT</pubDate>
            <atom:updated>2023-04-01T06:22:16.186Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lStfNOzD8PRnqzWWCmdVmg.png" /><figcaption>Inspired by <a href="http://www.obfuscationworkshop.io/">www.obfuscationworkshop.io</a></figcaption></figure><p><em>In this article I will show you how to strengthen ProGuard’s name obfuscation, making it harder for an attacker the reverse engineer your code and how this will help prevent many bugs created by incorrect obfuscation</em></p><p>I’ll tell you a secret: ProGuard is actually a code optimizer. One of the optimization’s side effects just happen to add some name obfuscation to the resulting byte code, namely the shorting and reusing of class and method-names. The actually benefit being, that the resulting binary is smaller and better compressible (smaller binaries can be loaded faster into the heap, ie. reduce latency).</p><h4>How does ProGuard’s Name Obfuscation work</h4><p>ProGuard uses dictionaries to define to what to rename a package, class or method. There is a default dictionary which just contains the letters a-z.</p><p>Let’s consider the following code with this lonely class:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*o-HUsvEDfrDr7nEYCplInw.png" /></figure><p>When optimizing with ProGuard, it will start by processing Foo.class. ProGuard will check its dictionary, the first entry being the letter a. There is no class with that name in this package, so this will result in Foo.class being renamed to a.class. Next the methods will be renamed: bar1() will turn into a() and bar2() into b() using the same strategy. A Java syntax representation of the resulting class would look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*sCtp-tNZognAYIWLR8ntXA.png" /><figcaption>Obfuscated version of class Foo</figcaption></figure><p>Now if you would add a new class Foobar.class it would be renamed to b.class and so on. If there are more than 26 classes in a package, the name gets longer: aa.class, ab.class, etc.</p><h3>Preventing Deterministic Name Obfuscation</h3><p>The name obfuscation <strong>process is deterministic</strong>. There is a defined ordering (I guess it is just lexicographical) in which the class are processed, so Foo.class would still be a.class and the methods would still be a() and b() respectively after adding a second class. That does not mean it will never change. If a class is added, which ordered, is in the middle of other classes the obfuscation mapping <em>will </em>change, but often than not the mapping stays similar over builds.</p><p>From a security standpoint this is not optimal. If an attacker knows that in version 1 of your app a.b() is e.g. your licence check logic, it will be easy to find that same logic in version 2 if it is still a.b().</p><h4>Providing Custom Name Obfuscation Dictionary</h4><p>ProGuard allows you to define the following dictionaries: (see the <a href="https://www.guardsquare.com/manual/configuration/usage">official manual for more info</a>)</p><pre><strong>-obfuscationdictionary </strong>method-dictionary.txt<br><strong>-packageobfuscationdictionary </strong>package-dictionary.txt<br><strong>-classobfuscationdictionary </strong>class-dictionary.txt</pre><p>The format of which are <a href="https://stackoverflow.com/a/10046282/774398">just a simple text file</a> with an entry each line, ignoring empty lines and lines starting with #</p><pre># A custom method dictonary<br><br>NUL<br>CoM4<br>COm9<br>lpt2<br>com5</pre><p>It is possible to have a little fun with these files. For instance, in the <a href="https://sourceforge.net/projects/proguard/">ProGuard distribution</a>, there are some examples of alternative dictionaries. <a href="https://github.com/facebook/proguard/blob/master/examples/dictionaries/windows.txt">This file contains names</a> which will make it impossible to extract the classes from the package (e.g .jar) on Windows because it would create illegal file names. Another version is optimized to enable the <a href="https://github.com/facebook/proguard/blob/master/examples/dictionaries/compact.txt">best possible compression</a> by using common small keywords in the byte code format. Another option is to use <a href="https://github.com/facebook/proguard/blob/master/examples/dictionaries/keywords.txt">Java keywords as class and method names</a> which is allowed in the byte code format creating very confusing stack traces.</p><p>Either way, this somewhat improves name obfuscation, but we still have the problem of it being fully deterministic.</p><h4>Randomizing the Dictionary</h4><p><a href="https://stackoverflow.com/users/492694/eric-lafortune">Eric Lafortune</a>, the creator of ProGuard (and it’s commercial counter-part DexGuard) intended the obfuscation to be deterministic (see this <a href="https://sourceforge.net/p/proguard/feature-requests/111/">features request about randomization of the dictionary</a>) but there is an easy trick to work around that: In our build tool, before executing ProGuard, we just generate a file with a random dictionary.</p><p>Using the Android Gradle build process as example, you could dynamically add a task which runs before ProGuard task itself:</p><pre>tasks.whenTaskAdded { currentTask -&gt;<br>    //Android Gradle plugin may change this task name in the future<br>    def prefix = &#39;transformClassesAndResourcesWithProguardFor&#39;</pre><pre>    if (currentTask.name.startsWith(prefix)) {</pre><pre>        def taskName = currentTask.name.replace(prefix,<br>                       &#39;createProguardDictionariesFor&#39;)</pre><pre>        task &quot;$taskName&quot; {<br>            doLast {<br>                createRandomizedDictonaries()<br>            }<br>        }<br><br>        //append scramble task to proguard task<br>        currentTask.dependsOn &quot;$taskName&quot;<br>    }<br>}</pre><p>Now in the task you would need to do the following:</p><ul><li>Read a template file with all possible dictionary entries</li><li>Shuffle the entries; do not pick 100% of the entries but a random amount between e.g. 60–90% so mappings cannot be easily converted between builds</li><li>Write the entries to a file</li><li>Reference the file in your ProGuard with -obfuscationdictionary</li><li>Repeat the steps for the class dictionary -classobfuscationdictionary</li></ul><h4>Additional Features</h4><p>Another feature I would suggest is the option to repackage all classes to a single package. This config would move all classes to a root-level package o</p><pre><strong>-repackageclasses </strong>&#39;o&#39;</pre><p>This can also be set dynamically with a similar logic described as above.</p><p>For easier debugging you can print out the assembled ProGuard config (when using multiple config files) with</p><pre><strong>-printconfiguration </strong>proguard-merge-config.txt</pre><h3>Consequence of using Random Name Obfuscation</h3><p>Be aware that each build variation <em>will have a practically unique obfuscation mapping</em>. So in an Android build, each <a href="https://developer.android.com/studio/build/build-variants.html">build variant</a> (flavor or build type) will create very different stack traces. So be careful to <em>keep all the mappings</em> for every version, flavor and build-type in Gradle and all <a href="https://maven.apache.org/pom.html#Dependencies">classifiers</a> in Maven.</p><p>This isn’t a disadvantage though. One bug which many Android developers experience at least once: <em>persisting of obfuscated names which makes migration impossible</em>. This usually happens when a <a href="https://github.com/FasterXML/jackson-databind">JSON databinding serializer</a> is used, which reads class and method names through reflection and converts them, or by using *.getClass().getName() is used with either SharedPreferences or Databases. The worst part is: this usually doesn’t get noticed because the name obfuscation mapping could stay the same for next couple of releases. So you are stuck with e.g.</p><pre>{<br>   &quot;xf&quot;: {<br>      &quot;a&quot;: &quot;Foo&quot;,<br>      &quot;ce&quot;: {<br>          &quot;tx&quot;: [{<br>              &quot;by&quot;: &quot;Foobar&quot;,<br>              &quot;bv&quot;: 137<br>          }]<br>      }<br>   }<br>}</pre><p>By forcing a different mapping each build, bugs like these will immediately surface, essentially <em>creating a </em><a href="https://en.wikipedia.org/wiki/Fail-fast"><em>fail-fast</em></a><em> for ProGuard mappings</em>.</p><h3>Summary</h3><ul><li>ProGuard’s name obfuscation is <em>deterministic</em>, therefore when the code only changes a little the <em>mappings mostly stay the same</em> over multiple releases</li><li>It is possible to create <em>randomized dictionary</em> for the obfuscation and tell ProGuard to use them, so every build will have a <em>unique mapping</em>, making it harder for an attacker to reverse engineer your code</li><li>Randomized name obfuscation also has the advantage of <em>acting as a fail-fast</em> so common ProGuard configuration issues will</li></ul><p><a href="https://www.guardsquare.com/manual/configuration/usage">ProGuard Manual: Usage | Guardsquare</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83b27b34c52a" width="1" height="1" alt=""><hr><p><a href="https://proandroiddev.com/improving-proguard-name-obfuscation-83b27b34c52a">Improving ProGuard Name Obfuscation</a> was originally published in <a href="https://proandroiddev.com/">ProAndroidDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Handling Proguard as Library Developer or in a Multi-Module Android Application]]></title>
            <link>https://proandroiddev.com/handling-proguard-as-library-developer-or-in-a-multi-module-android-application-2d738c37890?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/2d738c37890</guid>
            <category><![CDATA[proguard]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[android]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Sat, 20 Jan 2018 18:27:47 GMT</pubDate>
            <atom:updated>2023-02-26T11:30:39.254Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pIPLwP3QApfqsq3fJ5n_VQ.jpeg" /></figure><p>When starting a new project, all the Proguard configuration goes into that single proguard-rules.pro file. This is fine for the beginning or just simple apps, but gets unmaintainable when expanding to multiple modules.</p><h4>Modular Proguard Configuration</h4><p>Noticed that in current Android libraries you do not have to add any Proguard rules? That’s because the clever guys designing the Android Gradle plugin now support modular Proguard configurations (<em>I believe since v2</em>).</p><p>Even with local modules, push the rules to the respective module to keep Proguard contained, descriptive and maintainable. This has the main advantage that the configuration lifecycle follows the one of the module itself (e.g. when the module gets deleted all Proguard rules with it etc.)</p><p>Each dependency or module may package its own proguard.txt in the top level of it’s AAR package which will be appended to the main configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/597/1*Rv3I8L5fkr3NFT-gbfXbSg.png" /><figcaption>Example of the file layout of an .aar file containing proguard.txt</figcaption></figure><p>This will be supported with the following Gradle property:</p><pre>buildTypes {<br>    release {<br>        ...<br>        consumerProguardFiles &#39;proguard-rules.pro&#39;<br>    }<br>}</pre><p>where the proguard-rules.pro files contain the rules specific to that module. If you wonder how the merged configuration file looks, just add the following to your current config:</p><pre><strong>-printconfiguration </strong>proguard-merged-config.txt</pre><p>Note however, to <strong>use the most defensive rules possible</strong>, leaving the main module the fine-tuning. For instance if a library adds the -dontoptimze flag in its configuration file, there is no way for the app to enable optimization again. The merging strategy does not seem to be as sophisticated as the e.g. <a href="https://developer.android.com/studio/write/add-resources.html#resource_merging">resource merger</a>, meaning you have no way to force certain rules over others.</p><h4>Supporting Modular Proguard Config with Java Modules</h4><p>Unfortunately the current Android Gradle plugin (<em>v3</em>) does not support consumer Proguard files. This is due to the AarTransformer reading the proguard.txt from the archive but the JarTransformer does not. Hopefully the Android build team will add this in the future.</p><p>In the meantime you could use the following workaround for your <em>local Java modules:</em></p><p>Let’s say your module is called myJavaModule: Create a proguard-rules.pro file in the root as you would normally. Then just create a static link to it in your Proguard config in the app module:</p><pre>buildTypes {<br>    release {<br>        ...<br>        proguardFiles ... &quot;$rootProject.rootDir.absolutePath/myJavaModule/proguard-rules.pro&quot;<br>    }<br>}</pre><p>This is not the most sophisticated workaround, but once the Android DSL will support jar files no more configuration is needed (apart from remove the static link)</p><h3>Summary</h3><ul><li>Use consumerProguardFiles property in the Android Gradle DSL to set module specific Proguard rules (especially as a library developer)</li><li>Be as defensive as possible with those rules, since the consumer most likely cannot override them.</li><li>Currently, there is no direct support for modular configurations with Java modules</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d738c37890" width="1" height="1" alt=""><hr><p><a href="https://proandroiddev.com/handling-proguard-as-library-developer-or-in-a-multi-module-android-application-2d738c37890">Handling Proguard as Library Developer or in a Multi-Module Android Application</a> was originally published in <a href="https://proandroiddev.com/">ProAndroidDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Managing Logging in a Multi-Module Android Application]]></title>
            <link>https://proandroiddev.com/managing-logging-in-a-multi-module-android-application-eb966fb7fedc?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb966fb7fedc</guid>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[timber]]></category>
            <category><![CDATA[slf4j]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[logging]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Sun, 14 Jan 2018 11:43:28 GMT</pubDate>
            <atom:updated>2023-04-01T06:28:50.103Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f-KtBrTFGLj4TDlz8uu6Ug.jpeg" /></figure><p>In this article I will show you how we adapted our logging strategy to a massively grown project structure. In the first part I will go through the issues we had with the current setup and in the second part, what steps we took to improve them.</p><h3>Part I</h3><h4>Issue #1: Many Modules &amp; Libraries</h4><p>Our module layout consists of a growing number of about 15 Android modules and 4 Java modules. Additionally we have around 5 in-house libraries/dependencies we use and of course countless third-party libraries. <strong>A chaotic mixture of LogCat, SLF4J and a couple of home-brew</strong> loggers are used <strong>without any real centralized configuration</strong>.</p><h4>Issue #2: Using Custom Loggers</h4><p>Apart from LogCat we used a very basic custom facade called Flog. It would log the output to an internal file in addition to the console for better crash reports. There were clear rules when to use Flog and when Log but at that time we had 4 modules and different people working in the project. Over time, it was forgotten to use Flog and often times<strong> important log info was lost while finding specific bugs.</strong></p><h4>Issue #3: Inflexible SLF4J Android Binding</h4><p>Most Java projects we depend on use the common logging facade <a href="https://www.slf4j.org/">SLF4J</a>. There are multiple bindings for Android’s LogCat but they all share the same behavior: they respect the isLoggable(tag,level) check. This seems like the correct way of checking whether a message should be logged or not, but <strong>unfortunately it does not make much sense. </strong>The isLoggable method expects a tag and asks if a certain level might be logged, but this <strong>means every class that uses a logger must be defined</strong> (not even talking about how to know all those names)</p><h4>Issue #4: Logging in Production</h4><p>Ideally we want no log output in production. For one, we don’t want to reveal private or confidential data of our users or give away too much how the app works. Additionally, logging incurs, even if only small, a performance penalty we don’t want to accept in a production build were we want to show the user how amazingly fast our service is.</p><p>The current setup, which is just based on Proguard rules removing (part of) the logging statements, is not very stable. <strong>Changes in the config might go unnoticed</strong> and will break the protection. It so happened that a 3rd party lib added -dontoptimze to the Proguard configuration in its <a href="https://stackoverflow.com/questions/41572630/consumerproguardfiles-vs-proguardfiles">consumerProguardFile</a> which <a href="https://developer.android.com/studio/build/shrink-code.html#shrink-code">disables code removal</a>.</p><h4>Issue #5: Guidelines when to Log at what Level</h4><p>Like many projects, ours started with only 2 developers and was in rapid prototyping mode. There were no guidelines or discussions about logging, everyone assumed to be on the same boat. 3 years, 10k commits and 15 developers later, this turned out not be true. It wasn’t terrible but the decision when to use either info, debug or verbose seemed often times random.</p><h3>Part II</h3><p>In this second part I will show step-by-step how we improved on the points detailed above.</p><h4>Step 1: Choosing the Logger that’s right for you</h4><p>After analyzing our current issues we can deduce the following requirements for our new main logger:</p><ul><li>Centralized, stable and easy configuration</li><li>Not too different from the “Android way” of logging to make it easy for developers to adapt and to increase acceptance</li><li>Migration should be as easy as possible</li><li>It should be well-known and accepted in the community, in the sense that if a new team member joins there’s a chance that he or she worked with this framework.</li><li>Above all: it just should be <em>easy </em>and <em>simple </em>— in the end, it’s just logging.</li></ul><p>The first contender was <a href="https://www.slf4j.org/"><strong>SLF4J</strong></a>, <em>the </em>Java logging facade. Unfortunately this checked only one of our requirements: well-known and accepted. SLF4J allows centralized configuration, but it is not easy. If you use a <a href="https://mvnrepository.com/artifact/org.slf4j/slf4j-android">published binder library</a> there’s usually no way to change any behavior — if you want full control you have to implement it yourself (I think this requires 5 classes to implement, most of the stubs for Android). The gluing of the code works like in the old Java days: the library <a href="https://stackoverflow.com/questions/347248/how-can-i-get-a-list-of-all-the-implementations-of-an-interface-programmatically">searches through reflection</a> for a class that implements LoggerFactory in the classpath. The binding therefore works implicitly and seems like magic if you don’t know what it’s doing. The whole SLF4J framework offers a huge amount of features which most are irrelevant in an environment where everything runs on the same device. Finally, it has very different in handling compared to android.util.Log.</p><p>Our next contender was <strong>Jake Wharton’s </strong><a href="https://github.com/JakeWharton/timber"><strong>Timber</strong></a><strong> </strong>logger. It has easy centralized configuration: you just implement a so-called logging Tree and plant() it in debug builds. If no Tree is planted, logging is a <a href="https://en.wikipedia.org/wiki/NOP">no-op</a>. It is very similar to the API provided by android.util.Log with the additional convenience that no TAG has to be provided (Timber figures this out itself, and it works quite well, even in e.g. anonymous classes; I can however not comment on any performance implications this might have). With over <a href="https://github.com/JakeWharton/timber/stargazers">5k stars on GitHub</a> and just by the fact that it’s created by Android’s most popular developer Mr. Wharton ❤ himself, it is probably sure to say that most mid-level Android developers have at least heard of this library by now. Looking at the source code, this is <a href="https://github.com/JakeWharton/timber/blob/trunk/timber/src/main/java/timber/log/Timber.kt">just a single class</a> with a very simple interface to implement your own behavior. Timber seems to check all the boxes.</p><p>Timber additionally has also 2 very handy features: For one it supports logging without the need for string concatenation (<a href="https://www.slf4j.org/faq.html#logging_performance">SLF4J has a similar feature</a>). So instead of</p><pre>Log.<em>v</em>(<em>TAG</em>, &quot;this a simple Log.v message with &quot; + stringValue);</pre><p>you write</p><pre>Timber.<em>v</em>(&quot;this a simple Timber.v message with %s&quot;, stringValue);</pre><p>The advantage is that this will save you 3 allocations in the memory constraint world of Android, as explained by the android.util.Log‘s <a href="https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/util/Log.java">Javadoc</a></p><blockquote>Don’t forget that when you make a call like ‘Log.v(TAG, “index=” + i);’ that when you’re building the string to pass into Log.d, the compiler uses a StringBuilder and at least three allocations occur: the StringBuilder itself, the buffer, and the String object. (…) even more pressure on the gc. (…) you might be doing significant work and incurring significant overhead.</blockquote><p>And two, it supports a lot of very convenient lint checks. For example, it warns you if you, like explained above, use string concatenation instead of string formatting or just that the android logger is used. Most of the rules have quick-fix options making migration a charm:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*fhLdnqnB7NOXUwb9WQAyRQ.png" /><figcaption>Timber custom lint warning</figcaption></figure><p>The main drawback: <strong>Timber does not work with plain Java code </strong>since it is published as an <a href="https://developer.android.com/studio/projects/android-library.html">AAR</a>. I wish it would have an interface <em>Java </em>library and an Android implementation library which would also make handling dependencies with different versions of Timber easier.</p><p>For Java modules we kept SLF4J, since most third party Java libraries already use it and it is basically the standard way of logging in the plain old Java world. All the SLF4J calls will be routed through Timber with the <a href="https://github.com/patrickfav/slf4j-timber">slf4j-timber binding I implemented</a> (I later found out that there is already an <a href="https://github.com/arcao/slf4j-timber">implementation around</a> 🤦).</p><p><strong>Conclusion</strong>: Timber for Android, SLF4J for Java with Timber binding</p><h4>Step 2: Migrating our custom Logger</h4><p>This section might not apply to many projects, but it shows how easy it was to add custom behavior transparently to the new concept.</p><p>As mentioned above, one of our requirements is that we log important messages to an internal rolling log file which will be appended when our internal error tracker sends a crash report. Some of our features heavily suffer from fragmentation of Android devices, therefore this debugging info was often vital in finding some of the more obscure bugs. With Timber a new Tree was implemented and “planted” parallel to the DebugTree:</p><pre>public void onCreate() {<br>    Timber.<em>plant</em>(new Timber.DebugTree());<br>    Timber.<em>plant</em>(new TimberFileTree());<br>}</pre><p>The internal rule-set defines that Info, Warn, Error and WTF log levels will additionally be logged to the internal file. This makes this feature practically transparent for developers nobody has to decide when to use which logger since the usual Timber.i() will already do correct thing.</p><p><strong>Conclusion</strong>: if required, implement your own Tree</p><h4>Step 3: Disabling Logging in Production</h4><p>Logging in production is bad. It exposes possible confidential information and slows the application. Also</p><blockquote>(…) every time you log in production, a puppy dies. — <a href="https://github.com/JakeWharton/timber">Jake Wharton</a></blockquote><p>Since we now have a centralized logging config, we should make use of it. In our application’s onCreate() we plant the DebugTree <em>only in a debug build</em>:</p><pre>public void onCreate() {<br>    ...<br>    if (BuildConfig.<em>DEBUG</em>) {<br>        Timber.<em>plant</em>(new Timber.DebugTree());<br>    }<br>    Timber.<em>plant</em>(TimberFileLogTree.<em>get</em>());<br>}</pre><p>Apart from legacy or third party libraries your app will now be silent in production. To go a step further, and to prevent <a href="https://i.ytimg.com/vi/KEkrWRHCDQU/maxresdefault.jpg"><em>evil hackers</em></a> from reading your log messages (making it easier to understand the code), we let Proguard remove them for us. First is Timber itself:</p><pre><strong>-assumenosideeffects </strong>class timber.log.Timber {<br>    public static *** v(...);<br>    public static *** d(...);<br>}</pre><p>Then SLF4J logging:</p><pre><strong>-assumenosideeffects </strong>class org.slf4j.Logger {<br>    public *** trace(...);<br>    public *** debug(...);<br>}</pre><p>And finally all the legacy Android logging:</p><pre><strong>-assumenosideeffects </strong>class android.util.Log {<br>   public static *** v(...);<br>   public static *** d(...);<br>   public static *** i(...);<br>   public static *** w(...);<br>}</pre><p>Please note the following:</p><ol><li>We only remove debug and verbose logs since our internal file logger logs everything above that. This might not make sense in other projects, and you may safely remove all levels.</li><li>We remove practically everything from Android’s logging since this is basically just for legacy and third party libraries (which we don’t trust to be sensible when it comes to logging); Timber uses Log.println() so it won’t be affected by this</li><li>This will only work when Proguard’s optimize is enabled. <a href="https://developer.android.com/studio/build/shrink-code.html#shrink-code">Read more here</a>.</li><li>If an exception should be logged, that shouldn’t land in production builds just use debug or verbose like Timber.v(exception, &quot;&quot;)</li></ol><p><strong>Conclusion</strong>: a simple if and some Proguard rules will take care of that</p><h4>Step 4: Migration</h4><p>Here’s the annoying part: manually migration Log.* to Timber.*. Lint’s quick-fix option in Android Studio will help you, but it’s going to be a chore. In our case, we immediately migrated some of the core modules and will adapt on-the-fly when touching older classes.</p><p>During migration the following two issues popped up often:</p><ol><li>Leaving the TAG in the method signature like Timber.v(TAG, msg) which will break the log message, since the msg is now expected to be a placeholder for the first parameter, in this case TAG. A full-text search in your project for e.g. Timber.v(TAG will reveal these issues.</li><li>Since Timber uses <a href="https://docs.oracle.com/javase/1.5.0/docs/guide/language/varargs.html">varargs</a> as the last parameter, the Throwable parameter moved to the first place. Compare for instance Log.w(TAG, &quot;msg&quot;, exception) vs Timber.w(exception, &quot;msg&quot;)</li></ol><p>If there are any Java modules using other logging frameworks (or something home-brew), switch to SLF4J. If you never used it, here is a quick start. First add the dependency to the slf4j-api:</p><pre>implementation &#39;org.slf4j:slf4j-api:1.7.25&#39;</pre><p>then you can log with</p><pre>private final Logger logger = LoggerFactory.<em>getLogger</em>(MyClass.class.getSimpleName());<br>...<br>logger.debug(&quot;my message&quot;);</pre><p>in your main module (usually the app module), add the binder implementation to your dependencies</p><pre>implementation &#39;at.favre.lib:slf4j-timber:1.0.0&#39;</pre><p><strong>Conclusion</strong>: it’s going to be painful, just mind to caveats.</p><h4>Step 5: Logging Level Guideline</h4><p><em>This section is quite subjective and might or might not work with your project or team, so your mileage may vary.</em></p><ul><li><strong>Verbose/Trace</strong>: Either very chatty or only relevant for very specific edge cases. This level may include sensitive or confidential data if it helps debugging. <em>Example: view position when dragging.</em></li><li><strong>Debug: </strong>The baseline, when in doubt use this level. It may contain sensitive data but probably should not include confidential data. <em>Example: http network log</em></li><li><strong>Info</strong>: For very important and high level logs. This must not include any sensitive data. If in doubt, think if the message would be a problem if it would be accidentally logged in production. <em>Example: </em><em>onCreate of an Activity signalize start of a new screen.</em></li><li><strong>Warn: </strong>Issues which might not prohibit normal use, but to be aware of. <em>Example: network timeout in background request.</em></li><li><strong>Error</strong>: For exceptions or fatal states. In doubt, log the exception. <em>Example: any catch bock.</em></li><li><strong>WTF:</strong> Usually avoid; only useful if in the error recovery something happens or something that should basically never happen.</li></ul><h4>A Note for Library Developers</h4><p>Enabling logging is the responsibility of the main module (i.e. the app). <strong>Do not ever directly use </strong>Log.*; or worse, any home-brew logging faced. Either use <strong>SLF4J </strong>without binding or<strong> Timber</strong> without a planted<strong> </strong>Tree. With this configuration the library user has the full control over logging of all the dependencies/modules. Just add a section in your GitHub’s readme.md how to configure the logging framework.</p><h3>Conclusion</h3><p>We centralized our logging with <strong>Timber </strong>for Android modules and <strong>SLF4J </strong>with Timber binding<strong> </strong>for Java modules. We adapted our custom logger by implementing a custom Tree. With Proguard and a simple if we can easily remove all logs from the release builds. Timber’s own Lint quick-fixes will help migration a bit of which we do most on-demand when touching old code. By setting a clear guideline it should be clear when to use either log-level.</p><h4>References</h4><ul><li><a href="https://github.com/JakeWharton/timber">GitHub - JakeWharton/timber: A logger with a small, extensible API which provides utility on top of Android&#39;s normal Log class.</a></li><li><a href="https://github.com/patrickfav/slf4j-timber">patrickfav/slf4j-timber</a></li><li><a href="https://www.slf4j.org/manual.html">SLF4J Manual</a></li><li><a href="https://developer.android.com/studio/build/shrink-code.html">Shrink Your Code and Resources | Android Studio</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb966fb7fedc" width="1" height="1" alt=""><hr><p><a href="https://proandroiddev.com/managing-logging-in-a-multi-module-android-application-eb966fb7fedc">Managing Logging in a Multi-Module Android Application</a> was originally published in <a href="https://proandroiddev.com/">ProAndroidDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Security Best Practices: Symmetric Encryption with AES in Java and Android]]></title>
            <link>https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-7616beaaade9?source=rss-f5582e224131------2</link>
            <guid isPermaLink="false">https://medium.com/p/7616beaaade9</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[androiddev]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[encryption]]></category>
            <dc:creator><![CDATA[Patrick Favre]]></dc:creator>
            <pubDate>Sat, 06 Jan 2018 18:37:38 GMT</pubDate>
            <atom:updated>2023-02-26T11:43:57.236Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article I will bring you up to speed on the Advanced Encryption Standard (AES), common block modes, why you need padding and initialization vectors and how to protect your data against modification. Finally, I will show you how to easily implement this with Java avoiding common security issues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-74iYKGPthRrgTos1OK0LQ.png" /></figure><h4>What every Software Engineer should know about AES</h4><p>AES, also known by its original name Rijndael, was selected by the <a href="https://en.wikipedia.org/wiki/National_Institute_of_Standards_and_Technology">NIST</a> in 2000 to find a successor for the dated <a href="https://en.wikipedia.org/wiki/Data_Encryption_Standard">Data Encryption Standard</a>(DES). AES is a block cipher, that means encryption happens on fixed-length groups of bits. In our case the algorithm defines 128 bit blocks. AES supports key lengths of 128, 192 and 256 bit.</p><p>Every block goes through many cycles of transformation rounds. I will omit the details of the algorithm here, but the interested reader is referred to the <a href="https://en.wikipedia.org/wiki/Advanced_Encryption_Standard#High-level_description_of_the_algorithm">Wikipedia article about AES</a>. The important part is that the key length does not affect the block size but the number of repetitions of transformation rounds (128-bit key is 10 cycles, 256 bit is 14)</p><blockquote>Until May 2009, the only successful published attacks against the full AES were <a href="https://en.wikipedia.org/wiki/Side-channel_attack">side-channel attacks</a> on some specific implementations. (<a href="https://en.wikipedia.org/wiki/Advanced_Encryption_Standard#Security">Source</a>)</blockquote><h4>Want to encrypt more than one Block?</h4><p>So AES will only encrypt 128 bit of data, but if we want to encrypt whole messages we need to choose a block mode with which multiple blocks can be encrypted to a single cipher text. The simplest block mode is <a href="https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_Codebook_(ECB)">Electronic Codebook or ECB</a>. It uses the same unaltered key on every block like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*03BfQgtpQWi2M63Tf5vffA.png" /><figcaption>Image from Wikpedia</figcaption></figure><p>This is particularly bad since identical plaintext blocks are encrypted to identical ciphertext blocks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZlQ357RG_SGYaCAJwSYyhA.png" /><figcaption>Image encrypted with ECB block mode reveals patterns of the original (<a href="https://gist.github.com/patrickfav/13b2f727eaf91e3a72d87ac427485cb1">try yourself</a>)</figcaption></figure><p>Remember to <strong>never choose this mode unless you only encrypt data smaller than 128 bit. </strong>Unfortunately it is still often misused because it does not require you to provide an initial vector (more about that later) and therefore <em>seems </em>to be easier to handle for a developer.</p><p>One case has to be handled with block modes though: what happens if the last block is not <em>exactly </em>128 bit? That’s where <a href="https://en.wikipedia.org/wiki/Padding_(cryptography)"><strong>padding</strong></a><strong> </strong>comes into play, that is, filling the missing bits of the block up. The simplest of which just fills the missing bits with zeros. There is practically <a href="https://crypto.stackexchange.com/a/1488/44838">no security implication in the choice of padding</a> in AES.</p><h4>Cipher Block Chaining (CBC)</h4><p>So what alternatives to ECB are there? For one there is CBC which <a href="https://en.wikipedia.org/wiki/Exclusive_or">XOR</a>s the current plaintext block with the previous ciphertext block. <a href="https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Block_Chaining_(CBC)">This way, each ciphertext block depends on all plaintext blocks processed up to that point.</a> Using the same image as before the result would be noise not distinguishable from random data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YTbDSTLRsT4wUhS7Aj83cg.png" /><figcaption>Image encrypted with CBC block mode looks random</figcaption></figure><p>So what about the first block? The easiest way is to just use a block full of e.g. zeros, but then every encryption with the same key and plaintext would result in the same ciphertext. Also, if you reuse the same key for different plaintexts it would make it easier to recover the key. A better way is to use a random <a href="https://en.wikipedia.org/wiki/Initialization_vector"><strong>initialization vector</strong></a><strong> (IV). </strong>This is just a fancy word for random data that is about the size of one block (128 bit). Think about it like the <a href="https://security.stackexchange.com/questions/6058/is-real-salt-the-same-as-initialization-vectors"><strong>salt </strong>of the encryption</a>, that is, an IV can be public, should be random and only used one time. Mind though, that not knowing the IV will only hinder the decryption of the first block since the CBC XORs the ciphertext not the plaintext of the previous one.</p><p>When transmitting or persisting the data it is common to just prepend the IV to the actual cipher message. If you are interested on how to correctly use <a href="https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-and-android-part-2-b3b80e99ad36">AES-CBC check out part 2</a> of this series.</p><h4>Counter Mode (CTR)</h4><p>Another option is to use CTR mode. This block mode is interesting because it <a href="https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Counter_(CTR)">turns a block cipher into a stream cipher</a> which means no padding is required. In its basic form all blocks are numbered from 0 to n. Every block will now be encrypted with the key, the IV (also called <em>nonce </em>here) and the counter value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*Vte0wHeQ_sNgxFZSRZ-jbg.png" /><figcaption>Image from Wikpedia</figcaption></figure><p>The advantage is, unlike CBC, encryption can be done in parallel and all blocks are depended on the IV not only the first one. A big caveat is, that an IV <strong>must never be reused</strong> with the same key because an attacker can <a href="https://crypto.stackexchange.com/a/2993/44838">trivially calculate</a> the used key from that.</p><h4>Can I be sure that nobody altered my message?</h4><p>The hard truth: <em>encryption does not automatically protect</em> against data modification. It is actually a <a href="https://en.wikipedia.org/wiki/Padding_oracle_attack">pretty</a> <a href="https://en.wikipedia.org/wiki/Transport_Layer_Security#BEAST_attack">common</a> <a href="https://en.wikipedia.org/wiki/Lucky_Thirteen_attack">attack</a>. <a href="https://security.stackexchange.com/a/33576/60108">Read here</a> on a more thorough discussion about this issue.</p><p>So what can we do? We just add <a href="https://en.wikipedia.org/wiki/Message_authentication_code">Message Authentication Code (MAC)</a> to the encrypted message. A MAC is similar to a digital signature, with the difference that the verifying and authenticating key are practically the same. <a href="https://en.wikipedia.org/wiki/Authenticated_encryption#Approaches_to_authenticated_encryption">There are different variations</a> of this method, the mode that is <a href="https://crypto.stackexchange.com/a/224/44838">recommended by most researchers is called Encrypt-then-Mac</a>. That is, after encryption a MAC is calculated on the cipher text and appended. You would usually use <a href="https://en.wikipedia.org/wiki/Hash-based_message_authentication_code">Hash-based message authentication code (HMAC)</a> as type of MAC.</p><p>So now it starts getting complicated. For integrity/authenticity we have to choose a MAC algorithm, choose an encryption tag mode, calculate the mac and append it. This is also slow since the whole message must be processed twice. The opposite side has to do the same but for decrypting and verifying.</p><h4>Authenticated Encryption with GCM</h4><p>Wouldn’t it be great if there were modes which handled all the authentication stuff for you? Fortunately there is a thing called <a href="https://en.wikipedia.org/wiki/Authenticated_encryption">authenticated encryption</a> which simultaneously provides confidentiality, integrity, and authenticity assurances on the data. One of the most popular block modes that supports this is called <a href="https://en.wikipedia.org/wiki/Galois/Counter_Mode"><strong>Galois/Counter Mode</strong></a><strong> or GCM</strong> for short (it is e.g. also available as a <a href="https://tools.ietf.org/html/rfc5288">cipher suite in TLS v1.2</a>)</p><p>GCM is basically CTR mode which also calculates an authentication tag sequentially during encryption. This authentication tag is then usually appended to the cipher text. Its size is an important security property, so it should be at least 128 bit long.</p><p>It is also possible to authenticate additional information not included in the plaintext. This data is called <a href="https://en.wikipedia.org/wiki/Authenticated_encryption#Authenticated_Encryption_with_Associated_Data"><strong>associated data</strong></a><strong>.</strong> Why is this useful? For example the encrypted data has a meta property, the creation date, which is used to check if the content must be re-encrypted. An attacker could now trivially change the creation date, but if it is added as associated data, GCM will also verify this piece of information and recognize the change.</p><h4>A heated discussion: What Key Size to use?</h4><p>So intuition says: the bigger, the better — it is obvious that it is harder to brute force a 256-bit random value than a 128 bit. With our current understanding brute forcing through all values of a 128 bit long word would require <a href="https://security.stackexchange.com/a/6149/60108">astronomically amount of energy</a>, not realistic for anyone in sensible time (looking at you, NSA). So the decision is basically between (practically) infinite and infinite times 2¹²⁸.</p><blockquote>AES actually has three distinct key sizes because it has been chosen as a US Federal Algorithm Apt at being used in various areas under the control of the US federal government [including the military]. (…) So the fine military brains came up with the idea that there should be <strong>three </strong>”security levels”, so that the most important secrets were encrypted with the heavy methods that they deserved, but the data of lower tactical value could be encrypted with more practical, if weaker, algorithms. (…) So the <a href="http://en.wikipedia.org/wiki/National_Institute_of_Standards_and_Technology">NIST</a> decided to formally follow the regulations (ask for three key sizes) but to also do the smart thing (the lowest level had to be unbreakable with foreseeable technology)(<a href="https://security.stackexchange.com/a/19762/60108">Source</a>)</blockquote><p>The argument follows: an AES encrypted message probably won’t be broken by brute forcing the key, but by other less expensive attacks (not currently known). These attacks will be as harmful to 128-bit key mode as to the 256 bit mode, so choosing a bigger key size doesn’t help in this case.</p><p>So basically 128-bit key is enough security for most of every use case except <a href="https://en.wikipedia.org/wiki/Grover%27s_algorithm">quantum computer protection</a>. Also using 128 bit encrypts faster than 256 bit and the <a href="https://en.wikipedia.org/wiki/Rijndael_key_schedule">key-schedule</a> for 128-bit keys <a href="https://crypto.stackexchange.com/a/5120/44838">seems to be better protected</a> against <a href="https://en.wikipedia.org/wiki/Related-key_attack">related-key attacks</a> (however this is <a href="https://crypto.stackexchange.com/a/1554/44838">irrelevant to most real-world</a> uses).</p><h4>As a Side Note: Side Channel Attacks</h4><p><a href="https://en.wikipedia.org/wiki/Side-channel_attack">Side channel attacks</a> are attacks that aim to exploit issues <a href="https://crypto.stackexchange.com/questions/26186/is-there-a-cryptographic-algorithm-which-is-immune-to-side-channel-attacks/26187#26187">specific to certain implementations</a>. Encryption cipher schemes themselves cannot be inherently protected against them. Simple AES implementations may be prone to <a href="https://crypto.stackexchange.com/questions/5526/aes-timing-attacks">timing</a> and <a href="https://crypto.stackexchange.com/questions/55896/aes-cache-timing-attacks-and-nonce-based-operation-modes">caching </a><a href="https://cr.yp.to/antiforgery/cachetiming-20050414.pdf">attacks</a> <a href="https://www.tandfonline.com/doi/full/10.1080/23742917.2016.1231523">among others</a>.</p><p>As a <a href="https://codahale.com/a-lesson-in-timing-attacks/">very basic example</a>: a simple algorithm that is prone to timing attacks is an equals() method that compares two secret byte arrays. If the equals() has a quick-return, meaning after the first pair of bytes that don’t match it ends the loop, an attacker can measure the time it takes for the equals() to complete and can guess byte for byte until all match.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5PnoWA0GWv4NYG3vyZjYMA.png" /><figcaption>Code that may be vulnerable to timing attacks by using a quick return</figcaption></figure><p>One fix in this instance would be to use a <a href="https://stackoverflow.com/questions/37633688/constant-time-equals">constant-time equals</a>. Mind that it is often not trivial to write constant time code in interpreted languages like <a href="https://crypto.stackexchange.com/a/48878/44838">JVM languages</a>.</p><p>Timing and caching attacks on AES are not merely <a href="https://crypto.stackexchange.com/questions/3775/how-practical-are-side-channel-attacks-and-how-much-of-a-concern-are-they?rq=1">theoretical</a> and can even be exploited over a <a href="https://blog.ircmaxell.com/2014/11/its-all-about-time.html">network</a>. Although protecting against side channel attacks are mostly a concern of developers who implement cryptographic primitives, it is wise to get a sense of what coding practices may be detrimental to the security of the whole routine. The most general theme is, that the <a href="https://crypto.stackexchange.com/a/48877/44838">observable time-related behavior</a> should not depend upon secret data. Additionally, you should be carefully about what implementation to choose. For instance using Java 8+ with <a href="https://openjdk.java.net/">OpenJDK</a> and the default JCA provider should <a href="https://stackoverflow.com/questions/23058309/aes-ni-intrinsics-enabled-by-default">internally use</a> Intel’s <a href="https://en.wikipedia.org/wiki/AES_instruction_set">AES-NI</a> instruction set which is protected against <a href="https://crypto.stackexchange.com/questions/43563/does-aes-ni-offer-better-side-channel-protection-compared-to-aes-in-software">most timing and caching attacks</a> by being constant time and implemented in hardware (while still having good performance). Android uses it’s <a href="https://android.googlesource.com/platform/libcore/+/96b54bb/crypto/src/main/java/org/conscrypt/OpenSSLProvider.java">AndroidOpenSSLProvider</a> which internally may use AES in hardware (<a href="https://developer.arm.com/technologies/trustzone">ARM TrustZone</a>) depending on the <a href="https://en.wikipedia.org/wiki/System_on_a_chip">SoC</a>, but I&#39;m not confident it has the <a href="https://eprint.iacr.org/2018/621.pdf">same protection</a> as Intels pedant. But even if you facilitate hardware, other attack vectors are available, for instance <a href="https://en.wikipedia.org/wiki/Power_analysis">power analysis</a>. Dedicated hardware exist that is specifically designed to protect against most of these issues, namely a <a href="https://en.wikipedia.org/wiki/Hardware_security_module">hardware security module</a> (HSM). Unfortunately these devices usually cost upwards of multiple thousand dollars(fun fact: your <a href="https://en.wikipedia.org/wiki/Smart_card">chip based credit card</a> is also an HSM).</p><h4>Implementing AES-GCM in Java and Android</h4><p>So finally it gets practical. Modern Java has all the tools we need, but the crypto API might not be the most straight forward one. A mindful developer might also be unsure what length/sizes/defaults to use. <em>Note: if not stated otherwise everything applies equally to Java and Android.</em></p><p>In our example we use a randomly generated 128-bit key. Java will automatically choose the correct mode when you pass a key with 192 and 256 bit length. Note however, 256-bit encryption usually requires the <a href="http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html">JCE Unlimited Strength Jurisdiction Policy</a> installed in your JRE (Android is fine).</p><pre>SecureRandom secureRandom = new SecureRandom();<br>byte[] key = new byte[16];<br>secureRandom.nextBytes(key);<br>SecretKey secretKey = new SecretKeySpec(key, &quot;AES&quot;);</pre><p>Then we have to create our initialization vector. For GCM a 12 byte (not 16!) random (or counter) byte-array is <a href="http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38d.pdf">recommended by NIST</a> because it’s faster and more secure. Be mindful to always use a strong <a href="https://en.wikipedia.org/wiki/Pseudorandom_number_generator">pseudorandom number generator (PRNG)</a> like SecureRandom.</p><pre>byte[] iv = new byte[12]; //NEVER REUSE THIS IV WITH SAME KEY<br>secureRandom.nextBytes(iv);</pre><p>Then initialize your cipher. AES-GCM mode should be available to most modern JREs and <a href="https://developer.android.com/reference/javax/crypto/Cipher.html">Android newer than v2.3</a> (<a href="https://github.com/patrickfav/armadillo/issues/6">although only fully functional on SDK 21+</a>). If it happens to be not available install a <a href="https://www.bouncycastle.org/">custom crypto provider like BouncyCastle</a>, but the default provider is usually preferred. We choose an authentication tag of size 128 bit</p><pre>final Cipher cipher = Cipher.getInstance(&quot;AES/GCM/NoPadding&quot;);<br>GCMParameterSpec parameterSpec = new GCMParameterSpec(128, iv); //128 bit auth tag length<br>cipher.init(Cipher.ENCRYPT_MODE, secretKey, parameterSpec);</pre><p>Add optional associated data if you want (for instance meta data)</p><pre>if (associatedData != null) {<br>    cipher.updateAAD(associatedData);<br>}</pre><p>Encrypt; if you are encrypting big chunks of data look into <a href="https://docs.oracle.com/javase/7/docs/api/javax/crypto/CipherInputStream.html">CipherInputStream</a>, so the whole thing doesn&#39;t need to be loaded to the heap.</p><pre>byte[] cipherText = cipher.doFinal(plainText);</pre><p>Now concatenate all of it to a single message</p><pre>ByteBuffer byteBuffer = ByteBuffer.allocate(iv.length + cipherText.length);<br>byteBuffer.put(iv);<br>byteBuffer.put(cipherText);<br>byte[] cipherMessage = byteBuffer.array();</pre><p>Optionally encode it with e.g. <a href="https://en.wikipedia.org/wiki/Base64">Base64</a> if you require a string representation. <a href="https://developer.android.com/reference/android/util/Base64.html">Android does have a standard implementation</a> of this encoding, the JDK only from<a href="https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html"> version 8 on</a> (I would avoid <a href="https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/binary/Base64.html">Apache Commons Codec</a> if possible since it is slow and a messy implementation).</p><p>And that’s basically it for encryption. For constructing the message, the IV, the encrypted data and the authentication tag are appended to a single byte array. (in Java the authentication tag is automatically appended to the message, there is no way to handle it yourself with the standard crypto API).</p><p>It is best practice to try to wipe sensible data like a cryptographic key or IV from memory as fast as possible. Since Java is a language with <a href="https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)">automatic memory management</a>, we don’t have any guarantees that the following works as intended, but it should in most cases:</p><pre>Arrays.fill(key, (byte) 0); //overwrite the content of key with zeros</pre><p>Be mindful to not overwrite data that is still used somewhere else.</p><p>Now to the <strong>decrypt </strong>part; It works similar to the encryption. Initialize the cipher with the IV and add the optional associated data and decrypt:</p><pre>final Cipher cipher = Cipher.getInstance(&quot;AES/GCM/NoPadding&quot;);</pre><pre>//use first 12 bytes for iv<br>AlgorithmParameterSpec gcmIv = new GCMParameterSpec(128, cipherMessage, 0, 12);</pre><pre>cipher.init(Cipher.DECRYPT_MODE, secretKey, gcmIv);<br>if (associatedData != null) {<br>    cipher.updateAAD(associatedData);<br>}</pre><pre>//use everything from 12 bytes on as ciphertext<br>byte[] plainText = cipher.doFinal(cipherMessage, 12, cipherMessage.length - 12);</pre><p>That’s it! If you like to see a full example check out my <a href="https://github.com/patrickfav/armadillo?utm_source=android-arsenal.com&amp;utm_medium=referral&amp;utm_campaign=6636">GitHub project Armadillo</a> where <a href="https://github.com/patrickfav/armadillo/blob/master/armadillo/src/main/java/at/favre/lib/armadillo/AesGcmEncryption.java">I use AES-GCM</a> or this <a href="https://gist.github.com/patrickfav/7e28d4eb4bf500f7ee8012c4a0cf7bbf">minimal working sample on Gist</a>.</p><h4>Summary</h4><p>There are 3 properties we want for securing our data</p><ul><li><strong>Confidentiality: </strong>The ability to prevent eavesdroppers from discovering the plaintext message, or information about the plaintext message.</li><li><strong>Integrity: </strong>The ability to prevent an active attacker from modifying the message without the legitimate users noticing.</li><li><strong>Authenticity</strong> — The ability to prove that a message was generated by a particular party, and prevent forgery of new messages. This is usually provided via a Message Authentication Code (MAC). Note that authenticity automatically implies integrity.</li></ul><p>AES with Galois/Counter Mode (GCM) block mode provides all those properties and is fairly easy to use and is available in most Java/Android environments. Just consider the following:</p><ul><li>Use a 12 byte initialization vector that is never reused with the same key (use a strong <a href="https://en.wikipedia.org/wiki/Pseudorandom_number_generator">pseudorandom number generator</a> like SecureRandom)</li><li>Use a 128-bit authentication tag length</li><li>Use a 128-bit key length (you will be fine!)</li><li>Pack everything together into a single message</li></ul><h4><strong>Further Reading</strong></h4><p><a href="https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-and-android-part-2-b3b80e99ad36">Security Best Practices: Symmetric Encryption with AES in Java and Android: Part 2</a></p><h4>References</h4><ul><li><a href="https://github.com/patrickfav/armadillo">patrickfav/armadillo</a></li><li><a href="https://github.com/patrickfav/bytes-java">patrickfav/bytes-java</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7616beaaade9" width="1" height="1" alt=""><hr><p><a href="https://proandroiddev.com/security-best-practices-symmetric-encryption-with-aes-in-java-7616beaaade9">Security Best Practices: Symmetric Encryption with AES in Java and Android</a> was originally published in <a href="https://proandroiddev.com/">ProAndroidDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>