<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Lightrail - Medium]]></title>
        <description><![CDATA[The branded currency platform for customer growth. - Medium]]></description>
        <link>https://medium.com/lightrail?source=rss----6fc22cf3d350---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Lightrail - Medium</title>
            <link>https://medium.com/lightrail?source=rss----6fc22cf3d350---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 05:26:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/lightrail" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Getting Token Authentication Right in a Stateless Single Page Application]]></title>
            <link>https://medium.com/lightrail/getting-token-authentication-right-in-a-stateless-single-page-application-57d0c6474e3?source=rss----6fc22cf3d350---4</link>
            <guid isPermaLink="false">https://medium.com/p/57d0c6474e3</guid>
            <category><![CDATA[authentication]]></category>
            <category><![CDATA[jwt]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[api]]></category>
            <dc:creator><![CDATA[Peter Locke]]></dc:creator>
            <pubDate>Fri, 07 Jul 2017 16:06:39 GMT</pubDate>
            <atom:updated>2017-07-07T19:31:46.997Z</atom:updated>
            <content:encoded><![CDATA[<p>Single page application architecture is becoming more prevalent, yet many established patterns to implement authentication security and user experience have not caught up. Patterns used by traditional web applications do not cross over well, or at all, to a true stateless architecture where there is no server-side web session.</p><p>The popularity of JSON Web Tokens (JWTs) is well deserved. This fantastic evolution is a building block for solving these problems. As JWTs provide a simple, cryptographically secure means of exchanging information, they make stateless authentication possible. However a building block is just that: a place to start. In going through all the requirements to produce a truly professional SaaS authentication/authenticated user experience in the <a href="https://www.lightrail.com/">Lightrail</a> front-end (a stateless single page app backed by the same <a href="https://www.lightrail.com/docs/">RESTful API</a> that is published and exposed to external customers), we found that there were gaps in emerging best practices when it comes to a polished customer experience.</p><p>This post discusses in detail the JWT based authentication strategy and architecture we decided on, and the technical and user requirements that led us there. Key areas that are covered:</p><ul><li>Security versus convenience in authentication approaches</li><li>Multi-factor (2FA) authentication</li><li>The essential parts of the JWT payload</li><li>Providing a professional user experience with elegant session timeout and extension</li><li>Why we chose to split the JWT over two cookies</li></ul><p>A high level understanding of JWTs will be helpful in getting the most out of this post. There is an <a href="https://jwt.io/introduction/">excellent introduction at jwt.io</a> that compliments this post.</p><h3>Authentication: the Thankless Layer</h3><p>Logging into an application is a fundamental, crucial, and yet thankless experience. When it works correctly, most users gloss over its underlying complexity. They enter their username and password. Perhaps a piece of secondary authentication such as a code from a text message, TOTP hardware device, or an authenticator app if they have smartly set such a thing up and the service supports it. It is not part of the core product experience - just an expectation. Try talking to your users about salted hashes; they’ll more likely get hungry than think you’re doing a good job of protecting their data.</p><p>Authentication has to be secure but it also has to make sense in context of the application; the authenticated experience for Facebook is not the same as for your online banking site. And it shouldn’t be.</p><h3>Security and Convenience: the Clashing UX Requirements of Authentication</h3><p>When designing the authentication user experience, convenience and level of security are often at odds.</p><p><strong>More Secure</strong></p><ul><li>Secondary Authentication Method (2FA)</li><li>No persistent login ‘remember me’ (closing the tab/browser logs the user out)</li><li>Inactivity timeout (leaving the tab/window open but not using the site for X amount of time logs the user out)</li></ul><p><strong>More convenient for the user</strong></p><ul><li>Username and password only</li><li>Remain logged in across browser sessions</li><li>Remain logged in indefinitely regardless of activity</li></ul><h3>The Requirements We Chose</h3><p>Using the Facebook vs. online banking analogy again, Facebook and products like them lean to convenience. They want you in your account, serving you those ads, with as few hurdles as possible.</p><p>Your online banking site leans the other way, making darn sure you in fact want to log in and that you don’t stay logged in longer than necessary. We took the same security-centric approach with Lightrail, but with a focus on excellent user experience.</p><h4>End User Requirements</h4><ul><li>Username and password is supplemented with optional 2FA</li><li>Users remain logged in only as long as they have their browser open</li><li>Users only remain logged in only if they have been active within the last 30 minutes.</li><li>If users are inactive for 30 minutes, they are prompted to enter their password and continue their session.</li><li>Users get a different post-login user experience depending on their account type and user permissions.</li></ul><h4>Security / Best Practice Requirements:</h4><p>While the user won’t see any difference from these, we noted them as technical necessities for our stateless architecture and desired best practices.</p><ul><li>The front-end authenticates with the RESTful API in the same way as external integrators, meaning via supplying a valid JWT for the user on each request.</li><li>The server does not maintain any session state between calls. There is no stateful middleware layer between the front-end and API.</li><li>JavaScript/front-end should never have access to the full JWT</li><li>Everything HTTPS.</li></ul><p>With the requirements in hand, we set out to implement the two fundamental pieces of an authentication system:</p><ul><li>Logging the user in</li><li>Keeping them authenticated between requests</li></ul><p>Let’s dive into each.</p><h3>Logging in the user</h3><p>The majority of the traditional flow of registration and accepting a username and password remains unchanged and that’s good. If you’re implementing your own code to hash/salt a password or verify it, that’s a problem flag as it should not be architecture dependent.</p><p>The difference comes after the credentials are validated. In a traditional web architecture a session gets created — let’s see how that happens and what requirements it solves that we’ll need to solve differently:</p><h4>What We Can Learn From Traditional Sessions</h4><p>A successful authentication happy path results in the creation of an in-memory object on the server that persists across requests (the ‘session’). The user’s browser is given an HTTPOnly cookie that points to that session.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xPnE3qdGG9wq8gbefvKijQ.png" /><figcaption>Figure 1: traditional session creation</figcaption></figure><p>For each subsequent request the cookie is sent by the browser. If the session is still present on the server, the user is considered authenticated and the request continues. This flow enables a number of requirements essential to any web application, and that we’ll need for our SPA:</p><ul><li>The username and password only need to be supplied once and not for every request. No brainer.</li><li>If the user is inactive and the server does not get a request for that user for a set amount of time, they are no longer authenticated (as the session will be removed server side) and will be asked to re-login.</li><li>Every time a user is active (meaning makes a request to the server), their session length on the server is renewed.</li><li>If the user closes their browser, they are no longer logged in (as the session-identifying cookie will be gone).</li></ul><h4>A stateless implementation</h4><p>As the above shows, the server-side state provides a lot of the desired functionality from our requirements. So how do we make this work without it? This is where JWTs come in. After successful authentication, instead of creating a server-side session, we’ll give the browser a JWT we’ve created for that user. We’ll do this in the form of setting two cookies, each containing a portion of the JWT.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N3DLceC3hr2903H8zov4-Q.png" /><figcaption>Figure 2: Trading user credentials for a JWT</figcaption></figure><p>By splitting the JWT across two cookies, rather than using a single cookie or having the JWT stored in the browser local storage, we are able to meet our requirements without adding additional logic. Let’s see how:</p><h3>Keeping A User Authenticated with the Right UX: The Two Cookie JWT Approach</h3><p>It is important to understand how the JWT achieves authentication. From jwt.io:</p><blockquote><strong>Authentication</strong>: This is the most common scenario for using JWT. Once the user is logged in, each subsequent request will include the JWT, allowing the user to access routes, services, and resources that are permitted with that token.</blockquote><p>By including the cryptographically-guaranteed unmodified JWT on every request from the front-end, which contains identifying information for that user such as their uuid and allowed permissions, we’ve achieved a continually authenticated experience with no server-side state. Slick.</p><h4>The Cookie Split</h4><p>JWTs take the following form:</p><pre>header.payload.signature </pre><p>To implement our requirements, we put the header.payload in one cookie, and the signature in another, each with specific cookie attributes.</p><p><strong>Cookie 1:</strong></p><p>header.payload content with cookie attributes:</p><ul><li>secure</li><li>30 minute expiry</li></ul><p><strong>Cookie 2:</strong></p><p>signature content with cookie attributes:</p><ul><li>secure</li><li>HTTPOnly</li></ul><p>Over the following sections we’ll see how this split and these attributes enable the discussed requirements.</p><h4>The Authentication Transformation Layer</h4><p>To meet our technical requirement of the front-end and external API integrators using the same Lightrail RESTful API, a thin <em>stateless </em>authentication transformation layer is needed.</p><p>For external integrators supplying the JWT in the Authentication header, this layer is a passthrough no-op.</p><p>For requests that contain the two cookies (front-end requests), it reconstructs the JWT from the cookie content and places it Authentication header that the API expects before passing along the request. This layer also guards against CSRF (more on this later), and updates the header.payload cookie expiry by setting a new cookie on responses that contained valid JWT cookies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zwssp_fcvLG12uk3nuTgvA.png" /><figcaption>Figure 3: API and web front-end sharing same backing API</figcaption></figure><h4>User Roles and the JWT</h4><p>The concept of different users having different permissions is part of any modern authentication system. For example, a team administrator can add and remove team members, but a standard user cannot.</p><p>In our architecture this is accomplished through a roles list contained in the ‘payload’ section of the JWT.</p><pre>header.<br>{<br>//other payload data omitted for readability<br>&quot;roles&quot;: [<br>    &quot;teamAdmin&quot;,<br>    &quot;accountManager&quot;<br>  ]<br>}<br>.signature</pre><p>The roles (each is mapped to a list of allowed endpoints on the back end) provide two key functions:</p><ul><li>Calls to the API will fail for any type of request for which the user does not have permission.</li><li>As the roles are part of the JWT payload and therefore JavaScript readable in the header.payload cookie, the front-end can make user interface decisions based on a user’s roles.</li></ul><p><em>Important reminder: you can never trust anything client side readable not to be modified.</em> It is worth noting that if using a symmetric signing algorithm (the same key creates and validates) for your JWTs, the client side <strong><em>cannot</em></strong><em> </em>read the roles securely as it cannot verify the signature (you’d never have your secret key available client side).</p><p>In our implementation (we use symmetric signing) we determined that this is not a problem as the decisions the client-side is making are non sensitive (such as routing) and it is not exploitable server-side due to JWT cryptography. The user interface would become broken/unpopulated as any change to the JWT header.payload cookie content would make the digital signature invalid and all future API calls fail. Users cannot change their roles or elevate their privileges.</p><p>As a side note, JWTs do support asymmetric private/public key signing and verification with an <a href="http://kjur.github.io/jsrsasign/">all JavaScript JWT library</a>. A public key can be safely given and exposed client side. I would however urge caution if you feel this client-side signature validation pattern is needed as it may indicate a security risk in your architecture. Secure work (and data) should come from the server.</p><h4>2FA</h4><p>Implementing 2FA is reasonably painless thanks to roles in the JWT. When the user logs in and 2FA is required they are given a JWT with a role that only allows them to supply the secondary piece of authentication and not access any of their normal account functionality. The front-end will see the different role and prompt them for their secondary authentication code.</p><pre>header.<br>{<br>//other payload data omitted for readability<br>&quot;roles&quot;: [<br>    &quot;2FA&quot;,<br>  ]<br>}<br>.signature</pre><p>Upon successful verification of the secondary authentication code, the authentication server will include an updated JWT in the two cookies as part of the success response restoring the user’s normal set of roles. A ‘remember this device’ cookie will also be given so that they are not prompted for 2FA on every subsequent login for that browser for the given expiry period of the cookie (we chose 14 days).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RQXAeEScDZTIt8V7y3EOgA.png" /><figcaption>Figure 4: The login process when 2FA is required</figcaption></figure><h4>Other Important JWT Payload Content</h4><p>Beyond roles, the JWT payload contains what is necessary to the authentication process and security. Note that some of the keys and value format <a href="https://tools.ietf.org/html/rfc7519#section-4">are defined by RFC 7519</a>. By following the specification the JWTs remain compatible with JWT client libraries.</p><ul><li><strong>useruuid</strong>: fundamental, this ties the token to the particular user that is being authenticated.</li><li><strong>iat: </strong>the date/time of issue.</li><li><strong>exp</strong>: A date/time after which the server will no longer accept the JWT as valid. We chose 1 day. This is longer than any legitimate continued use of the Lightrail web application and a user will get a new JWT on each login. Note also that a side benefit is that we need no token refresh functionality (as the user re-entering their login credentials serves that purpose)</li><li><strong>jti</strong>: giving a token a unique identifier enables <em>optional </em>server-side operations like disabling/blacklisting a token if required.</li></ul><h3>Preventing CSRF</h3><p>If you’ve followed everything so far, you’d be rightfully worried about CSRF. The JWT, contained in the two cookies, will be submitted along with every browser request.</p><p>So what is stopping a malicious user from reading the API documentation, crafting a URL like <a href="https://product.your.com/api/dosomething/private,">https://product.your.com/api/dosomething/private,</a> obfuscating it in a shorturl, and tricking the user to click on it?</p><p>The crucial server side piece here is for the authentication layer to check for a specific request header that your JavaScript sets before it reconstructs the JWT from the submitted cookies. <strong><em>You must check this server side or your application will be open to CSRF</em></strong>. This reason this defence pattern works is because of the <a href="https://en.wikipedia.org/wiki/Same-origin_policy">same-origin policy (SOP)</a> restriction that only JavaScript can be used to add a custom header on Ajax requests, and only within its origin. <a href="https://www.owasp.org/index.php/CSRF_Prevention_Cheat_Sheet#Protecting_REST_Services:_Use_of_Custom_Request_Headers">OWASP describes this in more depth</a>.</p><h4>Same Site Cookies: a Welcome Improvement to CSRF Protection</h4><p>As of writing this post, a new type of cookie security attribute called ‘Same Site’ was still in draft. <a href="http://www.sjoerdlangkemper.nl/2016/04/14/preventing-csrf-with-samesite-cookie-attribute/">http://www.sjoerdlangkemper.nl/2016/04/14/preventing-csrf-with-samesite-cookie-attribute/</a> This attribute is specifically designed to prevent CSRF behaviour. SameSite cookies appear to be an ideal additional layer of security for this type of scenario.</p><h4>Why Not Have the Browser Client Send Back the JWT in the ‘Authorization’ Header Instead of Using Cookies?</h4><p>This decision is worth mentioning as it is a well documented pattern and we considered it. There are advantages, in that the server-side authentication layer would be simplified to just reading the Authorization header and CSRF would no longer be a factor.</p><p>Ultimately, we determined that the additional client side complexity to meet the desired requirements, along with the security considerations in letting the JavaScript have access to the full JWT, outweighed these benefits.</p><p>Having the signature in an HTTPOnly cookie means JavaScript never has access to the full JWT. The cookies also contain all information and timeout logic around authentication thereby providing an excellent single source of truth.</p><h3>Recapping Our Requirements</h3><p>We’ve seen how the JWT across two cookies meets our technical requirements for secure, stateless authentication. Let’s circle back to our user requirements and the rationale behind each cookie and its attributes:</p><h4>Username and password is supplemented with optional 2FA</h4><p><strong><em>Check</em></strong>. JWT role removing/restoring after login and 2FA verification accomplishes this. The front-end can read the JWT role(s) to make routing and display decisions for the user to prompt for 2FA (or not).</p><h4>Users remain logged in only as long as they have their browser open</h4><p><strong><em>Check</em></strong>. The signature cookie will be removed on browser close, and therefore the user will no longer be authenticated as the browser won’t have the full JWT to send to the server anymore.</p><h4>Users only remain logged in only if they have been active within the last 30 minutes</h4><p><strong><em>Check. </em></strong>Unless the user takes an action (thereby resulting in an API call and a new header.payload cookie with a new expiry), their header.payload cookie will expire and they will be no longer authenticated.</p><h4>Elegant handling and extension of session timeout</h4><p><strong><em>Check</em></strong>. The front-end checks for the existence of the header.payload cookie as it is <em>not</em> HTTPOnly. If it doesn’t exist, yet other working state remains, the user is prompted for their password. The password submission simply repeats the credentials-for-JWT handshake shown in Figure 2. Upon success, a new header.payload cookie is issued and the user continues where they left off.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/448/1*D8g7n7mBaIDC0YJgmhDMyw.png" /><figcaption>Figure 5: elegant handling of 30 minutes of inactivity</figcaption></figure><h4>Users get a different post-login user experience depending on their account type and user permissions</h4><p><strong><em>Check</em></strong>. The header.payload cookie and therefore roles can be read by the front-end and it can make routing and display decisions for the user — all with the safety of knowing that if the user attempts to modify their roles all calls to the server will be rejected.</p><h3>In Conclusion</h3><p>We’ve detailed a JWT based authentication layer for a stateless single page app that focuses on tight security and also provides an elegant authenticated user experience.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=57d0c6474e3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lightrail/getting-token-authentication-right-in-a-stateless-single-page-application-57d0c6474e3">Getting Token Authentication Right in a Stateless Single Page Application</a> was originally published in <a href="https://medium.com/lightrail">Lightrail</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prevent Business Intelligence Leaks by Using UUIDs Instead of Database IDs on URLs and in APIs]]></title>
            <link>https://medium.com/lightrail/prevent-business-intelligence-leaks-by-using-uuids-instead-of-database-ids-on-urls-and-in-apis-17f15669fd2e?source=rss----6fc22cf3d350---4</link>
            <guid isPermaLink="false">https://medium.com/p/17f15669fd2e</guid>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ecommerce]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[developer]]></category>
            <dc:creator><![CDATA[Peter Locke]]></dc:creator>
            <pubDate>Mon, 26 Jun 2017 22:43:06 GMT</pubDate>
            <atom:updated>2017-06-26T22:43:48.355Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s examine an innocuous looking URL that you may see when you are looking at your order history on some e-commerce site:</p><p>https://www.yourfavsite.com/account/orders?orderid=5963</p><p>Speaking strictly from an application security perspective, there is no problem here. As long you’ve done your job on the back end checking that the order’s user matches the logged in user, there is no security risk. You can’t look at other users’ orders, and they can’t look at yours.</p><p>The problem is that this one URL gives a surprising amount of insight into the underlying data, and therefore the business. Let’s see how.</p><h4>Leak #1: Overall Business Data Size</h4><p>If this is my (recent) order, there is a good chance that there are only about six thousand orders in that business’s system <em>across all users</em>. That’s a very sensitive piece of data that the business does not want to share.</p><h4>Leak #2: Overall Business Data Velocity</h4><p>If I worked for a competitor and was doing competitive research on this business, I would create a couple low value orders roughly an hour (or some other fixed time) apart. Let’s say the details pages of the two orders have the following URLs:</p><p>https://www.yourfavsite.com/account/orders?orderid=5963</p><p>https://www.yourfavsite.com/account/orders?orderid=5983</p><p>This shows me that the business is <em>doing roughly 20 orders per hour</em>. Again, very sensitive competitive information no business wants so easily determinable.</p><h4>Why This is So Common</h4><p>Certainly not every online business exposes data in this way. But this pattern is common enough that it warrants some attention. In fact, every ‘your first store’ or similar web programming language tutorial leads the learning developer directly down this path. You’ve probably already guessed it: <em>relational databases and generated auto-incrementing sequences.</em></p><p>Relational databases and relational data still power the vast majority of the transactional web and will continue to do so for the foreseeable future. Auto-incrementing numeric sequences as primary keys are a very standard, often default pattern for a relational database table. It’s easy to see why: the database does it for you and lookups are lightening fast. The first row gets an id of ‘1’, the second an id of ‘2’, and so on.</p><p>Every online business generally starts with a single relational database at the core of its architecture. Even as they grow and potentially introduce distributed components and NoSQL datastores and caches to meet load demands, it is likely that this core e-commerce data (orders, items, users) will remain relational. The data <em>is</em> relational and the data architecture itself using numeric sequences and relations is not wrong.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*NXqvFD2UZqtEbvFU8xS_GA.png" /><figcaption>A common e-commerce data architecture. ‘id’ is a generated, auto-inrementing sequence</figcaption></figure><p>So what to do? Luckily there is a good solution that doesn’t require data re-architecture or moving away from auto-incrementing sequences.</p><h4>The Solution: UUIDs to the Rescue</h4><p>Changing active data is hard, but adding a new column to a table in a relational database should be doable without downtime even for a large table under load. Of course you should check the documentation for your particular DB and version to add and populate the column in such a way as to minimize production impact.</p><p>Before change, a excerpt of data from our example data architecture could be as follows. There will naturally be many other columns in these tables (they have been omitted for diagram readability).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*q4xc4OL5c2RtjENawhxL5g.png" /><figcaption>A snippet of relational data. ‘id’ always increments per row</figcaption></figure><p>The column we want to add (to any table that will be addressed via the client side) is a UUID.</p><p><a href="https://en.wikipedia.org/wiki/Universally_unique_identifier">A UUID is a universally unique alphanumeric identifier</a> that says nothing about the underlying data. While any long unguessable alphanumeric string will work, UUIDs are a great choice as their generation is natively supported in most modern programming languages <em>and even in databases themselves.</em> This makes populating the column very easy and architecturally flexible as it can be done in code, or right in the database automatically on new inserts, depending on your needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/681/1*gydmE18HLP-6XTpl5qqJaw.png" /><figcaption>Our same relational data but now we have UUID on Order and User</figcaption></figure><p>This example data illustrates that a UUID is not needed on every table, only those that are referenced from the client side (we are assuming here that OrderItems are shown in context with an order and not individually). It is completely fine (and more efficient) for the back end to continue to use the numeric primary/foreign key relationship to get order item information for an order, or look up details for an order item directly, as those ids are never shown to the user.</p><h4>Update your Client Side Driven Lookups</h4><p>Once you have UUIDs on the relevant tables (don’t forget to add appropriate indexes to these tables), it is a matter of combing through your front end for anywhere an ‘id’ is exposed and updating it to UUID, and updating the back-end lookup code appropriately to expect and use UUID for database lookups.</p><p>Here is how our two order history URLs look now, giving away nothing:</p><pre>https://www.yourfavsite.com/account/orders?orderuuid=f137804d-77d6-4f03-b808-834833aa96f2</pre><pre>https://www.yourfavsite.com/account/orders?orderuuid=3c3734de-3c38-4744-95ea-de64e52e37bd</pre><p><strong>Note:</strong> it is often necessary to have the back end continue to support old ‘id’ based URL patterns as well, as we’ll see in the next section: links in the wild.</p><h4>Remembering Links in the Wild — User Reports and Emails</h4><p>It is tempting to remove back-end support completely for ‘id’ coming in as a parameter. Before you do however, consider if there are any links in the wild that point back to these locations. Things like generated user reports and sent transactional emails that point back to order history are prime candidates; the links are persisted and if you remove support for the old ‘id’ pattern, they will break. There is no business risk to allowing the back end to continue to support the old patterns where needed.</p><h4>Dealing With Your API</h4><p>Uh oh! You have a public facing API that exposes a sensitive database id in its request or response data. This is a bit more of a challenge as you can’t just remove it and replace it like you did on the front end without risk of breaking your API integrators’ build-ins. The best strategy here is to work with the appropriate internal stakeholders as quickly as possible to come up with a deprecation plan. It will generally involve most or all of these steps; the details and timelines are very business dependant depending on the adoption and importance of the given API.</p><ol><li>Add ‘UUID’ alongside every place the ‘id’ is exposed in the request / response with a minor version release. Adding things to an API should be a non breaking change.</li><li>Communicate with your API users about the deprecation of ‘id’ and the timelines for the breaking change. Encourage them to start using ‘UUID’.</li><li>Add as much logging as you can to determine the use of the offending endpoints (especially if ‘id’ can be a request parameter)</li><li>Remove ‘id’ through an appropriate breaking change update (such as a major version update)</li><li>(Finally) end of life the version containing ‘id’ as per your deprecation plan communications.</li></ol><h4>Final Thoughts</h4><p>It is worth the time it takes to look through your online offering for this pattern. It doesn’t just apply to URLs; it applies to the entire client side (hidden form data, cookie values, APIs as discussed, etc.) — URLs are just by far the most visible place. And while we used e-commerce data as an example, it applies to any database object for which count or velocity is business sensitive.</p><p>Don’t be lulled into a false sense of security if your sequences are not auto-incrementing sequentially or if your data is partitioned. While the risk is lessened, as long as there is <em>any predictable pattern </em>in your exposed ids, some information is being leaked and the pattern can be interpreted.</p><p>If you find some, don’t panic; assess the business risk internally, determine the size and scope of the change, and prioritize. Happy building!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17f15669fd2e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lightrail/prevent-business-intelligence-leaks-by-using-uuids-instead-of-database-ids-on-urls-and-in-apis-17f15669fd2e">Prevent Business Intelligence Leaks by Using UUIDs Instead of Database IDs on URLs and in APIs</a> was originally published in <a href="https://medium.com/lightrail">Lightrail</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Activity Aware IDS for AWS: The simple way to stay aware of activity in your AWS Account]]></title>
            <link>https://medium.com/lightrail/activity-aware-ids-for-aws-the-simple-way-to-stay-aware-of-activity-in-your-aws-account-2c1c6f2335fc?source=rss----6fc22cf3d350---4</link>
            <guid isPermaLink="false">https://medium.com/p/2c1c6f2335fc</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[intrusion-detection]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Jamie Starke]]></dc:creator>
            <pubDate>Tue, 25 Apr 2017 18:57:32 GMT</pubDate>
            <atom:updated>2017-04-25T18:57:31.826Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KSBO97sM5CAxDjpSRghyag.jpeg" /></figure><p>We’re open sourcing the internal tool we use to notify our team of suspicious activity in our AWS Account. Given our least privilege approach to IAM policies, <a href="https://github.com/Giftbit/activity-aware-ids-aws">Activity Aware IDS for AWS</a> provides an integral part of our configuration debugging and intrusion detection (IDS).</p><p>If you’re already sold, feel free to check out the <a href="https://github.com/Giftbit/activity-aware-ids-aws">Repository</a>, or read on to find out more about the problems it solves, how it works, and how it can help you maintain the security of your AWS account.</p><p>Activity Aware IDS helps you be more aware of activity in your AWS account, including those that might suggest potential account compromises. In this article, we will discuss the common use cases for Activity Aware IDS for AWS, an overview of its architecture and how you can start using it today. Before we get to that, it’s important to understand the security threats you face as an AWS customer, your responsibility in protecting against them, and overview of the principle of least privilege as a best practice in thinking about security and access control.</p><h3>The Importance of the Principle of Least Privilege</h3><p>A decent understanding of the principle of least privilege is vital when talking about security and access controls. In this section, we will briefly describe the principle of least privilege and work through an example of working towards this principle.</p><p>Activity Aware IDS is most effective when following the principle of least privilege, that is, that in any system, any identity (users, programs, systems, etc.) granted privileges to access resources or information, should be granted only the minimum privileges necessary to perform their tasks. In the Activity Aware IDS default configuration, it will inform your team when users and roles are attempting to use actions or access resources beyond their privileges. It is possible to configure Activity Aware IDS to notify you of AWS API actions, even when these actions are permitted, but this requires knowledge of the specific actions for which your team wants to be informed.</p><p>As an example of following the principle of least privilege, let us look at a recent scenario we faced. We wanted to back up the logs from our instances into an S3 bucket so that if anything happened to one of our instances, we could always get access to its recent logs for investigating the root cause.</p><p>To allow instances to upload their logs, we could have used the Identity and Access Management (IAM) system to attach the AmazonS3FullAccess Managed policy to the roles associated with the EC2 instances. With those permissions, they could easily have uploaded their logs, but they would also be able to do other things, like Reading or Deleting the log files. In the case that this instance had these extra privileges, and for some reason, it attempted to delete all the log file, it would be successful. Under the default configuration, we would not be informed of this because the instance only performed actions it was granted. Instead, we could give them an inline policy like the following:</p><pre>{ &quot;Version&quot;: &quot;2012-10-17&quot;, &quot;Statement&quot;: [ { &quot;Effect&quot;: &quot;Allow&quot;, &quot;Action&quot;: &quot;s3:PutObject&quot;, &quot;Resource&quot;: &quot;*&quot; } ] }</pre><p>This would restrict the instances to only be able to upload to an S3 bucket, but not allow them to Read or Delete from the buckets. This is still not least privilege because, although an instance can only upload files, it can upload them to <strong>any</strong> bucket in your account. Instead, we could restrict the privilege to a specific bucket:</p><pre>{ &quot;Version&quot;: &quot;2012-10-17&quot;, &quot;Statement&quot;: [ { &quot;Effect&quot;: &quot;Allow&quot;, &quot;Action&quot;: &quot;s3:PutObject&quot;, &quot;Resource&quot;: &quot;arn:aws:s3:::mybucket/*&quot; } ] }</pre><p>So now we’ve restricted our instances to being able to upload only to a specific bucket, we must be good right? Well, we wanted to find the logs for a specific instance, so we wanted them to put their logs into a unique prefix for each instance. For this, we used a prefix that was a combination of the unique id for the role associated with the primary role of the instance, combined with the instance id, like role-id:instance-id. This gives us something special when it comes IAM policies. We can make the above policy even more restrictive:</p><pre>{ &quot;Version&quot;: &quot;2012-10-17&quot;, &quot;Statement&quot;: [ { &quot;Effect&quot;: &quot;Allow&quot;, &quot;Action&quot;: &quot;s3:PutObject&quot;, &quot;Resource&quot;: &quot;arn:aws:s3:::mybucket/${aws:user_id}/*&quot; } ] }</pre><p>In the policy above, ${aws:user_id} is what is known as an <a href="http://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_variables.html#policy-vars-infotouse">IAM policy variable</a>. In this case, this specific variable is equivalent to the unique identifier for which ever role, user, or other identity, is making a particular request. So in this example, it would be coming from an EC2 instance. If you consult the table from the <a href="http://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_variables.html#policy-vars-infotouse">IAM policy variables</a> page, you will find that for an EC2 instance, this resolves to role-id:ec2-instance-id, which is what we used for the prefix above. Thus, our instance is only able to upload its log files into a specific bucket, with the specific prefix for its instance. Under the default configuration of Activity Aware IDS, if this instance attempted to perform any other action on this or any other bucket, or upload its logs to a different prefix, a notification would be sent to the company Slack channel, thus informing the team of the aberration.</p><p>So now we must be good right? Well, that’s the hard thing about the principle of least privilege. If we happened to know that the instances only upload their logs at a specific time, we could reduce it’s privilege even further. The purpose of the principle of least privilege is to reduce the attack surface, that is, the different contexts that a malicious user could attempt to cause damage to your systems. This damage could be caused by creating bad data, getting a hold of your good data, using your resources like your EC2 instances, or creating EC2 instances that get charged to your account. But since we want to backup our logs frequently throughout the day, for all reasonable intents and purposes, we will call this example done.</p><p>Setting up proper IAM permissions like above isn’t just good practices, it’s one part of what AWS customers are responsible in the Shared Responsibility Model. Let us explore this a little further.</p><p>As a customer in any cloud provider, it’s important to understand your responsibilities. In this section, we’ll cover AWS’ Shared Responsibility Model, and overview your responsibilities as an AWS Customer.</p><p>When securing your resources, the cornerstone of security in the AWS Cloud is the <a href="https://www.youtube.com/watch?v=RwUSPklR24M">Shared Responsibility Model</a>. At a high level, this is a delineation between what AWS takes responsibility to secure, and what you as an AWS Customer are responsible for securing. Most simply, AWS takes care of security OF the cloud, and you are responsible for security IN the cloud.</p><p>For the AWS Side of this, they take responsibility for securing the building blocks used to compose your systems. These include the Compute (EC2 hosts), Storage (S3 infrastructure), Databases (RDS Hosts), Networking infrastructure, and so on. That’s not to say that you can just throw all of your customers’ credit card data into an S3 bucket, and your security responsibility is complete.</p><p>As an AWS Customer, you are responsible for maintaining the security of the operating systems of your EC2 instances, your application, network and firewall configurations, identity and access management, encryption, and customer data. A big part here is that AWS provides the building blocks, like the Storage (S3), and Encryption (KMS) to protect the security of this data, but it’s your responsibility to use those services properly, and rotate your access keys regularly. If we stored our data with too broad of permissions, and some attacker gained access to our customer data and leaked it, we, as the AWS customer, are responsible for that.</p><p>If this doesn’t sound obvious, or you just need to review, I recommend you check out <a href="https://www.youtube.com/watch?v=RwUSPklR24M">The AWS Shared Security Responsibility Model in Practice</a> video as a starting point.</p><p>Now that we have a better understanding of what we as AWS Customers are responsible for security-wise, let us go on to an overview of the potential security threats you need to consider when operating in AWS.</p><h3>Security Threats in the AWS Cloud</h3><p>When trying to secure your resources in the AWS Cloud, it’s valuable to understand what kind of security threats you’re likely to face. In this section, we will briefly cover the different types of security incidents you need to consider.</p><p>Although there are a number of different potential security threats you need to be aware of in the AWS Cloud, they have been nicely broken down into the <a href="http://www.slideshare.net/AmazonWebServices/you-cant-protect-what-you-cant-see-aws-security-monitoring-compliance-validation-from-adobe-sac309#9">3 major types of security incidents in AWS</a> by Adobe. These are <strong><em>Infrastructure Impact</em></strong><em>, </em><strong><em>Host Compromise</em></strong><em>, and </em><strong><em>Account Compromise</em></strong>.</p><p><strong><em>Infrastructure Impact</em></strong> includes external attacks on the underlying infrastructure of your application. This type of attack largely consists of Distributed Denial of Service (DDoS) attacks, where an attacker sends a large volume of traffic at your site. The main objective of this type of attack is to occupy your resources with garbage traffic, so that legitimate traffic can not get through, thus taking your site offline. DDoS Protection tools, like the new <a href="https://aws.amazon.com/shield/">AWS Shield</a>, can help to alleviate these types of threats, by identifying common patterns in the DDoS traffic, and blocking it before it gets to your servers.</p><p><strong><em>Host Compromise</em></strong> involves using a technique like command injection to gain access to your existing resources, such as your EC2 instances. When an attacker gains access to one of your instances, for example, they are usually targeting one of two things: Using the instance for its compute resources, such as BitCoin Mining; or Using the instance to gain access to its data or approach another instance that is likely to have valuable data, called an Advanced Persistent Threat (APT). In the case that the attacker can’t find valuable data on the instance they gain access to, they will sometimes use that instance to look for other potentially vulnerable instances on the same network (or VPC). Host-based intrusion detection and intrusion prevention are the most common methods of alleviating this type of threat. These protections regularly involve placing some software or agent on your host, which scans incoming and outgoing traffic to look for common attack patterns, and informing you and/or stopping this traffic automatically.</p><p><strong><em>Account Compromise</em></strong> involves an attacker gaining access to some identity (either users or roles on an instance), and then using that identity for the data it has access to or the resources it can create. As in the Host Compromise case above, if the attacker is interested mining bitcoins, they might spin up a number of large (potentially 4xLarge or 8xLarge) instances, using them to mine bitcoins, then leave you with the bill for the instances. Alternatively, if they’re interested in the data, existing safeguards might not allow them to export a snapshot of your database directly, for example, but if they can create an instance that has access to the database, they can export it from there.</p><p>When it comes to protecting against Infrastructure Impact and Host Compromise, there seems to be no shortage of tools to help with these threats. When it comes to Account Compromise, the solution space seems to be a lot more open. This is where Activity Aware IDS for AWS fits in. When an attacker is attempting to determine the permissions of a compromised identity, they will likely be denied access to a number of the actions and resources they attempt to use while probing the identity. Activity Aware IDS notifies you of these denials where your team will notice them most, like Slack.</p><p>Now that we have a better idea of the common types of security incidents that might occur in the AWS Cloud let us look at the use cases for Activity Aware IDS for AWS.</p><h3>Activity Aware IDS for AWS Use Cases</h3><h3>Intrusion Detection</h3><p>As we covered briefly in the last section, Activity Aware IDS for AWS can help you discover Account Compromises. This works by using CloudTrail, and the principle of least privilege. CloudTrail is essentially a log of all of the activity in your account, and the principle of least privilege, as we covered above, means giving any identity (users and roles) only access to the permissions it needs to fulfill its tasks.</p><p>Now let us imagine that a member of your team, Alice, had her credentials compromised by a malicious attacker Mallory. Mallory might want to see what Groups and Roles they now have access to through Alice’s account. For this example, let’s assume that Alice doesn’t have access to view the Groups she is associated with, or at the very least, doesn’t have access to view the policies attached to those groups. If Mallory attempts to view the policies of those groups, she will get denied access, which gets logged to CloudTrail. At this point, Activity Aware IDS receives the denial log, converts it into a friendly format, and sends it to a Slack Channel monitored by your team. Once the message arrives at Slack, your team will see that there are strange “Access Denied” messages associated with Alice, speak with her about the denial, to find out that she is not performing the actions, and replace her credentials.</p><h3>Principle of Least Privilege Debugging</h3><p>In the example above, the attacker Mallory was blocked from performing actions due to the principle of least privilege. Although the principle of least privilege is a common recommendation regarding security, it can be difficult in finding the exact set of permissions that a User or Role should have. Activity Aware IDS can also assist with this.</p><p>Imagine that you are deploying a new service, and you want to give it permissions following the Principle of Least Privilege. The easiest place to start with the principle of least privilege is to create a role with no permissions. Now lets assume that the system needs to send logs to CloudWatch Logs (this is a common requirement for AWS Lambda). When the system attempts to create a new Log Group, for example, it will get denied, because the role has no permissions. This denial will get logged to CloudTrail, then Activity Aware IDS for AWS will send the denial to your Slack, where you can see the specific action being attempted, the Role attempting to perform the action, and even the arn of the specific resource it’s trying to perform the action on. At this point, you can add a statement to the policy for the role allowing it access to “CreateLogGroup” and even provide a fairly restrictive resource description.</p><h3>Architectural Overview</h3><p>There is usually value in understanding, at least at a high level how systems you use work. In this section, we’ll briefly overview the architecture of Activity Aware IDS, its major components, and how they work together.</p><p>The architecture of Activity Aware IDS for AWS is fairly simple. It’s composed mainly of 3 pieces: <strong>Source Lambdas</strong>, <strong>Destination Lambdas</strong>, and an <strong>SNS Topic</strong> for message passing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/0*EB5P08N9pfvZYp3T." /></figure><p>Above is an architectural overview of Activity Aware IDS for AWS.</p><p>On the left side, we have <strong>Sources</strong>. These are the services that generate the events of which you want to be aware. Each of these services is coupled with a specific lambda function that knows the structure of events for that service, converts them into a common message format, and sends them to the SNS Topic. In the initial release, we have a CloudTrail source, but we’ve designed the system to be extensible with additional sources, such as VPC Flow logs.</p><p>On the right side, we have <strong>Destinations</strong>. These are the services or methods of communication by which you want to be informed. Each of these destination services is coupled with a specific lambda function that is subscribed to the SNS Topic, knows how to interpret the common message format, and send a message to the given destination service. In the initial release, we have a Slack destination, but like with Sources, the system is designed to be extensible, and support additional sources, such as SES for email, PagerDuty, and more.</p><p>The <strong>SNS Topic</strong> is a simple, but central messaging hub in Activity Aware IDS. It receives messages from the source lambdas and sends them out to the subscribed destination lambdas.</p><p>So there you have it, it’s fairly simple, but with its extensibility, it can also be quite powerful.</p><h3>Getting Started</h3><p>If you’ve made it this far, you’re probably still interested, so why don’t you give it a shot. Check out our <a href="https://github.com/Giftbit/activity-aware-ids-aws#getting-started">Getting Started guide</a> for the requirements you’ll need, and how to install Activity Aware IDS for AWS in your AWS account.</p><h3>How you can help</h3><p>Do you have a source you’d like to get notified about in your Slack? Maybe you’d love to have your CloudTrail events go to a different destination, like Email? Or maybe you found a bug in the system? We love getting all of this feedback. Be sure to <a href="https://github.com/Giftbit/activity-aware-ids-aws/issues/new">add an issue</a> for your Source and Destination requests, as well as any bugs you’ve located in our <a href="https://github.com/Giftbit/activity-aware-ids-aws/issues/new">Issues tracker</a>.</p><p>Feel like getting your hands dirty? If you want to contribute to the project, we’d be happy to have it. <a href="https://github.com/Giftbit/activity-aware-ids-aws/issues/new">Create an issue</a> for it first, so that we might give you pointers, and so that others know if a feature they’re excited about is being worked on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c1c6f2335fc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lightrail/activity-aware-ids-for-aws-the-simple-way-to-stay-aware-of-activity-in-your-aws-account-2c1c6f2335fc">Activity Aware IDS for AWS: The simple way to stay aware of activity in your AWS Account</a> was originally published in <a href="https://medium.com/lightrail">Lightrail</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Securely Manage System Configuration Using AWS]]></title>
            <link>https://medium.com/lightrail/how-to-securely-manage-system-configuration-using-aws-9261cb332f8f?source=rss----6fc22cf3d350---4</link>
            <guid isPermaLink="false">https://medium.com/p/9261cb332f8f</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[encrypted-system-config]]></category>
            <category><![CDATA[application-config-aws]]></category>
            <category><![CDATA[amazon-s3]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Tim Jordison]]></dc:creator>
            <pubDate>Tue, 25 Apr 2017 18:54:46 GMT</pubDate>
            <atom:updated>2017-03-22T16:50:14.788Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HZK7Pqgjzr6S7Qz-frqc-g.jpeg" /></figure><p>Safely managing highly sensitive configuration is a problem that must be solved for nearly every non-trivial application.</p><p>This article describes the architecture we implemented at Giftbit to store and access sensitive system configuration. Our approach ensures that the configuration is stored in an encrypted format while providing the application a secure method to access it.</p><p>Configurations such as third party API keys, database passwords and other sensitive system parameters are secrets that, if leaked, can have catastrophic repercussions. Although storing these secrets plays a critical role in application security, most implementations still fall short by failing to adhere to the <a href="https://en.wikipedia.org/wiki/Principle_of_least_privilege">principle of least-privilege</a>. The principle of least-privilege states that a user or system has permission to access only those resources it legitimately needs to perform its function. This means that such systems need very fine-grained access control measures.</p><p>The problem is challenging, as often configuration secrets are needed very early in the application lifecycle, and they themselves should protected by an encryption key. This creates a cyclical chicken and egg type problem of how to securely supply the secret to the application that protects the other secrets. The approach outlined is for an application running on AWS and is achieved by leveraging Amazon’s <a href="https://aws.amazon.com/s3/">Simple Storage Service (S3)</a>, <a href="https://aws.amazon.com/kms">Key Management Service (KMS)</a>, and <a href="https://aws.amazon.com/iam/">Identity Access Management (IAM)</a> services.</p><h3>Considerations</h3><p>When deciding what approach to use for storing sensitive system configuration, there are several important considerations:</p><ul><li>How is read and write access to the configuration files controlled?</li><li>How is the configuration updated?</li><li>How does the architecture function across different environments?</li></ul><p>The approach described in this article will aim to answer these questions while adhering to IAM security best practices.</p><h3>Architecture</h3><p>The architecture at a high level is to store the configuration in an S3 bucket encrypted under a KMS key. The application, running Amazon’s Elastic Cloud Compute (EC2) or AWS Lambda, will read the configuration from S3 on start-up. The application is assigned an IAM role which has a policy that grants read-only access to the bucket, and permission to decrypt using the KMS key. Lastly, a policy is placed on the bucket which restricts access to only specific allowed IAM roles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/566/0*rZsJXiyKjcEMU5SH.png" /></figure><h3>How KMS and IAM Solve the Chicken and Egg Problem</h3><p>The configuration files need to be stored in an encrypted format and decrypted by the application. The primary challenge here is discerning how to securely pass the decryption key to the application. Attempting to protect this decryption key creates a cyclical problem; however, this issue is avoided utilizing KMS and IAM.</p><p>KMS is used to encrypt and decrypt the configuration files. When an application is running on EC2, it has an instance profile, which is used to associate an IAM role with the application. The instance profile causes AWS actions that are taken from the application’s code (such as via the AWS Java SDK) to assume the IAM role’s privileges. This role can be setup to have access to the KMS key which means when the application requests the configuration from S3, it can automatically decrypt the contents. Utilizing the instance profile and IAM permissions solves the awkward problem of having to securely supply a decryption key to the application.</p><h3>IAM Setup</h3><p>To obey the principle of least-privilege, there are three roles that must be created in IAM: A role which will be associated with the application’s EC2 instance; a role which will be used by developers to update the configuration; and a role which will be used by developers to read from the configuration. It is important that these three roles exist since neither a developer nor an application should have access to resources or actions outside of those they require to perform their task.</p><p>The need for the first two roles is self-evident, but the third is less obvious and is to be used under the very rare circumstances where a developer may need to read the configuration. It might be tempting to use the root AWS user to perform such a duty, but to adhere to <a href="http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html">IAM best practices</a> this should be avoided. Using the root user for this purpose goes against the principle of least-privilege.</p><h3>KMS Setup</h3><p>Creating a KMS encryption key can be done through the IAM Management Console. It is possible to create the key with an external key material origin; however, this should only be done if required for the implementation. The key administrative permissions must be locked down to IAM users who will perform such actions and not granted to the EC2 instance role. The key usage permissions is where the three roles described in the IAM Setup section must be assigned. Below is a screenshot of the first step of creating a KMS key.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Wp--lIrNCmsQsbtu.png" /></figure><h3>S3 Setup</h3><p>Creating the S3 bucket can be done through the S3 Management Console. Once the bucket is created, a bucket policy must be added to limit access to the bucket. This is done via the bucket properties permissions section. The following is an example of working bucket policy for this architecture.</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;Statement 1: Ensure config is encrypted on upload&quot;<br>            &quot;Effect&quot;: &quot;Deny&quot;,<br>            &quot;Principal&quot;: &quot;*&quot;,<br>            &quot;Action&quot;: &quot;s3:PutObject&quot;,<br>            &quot;Resource&quot;: &quot;arn:aws:s3:::CONFIG_BUCKET_NAME/*&quot;,<br>            &quot;Condition&quot;: {<br>                &quot;StringNotLike&quot;: {<br>                    &quot;s3:x-amz-server-side-encryption-aws-kms-key-id&quot;: &quot;KMS_KEY_RESOURCE_ARN&quot;<br>                }<br>            }<br>        },<br>        {<br>            &quot;Sid&quot;: &quot;Statement 2: Restrict role which writes configuration to only write actions.&quot;,<br>            &quot;Effect&quot;: &quot;Deny&quot;,<br>            &quot;Principal&quot;: &quot;*&quot;,<br>            &quot;NotAction&quot;: [<br>                &quot;s3:ListBucket&quot;,<br>                &quot;s3:PutObject&quot;,<br>                &quot;s3:PutObjectAcl&quot;,<br>                &quot;s3:PutObjectVersionAcl&quot;,<br>                &quot;s3:GetBucketLocation&quot;,<br>                &quot;s3:GetBucketVersioning&quot;<br>            ],<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:s3:::CONFIG_BUCKET_NAME&quot;,<br>                &quot;arn:aws:s3:::CONFIG_BUCKET_NAME/*&quot;<br>            ],<br>            &quot;Condition&quot;: {<br>                &quot;StringLike&quot;: {<br>                    &quot;aws:userid&quot;: &quot;ADMIN_WRITE_IAM_ROLE_ID:*&quot;<br>                }<br>            }<br>        },<br>        {<br>            &quot;Sid&quot;: &quot;Statement 3: Restrict roles which reads configuration to only read actions.&quot;,<br>            &quot;Effect&quot;: &quot;Deny&quot;,<br>            &quot;Principal&quot;: &quot;*&quot;,<br>            &quot;NotAction&quot;: [<br>                &quot;s3:ListBucket&quot;,<br>                &quot;s3:GetObject&quot;,<br>                &quot;s3:GetObjectAcl&quot;,<br>                &quot;s3:GetObjectVersionAcl&quot;,<br>                &quot;s3:GetBucketLocation&quot;<br>            ],<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:s3:::CONFIG_BUCKET_NAME&quot;,<br>                &quot;arn:aws:s3:::CONFIG_BUCKET_NAME/*&quot;<br>            ],<br>            &quot;Condition&quot;: {<br>                &quot;StringLike&quot;: {<br>                    &quot;aws:userid&quot;: [<br>                        &quot;APPLICATION_IAM_ROLE_ID:*&quot;,<br>                        &quot;ADMIN_READ_IAM_ROLE_ID:*&quot;,<br>                    ]<br>                }<br>            }<br>        },<br>        {<br>            &quot;Sid&quot;: &quot;Statement 4: Deny anyone else&quot;,<br>            &quot;Effect&quot;: &quot;Deny&quot;,<br>            &quot;Principal&quot;: &quot;*&quot;,<br>            &quot;NotAction&quot;: [<br>                &quot;s3:GetBucketLocation&quot;,<br>                &quot;s3:GetBucketTagging&quot;<br>            ],<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:s3:::CONFIG_BUCKET_NAME&quot;,<br>                &quot;arn:aws:s3:::CONFIG_BUCKET_NAME/*&quot;<br>            ],<br>            &quot;Condition&quot;: {<br>                &quot;StringNotLike&quot;: {<br>                    &quot;aws:userid&quot;: [<br>                        &quot;APPLICATION_IAM_ROLE_ID:*&quot;,<br>                        &quot;ADMIN_WRITE_IAM_ROLE_ID:*&quot;,<br>                        &quot;ADMIN_READ_IAM_ROLE_ID:*&quot;,<br>                        &quot;ROOT_ACCOUNT_ID&quot;<br>                    ]<br>                }<br>            }<br>        }<br>    ]<br>}</pre><p>The variables that must be replaced for this policy to work are the following:</p><ul><li>CONFIG_BUCKET_NAME</li><li>KMS_KEY_RESOURCE_ARN</li><li>ADMIN_WRITE_IAM_ROLE_ID</li><li>ADMIN_READ_IAM_ROLE_ID</li><li>APPLICATION_IAM_ROLE_ID</li><li>ROOT_ACCOUNT_ID</li></ul><p>The ids of the IAM roles can be retrieved using the AWS Command Line Interface via the following commands.</p><pre>aws iam get-role --role-name TheNameOfTheRole</pre><h3>Dissecting the S3 Bucket Policy</h3><p>The policy contains four statements.</p><ul><li>The first statement ensures that any files uploaded to the bucket are server-side-encrypted using the correct KMS key. This is really useful since it will ensure security standards are maintained.</li><li>The second statement limits the role which will write to the bucket to only actions required for write operations.</li><li>The third limits the roles which will read from the bucket to only actions required for read operations.</li><li>The fourth and final statement restricts any other user or role from accessing the bucket. The reason &quot;NotAction&quot;:&quot;s3:GetBucketLocation&quot; is used rather than &quot;Action&quot;:&quot;s3:*&quot;, is that whenever a developer visits S3 in their browser, S3 in turn calls &quot;S3:GetBucketLocation&quot; on all buckets; this action results in an Access Denied message in CloudTrail, which can get noisy if the AWS account is configured to notify Access Denied and Authorization Failures for security purposes. See our <a href="http://blog.giftbit.com/activity-aware-ids-for-aws">recent article</a> for a simple way to stay informed of activity in your AWS account.</li></ul><p>A bonus of using S3 to store configuration files is that audit logging can trivially be enabled through the S3 bucket properties. Audit logging makes it easy to tell when files were accessed or updated and which user performed such actions, which assists greatly in a thorough Intrusion Detection System approach.</p><h3>Fetching Configuration Files from the Application</h3><p>The name of the S3 bucket should be stored in the non-sensitive application configuration. It is a good idea to fetch the credential files during application start-up to avoid incurring the cost of retrieving them from S3 every time they need to be accessed. Another motivation for fetching during start-up is as with any sensitive information, it is always preferable to minimize the number of times it is sent over any part of the network.</p><p>A straightforward approach is to store the files in a JSON format which makes them easy to parse. For example, the file stored in S3 could be called example_config.json with the following contents.</p><pre>{<br>            &quot;secretKey&quot;:&quot;abcdef-123456&quot;<br>        }</pre><p>Within the application code, an object will exist which will be populated on load. To carry on with the example the corresponding object might look like this:</p><pre>@Singleton<br>        class ExampleConfig {<br>            static String FILE_NAME_IN_S3 = “example_config.json”<br>            String secretKey<br>        }</pre><p>The following Groovy snippet is an example of how an application config object could be populated using AWS’s S3 Java SDK.</p><pre>ExampleConfig populateExampleConfig(String s3Bucket) {<br>            ExampleConfig exampleConfig = ExampleConfig.getInstance()<br>            AmazonS3Client client = new AmazonS3Client()<br>            S3Object object = client.getObject(s3Bucket, exampleConfig.FILE_NAME_IN_S3)<br>            def json = new JsonSlurper().parse(object.getObjectContent())<br>            exampleConfig.secretKey = json.secretKey<br>            return exampleConfig<br>        }</pre><p>The above example highlights the important fact that no AWS credentials need to be supplied to the application for it to be able to access the secure configuration in S3. When the applications calls S3 using as the AWS SDK, the applications ‘Instance profile credentials’ will be used since none were provided. Again, this is made possible by the IAM role associated with the EC2 instance. For more information on how the application receives its AWS credentials see <a href="http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html">Working with AWS Credentials</a>.</p><p>A rather nice benefit of this architecture is that it is easy to add an endpoint to update and reload the configuration on the fly, without needing to restart the application. The endpoint can simply re-populate the applications config objects using the same method that is used during application startup.</p><h3>Configuration Across Environments</h3><p>It is crucial to use separate S3 buckets and KMS keys for at least production and nonproduction environments. It is also best practice to have different AWS accounts for different environments. Configuration will likely differ between environments so separating them allows production to be completely locked down, whilst granting the development team freedom to be more lenient on the other environments. This allows developers to move quickly without being constrained with inadequate permissions.</p><p>There are also likely differences in how a developer will run the application locally compared to the application running on EC2. For example, when a developer runs the application locally, suppose the AWS credentials used belong to an IAM user for that developer. The application running on their local machine will need to fetch the configuration files from the development S3 bucket. In this case, the IAM user could be given one of the roles described earlier which would allow them to read from the bucket.</p><h3>Managing the AWS Infrastructure</h3><p>The roles required for this architecture are only given permissions to perform their specific tasks. A question arises around how to manage this infrastructure. While trying to adhere to principle of least-privilege, the first cut would be to avoid using the root AWS user to manage such infrastructure. Instead, an IAM user should be created for these purposes which has permission to manage the IAM users, the KMS keys, and the S3 bucket policies. Since this user will have elevated permissions, it is a good practice to both restrict access to a physical FOB and have policies in place to be notified of suspicious activity.</p><h3>Why This Architecture Is Well Suited for the Job</h3><p>The motivation of this architecture is to enable fine-grained access control, adhering to the principle of least-privilege, with the goal of reducing the surface area of attack vectors. The approach outlined ensures that sensitive configuration files are stored in an encrypted format, and that only the IAM roles that need to read or update files are allowed to. Moreover, this approach also provides the additional benefits of S3 logging, enabling visibility into who is accessing such resources, as well as a secure infrastructure that ensures security measures are followed and stay up-to-date. Maintaining a secure system overall is an ongoing task, and the proper handling of sensitive configuration is a core component in an ever-evolving security landscape.</p><p><em>Originally published at </em><a href="http://blog.giftbit.com/how-to-securely-manage-system-configuration-using-aws"><em>blog.giftbit.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9261cb332f8f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lightrail/how-to-securely-manage-system-configuration-using-aws-9261cb332f8f">How To Securely Manage System Configuration Using AWS</a> was originally published in <a href="https://medium.com/lightrail">Lightrail</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>