<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Axel Springer Tech on Medium]]></title>
        <description><![CDATA[Stories by Axel Springer Tech on Medium]]></description>
        <link>https://medium.com/@axelspringertech?source=rss-7d3ea0a1bc11------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*IpHZcEDKV5HCqDFpisOp6g.gif</url>
            <title>Stories by Axel Springer Tech on Medium</title>
            <link>https://medium.com/@axelspringertech?source=rss-7d3ea0a1bc11------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 04:36:19 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@axelspringertech/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Help Your QA Testing Your App]]></title>
            <link>https://medium.com/axel-springer-tech/help-your-qa-testing-your-app-6e7c64581c15?source=rss-7d3ea0a1bc11------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e7c64581c15</guid>
            <category><![CDATA[error-handling]]></category>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[upday]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Axel Springer Tech]]></dc:creator>
            <pubDate>Tue, 17 Mar 2020 11:38:20 GMT</pubDate>
            <atom:updated>2020-03-17T11:43:53.713Z</atom:updated>
            <content:encoded><![CDATA[<h3>QA Testing Your App</h3><p>By <a href="https://medium.com/@aniakarpiuk">aniakarpiuk</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*czi6pUBf2cNLXC2kj9b4qw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@maxcodes?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Max Nelson</a> on <a href="https://unsplash.com/search/photos/coding?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>When developing Android applications, it is a good practice to support slow connections and error handling. This means, there is a loading screen shown, when data is downloaded from the server and an error screen visible, when something went wrong. It might be fine when you are a developer and you are always able to delay or mock responses when implementing those screens, to see the output… but what when you are in QA or a PO who are requested to test the new developments?</p><p>QA always has its own ways to test behaviour but it sometimes can be troublesome or not all cases can be easily covered. We, as developers, can help with that with not too much additional work required. I don’t have to mention that a smile on a QA face is priceless.</p><p>This proposal depends on the tech stack you use. Our basic library for requests is <strong>OkHttp. </strong>We also use it to prepare additional features for testing our app.</p><p>The basic idea is to use custom <a href="https://square.github.io/okhttp/interceptors/">interceptors</a> to emulate request errors or delays. It doesn’t need to be anything fancy. You can implement a basic UI to setup how often the requests should fail. Then the interceptor will read this value and emulate an error accordingly. Here is a basic implementation of it.</p><p>But first prepare the UI to configure failures or delays. You can use the debug mode and show it in your app settings (if the app is not in release flavour):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*L4A_5fGjqshwwpWs" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/0*jylZEXaUGyxautF3.png" /></figure><p>Next, prepare an interceptor which will modify responses according to the new settings (you can use regular preferences to pass values between objects).</p><p>The delay intercept method could look like this:</p><pre>@Throws(IOException::<strong>class</strong>)<br><strong>override fun </strong>intercept(chain: Interceptor.Chain): Response {<br>    <strong>val </strong>request = chain.request()</pre><pre>    <strong>try </strong>{<br>        Thread.sleep(preferences.debugRequestDelay)<br>    } <strong>catch </strong>(e: InterruptedException) {<br>        Log.w(&quot;ErrorInterceptor&quot;, &quot;Thread is not tired, doesn&#39;t want to go to sleep: $e&quot;)<br>    }</pre><pre>    <strong>return </strong>chain.proceed(request)<br>}</pre><p>…and Error interceptor:</p><pre>@Throws(IOException::<strong>class</strong>)<br><strong>override fun </strong>intercept(chain: Interceptor.Chain): Response {<br>    <strong>val </strong>request = chain.request()<br>    <strong>val </strong>response = chain.proceed(request)</pre><pre>    <strong>return if </strong>(shouldFail(request))<br>        createErrorResponse(request, chain.connection()!!.handshake())<br>    <strong>else<br>        </strong>response<br>}<strong>private fun </strong>shouldFail(request: Request): Boolean =<br>    preferences.debugFailureProbability &gt; randomInt<strong>private val </strong>randomInt: Int<br>    @IntRange(from = 0, to = 99)<br>    <strong>get</strong>() =<strong> </strong>Random().nextInt(100)<strong>private fun </strong>createErrorResponse(request: Request,<br>                                handshake: Handshake?): Response =<br>    Response.Builder()<br>            .request(request)<br>            .message(&quot;Fake debug error&quot;)<br>            .body(createEmptyBody())<br>            .protocol(Protocol.HTTP_2)<br>            .code(code)<br>            .handshake(handshake)<br>            .build()</pre><p>Now you are ready to add interceptors when you create the OkHttpClient:</p><pre>OkHttpClient<br>        .newBuilder()<br>        ....<br>        .addInterceptor(delayInterceptor)<br>        .addInterceptor(errorInterceptor)<br>        .build()</pre><p>And that’s it!</p><p>Kudos to <a href="https://medium.com/@tpolansk">Tomek</a>, one of our former Android devs, who introduced this feature but was too shy to share it with the world.</p><p>This article was originally published on <a href="https://medium.com/upday-devs">upday devs blog</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e7c64581c15" width="1" height="1" alt=""><hr><p><a href="https://medium.com/axel-springer-tech/help-your-qa-testing-your-app-6e7c64581c15">Help Your QA Testing Your App</a> was originally published in <a href="https://medium.com/axel-springer-tech">Axel Springer Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimizing DynamoDB WCU utilization in EMR-Hive]]></title>
            <link>https://medium.com/axel-springer-tech/optimizing-dynamodb-wcu-utilization-in-emr-hive-e0c7bf1b7eee?source=rss-7d3ea0a1bc11------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0c7bf1b7eee</guid>
            <category><![CDATA[apache]]></category>
            <category><![CDATA[dynamodb]]></category>
            <category><![CDATA[upday]]></category>
            <category><![CDATA[apache-airflow]]></category>
            <category><![CDATA[apache-hive]]></category>
            <dc:creator><![CDATA[Axel Springer Tech]]></dc:creator>
            <pubDate>Wed, 11 Mar 2020 10:15:47 GMT</pubDate>
            <atom:updated>2020-03-11T10:15:47.613Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*B8mYVE_C9_0__za6.jpeg" /></figure><p>By <a href="https://medium.com/@somasundaramsekar.1986">Somasundaram Sekar</a></p><h3>User engagement score</h3><p>Upday’s passion for providing high-quality, personalized content to the user means, continuous monitoring of <em>User-Content</em> interaction like swipes, clicks and time spent on the content itself and converting those into <em>user engagement scores, w</em>hich, can then be used for improving the <em>UX</em> by surfacing better content relevant to the user.</p><p>For us the engineering team this means generating <strong>“~ 3 Billion”</strong> engagement scores every day and stashing them into a highly scalable, highly available, low latency storage. So DynamoDB it was.</p><h3>The Data pipeline</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/0*7OUQH3Wo1tArNNG6.png" /><figcaption>Engagement score Airflow DAG</figcaption></figure><p>Our data seamlessly flows through various parts of our systems co-ordinated by <a href="https://airflow.apache.org/">Apache Airflow</a> based data pipeline.</p><p>The User engagement score pipeline generates the scores and also write them to the DynamoDB table.</p><p>The <em>Read</em> workload on the table is predictable, as you can see from the (<em>ConsumedReadCapacityUnits/second)</em> Cloudwatch graph. Hence we have a Provisioned Read Capacity Units that also serves to cover the occasional bursts in the reads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IsyqEx411tG_jlRQ.png" /><figcaption>Predictable Read Capacity Units consumption</figcaption></figure><p>The <em>Write</em>, though, is batched and happens only once a day and we needed to have predictability on both cost and the performance. The pipeline writes to the DynamoDB table from an EMR cluster running Apache Hive application using <a href="https://github.com/awslabs/emr-dynamodb-connector">emr-dynamodb-connector</a>.</p><p>To have better control over the write throughput we bump <em>Provisioned Write Capacity Units(WCU)</em> of the table to <strong><em>30000</em></strong> before we run the Hive query and upon completion, we change it back to “<strong><em>1 WCU</em></strong>”.</p><h3>The Problem</h3><p><em>Alles gut </em>until<em> </em>we started noticing that the provisioned capacity was underutilized at times and the job taking 2x more time to complete.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PAm9N0keICnpVxYk.png" /><figcaption>Underutilized Write capacity units</figcaption></figure><p>Upon analyzing the Hive logs, it was evident that the issue was due to idle mappers caused by Tez’s “<em>split grouping”</em> which grouped the files into fewer <em>Splits</em>, affecting the number of records being written parallelly into table and consequently the consumed WCU.</p><h3>Tez and the split count</h3><p><em>With Amazon EMR release 5.x , Tez is the default execution engine for Hive instead of MapReduce. Tez provides improved performance for most workflows.</em></p><p>Tez, among other parameters, uses <em>tez.grouping.max-size &amp; tez.grouping.min-size</em> <a href="https://cwiki.apache.org/confluence/display/TEZ/How+initial+task+parallelism+works">to calculate the split and inturn the parallelism</a>.</p><p>This was the root cause of underutilization of WCU in our case, the gzipped CSV files were grouped into splits fewer than the number of map slots available, thus affecting the parallelism.</p><h3>Solution</h3><p>The solution was to control the parallelism(a.k.a Mappers in this case) and to set the split count to be much higher than the available mappers for proper WCU utilization.</p><p>We noticed that the utilization was maximum when the number of mappers was around the 30s’, so we set the number of mappers to 30 and the <em>tez.grouping.split-count </em>to<em> </em>the number of files available in the S3 location, which is always way higher than 30, at least 10 times more.</p><pre># Hive configuration properties <br>SET tez.grouping.split-count = [file count];<br>SET mapreduce.job.maps = 30;</pre><h3>Summary</h3><p>With the above configurations in place, the utilization of WCU improved significantly and remained consistent, as evident in the Cloudwatch graph below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lEG_XlqzmLMnXYCO.png" /><figcaption>Better utilization of WCU</figcaption></figure><p>Though Tez’s default grouping behavior optimizes the number of map tasks for better utilization of resources, it turned counterproductive <em>in this particular case</em> because of the need for higher throughput while writing to the DynamoDB and availability of the configuration properties to tune this behavior came very handy.</p><p><strong>This story was originally published on the </strong><a href="https://medium.com/upday-devs/optimizing-dynamodb-wcu-utilization-in-emr-hive-c29d89760069"><strong>upday devs blog</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0c7bf1b7eee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/axel-springer-tech/optimizing-dynamodb-wcu-utilization-in-emr-hive-e0c7bf1b7eee">Optimizing DynamoDB WCU utilization in EMR-Hive</a> was originally published in <a href="https://medium.com/axel-springer-tech">Axel Springer Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Pass the upday Coding Test]]></title>
            <link>https://medium.com/axel-springer-tech/how-to-pass-the-upday-coding-test-dc353150c41c?source=rss-7d3ea0a1bc11------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc353150c41c</guid>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[upday]]></category>
            <category><![CDATA[interview-tips]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Axel Springer Tech]]></dc:creator>
            <pubDate>Fri, 06 Mar 2020 08:59:40 GMT</pubDate>
            <atom:updated>2020-03-06T09:00:10.131Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fvfJhnELV4GTYkH3PZJHkg.png" /></figure><p><strong>By </strong><a href="https://medium.com/@dankift"><strong>Daniel Kift</strong></a></p><h3>The Coding Assignment</h3><p>Like almost every other tech company, one stage of the upday hiring process involves asking candidates to write some code. Each company has their own way of assessing a candidate’s skills.</p><p>Some companies ask candidates to invert a binary tree on a whiteboard against the clock. Others involve competitive coding style exercises on platforms like <a href="https://www.hackerrank.com/">HackerRank</a>.</p><p>We prefer to ask candidates to do a task that they would typically do in their day-to-day role. In our case, this means writing a Spring Boot app with a RESTful API for creating, reading, updating, and deleting news articles.</p><p>Here are some tips for what we look for in this part of the process.</p><h3>The Important Stuff</h3><h4>Tests</h4><p>The only one way to get an instant rejection is to submit an application without any tests. We need to know that the code works before clients start using it.</p><p>The following types of test should give us some confidence:</p><ul><li><strong>Unit tests<br></strong>These tests should test individual units of code, run quickly by mocking slow collaborators (e.g. those doing IO like reading from databases, disk, making network requests and so on), and cover multiple paths and edge-cases</li><li><strong>REST tests<br></strong>These tests involve sending REST requests via something like <a href="http://rest-assured.io/">REST-assured</a> or a regular HTTP client to ensure that the individual units integrate with each other correctly and the application works as a whole.</li></ul><p>To ensure the application is well tested, we look for a few things:</p><ol><li><strong>Strong assertions<br></strong>Tests must include an assertion or mock verification. Weak assertions like isNotNull() are rarely sufficient to know that your code is behaving correctly.</li><li><strong>Multiple cases<br></strong>Tests should not just check the happy path. Consider different paths, exceptional cases, and boundary values</li><li><strong>Clarity<br></strong>Are test names clear? Following test <a href="https://dzone.com/articles/7-popular-unit-test-naming">naming conventions</a> can help.<br>Does the test use <a href="https://medium.com/@arleypadua/builder-pattern-applied-to-testing-60e009c427c6">patterns</a> to make clear what is being tested</li><li><strong>Independence</strong><br>Tests should be independent. They should not need to run in any order, and clear up any state created during their execution so that they don’t influence other tests</li><li><strong>A Green suite<br></strong>Last but not least, the tests should all be passing</li></ol><p>A specific test coverage metric (e.g. 80%) isn’t so important to us. We don’t need you to test whether Spring Boot or the Java standard libraries work, but the core business logic should be tested.</p><h4>REST</h4><p>Once an API is being used by clients in production, it becomes very difficult to change or remove it. So getting the API design right in the first case is important. The things we look for in this area are:</p><ul><li>A<strong> resource-oriented </strong>API (see <a href="https://cloud.google.com/apis/design/resources">https://cloud.google.com/apis/design/resources</a>)</li><li>The correct <strong>HTTP methods</strong> should be used. GETs for idempotent requests that can be cached. POST/PUTs for creating new resources, DELETEs for deleting resources and so on</li><li>Sensible<strong> response status codes</strong> should be used as well. 2xx for success, 4xx for user errors, 5xx for server errors</li><li>Ideally there will be some mechanism for <strong>versioning</strong> in case we need to make a backwards-incompatible change to the API</li></ul><p>Hyperlinks / HATEOAS for discoverability of other APIs would also be great but is not expected.</p><h4>Architecture / Separation of Concerns</h4><p>Once the API is defined, some business logic and data-access code will be needed. These concerns should be split into separate layers. This helps guide implementations towards following the Single Responsibility Principle. It also makes it easier to test and evolve layers independently.</p><ul><li><strong>controllers</strong> should take care of REST / HTTP concerns.</li><li><strong>services </strong>should contain the business logic of the application.</li><li><strong>repositories</strong> should handle data access.</li></ul><p>Typically requests will flow from layer to layer. Layers further down (e.g. repositories) should not depend on layers further up (e.g. controllers).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/401/0*BzePyVtwJl06y2fN.png" /><figcaption>Application Layers</figcaption></figure><p>Tools like <a href="https://github.com/jQAssistant/jqa-spring-plugin#jqassistant-spring-framework-plugin">jQ</a> can help enforce this separation (and other best practices).</p><p><strong>Packages structure</strong> could be an entire <a href="https://dzone.com/articles/package-by-layer-for-spring-projects-is-obsolete">topic</a> on its own. For a coding assignment, whether you slice your application vertically around business capabilities (preferred), or horizontally around these layers (model, view, controller) isn’t as important as whether the code has a sensible structure and is easy to navigate.</p><h4>Error Handling</h4><p>Does the application behave appropriately when errors occur. Those errors may be:</p><ul><li><strong>client errors</strong> — invalid client requests (e.g. malformed request body, missing parameters, etc.)</li><li><strong>server errors</strong> — like unreachable resources, null pointers, etc.</li></ul><p>Error handling doesn’t need to be exhaustive, but we’d like candidates to show that they have considered some error cases and know how to deal with them.</p><p>Error responses should be useful for users and developers trying to use your API. They should also not expose internal implementation details, like stack traces, that could alert malicious users of potential vulnerabilities in your code.</p><h4>Documentation</h4><p>Is a simple <strong>README</strong> provided, including information like:</p><ul><li>How to start the application</li><li>How to call the APIs. Providing this information via <a href="https://swagger.io/">Swagger</a> or <a href="https://spring.io/projects/spring-restdocs">Spring REST Docs</a> would be even better.</li><li>What features weren’t included due to time constraints, and what you might have done given more time</li></ul><h3>The Nice to Haves</h3><h4>Dependencies</h4><ul><li>Are only necessary dependencies imported?</li><li>Are transitive dependencies from spring-boot-starters preferred?</li><li>Are dependencies appropriately scoped (e.g. to ensure test dependencies aren’t packaged in the final JAR)?</li><li>Are recent versions of dependencies used (e.g. to reduce the likelihood of using libraries with vulnerabilities)?</li></ul><p><a href="https://start.spring.io/">https://start.spring.io</a> is a good place to start for setting up appropriate dependencies.</p><h4>Care</h4><p>Some care should be taken when writing and committing the code, for example:</p><ul><li>Is the code formatted?</li><li>Is there any commented out code or unfinished TODOs?</li><li>Does the code contain typos or spelling mistakes?</li><li>Does the code follow code conventions of the language used, whether it be <a href="https://www.oracle.com/technetwork/java/codeconvtoc-136057.html">Java</a> or <a href="https://kotlinlang.org/docs/reference/coding-conventions.html">Kotlin</a>?</li><li>Is a <a href="https://git-scm.com/docs/gitignore">.gitignore</a> file included?</li><li>Do methods and variables have meaningful names?</li><li>Are classes and methods small and focused, with a single responsibility?</li></ul><p>IDE plugins like <a href="https://www.sonarlint.org/">sonarlint</a> can help you spot some of these issues.</p><h4>How Easily Can We Run the App?</h4><p>The most basic requirement is that the application should start. Ideally it will <strong>start with one click or command</strong>, and an instruction manual is not needed to get it running.</p><p>Tools like <a href="https://docs.docker.com/compose/">docker-compose</a> or <a href="https://www.testcontainers.org/">Testontainers</a> can help here. An in-memory database like <a href="https://www.h2database.com/html/main.html">H2</a> is also fine.</p><h4>Data Access</h4><p>Lots of different tools and approaches are valid (e.g. JPA, JDBC, jOOQ). The chosen approach isn’t as important as whether the data model makes sense.</p><p>Similarly, the database technology used isn’t as important as the ability to describe why it is a good choice (typically in a follow up interview).</p><p>Depending on the choice made, extra points are available for thinking about things like paging and sorting, and dealing with common problems (e.g. n+1).</p><h4>Logging &amp; Metrics</h4><p>Is it possible to <strong>understand what is happening in the application at runtime</strong>, so that it can be operated successfully in production?</p><ul><li>Are exceptions logged with their stack traces to help diagnose issues?</li><li>Is there a health check API?</li><li>Are custom application <a href="https://micrometer.io/">metrics</a> sent?</li></ul><h3>Summary</h3><p>Technical skills are not the most important quality we look for in a new team member. We see a <strong>willingness to learn, ability to communicate, and empathy</strong> as more important.</p><p>That means, a perfect solution to the homework assignment isn’t expected. We also try to adapt our review based on the experience level of the candidate. But, we would like to see that you have a good foundational knowledge of REST, Spring, and Java or Kotlin so that you are successful when you start working with us.</p><h4><strong>This article was originally published on </strong><a href="https://medium.com/upday-devs/how-to-pass-the-upday-coding-test-809b8a93ba2f"><strong>upday devs blog</strong></a><strong>.</strong></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc353150c41c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/axel-springer-tech/how-to-pass-the-upday-coding-test-dc353150c41c">How to Pass the upday Coding Test</a> was originally published in <a href="https://medium.com/axel-springer-tech">Axel Springer Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Write for Axel Springer Tech Blog]]></title>
            <link>https://medium.com/axel-springer-tech/write-for-axel-springer-tech-blog-9e9ece03152b?source=rss-7d3ea0a1bc11------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e9ece03152b</guid>
            <category><![CDATA[axel-springer]]></category>
            <category><![CDATA[writers-guide]]></category>
            <dc:creator><![CDATA[Axel Springer Tech]]></dc:creator>
            <pubDate>Thu, 22 Aug 2019 13:11:56 GMT</pubDate>
            <atom:updated>2019-11-20T13:35:07.912Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to publish your ideas, insights and knowledge</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lbfB67tm2GlRER3BpzJrGw.png" /></figure><p>On the Axel Springer tech blog, tech and product folks are giving insights on their work across the Axel Springer group. The goal of the site is to aggregate tech knowledge from all employees across the Axel Springer group, give a voice to those employees who are working on great, innovative products and, of course, show people outside the Axel Springer group what we have to offer as a tech company.</p><p>You’re working in tech and want to share some insights about your work, products or team? Great, we’d love to host your article on our tech blog. Here’s a guide on how to post on the tech blog:</p><h3><strong>Contribution Guide</strong></h3><h4><strong>Topics that we are interested in</strong></h4><ul><li>Software Engineering / Programming</li><li>Machine Learning / Data Science</li><li>Design (UI/UX)</li><li>Agile &amp; Culture</li><li>DevOps</li><li>CyberSecurity</li><li>Innovation topics</li><li>Product Management</li><li>Business Use Cases (tech-related)</li><li>And many more…</li></ul><h3><strong>Why should we write blog posts?</strong></h3><ul><li>It’s good for our open source strategy. If you plan or already open-sourced something on Github repos such as on Axel Springer (<a href="https://github.com/axelspringer">https://github.com/axelspringer</a>), Axel Springer Ideas Engineering (<a href="https://github.com/as-ideas">https://github.com/as-ideas</a>), idealo (<a href="https://github.com/idealo">https://github.com/idealo</a>), StepStone (<a href="https://github.com/stepstone-tech">https://github.com/stepstone-tech</a>) and many more, it’s a good practice to write about this and share it with the community. You can also post this on Reddit.</li><li>It’s good for recruitment. Candidates can read what we do and you can help us find new amazing colleagues who can shape our company.</li><li>Good for you! In a blog post sometimes you need to write about very complex problems and it’s always better to write those things down in a very easy fashion. Then you can ensure that you understood the problem.</li><li>You can see the blog post as a personal journal where you store your knowledge. Those blog posts can then be read by new team members as well.</li><li>You’ll be famous 😉.</li></ul><h3><strong>How to contribute</strong></h3><h4><strong>1.</strong> <strong>New article which has not been published on Medium yet:</strong></h4><ul><li>Create an account on Medium.</li><li>Create a “New Story” (Write your story on Google Docs and have it reviewed before by your team members e.g. spellcheck, consistency etc.).</li><li>Add your story to the Axel Springer Tech publications (you need to contact one of the Editors to be added to the Axel Springer Tech blog as Writer).</li><li>Wait until we review it. We might do light editing e.g. on the title or content and then publish it. We will also publish it on our social channels such as Twitter.</li></ul><h4><strong>2.</strong> <strong>Article already published on Medium e.g. idealo/StepStone/Spring-Media Tech blog:</strong></h4><ul><li>Send us the link to your story.</li><li>We will copy the story and do some light editing e.g. on the title or content, add attribution to the original article.</li><li>We will repost your article with the <a href="https://medium.com/u/7d3ea0a1bc11">Axel Springer Tech</a> account.</li></ul><h3><strong>Useful link</strong></h3><ul><li>FreeCodeCamp Guidelines for submissions. Good tips on how to write a blog post: <a href="https://www.freecodecamp.org/news/how-to-get-published-in-the-freecodecamp-medium-publication-9b342a22400e/">https://www.freecodecamp.org/news/how-to-get-published-in-the-freecodecamp-medium-publication-9b342a22400e/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e9ece03152b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/axel-springer-tech/write-for-axel-springer-tech-blog-9e9ece03152b">Write for Axel Springer Tech Blog</a> was originally published in <a href="https://medium.com/axel-springer-tech">Axel Springer Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What do UX Hack Days look like at idealo ‍‍]]></title>
            <link>https://medium.com/axel-springer-tech/what-do-ux-hack-days-look-like-at-idealo-fbc3d74f15d?source=rss-7d3ea0a1bc11------2</link>
            <guid isPermaLink="false">https://medium.com/p/fbc3d74f15d</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[product]]></category>
            <category><![CDATA[ui]]></category>
            <category><![CDATA[design]]></category>
            <category><![CDATA[ux]]></category>
            <dc:creator><![CDATA[Axel Springer Tech]]></dc:creator>
            <pubDate>Tue, 20 Aug 2019 08:15:15 GMT</pubDate>
            <atom:updated>2019-08-20T08:15:15.759Z</atom:updated>
            <content:encoded><![CDATA[<h4>Summary of our UX innovation week <a href="https://twitter.com/idealoTech">@idealoTech</a> (Part 1)</h4><h4>By <a href="https://medium.com/u/30b0391d5ddd">David Paschke</a> (Senior UX / UI Designer @ idealo)</h4><p><em>Last year the UX department here at </em><a href="https://www.idealo.de/"><em>idealo.de</em></a><em> decided to do hack days twice a year: In spring and fall we engage with topics that are outside our day-to-day business. We want to think outside the box, do research and change our way of working. Furthermore, we’d like to mix colleagues that do not necessarily work together on a daily basis and learn from each other. The goal is to keep it fresh — everything we do for fun will also have an impact on real projects.</em></p><p><em>We call our hack days </em><strong><em>FUX days</em></strong><em>. FUX stands for “Fantastic UX” — in German that word sounds a lot like the animal fox (“Fuchs”), so we already have our mascot.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C8ZD6_poGwsBn4O4MLTz4g.jpeg" /><figcaption>A fux in its natural environment</figcaption></figure><h3>Topics, ideas, process</h3><p>Our FUX days started with a casual planning phase — everybody was invited to come up with topics that are allowed to span from abstract projects on a meta level to geeking out in detail on microinteractions. We split into two teams: Team 1 assigned themselves the task to rethink and redesign some parts of idealo, whilst Team 2 had the task of diving deep into the idealo customer journey.</p><p>In this article we want to show the results of team 1.</p><p><strong>At first, we came up with new concepts.</strong> What could idealo be besides a website and an app? Could the core functionality be integrated into a VR or AR tool? Should we open an idealo popup store? What would the idealo website feel like if it would behave more like an app?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GEHIOSY2z4iF2jaUM8HP-w.jpeg" /><figcaption>Lots of ideas on cliché sticky notes</figcaption></figure><p><strong>The second part was about style and brand:</strong> Rethinking the qualities that support the idealo design of now and redesign the idealo image of tomorrow. What would idealo look like if it was founded in 2018 and not 18 years ago? Can we design a flexible, fresh brand for an idealo that offers direct buying only?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t4Yo40MC__gGqGK8g1gbJQ.jpeg" /><figcaption>Giving each other feedback was key</figcaption></figure><h3>Results</h3><p>Here is the outcome — everybody on team 1 worked on one or more projects that we want to exhibit here. Below you’ll find those projects and what the authors have to say about it.</p><h4>App Patterns at Web</h4><p>by <a href="https://twitter.com/nomisum">Simon Schmidt</a></p><p><em>Simon says<br></em>One thing that disturbs me the most of our current webpage is the disconnectedness. The link between pages and areas is not seamless but clunky, highlighted even more through the lack of transitions and animations.With my exploration I tried to combine different ideas into one cohesive concept.</p><ul><li><a href="https://medium.com/amazing-stuff/complexion-reduction-a-new-trend-in-mobile-design-cef033a0b978">Complexion Reduction</a></li><li>Search as integral center of UX</li><li>Tag based search</li><li>Seamless transitions between page types</li><li>Joy through polish — consistent use of elements, shadows, attention to detail</li></ul><p>To save time I decided to use our current iOS design approach as a basis for key elements.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FWhGf8B4wRLI%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DWhGf8B4wRLI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FWhGf8B4wRLI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/36a66239c13ee029961a4b8fef39cf81/href">https://medium.com/media/36a66239c13ee029961a4b8fef39cf81/href</a></iframe><h4>Dashboard</h4><p>by Andrea Kessler</p><p><em>Andrea says<br></em>In the web, idealo users don’t have a personal place where they can overview their saved products, their orders and also their personal stuff.</p><p>idealo has a lot of knowledge about users and a lot more about the products they want — but doesn’t make these facts (and the combination of them) reachable and visually appealing for the users. <br>The playful designed dashboard brings more benefit and joy of use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w-HdpSVZIkRnAM2Pp960jQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vpEvjQWpolb5FWuVsF358g.jpeg" /><figcaption>Dashboard in dark and light mode</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gteRRCS22xfufxe1jmX4jA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fMLeyga6luhZWoVFyPp4sA.jpeg" /><figcaption>Product details in dark and light mode</figcaption></figure><h4>The line is the key</h4><p>by Xosé Lustres</p><p><em>Xosé says<br></em>idealo is a complex product and sometimes the user is lost on crowded pages with a lot of information about offers and products. On the other hand this complexity is our best value and part of our identity (Vielfalt). The proposed solution brings together brand and complexity positively, the complexity is hidden behind our brand element (orange bar) and it builds a strong key visual. Every time the user reach this orange bar wonderful things will happen, it will unfold to the user relevant features/content depending of the context. In this way the brand element is more valuable and we encourage the user to interact with us.</p><p>Here is a <a href="https://projects.invisionapp.com/share/KQGJVJIRD6A#/screens/286733988_Sport_Page">basic prototype</a> to show the line behaviour.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Atz1mF4Vgzp0GTzyqp1juQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a3Fs3xKl3kNPksxAAb40gQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lLUpk4PAub-kmqhkjNVxpQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PgtW46W2B1oPCqypnp72Nw.png" /></figure><h4>Fully customized</h4><p>by <a href="https://twitter.com/pixelfunk">David Paschke</a></p><p><em>David says<br></em>idealo is offering every kind of product — it’s like a general store. Imagine idealo would look specialized, depending on the category you are viewing.<br>A lot of people are taken to idealo via SEO / google searches. Let’s say you are looking for “good running shoes” at google, one of the results is “running shoes at idealo”, you click on it and enter idealo. In this case idealo would be styled like a shoe store with big imagery, fresh colors and bold texts. Thanks to this customization users get a feeling of trust and get into shopping mode. Also the content is tailor made in this case: In addition to shoes (that’s what you were looking for in the first place), we are offering you running cloth and also electronic products that you could need for your next run.</p><p>In the second picture you see idealo in an audio equipment mode. The basic elements are the same but the visual style has changed: tiles get bigger, everything is more rectangular, the color scheme gets darker etc. Those style-differences between the shoe and audio style should be easy to realize — it can be done with pure CSS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_jmpsPrbTpwKxdVr8w-4bg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S2EC3k0ucuMrafNBKlhzaw.png" /><figcaption>idealo in shoe and audio equipment mode</figcaption></figure><h4>idealo Homepage Redesign</h4><p>by Daniela Strauch</p><p><em>Daniela says<br></em>In The FUX-Tage we were given the chance to be creative and innovative in the creation of designs. During this process, many ideas and questions came up, however, one of the key questions for me was <em>“How could idealo look like in 2025?”.</em> That was my starting point. My inspiration came from the idea that idealo could be based on user interests, that means that the content on the homepage would be based on the interests previously acquired by a “survey” made at the time of registration and also in the recently viewed products (personalized for each user). As the main focus, the Homepage would have the TV Campaign videos playing in the header with an interactive Search Field. After will come to a Carrousel Gallery with offers organized by Themes and finally a featured product.</p><p>The main focus of the design lies in 2 aspects — color and trend. For me it was important to show that idealo can be more modern and less technical by the use of bold colors, shadows, gradients and tons of White.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TEBsgwBXriTTuH8BdFFYtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eyo2XOeZHzVkOdpjXtPlCA.png" /><figcaption>Two versions of Daniela’s homepage redesign</figcaption></figure><h4>Offer List Redesign</h4><p>by Daniela Strauch</p><p><em>Daniela says<br></em>As a second design, I wanted to redesign one of the most important parts of the website and make it more <em>attractive and interactive</em>.</p><p>In order to reach those standards, I decided to divide the offer list into sections. The left section consists of the product image gallery (almost 2 times bigger than the live version) that gives more attention to the product and arouse interest. Underneath we show the direct buyable offer and further product information. The right section shows the offers on tiles — that way they have more individuality and presence on the site. In comparison to the live version, this design focuses on 2 things: product image and price comparison.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*35pHLVhLRn_4fEU0CbD1ZQ.png" /></figure><h4>Blog Page</h4><p>by Paolo Pedretti</p><p><em>Paolo says<br></em>idealo as a blog page: get inspired, get informed and find anything you need. idealo has a lot of knowledge — I’d like to share this knowledge about products and trends with our users in the form of a blog. A place where users can refer to and be confident to find transparent, useful and independent information for their purchases. Let’s bring the content in focus to inform, guide and inspire our users.</p><p>If you already know what you need just type the product name in the search field. Tags that are relevant to the product you are looking for will populate the top of the page. Drag end drop the tags you are interested in in the box below to make up your perfect product selection. We will find the best offer for you.</p><p>Do you want to have a broader overview? Just reveal more offers clicking on the drawer below the main offer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5puK-8-l4Qj6kYoSUIqFGQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5ktg9Pl_eksW6Q8DZjG0yA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RsWKrtoT1ijBqplY0XuriQ.jpeg" /><figcaption>Homepage, playful filtering and offer drawer</figcaption></figure><h3>Impressions</h3><p>Our FUX days Vol. 1 in a nutshell — from working together, lots of feedback sessions and a small exhibition in the end.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RxHCq05DPdN-Cfq_v7Pkuw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c4OeDj-WGSRnmkpy-DZN1A.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9WK4UPjkjEdX2KTRZwk8uA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*va30d3c6wHjYJmeH1ILY9g.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xNCoFkuooroeJwn9haTdMQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mR8pE-vr7lk7hDbouiYxig.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*72m9gZadBOE9sVWTgcZABA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*auGCoeNIWoERJBrLcr3h1w.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pToC_OESi3cebmSmT8DXdg.jpeg" /></figure><h3>What’s next?</h3><p>The FUX days were super fun — everybody is still excited about this kind of hackathon. Internal feedback from a lot of co-workers was also great and helpful. There will be more FUX days in the near future — so stay tuned!</p><p>If you found this article useful, give me a high five 👏🏻 so others can find it too, and share it with your friends. Follow me here on Medium (<a href="https://medium.com/@david.paschke">David Paschke</a>) or on Twitter (<a href="https://twitter.com/pixelfunk">@pixelfunk</a>) to stay up-to-date with my work. Thanks for reading!</p><p><strong>This article was originally published on the </strong><a href="https://medium.com/idealo-tech-blog"><strong>idealo Tech Blog</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fbc3d74f15d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/axel-springer-tech/what-do-ux-hack-days-look-like-at-idealo-fbc3d74f15d">What do UX Hack Days look like at idealo 👩🏻‍🎨👨🏼‍🔬</a> was originally published in <a href="https://medium.com/axel-springer-tech">Axel Springer Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Learning for Classifying Hotel Aesthetics Photos]]></title>
            <link>https://medium.com/axel-springer-tech/deep-learning-for-classifying-hotel-aesthetics-photos-8f709612b5dc?source=rss-7d3ea0a1bc11------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f709612b5dc</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Axel Springer Tech]]></dc:creator>
            <pubDate>Tue, 06 Aug 2019 11:57:33 GMT</pubDate>
            <atom:updated>2019-08-06T11:57:33.452Z</atom:updated>
            <content:encoded><![CDATA[<h4>Hotel comparison site idealo.de trained two Deep Neural Networks to automatically assess the aesthetic and technical quality of millions of hotel images</h4><h4>By <a href="https://medium.com/u/569204318fae">Christopher Lennan</a> (Senior Data Scientist @ idealo) and <a href="https://medium.com/u/4ff6d2f67626">Dat Tran</a> (Head of AI @ Axel Springer AI)</h4><p>Most people love to travel. The desire to experience something unfamiliar, such as learning about a new culture or meeting different people, is something that probably all of us have felt in our lives. However, travel means making travel arrangements, which usually includes booking a hotel room. Finding a hotel room is however not always simple. Many factors influence our decisions, including budget, reviews, and the way the hotel offer is presented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*-qDjlKt7jK_Ri6gt.png" /></figure><p>This also includes the photos, which try to paint a favorable image of the property while presenting useful information. At <a href="https://hotel.idealo.de/">hotel.idealo.de</a>, we help people find the right hotel by providing high-quality hotel price comparisons.</p><h3>Not only the price matters but also the visual aspect</h3><p>We’ve all heard that “a picture is worth a thousand words”. That definitely applies to our hotel price comparison service. Each hotel supplies us with dozens of images, presenting us with the challenge of choosing the most “attractive” image for each pitch on our offer comparison pages since photos can be just as important for bookings as reviews. Given millions of hotel offers, we end up with more than 100 million images which require an “attractiveness” assessment.</p><p>This post presents how we use deep learning to automatically assess the quality of images. In particular, we examine two important dimensions. First, we’ll show you the aesthetic aspect which cares about the visual beauty in the image, which can be quite subjective. Second, we take a look at the technical aspect, which focuses on things such as sharpness, brightness, distortion, and other factors.</p><h3>How we trained an algorithm to see what’s beautiful</h3><p>Teaching and training a computer to decide what makes a good hotel photo is a hard problem; it’s even tough for humans. Substantial work has already been done in this field. For example, <a href="https://devblogs.nvidia.com/understanding-aesthetics-deep-learning/">EyeEm tackled the problem as a classification problem</a>, training a VGG-11 model with a hinge loss on their dataset in triplets form: query, positive, and negative class. Their main idea took advantage of the image similarity which resulted from the ordering in the triplets. Other solutions are presented as regression problems such as the <a href="https://www.youtube.com/watch?v=qGotULKg8e0&amp;t=1014s">ones used by Expedia</a>.</p><p>Overall, all the solutions so far only focus on the aesthetic aspect, not really the technical side. Moreover, they also need to a lot of data to train the model, particularly true in EyeEm’s case. These other solutions also fail to consider that one single photo can comprise many different opinions on the visual part since beauty is subjective. For example, in figure 1 we have a hotel bedroom. One person might give it a high rating because the photo actually looks quite nice. Another person might not really like it as there would be too many items in the photo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*wfAxYciasuTcMKUz.jpg" /></figure><p>To overcome these challenges, we implemented two separate classifiers based on Google’s research paper “NIMA: Neural Image Assessment” [Talebi and Milanfar 2017]. NIMA consists of two Convolutional Neural Networks (CNNs) that aim to predict the aesthetic and technical quality of images, respectively. The code for our implementation can be found on <a href="https://github.com/idealo/image-quality-assessment">Github</a>.</p><p>The special feature of NIMA is the use of the Earth Mover’s Loss (EML) as the loss function, which differs from the Categorical Cross Entropy (CCE) loss generally applied in Deep Learning classification tasks. The EML can be understood as the amount of “earth” that needs to be moved to make two probability distributions equal, as shown in figure 2. A useful attribute of this loss function is that it captures the inherent order of the classes.</p><p>For our image quality ratings, the scores 4, 5, and 6 are more related than 1, 5, and 10. In other words, we want to punish a prediction of 4 more if the true score is 10 than when the true score is 5. CCE does not capture this relationship, which is often not required in object classification tasks (e.g. misclassifying a tree as a dog is as bad as classifying it as a cat).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ONiKFlWM4UpUpRwd.png" /></figure><h3>Training</h3><p>We tried transfer learning in an early iteration to train our aesthetic and technical classifiers on two public datasets, the AVA dataset [Murray et al. 2012] for the aesthetic model and the TID2013 dataset [Ponomarenko et al. 2013] for the technical model. We used the MobileNet architecture with ImageNet weights for the model and replaced the last dense layer in MobileNet with a dense layer that outputs to 10 classes (scores 1 to 10). We also used VGG16 but dropped it due to slower inference speed.</p><p>In order to use the EML in the NIMA model, we need a distribution of ratings across all ten score classes for each image. For the AVA dataset, used to train the aesthetic classifications, these distribution labels already exist whereas for the TID2013 dataset, we needed to infer the distribution used for technical classifications from the mean score given for each image. For more details on our distribution inference check out our <a href="https://github.com/idealo/image-quality-assessment">GitHub repo</a>. The inferred distributions then form the ground truth labels used for training, as depicted in figure 3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*xyt54r8LiYYI1xGw.png" /></figure><h4><strong>Fine-tuning stages</strong></h4><p>We train the models in a two stage process:</p><ol><li>We start by training only the last dense layer with a higher learning rate. This ensures that the newly added random weights are adjusted to the ImageNet convolutional weights. This burn-in period mitigates the risk of juggling around the convolutional weights at training start and consequently slowing down the training process.</li><li>After the burn-in period we train all weights in the CNN with a low learning rate, as figure 4 shows.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_QVap1ma_QSo61sT.png" /></figure><p>The train and validation losses for both the aesthetic and technical model level out after 5 and 25 epochs, respectively. This provides a good indicator that the newly added weights have learned to classify aesthetic and technical quality as good as possible, all of which means it is time to start training all weights.</p><p>A significant drop in loss occurs for the aesthetic classifier once we also start training the convolutional weights (dashed lines in left graph above). This indicates that we adjust the convolutional weights significantly for the aesthetic classification task. The drop in loss is smaller for the technical classifier, which seems counterintuitive, since the technical image quality should be object agnostic and the ImageNet weights are optimised to recognise objects. The small drop might be due to the very small learning rate required to regularise training on the small TID2013 dataset.</p><p>You can find all hyper-parameters used for training on <a href="https://github.com/idealo/image-quality-assessment">our GitHub repo</a>.</p><h4><strong>Infrastructure</strong></h4><p>We used Keras and Tensorflow as our main backend for training the models. We further containerized everything into Docker and then trained it on AWS using the p2 (NVIDIA Tesla K80) and p3 (NVIDIA V100) instance. GPU support enabled us to achieve a 30-fold speedup in training time versus CPU. Figure 5 describes our training and evaluation workflow which we also <a href="https://github.com/idealo/nvidia-docker-keras">open-sourced on Github</a>. Basically, we launch an EC2 instance with Docker Machine and then train the model via a remote EC2 training script. After the training is done, the training outputs (logs and best model weights) are stored on S3 in a time stamped folder. This model can then be used for evaluation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*aMUaDtLwvZt-YjbG.png" /></figure><h3>Results</h3><p>The predictions shown in figure 6 indicate that the aesthetic classifier correctly ranks the images from very aesthetic (leftmost image with sunset) to least aesthetic (boring hotel room on the right).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*8uTZtwMK78wLRRcZ.png" /></figure><p>Similarly for technical classifications, the classifier predicts higher scores for undistorted images (first and fourth image from left), versus images with jpeg compression (second and fifth) or blur (third and sixth), as you can see in figure 7.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*2-CWbi_ZpkqsEsgt.png" /></figure><h4><strong>Hotel specific data is key to our model</strong></h4><p>Training the aesthetic and technical classifiers on public datasets already gave us quite impressive results. However, we’ve recognized that applying it to our hotel images could be problematic, since it lacked some kind of domain-specific learning pertinent to the aesthetic model. For example, figure 8 shows the ranking for hotel bedrooms. Although the ranking seems to be correct in terms of visual aspect, you can see that the distance from the best image to the worst image is relatively small. This can be quite problematic in production as there’s no clear boundary for determining a good image, for example, is the score 8 a good hotel image or is 5 a good score?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*EEYbq0AFppO77XCh.png" /></figure><p>This result can also be seen in figure 9. The model doesn’t seem to capture classes with very high or low ratings but is more tied in the center between class 4 to 6.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nUDpWG008Ma_4jlJ.png" /></figure><p>Another problem with transfer learning can be seen in figure 10. The image with the best ranking is a view from the balcony. Sometimes some listings with restrooms rank as the best images. Although visually there is no problem with it, we rather want to have the living room or sleeping room as the first image. This problem occurs because the model is trained on the AVA dataset, which not only includes photos with rooms but also photos with categories like food, landscape, people and many more others. Therefore, we decided to further fine-tune our model with self-labeled data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*cizf8JNNHyQwRe5y.png" /></figure><p>We created a simple labeling application and collected aesthetic ratings for 1000 images. We then fine-tuned the aesthetic model on 800 training images and then evaluated it on the rest. Our next iteration enabled us to clearly cover the higher and lower classes well, as shown in figure 11, which resulted in a better discriminatory power.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dV2XUpy6gwjoQmiP.png" /></figure><p>Moreover, when applying the model with our hotel specific data, figure 12 shows a much better ranking compared to the results we got in figure 10. The best image now shows a nice cozy living room instead of a view from the balcony, which doesn’t appear in the top 12 images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*ZmJA4pvqPfhNVjS2.png" /></figure><h3>Interpreting neural networks</h3><p>Deep neural networks work impressively well on many computer vision tasks. However, interpreting those results is generally difficult given their black-box nature. In order to gain a better understanding as to how the CNN assesses aesthetics for an image, we visualised the learned convolutional filter weights and output nodes in Aesthetic MobileNet. We used the awesome <a href="https://github.com/tensorflow/lucid">Lucid</a> library for all visualisations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R611QG80yh7awA-s.png" /></figure><p>Earlier convolutional layers are generally associated with simpler structures, like edges, wave patterns, and grids. The images above in figure 13 show patterns associate with six filters in layer 23 of MobileNet:</p><ul><li>The six images in the top row are generated from the original MobileNet ImageNet weights (ImageNet MobileNet)</li><li>The middle row images are generated from the MobileNet weights fine-tuned on the AVA dataset for aesthetic ratings (Aesthetic MobileNet first iteration)</li><li>The bottom row images are from the second iteration MobileNet Aesthetic</li></ul><p>We can see from the filter visualisations that the earlier convolutional filters are little affected throughout fine-tuning, since they are very similar to the original ones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*knSREGlrmxiB15ba.png" /></figure><p>The learned shapes are more complex for mid convolutional filters shown in figure 14. They resemble interwoven structures like fur or a grid with eyes. At this level, the filters only seemed markedly affected in the second iteration model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*auZ12n79mO373uqK.png" /></figure><p>The later convolutional layers in figure 15 show even more complex structures that resemble animals and tree like shapes. We can see that the filters for Aesthetic MobileNet differ significantly from the ImageNet ones, as they seem to be less focussed on objects, e.g. no animal shapes in the fourth filter from the left.</p><h3>Our learnings</h3><p>In this article, we presented our business challenge for automatically assessing the quality of images in both aesthetic and technical dimensions. We showed that trained aesthetic and technical models could successfully rank images according to aesthetic and technical quality. We also concluded that transfer learning is great way to bootstrap deep learning projects. However, domain specific data proved to be the key to improve our results. Moreover, we learned that not much additional data is needed to fine-tune our model as opposed to other solutions used so far. We also open-sourced our code.</p><p>Check out our <a href="https://github.com/idealo/image-quality-assessment">repo on Github</a> if you’re interested to try it out.</p><h3>References</h3><p>[Talebi and Milanfar 2017] <a href="https://arxiv.org/abs/1709.05424">NIMA: Neural Image Assessment, arXiv:1709.05424, 2017</a></p><p>[Murray et al. 2012] <a href="https://ieeexplore.ieee.org/document/6247954">AVA: A Large-Scale Database for Aesthetic Visual Analysis, Computer Vision and Pattern Recognition (CVPR), 2012</a></p><p>[Ponomarenko et al. 2013] <a href="https://ieeexplore.ieee.org/document/6623960">Color image database TID2013: Peculiarities and preliminary results, European Workshop on Visual Information Processing (EUVIP) 2013</a></p><h3>Acknowledgments</h3><p>This is joint work with <a href="https://www.linkedin.com/in/hao-nguyen-015252169/">Hao Nguyen</a>.</p><p><strong>This article was originally published on the </strong><a href="https://devblogs.nvidia.com/deep-learning-hotel-aesthetics-photos/"><strong>NVIDIA Developer Blog</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f709612b5dc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/axel-springer-tech/deep-learning-for-classifying-hotel-aesthetics-photos-8f709612b5dc">Deep Learning for Classifying Hotel Aesthetics Photos</a> was originally published in <a href="https://medium.com/axel-springer-tech">Axel Springer Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Minimum Viable (Data) Product?]]></title>
            <link>https://medium.com/axel-springer-tech/what-is-minimum-viable-data-product-6e6c4595db9b?source=rss-7d3ea0a1bc11------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e6c4595db9b</guid>
            <category><![CDATA[product]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[lean-startup]]></category>
            <category><![CDATA[technolgy]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Axel Springer Tech]]></dc:creator>
            <pubDate>Tue, 06 Aug 2019 11:46:58 GMT</pubDate>
            <atom:updated>2019-08-06T11:59:01.111Z</atom:updated>
            <content:encoded><![CDATA[<h4>🤖 A personal view on what MVP means for machine learning products…</h4><h4>By <a href="https://medium.com/u/4ff6d2f67626">Dat Tran</a> (Head of AI @ Axel Springer AI)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p8FDgSm9-1suFSBddROjGw.jpeg" /></figure><p>A couple of months ago I left Pivotal to join <a href="https://www.idealo.de/">idealo.de</a> (the leading price comparison website in Europe and one of the largest portals in the German e-commerce market) to help them integrating machine learning (ML) into their products. Besides the usual tasks like building out the data science team, setting up the infrastructure and many more administrative stuff, I had to define the ML powered product roadmap. And associated with this was also the definition of a Minimum Viable Product (MVP) for machine learning products. The question I often face though, here at idealo and actually also back at my time at Pivotal, is what actually a good MVP means? In this article, I will shed some lights on the different dimensions of a good MVP for machine learning products drawing in the experiences that I’ve gained so far.</p><h3>What is a MVP?</h3><p>At Pivotal Labs, I was exposed with the lean startup thinking popularized by <a href="https://medium.com/u/d2f31bf094c6">Eric Ries</a>. Lean startup is basically the state-of-the art methodology for product development nowadays. Its central idea is that by building products or services iteratively by constantly integrating customer feedback you can reduce the risk that the product/service will fail (build-measure-learn).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*J1mb7rDqd5pbtfCTHU1RGQ.png" /></figure><p>An integral part of the build-measure-learn concept is the MVP which is essentially a “version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort”. One well known example is to validate whether mobility will be successful or not (see image below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hJYxnfs6npD09IuX4606OQ.png" /></figure><p>We essentially start with the smallest effort to test the idea. In this case, we just take two wheels and a board. Then we ship this to the market and get feedback to continuously improve our product by adding more complexity to it. In this case we ended up with a car that integrates the consumer’s feedback. A well known example is Airbnb. In 2007, Brian Chesky and Joe Gebbia wanted to start their own business but also couldn’t afford the rent in San Francisco. At the same time, there was a design conference coming to town. They decided to rent out their place to conference attendees who didn’t find a hotel nearby. What they did was to take pictures of their apartment, put it online on a simple website (see image below) and soon they had three paying guest for the duration of the conference. This little test gave them valuable insights that people would be willing to pay to stay in someone else’s home rather than a hotel and that not just recent college grads would sign up. Afterwards they started Airbnb and the rest is history (if you want to read more successful MVP stories check out this <a href="https://speckyboy.com/successful-minimum-viable-products/">link</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*snvq6zkHLQVjxBsS9MDSrA.png" /></figure><p>In contrast to this, the other approach is by constructing the car one by one from the wheels up to the chassis without ever shipping it once. However, this approach is very costly. At the end of the day, we might ship a product that a customer doesn’t want. For example, let’s take Juicero as an example. They raised $120m from investors to create a well designed juice-squeezing machine which they released after some time of development for a very high price (initially it was originally priced at $699 and then subsequently reduced to $399). Along with the machine, you could also buy juice packs filled with raw fruits and vegetables priced at $5–7. Maybe some of you might know but the company already shutdown because it didn’t realize that you didn’t really need a high-priced juice-squeezing machine to squeeze out the juice packs. They didn’t really understand their customers. A simple user research would have made them aware that you don’t need an expensive machine to squeeze out the juice packs but two healthy hands would haven been enough.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UtA2dTdXrgl454pcrAMvZQ.png" /></figure><h3>How does the MVP concept relate to machine learning products?</h3><p>The MVP concept can also be applied to machine learning because at the end of the day, machine learning is also part of the overall product or the end product itself. In terms of this, there are three important dimensions that I consider as important:</p><h4>1. Minimum Viable Model</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*59vRc8MeLFWQbvZ2RSbnQg.png" /></figure><p>One important part of the ML product is the modelling exercise itself. Let’s assume we have a classification problem where we want to classify some data into pre-defined categories such as <a href="https://medium.com/@timanglade/how-hbos-silicon-valley-built-not-hotdog-with-mobile-tensorflow-keras-react-native-ef03260747f3">hotdog vs. not hodog</a>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*pehSJ49MtD7Pt4kEVeWLow.png" /></figure><p>A possible approach to solve this classification problem would be to take a neural network with one hidden layer. We would next train and evaluate the model. Then depending on the results, we might want to keep improving our model. We then would add another hidden layer and then do the same modelling exercise again. Then depending on the results again we might add more and more hidden layers. This approach is fairly straight-forward and actually the best solution to solve the hotdog vs. no hotdog problem because no feature engineering is needed (we essentially can take the original image as input data). However, for most of the classification problems out there, unless they are not specialized problems like the ones that we face in computer vision or natural language understanding, this is not the best way to solve this kind of problem. The major drawback of deep learning is generally its interpretability. It’s generally hard to interpret the results of a neural network depending on what kind of network you use. Moreover, you fairly spend a huge amount of time in tuning the parameters of a neural network where the gain in model performance is negligible.</p><h4>Start simple and establish a baseline…</h4><p>A more reasonable approach for most of the classification problems is rather to start with a linear model like logistic regression. Although in many real-world applications the assumption of linearity is unrealistic, logistic regression does fairly well and serves as a good benchmark aka baseline model. Its main advantages are also interpretability and you get the conditional probabilities for free which is very convenient in many cases.</p><p>Then to improve the model and also to relax the linearity assumption, you could use tree-based models. In this case, there are two broad categories, bagging and boosting. Under the hood, both use decisions trees but with a different way of training it. Finally, when all options are exhausted and you would like to keep improving your model, we could then utilize deep learning techniques.</p><h4>2. Minimum Viable Platform</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b4DHTNXzPO5X9Sswr0ItPA.png" /></figure><p>During my time at Pivotal Labs, I worked on a lot of projects to help Fortune 500 companies to help them getting started on their data journey. One of the common thing they share is that a lot of those projects started with a huge investment into their infrastructure. They spend a lot of money on buying big data platforms, the so called “data lakes”. Then after buying them, they started to load the data into their data lakes without even thinking of potential use cases. Then they heard about something called Apache Spark and added this to the layer as well. Now since AI has been the next big thing, they’ve also started to purchase GPUs and put deep learning frameworks such as TensorFlow on top of this. Sounds great right to have all (cool) tools at one place? The biggest problem, however, was that after putting all the data into their data lake, the data was just not right for the use case. They either didn’t collect the right data or it didn’t just exist that can support the potential use case.</p><h4>Use the cloud to bootstrap your “data” initiatives…</h4><p>A more reasonable approach would be not to think of hardware/software but more of solving the problem first. With this approach they can realize early on what data is needed in order to solve the problem and can also counteract against any data error. Other than that, there are a lot of machine learning problems, which I’ve seen so far, that can still realistically be solved on a local machine. They don’t really need to do this huge investment into their infrastructure. And in case, if data is really big, they could use cloud providers like AWS or Google Cloud where they can spin up a Spark cluster very easily. And if they have a deep learning problem, there would be many options as well. Either they can use the already mentioned cloud providers or services like <a href="https://www.floydhub.com/">FloydHub</a> which offers them a Platform-as-a-Service (PaaS) for training and deployment of deep learning models on the cloud.</p><h4>3. Minimum Viable (Data) Product</h4><p>The last point I want to talk about is the data product itself. Essentially, there are many examples for data products such as chatbots, spam detectors and many others — the list is long (check out <a href="https://medium.com/u/9766caca50be">Neal Lathia</a>’s excellent article for more ML product examples). But in this case, I will focus on recommendation services as I work in the e-commerce domain now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Nw6MFuPGF3vrJ_kLa1QBw.png" /></figure><p>One way to start with setting up a recommendation service is to use simple similarity algorithms and then continue with matrix factorization techniques. Finally, we could also try more complex models involving deep learning methods (e.g. deep structured semantic model). However, this approach is not really recommended. Recommendations can take many forms. Recommendations, for example, given by your friends are also recommendations or the top 100 most liked products are also recommendations. Not every sophisticated algorithm that we use will also lead to success but it has to be tested. In fact, don’t be afraid to start your recommendation service without machine learning.</p><h4>Do the right thing…</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/0*Lp5OdIDMhiuEV53j" /></figure><p>So the right approach would be first to set up the A/B testing framework and evaluation metrics (e.g. bounce rate or click rate) and then start with a simple approach like top products. Then after testing that users are really inclined to click on those recommendations (sometimes they have to get used to this first especially if it’s a new product feature) and eventually also might buy those recommended items, we could try it out with more sophisticated approaches like collaborative filtering techniques. For example, we could create a recommendation based on users who bought this item are also interested in those items or users who viewed this item are also interested in those items. The variety of options are endless for “users who … this item are also interested in those items”.</p><h3>Summary</h3><p>In this article, I gave my personal view on what MVP means for machine learning products. In essence, it is just about starting small and then iterate. Moreover to give a clearer understanding of what I mean with MVP for machine learning products, I discussed three major dimensions that I find critical for a good MVP data product:</p><ul><li>a minimum viable model,</li><li>a minimum viable platform,</li><li>and a minimum viable (data) product.</li></ul><p>Hopefully, in your next machine learning project you will keep those three dimensions in mind as well. If you found this article useful, give me a high five 👏🏻 so others can find it too, and share it with your friends. Follow me here on Medium (<a href="https://medium.com/@datitran">Dat Tran</a>) or on Twitter (<a href="https://twitter.com/datitran">@datitran</a>) to stay up-to-date with my work. Thanks for reading!</p><p><strong>This article was originally published on the </strong><a href="https://medium.com/idealo-tech-blog"><strong>idealo Tech Blog</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e6c4595db9b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/axel-springer-tech/what-is-minimum-viable-data-product-6e6c4595db9b">What is Minimum Viable (Data) Product?</a> was originally published in <a href="https://medium.com/axel-springer-tech">Axel Springer Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>