<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Minted Labs on Medium]]></title>
        <description><![CDATA[Stories by Minted Labs on Medium]]></description>
        <link>https://medium.com/@mintedlabs?source=rss-4fd4c3ff70c9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Eh1A-5ettmMx94FckznTVQ.jpeg</url>
            <title>Stories by Minted Labs on Medium</title>
            <link>https://medium.com/@mintedlabs?source=rss-4fd4c3ff70c9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 22:53:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mintedlabs/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Creating training datasets at scale]]></title>
            <link>https://medium.com/minted-labs-blog/creating-training-datasets-at-scale-5d09cf69de60?source=rss-4fd4c3ff70c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d09cf69de60</guid>
            <category><![CDATA[minted]]></category>
            <category><![CDATA[metadata]]></category>
            <category><![CDATA[attribution]]></category>
            <category><![CDATA[qa]]></category>
            <category><![CDATA[image-classification]]></category>
            <dc:creator><![CDATA[Minted Labs]]></dc:creator>
            <pubDate>Fri, 10 Nov 2017 02:21:20 GMT</pubDate>
            <atom:updated>2017-11-17T06:56:16.011Z</atom:updated>
            <content:encoded><![CDATA[<p>To a bride shopping for the perfect wedding invitation, these wedding invitations might look like five very different design approaches to announce the big day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/0*moXUX5BYXG5R1cld." /></figure><p>But to us at Minted, they are comprised of hundreds of thousands of data points collected from a series of questions that hit every tiny detail of the card. These questions include specifics around layout, image details–color, shape, size–font style, and nearly everything you see in the design.</p><p>The tricky part is, once the first set of questions is answered, there are constantly new waves of questions coming in, all of which need to be answered in a consistent way across Minted’s massive design archive. Our ability to not only attribute the designs in a consistent manner over time, but also to go back and <em>reattribute</em> the whole archive with new values on a regular basis, is core to helping us navigate what we have on file and understand trends, and present the most relevant content to our customers.</p><p>So how to do this? The scale of our archive makes a purely manual option too expensive, so we have started to use image recognition processes to meet the business needs.</p><p>But because the questions being asked evolve on a regular basis, we need a repeatable, efficient process that can efficiently generate accurate yet sufficiently-sized training datasets from which the classification models can learn.</p><h4><strong>Selecting a vendor</strong></h4><p>To start, we knew we wanted to hire a vendor who specializes in this sort of work. Consistency and QA are mission critical to ensuring a sophisticated product, and it’s nearly impossible (from a resource and budget standpoint) to do it all in house. We originally thought we’d have an added layer of complexity in finding a firm with design experience, but ultimately deprioritized that in favor of working with a stable workforce we could train, that is using a tested workflow optimized for quality output.</p><p>In the end, we chose<a href="http://samasource.org/"> Samasource</a>, a company with deep expertise in image attribution that has a well thought out QA process, a collaborative web application called “SamaHub” that enables that process, and an agent turnover rate that is eye-poppingly low. Because their turnover is so low, our training process has gotten easier with each project as quality has gone up, and we expect that trend to continue.</p><p>It’s worth noting here that Samasource, while delivering top quality results, is also a nonprofit that gives work to people living in poverty. The mission-alignment with Minted was an added bonus for us.</p><h4><strong>Getting to 100%</strong></h4><p>The majority of our QA process runs on the SamaHub application, and that gets us most of the way there. We manage the Samasource leg of the process to a 95% accuracy goal. But we have also created a final review process on our end that truly polishes up our confidence and gets us to the 100% accuracy we need to train the models.</p><h4><strong>Detailed end-to-end process</strong></h4><p>Each project has three phases.</p><ol><li>Develop and implement a training program for the agents.</li><li>The agents attribute the images during the production phase. Throughout this phase, the field account manager and the Samasource PM conduct two cycles of QA tests while we do some QA on our end as well. Coming out of this phase, the attribution accuracy is at 95%.</li><li>Once all images have been attributed, we upload the data into our data warehouse and do a final round of QA. Coming out of this phase, the attribution accuracy is at 100%.</li></ol><h4><strong>1. Training</strong></h4><p>Training typically takes two weeks. For the training materials, we generate the list of questions along with example correct and incorrect responses. Two important considerations are which questions we should ask and how best to phrase them.</p><p>Winnowing the list of questions is only part of the work — we also have to be mindful of how best to phrase them so that they are free of common cultural frames of reference or educational background. For example:</p><ul><li>What is considered “heavy” use of geometric shapes?</li><li>How can we clearly explain what “abstract art” is?</li><li>How would you describe, in words, whether a design has “polka dots”? How are those different from other dots? From confetti dots?</li></ul><p>Once we’ve settled on the questions and the scoring scheme, the Samasource PM sets up the project on the SamaHub platform. The Samasource PM reviews the training materials with the field account manager who in turn trains the team lead and agents. Agents attribute a random sample of images and the field account manager lets us know if the training materials need any clarification or refinement.</p><h4><strong>2. Production and QA</strong></h4><p>The duration of the production phase depends on three things:</p><ul><li>The number of images</li><li>The number of questions</li><li>The complexity of the questions (one of the more difficult ones that we asked in a recent project on Fine Art Prints attribution was “Please describe this piece of art to a blind person”)</li></ul><p>Three levels of QA tests are conducted throughout this phase, which you can see in the diagram below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EkVZnQX0ipqyOf5v." /><figcaption><em>A multi-tiered QA system and feedback loops ensure high-quality output.</em></figcaption></figure><ol><li>Starting from the Uganda delivery center, at the end of each production day, the field account manager reviews 100% of the tasks. If the tasks do not meet a 95% accuracy threshold, they are sent back to the agents for rework and feedback is provided. If they do meet the threshold, the tasks are delivered to the Samasource PM.</li><li>The Samasource PM does QA twice a week on a random sample (usually 5%) of the tasks. If the sample does not pass the 95% accuracy threshold, the Samasource PM investigates and determines if the tasks should be rejected. If rejected, the tasks are sent back for rework and feedback is provided to the Uganda team.</li><li>Tasks that pass the Samasource PM’s QA are delivered to us on a weekly basis. We do weekly QA on a random sample (usually 5%) of the tasks. If the sample does not pass the 95% accuracy threshold, we determine if rework is needed. If it does pass, then the tasks are considered accepted.</li></ol><p>All QA in this phase is done on the SamaHub platform, which provides a consistent methodology for all process participants to QA, provides automated reporting, and allows all process participants to see the aggregate and detailed work others have done. The screenshot below demonstrates how this process works:</p><ul><li>Each image is shown one at a time along with all the questions.</li><li>For each question, we give it a thumbs up or thumbs down, and if applicable, leave comments explaining what the correct answers should be.</li><li>After reviewing all images, the SamaHub platform calculates whether the tasks have met the 95% accuracy threshold.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vT1O8JuQoe7wD3kp." /><figcaption><em>Review process in SamaHub. We loop through one image at a time and go through all questions for each.</em></figcaption></figure><p>So far, Samasource’s multi-step QA process has been effective By the time the tasks reach us, they’ve all met the 95% accuracy threshold.</p><h4><strong>3. </strong>Data storage and final review</h4><p>While a 95% accuracy rate is high, we need 100% accuracy to build robust models. The challenge, of course, is how to achieve that. To truly guarantee a 100% accuracy rate, you would have to review every value for every image, at which point you are basically doing the work yourself.</p><p>We have found a way to do a 100% accuracy rate review, and it’s not as bad as you would think on the surface. The two key things are:</p><ul><li>You have to start with a data set that is already at the 95% level.</li><li>You have to create a visual review process where you can quickly scroll through images without any button pushing.</li></ul><p>What this allows you to do is to choose a particular attribute and a value, display all the images on the screen, and then quickly scroll through them, looking for errors. We have found that when you are looking for only a single value at a time, the misclassifications jump out at you immediately, even when the images are going by on your screen at a very high rate.</p><p>The only thing that requires work is the mitigation when a misclassification is found. In that case, you have to stop, note down the error, and continue on. This is why it is important to start with a 95% accurate dataset.</p><p>We load the raw Samasource data into Redshift and conduct the final QA review on our business intelligence platform, Looker.</p><p>This process is the reverse of what was done on the SamaHub platform: instead of looping through one image at a time and reviewing all questions for each, we loop through one question and value at a time and scan through all images for each.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jptyCy6HTPxQySIE." /><figcaption><em>Review process in Looker. Each question is configured as a Looker “dimension”. We loop through one dimension at a time and scroll through all images to identify any misclassifications.</em></figcaption></figure><h4><strong>Key learnings</strong></h4><p>To have confidence in the models we build, we need to be sure we have good training data. As we’ve described — and it should come as no surprise to most — a lot of work is needed to ensure this. To summarize our key learnings:</p><ul><li>Training will take time, but that’s easy — the hard part is nailing down a framework that supports a solid QA system with multiple feedback loops.</li><li>Solely relying on the vendor to do QA is usually not enough. You’ll need to play an active role in the QA process, providing regular feedback to the agents but also setting up a final QA process on your end to achieve that 100% accuracy rate.</li></ul><p><em>Minted is a design marketplace which sources all of its submissions through open competitions. Each competition, called a ‘design challenge’, attracts thousands of graphic designs, textile designs or artwork and millions of votes from consumers — providing Minted with a great deal of data about design.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d09cf69de60" width="1" height="1" alt=""><hr><p><a href="https://medium.com/minted-labs-blog/creating-training-datasets-at-scale-5d09cf69de60">Creating training datasets at scale</a> was originally published in <a href="https://medium.com/minted-labs-blog">Minted Labs Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LAB SNACK! How the Presidential Election affected our holiday cards in 2016]]></title>
            <link>https://medium.com/minted-labs-blog/lab-snack-how-the-presidential-election-affected-our-holiday-cards-in-2016-fac278a7200b?source=rss-4fd4c3ff70c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/fac278a7200b</guid>
            <category><![CDATA[text-mining]]></category>
            <category><![CDATA[2016-election]]></category>
            <category><![CDATA[minted]]></category>
            <category><![CDATA[holiday-cards]]></category>
            <dc:creator><![CDATA[Minted Labs]]></dc:creator>
            <pubDate>Fri, 10 Nov 2017 02:20:07 GMT</pubDate>
            <atom:updated>2017-11-27T19:54:46.130Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s no secret that what’s going on in the world around us often works its way into our everyday lives. The 2016 Presidential Election took place right smack in the middle of the 2016 holiday card buying season. We were amazed to see how it affected the greetings people chose to incorporate into their cards.</p><p>Here’s what the data showed us:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xtuZug5wxnKPNfkTEF2eWw.png" /><figcaption>SOURCE: MINTED LABS</figcaption></figure><p>As you can see, “Cheer” wasn’t exactly the term people were looking for that season (dipping down to -14% post-election, yikes!). Words like “Hope” remained strong throughout the season at +9% but in the end, it was the message of “Peace” that prevailed. The word was already trending pre-election at +8% in holiday message but jumped to +14% soon after the votes were in.</p><p>It’s been quite a year in politics, natural disasters, and loss of several music icons. Take a minute to send a note to someone you love–or a perfect stranger! We could all use a little Cheer, Hope, and Peace.</p><p><em>Minted is a design marketplace which sources all of its submissions through open competitions. Each competition, called a ‘design challenge’, attracts thousands of graphic designs, textile designs or artwork and millions of votes from consumers — providing Minted with a great deal of data about design.</em></p><p><em>Lab Snacks are fun facts mined from a database of millions of Minted moments that take place online every day.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fac278a7200b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/minted-labs-blog/lab-snack-how-the-presidential-election-affected-our-holiday-cards-in-2016-fac278a7200b">LAB SNACK! How the Presidential Election affected our holiday cards in 2016</a> was originally published in <a href="https://medium.com/minted-labs-blog">Minted Labs Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What font trends can tell us about consumer preferences]]></title>
            <link>https://medium.com/minted-labs-blog/what-font-trends-can-tell-us-about-consumer-preferences-9ddcbfd5504?source=rss-4fd4c3ff70c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ddcbfd5504</guid>
            <category><![CDATA[minted]]></category>
            <category><![CDATA[fonts]]></category>
            <category><![CDATA[typography]]></category>
            <dc:creator><![CDATA[Minted Labs]]></dc:creator>
            <pubDate>Fri, 10 Nov 2017 02:19:11 GMT</pubDate>
            <atom:updated>2017-11-17T06:56:52.764Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="http://www.nbc.com/saturday-night-live/video/papyrus/3594574?snl=1">Ryan Gosling’s feelings about the Papyrus font</a> reminded us of many conversations we have witnessed at Minted, where people are deeply passionate about fonts. Minted is a design marketplace which sources all of its submissions through open competitions. Each competition, called a ‘design challenge’, attracts thousands of graphic designs, textile designs or artwork and millions of votes from consumers — providing Minted with a great deal of data about design.</p><p>Discussions about typography are part of our day-to-day both within our team meetings and our artist community. While it’s easy to see there have been major changes in the types of fonts we use from decade to decade–styles we used in the 60s look very different from those in the 70s and 80s–it’s less apparent that our font preferences have changed significantly in shorter timeframes of 3–4 years.</p><p>Because we’re as obsessed with data as we are with fonts, we mined through submissions from <a href="https://www.minted.com/participate/overview?it=utility_nav">past Minted art and design challenges</a> and created a visual to illustrate some of the trends we’ve seen, how quickly we’ve seen them, and what they can tell us about consumer behavior.</p><p>For context, Minted’s design archive consists of 175,000 designs from over 10,000 artists across 60 countries, and hundreds of cities. We included submissions that were voted into the permanent assortment as well as those that were not. In short: this database is incredibly diverse.</p><p>It’s also important to mention the data associated with these submissions is well attributed. <a href="https://minted-labs-blog.minted.com/creating-training-datasets-at-scale-5d09cf69de60">Our team spends a lot of time on metadata creation and maintenance for all our images</a>, and typography attribution in particular is something in which we have a lot of confidence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zO-AG9TPuxqqi_sD-jxqMw.png" /><figcaption>SOURCE: MINTED LABS</figcaption></figure><p>As you can see in the graph, fonts do indeed change quite a bit over a short amount of time. To illustrate this, we looked at the percent of submissions we receive every year represented by font. Then, we looked at the percent change in each font’s share–or how often it came up in the entire submission pool–over time. So, for example, if a font represented 10% of all submissions in 2009, and represented 16% in 2016, that would be a change of 6%. Then we plotted each point on the graph next to its corresponding font (as pictured).</p><p>At Minted, our assortment process is driven by our community, not by our own judgment of what is or is not “on trend” by industry standards. With that in mind, we like to take what we learn from our community and apply it to what’s going on in consumer preferences overall.</p><p>As Minted Founder &amp; CEO Mariam Naficy explains, “Art and design often signal societal values.” Looking at the graph, we see, for example, that hand-lettered fonts have been on the rise for a few years, which Naficy says “is closely linked to consumer interest in DIY, handmade, and artisanal goods.” More formal, scripted fonts have been on the decline, following consumers’ movement away from anything that feels “too corporate” or structured.</p><p>It’s easy for the untrained eye to overlook the specifics in font styles. But take a closer look next time you see an unusual font. These subtle changes can often tell us a lot about bigger trends, and give us insight into what might come next.</p><p><em>Minted is a design marketplace which sources all of its submissions through open competitions. Each competition, called a ‘design challenge’, attracts thousands of graphic designs, textile designs or artwork and millions of votes from consumers — providing Minted with a great deal of data about design.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ddcbfd5504" width="1" height="1" alt=""><hr><p><a href="https://medium.com/minted-labs-blog/what-font-trends-can-tell-us-about-consumer-preferences-9ddcbfd5504">What font trends can tell us about consumer preferences</a> was originally published in <a href="https://medium.com/minted-labs-blog">Minted Labs Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>