<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by magicpin Engineering on Medium]]></title>
        <description><![CDATA[Stories by magicpin Engineering on Medium]]></description>
        <link>https://medium.com/@magicpineng?source=rss-19acbe52d911------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*2yPomcMQh5zsnyMRNX0ApQ.png</url>
            <title>Stories by magicpin Engineering on Medium</title>
            <link>https://medium.com/@magicpineng?source=rss-19acbe52d911------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 00:23:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@magicpineng/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Introducing magicpin’s Local Search]]></title>
            <link>https://magicpineng.medium.com/introducing-magicpins-local-search-dfef42415c75?source=rss-19acbe52d911------2</link>
            <guid isPermaLink="false">https://medium.com/p/dfef42415c75</guid>
            <dc:creator><![CDATA[magicpin Engineering]]></dc:creator>
            <pubDate>Tue, 30 Mar 2021 12:23:33 GMT</pubDate>
            <atom:updated>2021-03-31T03:42:35.867Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vPb6fdnoHCQ_xd4P.png" /></figure><h3>Background</h3><p>magicpin’s vision is to make offline commerce magical. We aim to provide the same convenience to users when they shop from their local neighborhood shop as they are used to with a number of ecommerce players. However, building rich consumer experiences for the offline world is difficult primarily due to the fact that offline retail doesn’t have real time data flowing through. Wouldn’t it be helpful if you could know if your favorite pasta is in stock at your neighborhood stores and then get it delivered to your doorstep in an hour? Local neighborhood shops have an amazing depth of assortment and really understand the neighborhoods they serve. For eg. You can find pretty much any medicine at chemist shops in Green Park given its proximity to AIIMS, similarly Nehru Place in Delhi is known for its computer hardware assortment and competitive pricing. These local merchants have great merchandising skills and deep ties to the supplier ensuring the latest assortment, however from a user’s perspective it is time consuming to know what is available and at what price. Looking for the latest ink tank printer for your home office but don’t want to go to Nehru Place — we have a solution for you.</p><h3>How do you search for availability and prices of products at my neighbourhood stores?</h3><p>magicpin’s local search is powered by cutting edge machine learning that enables you to search for products in your neighbourhood returning relevant pricing and availability information.</p><p>Last year when we started thinking about this feature we brainstormed multiple approaches to build an index of products available at different merchants. A straight line solution is to ask merchants to provide us with a catalog of products with their prices and inventory, similar to what marketplace sellers do for Amazon or a Flipkart. However this straight line approach is high touch and requires the merchant to do all the heavy lifting. It is also hard to guarantee freshness of results in such a model since we are dependent on the merchant pushing the data to us. Though deterministic with respect to process, we realized that this solution wouldn’t scale. Alternatively, there have been a few interesting approaches to a build merchant specific product catalog like <a href="https://www.pointy.com/us">Pointy</a>. Most of these solutions extract data using barcode scanners or by asking users to scan QR codes on receipts. These work well in more mature ecosystems (US, EU) where retailers have access to these tools. In a market like India there are very few retailers with barcode scanners and even fewer with POS that print QR codes that encode receipt data and not just the total amount to be paid. However, with the advent of GST more and more merchants are printing receipts and given that a lot of magicpin users upload receipts through our app we thought it would be interesting to reverse engineer the merchant catalog through the receipts that we get.</p><h3>A look at the underlying technology</h3><p>Our main focus in building a catalog and enabling search was to use receipts that a user uploads to our app to earn magicpin points. We realised that these receipts don’t capture 100% of merchant transactions and to plug that gap we built software that a merchant could install on their terminal and send us anonymised receipts data. The BPP software, as it is known internally, is available as an add-on to the merchant and they can install the software to their terminals with a single click. The software understands different kinds of receipts formats and asynchronously sends us anonymised receipts data whenever a merchant prints a receipt. The diagram below summarises the overall flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5tGKLlueH8NYWt8B" /></figure><p>Once that receipt text reaches our system the next step is to understand that text by mapping it to SKUs. The text in the bill is often written in shorthand and different merchants/POS systems print it out differently. At magicpin, we are big believers in leveraging open source ML models and techniques. Given our constraints we are not trying to build the next best model to do text classification but looking to build features for our users that combine the best models for a feature with our unique data about offline retail.</p><p><strong>BERT #FTW — How we use BERT to do the heavy lifting</strong></p><p>The current problem that we are aiming to solve is well studied in the field of named entity recognition. The text from the receipts is processed line by line in order to look for potential products. We use a popular pre-trained Transformer based model BERT in a named entity recognition setting. We defined the named entities as “Product”,”Brand”, “Quantity”, “Other”, and manually labelled 200000 lines from receipts for use in this supervised fine tuning of the BERT model.</p><p>The challenge with most fixed vocabulary deep learning models such as BERT is how to account for out of vocab domain words, such as “aashirvad atta” which are very common in the receipts we process. We experimented with external augmentation of word embeddings for these out of vocab words. One approach we tried was to append each word’s BERT embedding with word2vec embeddings (trained on a small corpus of sentences mentioning most of the important domain specific words) before the classifier (fine tuning layer) layer of BERT. Using this augmentation procedure we see a bump in performance of about 5–6 % in both precision and recall, pushing up our precision to 88% and recall to 85%.</p><p>Once we tag text with high confidence from the receipts we still need to map them to a pre-defined dictionary of grocery products in order to build a catalog. For this last step we used a technique based on Locality Sensitive Hashing.</p><p>The extracted products (entities) are mapped to a database of grocery products using LSH’s approximate search method. The hashing method we used is the min-hash with 128 permutations. We obtain a similarity score for every piece of text tagged by BERT with item names in our product dictionary from LSH. We automatically add all the extracted products (from receipts) which have a similarity score above our fixed threshold to the merchant’s catalog.</p><p>This way our catalog management system builds a catalog for the merchant based on items purchased at the merchant in the past and is able to automatically add new items added or updated due to seasonal change, festivities or any other such reason.</p><h3>What do I see as the user?</h3><p>The end result is something similar to what you see below — we are able to extract items from receipts and make it searchable for the user. We realize we are just scratching the surface with respect to the potential applications that we could build on this offline retail data. If problems like this excite you, please reach out to us and help us make offline shopping more convenient.</p><p>Try it out by typing the product in the “Add your items” box on this <a href="https://magicpin.in/magicstore/1083632/#/">link</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/0*GuZIaun8gUzmCds0" /></figure><h3>What’s Next?</h3><p>We are constantly striving to build a suite of tools that can power small businesses in the digital age and help them generate demand. A comprehensive catalog of products and with inventory and pricing is a vital building block. Our approach has unlocked a lot of search based demand for these small businesses but we are constantly looking to improve our catalog and also the search that sits on top of it. We are looking to build query understanding models to help us navigate queries like ‘Pasta’ where a user could mean either ‘Pasta’ in grocery or ‘Pasta’ as a dish.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dfef42415c75" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Neural Nets to learn the best images to discover a new local establishment]]></title>
            <link>https://magicpineng.medium.com/using-neural-nets-to-learn-the-best-images-to-discover-a-new-local-establishment-33be384ecabc?source=rss-19acbe52d911------2</link>
            <guid isPermaLink="false">https://medium.com/p/33be384ecabc</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[magicpin Engineering]]></dc:creator>
            <pubDate>Tue, 26 Jun 2018 06:56:27 GMT</pubDate>
            <atom:updated>2018-06-26T06:56:27.045Z</atom:updated>
            <content:encoded><![CDATA[<p>A well-known saying amongst chefs is, “You eat first with your eyes” and nothing captures this inherently social experience better than a picture. magicpin is a social network for local experiences and pictures are central to sharing this experience on the platform. In this day and age of high res smartphone cameras the best way to checkout a trendy neighbourhood eatery is through pictures, before of-course you team-up with your friends to group buy a magicpin voucher at an amazing price :). Our users spend close to 30 min a day exploring and discovering trending local eateries, pubs, salons and even grocery stores through user uploaded pictures of a place. In this post we will describe how we built our system to auto feature the best and most interesting pictures uploaded by users.</p><p>A user uploads a picture whenever they transact at a local establishment and claim a cashback. We select the best pictures uploaded by users and feature them on the merchant page and also in the feed of the users in the same locality. Rating pictures at best is a subjective task and we try to mitigate the issue by having multiple analysts rate and re-rate the picture. When we started out our team of analysts was solely responsible for selecting the pictures to feature but as our user base and transactions grew keeping up with the pictures became next to impossible. Recent developments in the field of neural nets have greatly improved the performance of the state of the art image classification systems. In this post we will describe how we built and tested different machine learning techniques on a 1 Million+ image dataset progressing from classic ML techniques like Random Forest to multi-million parameter CNNs. To gain a thorough in-depth understanding of some these models we recommend going through the material in <a href="http://cs231n.stanford.edu/">this course on CNNs</a></p><p>The main objective of this system is to develop a machine-learning based model capable of classifying pictures uploaded by users. Like with almost all such systems the model sits in the middle of a much larger system that is responsible of other tasks like data preparation, evaluation etc. The diagram below describes one such system at a high level.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/834/0*hVL1ZSsYq9eH5-cq" /></figure><p><strong>Dataset — 1 Million manually rated images</strong></p><p>Our first dataset comprised of 1 Million user uploaded images and the corresponding transactions(image metadata) that has been rated on a scale of 1–5 by our human analysts. We went on to extract 20+ features from the image metadata like the time, day, location, transaction amount etc. The intuition was to build a classic machine-learning model like SVN to establish a baseline and compare the performance with neural nets. In the section below we describe the first cut of our models and how we iterated on the datasets and progressed to neural nets.</p><p>This step also involved the following data massaging steps -</p><ol><li>Replace missing data entries with mean values</li><li>Convert string data into numerical format using one-hot encoding or label-encoding.</li><li>Feature Engineering — Categorize createTime into 4 group based on hour of transaction (morning,afternoon,evening,night),weekday,month. Convert continuous attributes like cost for two, transaction amount, cash back amount into 1,2,3, and 4.Create columns to capture proximity in user home location and merchant location.</li></ol><p><strong>First Cut — Classification models based on techniques like Random Forest and SVN</strong></p><p>For our first cut we decided to train 9 different classification models with the objective to predict one of the 5 classes of rating (1,2,3,4 or 5). The results of the different models are below. All these models were implemented using Scikit learn or LibSVM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/0*hLBpDOl0uzVnWWyM" /></figure><p>The last column indicates the precision on the training set. Further the accuracy on the test set for the best performing models (Random Forest/Decision Tree) was even lower at close to 40%. To understand the reason for this lack of generalization we decided to plot the rating vs different attributes. The below figure captures why none of these models worked as expected for this problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/0*YGUK_quM-SOB4OX4" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/0*vVuP9HM_PHZ0eaTF" /></figure><p>Looking at the picture it is obvious that we need to use neural nets for this task since it incorporates attributes of the images rather than working only on the metadata like in the case of the above models.</p><p><strong>Migration to Deep Learning — CNNs</strong></p><p>In order to improve the system the next step was to train neural nets. For this we expanded our dataset to be 1.5M images and split it into training, validation and test set.</p><p><strong>Training via Transfer Learning</strong></p><p>To train our CNNs we first started with transfer learning on existing image classification models. Neural nets work best when they have many parameters, making them powerful function approximators. However, this means they must be trained on very large datasets. Because training models from scratch can be a very computationally intensive process requiring days or even weeks, we decided to start with transfer learning on the following pre-trained models on the ILSVRC-2012-CLS image classification dataset.</p><p>We did transfer learning on the following model -</p><ul><li>VGG 16</li><li>MobileNet V2 1.4</li><li>Inception V3</li><li>ResNet50</li><li>InceptionResNetV2</li><li>Xception</li><li>NASNet-Mobile</li><li>NASNet-Large</li><li>DenseNet 201</li></ul><p>We worked with both popular models like vgg16,inceptionV3 etc. to the state of the art models like Xception,NASNet etc.</p><p>Below are the training and test results for these models -</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6c6166fe070569a9e3e79a4c9d6fad85/href">https://medium.com/media/6c6166fe070569a9e3e79a4c9d6fad85/href</a></iframe><p>Surprisingly the test accuracies are lower than what we achieved with Random Forest. To understand these results, we plotted the confusion matrix for Inception V3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/0*igxV64BJ4cTbPyEq" /></figure><p>The model is unable to predict any images for class 1,3 and 5 and to understand why we looked at the distribution in our training set for 500000 images.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b5435315faabbd631afbce874e4e2422/href">https://medium.com/media/b5435315faabbd631afbce874e4e2422/href</a></iframe><p>The skewness in the distribution of images of each class resulted in the model not being able to generalize. So all images of classes (1–2–3) were predicted as 2 and images of classes (4–5) were predicted as 4.</p><p><strong>Deep Learning Pipeline Enhancements</strong></p><p>To improve the performance of the above models we decided to explore the following:</p><ol><li><strong>Eliminate skewness in the training data</strong> — We built a training set with equal number of images in the all the classes.</li><li><strong>Binary Classification instead of categorical classification</strong> — Our original aim was to classify images as {1,2,3,4,5} however for our system classifying images as {1,0} would suffice. We aggregated images rated {1,2} into the label {0} and images rated {4,5} into the label {1}. We decided to remove class 3 from dataset to improve the generalization power of the model.</li><li><strong>Fine Tuning the models instead of Transfer Learning</strong> — In transfer learning the weights of the pre-trained model are not changed, we only add our own last layer to do prediction on previously learned technique. However when we fine-tune a model, we make the model do the entire backpropagation again which ensures weights are adjusted with respect to our dataset. The advantage is the extremely high number of trainable parameters however the process of training with fine tuning takes a substantially longer time than transfer learning.</li></ol><p>We trained all the above models again using Fine Tuning on a dataset with 12k images and selected the best 3. The 3 best models — NASNet-Mobile, InceptionResNetV2, and Xception — were trained on our entire data set. Below is the training, validation and test accuracy.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4e5cc4d007ca6970f7a8f52256a63ccf/href">https://medium.com/media/4e5cc4d007ca6970f7a8f52256a63ccf/href</a></iframe><p>The above results were obtained using ‘Adam’ model optimizer and learning rate of 0.001. On changing the hyperparameters to ‘RMSProp’ for model optimizer and a learning rate of 0.00001 the results improved further.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7dc927da5d585dc5c3d8051c4c3af45a/href">https://medium.com/media/7dc927da5d585dc5c3d8051c4c3af45a/href</a></iframe><p><strong>Conclusion</strong></p><p>Moving to a deep learning based system to identify the best images has led to substantial reduction in the workload of our analysts and led to a increased number of images that we can show to our users. At the same time we realize neural nets like any machine learning technique are far from perfect and its shortcomings are well documented and reported. We will continue our endeavours to make this system better — investing substantially in a hybrid system of analysts and neural nets. Ping us if you have questions about what we did or would like to tell us how we could have done something better? If you want to come and build with us, even better!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33be384ecabc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building ARQuests — AR Experience for Indian users]]></title>
            <link>https://magicpineng.medium.com/building-arquests-indias-first-ar-experience-d82927f42e01?source=rss-19acbe52d911------2</link>
            <guid isPermaLink="false">https://medium.com/p/d82927f42e01</guid>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[ar]]></category>
            <dc:creator><![CDATA[magicpin Engineering]]></dc:creator>
            <pubDate>Mon, 26 Mar 2018 10:53:26 GMT</pubDate>
            <atom:updated>2018-03-27T06:38:45.228Z</atom:updated>
            <content:encoded><![CDATA[<p>Tim Cook famously <a href="https://www.independent.co.uk/life-style/gadgets-and-tech/features/apple-iphone-tim-cook-interview-features-new-augmented-reality-ar-arkit-a7993566.html">mentioned</a> that AR could be as big as the iPhone and over the past year the AR/VR community has matched his enthusiasm with tremendous progress in technology on both hardware and software. Pokemon Go showcased what was <a href="https://www.newyorker.com/tech/elements/pokemon-go-will-make-you-crave-augmented-reality">possible</a> with AR and needless to say like a lot of developers we are also very excited about AR and its potential in furthering our mission of making local experiences magical. Users on our app currently share their experiences by clicking a selfie and we wanted to leverage AR to build more ways to share those experiences. AR with its promise to seamlessly blend the digital with the physical surroundings resonated with our quest to build features that allowed our community to engage with their physical surroundings at thousands of magicpin partner merchants.</p><p>AR tech has made a lot of progress in the last year but launching an AR feature across iOS and Android still has a lot of challenges — hardware fragmentation, SLAM capabilities etc. In this post we will share some of the challenges and implementation details of the <a href="https://magicpin.in/ar-quests/">AR Quest</a> feature we launched last December.</p><p><strong>Current state and Challenges in implementation</strong></p><p>A number of companies have incorporated augmented reality into their applications — Facebook and SnapChat launched face covers, Apple and Samsung now have AR emojis, and Pixel has AR stickers. Most of the tech Facebook and Snap have developed isn’t open sourced or available in the form of a SDK. In addition to pouring over blogs on how these giants have built these experiences we started looking at number of available SDKs.</p><p>On iOS, Apple’s ARKit is feature-rich and has effective tracking — it was a straight-line decision to use it for iOS. Android on the other hand, is a totally different story — how we wish for a widely supported feature rich SDK like ARKit. In the second half of 2017, Google launched ARCore, however by November it was limited to a few high end devices. For Android development we started looking at a number of 3rd party SDKs. Below are some high level thoughts about them. There are extensive <a href="https://thinkmobiles.com/blog/best-ar-sdk-review/">comparisons</a> available but to test our use-case we ended up doing a POC with most of them. Though it is time consuming it gave us a good handle on what was possible with each and most importantly how they performed of different handsets. Performance on different handsets is often the hardest to predict and there are no good benchmarks available.</p><ul><li><strong>Vuforia</strong> — Widely used, supports tracking of markers, lacks marker less tracking.</li><li><strong>KudanCV </strong>— Supports both marker and marker-less tracking, but lacks plane surface detection. Can be integrated stand alone or with Unity</li><li><strong>EasyAR</strong> — Supports SLAM but has no Unity integration out of the box.</li><li><strong>Wikitude</strong> — Supports markerless tracking.</li><li><strong>ARToolKit</strong> — Open Source, supports marker tracking.</li><li><strong>ORG_SLAM</strong> — Open source library (GPLv3) written in C++ for mobile, supports marker-less tracking.</li></ul><p>In addition to deciding which SDK to use there were a number of other open issues to address like support for animated 3D objects and size of the APK. Android natively does not support rendering of 3D objects, especially if they have animations. This was necessary for the interaction we were planning to build. Most of the above libraries have a high storage footprint and this caused the size of our application to increase to 30MB from about 6MB. Another challenge was our desire to record video when the user is trying to find the hidden 3-D object without comprising the frame rate — this was easier said than done, especially in Android.</p><p><strong>Implementation overview</strong></p><p><strong>Selecting the Core</strong></p><p>As we set-up for our POC with different SDKs we decided on some standard parameters to evaluate the SDKs — performance and compatibility with different hardware, SLAM abilities, marker-less support, documentation, ease of development and integration, and finally cost. Based on the POC, we decided to use KudanCV. In our tests it was only that came through on all our parameters especially SLAM tracking abilities, support for marker-less tracking and ease of integration. Two other SDKs deserve a mention — Vuforia and ORG_SLAM. Vuforia is very popular in the AR community and not without reason. However for us, the lack of marker-less tracking was a deal-breaker. (Vuforia very recently released Ground plane detection and mid air detection, which could be a potential solution for us). ORG_SLAM on the other hand, stood out for its tracking and positioning (SLAM) abilities. However, the library ships with GPLv3 license that makes it tough to use in a non open-source application. Also, it had a substantially higher setup cost than others, porting the library, making sure it runs on various hardware platforms and optimising it for performance.</p><p><strong>Game Engine</strong></p><p>Initially we decided not to use a game engine primarily to prevent an increase in the size of the APK. However, as we were designing the UX of ARQuest we realized that 3D objects, animations and particle effects would be necessary for an engaging experience. None of these are natively supported by the Android framework and implementing these from scratch would have been a long process. To overcome all of these we decided to use Unity3D — this made the object + animation rendering really easy, and also opened up an entirely new domain of features like particle effect that enabled us to make ARQuest more realistic and engaging. KudanCV supports out of the box Unity integration which made the whole Unity integration process seamless.</p><p><strong>Reducing APK size</strong></p><p>The increase in application size was a major concern for us. The existing magicpin app at that time was only 6MB and we took immense pride in packing so many features into such a compact bundle. We wanted to make sure that we don’t bloat the size of the application since it adversely impacts downloads.</p><p>Our first iteration with AR and Unity increased the APK to an unacceptable 50MB. We started to look for ways to bring this down and our first step was to split the APK according to platforms, one for ARM and the other for x86. This reduced the size to 33MB for each variant.</p><p>Unity provides some guidelines and strategies to reduce size — one of them is to extract objects and textures into a an APK expansion file — this gave us another 5MB size reduction. Also, this process is easy if you are building your final APK from Unity itself, but if you have to merge it into an existing application build, there are a number of configurations that need to be taken care of. Some of these are not very well documented and we had to spend time with out app to make sure it doesn’t inadvertently break something else.</p><p>At this point we started looking at techniques to reduce the size of Unity and KudanCV core — since they were contributing most of the size increase. Unity has some guidelines in reducing the APK to below 40 MB but we had a more ambitious goal of keeping it under 10 MB. One of our ideas was to extract the Unity and KudanCV from our APK and instead package them as Android Framework. Our current implementation loads Unity related files and libraries only when required in the app while ensuring rest of the app continues to work without it. The entire process took us two iterations — the first one reduced the size from 33 MB to 20 MB, and the second one from 20MB to 10MB. The APK on play store currently is 10.8 MB. This entire process is pretty detailed and we will share more in a separate post.</p><p><strong>Recording Gameplay</strong></p><p>Part of magicpin’s appeal is the ability for a user to share her experience with the community. We wanted to give people the ability to share their AR experience with the community and for this needed to record and playback video as the user went about finding the hidden objects. This is natively supported in iOS but the achieving the same in Android with acceptable performance is not a straight line task.</p><p>A quick way would be to take screenshots of the activity or bitmaps of the root view and then convert it into video. This approach though easy to implement is very CPU intensive and leads to massive drop in frame rate. Implementing this resulted in the ARQuest frame rate dropping from 30fps to just 2fps. Another alternative is to use the screen recording functionality of Android. This is efficient but it records everything displayed on the screen including notifications and also required us to ask for scary permissions from the user. Unity also support frame captures, i.e., every thing that the unity camera sees can be captured into a bitmap, which can be convert into video (using ffmpeg). This feature requires double rendering of the 3D space, first when everything is being rendered on the screen, and second when everything is being rendered on the frame buffer that is being used to save bitmaps. This is better than the former methods, but it does result in some frame rate deterioration. As we implemented this we found that the performance depends significantly on the hardware capabilities since there is high frequency content switching and memory transfer between unity and java.</p><p>Another technique used to record video is to use OpenGL. We found that the OpenGL ESv3 feature — glBlitFrameBuffer() enables the OpenGL pipeline to copy everything into another secondary frame buffer. This method avoids double rending, thus ensuring no degradation in the frame rate. The data from the secondary frame buffer can be directly used to create a video using an Android video encoder. This is the approach we finally settled on to record gameplay video.</p><p><strong>Parting Thoughts</strong></p><p>Building ARQuest has been a very exciting journey into the world of AR. We uncovered a lot of pitfalls and experimented with a number of solutions to come up with an engaging experience. As a team we were exposed to game development and got to dabble with Unity. We are continuing to experiment with ARQuest and plan to introduce a number of new objects and game plays. Like always reach out to us if you would like to know more or have thoughts on how we could do this better. If building engaging experiences for millions of users is your thing, come join us!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d82927f42e01" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[In-depth look at a Scalable & Robust Data Stream Processing pipeline using Golang]]></title>
            <link>https://magicpineng.medium.com/in-depth-look-at-a-scalable-robust-data-stream-processing-pipeline-using-golang-processing-500k-9e68310a0675?source=rss-19acbe52d911------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e68310a0675</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[data-streaming]]></category>
            <dc:creator><![CDATA[magicpin Engineering]]></dc:creator>
            <pubDate>Mon, 22 Jan 2018 08:39:27 GMT</pubDate>
            <atom:updated>2018-12-11T11:13:05.311Z</atom:updated>
            <content:encoded><![CDATA[<p>Over the last 6 months we have seen tremendous growth at magicpin. We have grown 6x in both users and merchants and the average session time is up more than 50%. All this growth has resulted in an exponential increase in the data generated — transactions, reviews, likes, selfies, check-ins, voucher redemption and more. Right from the outset we realised how important this data in ensuring an engaging user experience. From surfacing the best restaurants based on a users interest to personalising the cash back deals for our users — this is the data powers them and more. In this post we will focus on a critical component in building these features — a robust and scalable system to process and ingest diverse data streams and serve recommendations generated based on them.</p><p>Like a lot of consumer tech companies we have a number of data streams with different sources of origin and different magnitude of scale. We wanted to build a system that could handle all of them, from the clickstream data generated by the app and site, to transactions performed by the user and the voucher, deals booked by those users. We also wanted to insulate upstream systems from this stream ingestion system, and ensure that the upstream systems to operate in a ‘fire and forget’ mode. Given our usage and growth, we needed the system to be able to handle thousands of request/s in parallel at best (concurrently at worst) and support async calls. An architecture that really impressed us was the <a href="https://medium.com/netflix-techblog/evolution-of-the-netflix-data-pipeline-da246ca36905">Keystone Pipeline</a> at Netflix . As stream processing was central to this, we quickly settled on Kafka being the central queue. Further as in the blog above we preferred upstream systems to push data over HTTP and avoid writing directly to Kafka since that would mean a lot of changes to the upstream services especially around error handling, retries and making calls async. Additionally, we introduced a web socket endpoint for the app to be able to send the real-time data to the system without the overhead of HTTP. On the data-storage side, we decided to use Cassandra and Elastic. On top of these data stores we built our personalisation and recommendation engine that generates recommendations using Spark’s ML library. We will go into details about our recommendation engine in a separate post.</p><p>At magicpin we have a lot experience in scaling Java based MVC apps. At same time we also knew how finicky it can be to tune JVM and as a company, we make every effort to stay on top of new and emerging technologies and continously ask ourselves, can we be doing this better? As we were debating the implementation of this system we realised that a language thought with parallelism and concurrency from the ground up might be a better fit. Researching more led us to this awesome talk ‘<a href="https://www.youtube.com/watch?v=cN_DpYBzKso">Concurrency Is Not Parallelism</a>’ and made us excited to explore Go more. The fundamentals of Golang and how constructs like ‘Channels’ make parallel programming more elegant excited us and after reading this blog post — <a href="http://marcio.io/2015/07/handling-1-million-requests-per-minute-with-golang/">Handling-1-million-requests-per-minute-with-golang</a> we were convinced that Golang is the way to go!</p><p>A couple of weeks later, after an initial crash course in Golang, we came away more impressed with how elegant and effective (both in lines of code and performance) Golang was. We started modeling our architecture and drew a lot of inspiration from the awesome Malwarebyte’s Golang article and came up with a worker pool architecture that streamed data from upstream sources through Kafka into ES and Cassandra. We wrote Spark jobs on top of this data and wrote the resulting recommendations back to Elastic/Cassandra.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*njCQWsbrRcn4-LcWZhcMBg.png" /></figure><p>The ingestion service is built around a HTTP worker pool and delegator/router which passes incoming jobs to idle workers. Each worker runs its own Go Routine pushing the data onto a Kafka topic. Once the job is completed, it returns to the pool. Simple and effective. Coupling this with an efficient and effective NoSQL store (Cassandra worked well for our use-case) we are to process 5k req/s on each 4vCPU machine. In addition to ingesting data, different consumers are colocated with our HTTP servers. These consumers perform a number of operations like cleaning, massaging, and aggregating the data before pushing them into the recommendation pipeline and/or other data sinks(eg. ES).</p><p>The REST API that exposed the output of the recommendation/personalisation engine uses Golang’s Gorilla mux package for routing and dispatch. The http package, which is used by Gorilla (and practically every other web framework for Golang) ensures that there is one Go routine per HTTP request. On the API side we could get a single 1 vCPU instance to serve 600 req/s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OBcWanZPmrJdRZQ5vc7OuA.png" /><figcaption>Server — Requests processed per second</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uTCoV2eBjE6usOC9IgXAqA.png" /><figcaption>API — Requested served per second</figcaption></figure><p>All of our Golang apps are built using the twelve-factor app methodology which makes them very easy to be containerized using Docker. To further optimize our hardware cost we deploy these containers on Kubernetes and leverage auto scaling to spin-up containers on-demand as the traffic spikes.</p><p>Our transition and experiment with Golang has been very fruitful. We have started to see if we can move other micro-services and middleware in our stack to Golang. At the same time we realize that Golang is not a silver bullet. We carefully look at our services and keep asking one fundamental question — Can this be better? It’s a constantly evolving process and one that takes careful research and planning. As we continue to grow we are excited to figure out how to scale these services to 10x the current level. Ping us if you have questions about what we did or would like to tell us how we could have done something better? If you want to come and build with us, even better!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e68310a0675" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hello World]]></title>
            <link>https://magicpineng.medium.com/hello-world-6470157192a?source=rss-19acbe52d911------2</link>
            <guid isPermaLink="false">https://medium.com/p/6470157192a</guid>
            <category><![CDATA[announcements]]></category>
            <dc:creator><![CDATA[magicpin Engineering]]></dc:creator>
            <pubDate>Thu, 18 Jan 2018 07:26:45 GMT</pubDate>
            <atom:updated>2018-01-22T08:13:58.843Z</atom:updated>
            <content:encoded><![CDATA[<p>Nothing should start without a good ol’ — ‘Hello World’</p><p>We plan to use this space to share our engineering learnings and contribute to the open source community. We at magicpin have been big proponents of open source software and have achieved amazing scale using the best-in-class open source components, we hope to use this platform to contribute something back.</p><p>Looking forward to learning from the larger developer community and hope some of our experiences are useful for developers on similar journeys.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6470157192a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>