<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by CUUUPID on Medium]]></title>
        <description><![CDATA[Stories by CUUUPID on Medium]]></description>
        <link>https://medium.com/@cuuupid?source=rss-29dfcc1e4a6c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*5WGRwhA3HSOxe31YpPyriA.jpeg</url>
            <title>Stories by CUUUPID on Medium</title>
            <link>https://medium.com/@cuuupid?source=rss-29dfcc1e4a6c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:54:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@cuuupid/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Best Things in Life Don’t Scale]]></title>
            <link>https://cuuupid.medium.com/the-best-things-in-life-dont-scale-64b45c9fd502?source=rss-29dfcc1e4a6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/64b45c9fd502</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[metaverse]]></category>
            <category><![CDATA[web3]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[CUUUPID]]></dc:creator>
            <pubDate>Sun, 15 May 2022 04:40:08 GMT</pubDate>
            <atom:updated>2022-05-15T04:40:08.011Z</atom:updated>
            <content:encoded><![CDATA[<p>The biggest buzzwords in tech will fail because they just don’t scale.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j6wTMaMGvUExXbO1" /></figure><p>As we rapidly innovate with new technology, we have to keep in mind whether or not our innovations are <em>practical</em> and <em>feasible. </em>In a world where we count users in thousands and runtime in milliseconds, it’s more important than ever to ensure new tech scales.</p><p>That’s why it’s shocking that the three biggest buzzwords in tech completely <strong>fail to scale</strong>.</p><p>Of course, I’m talking about the three categories that seemingly every bit of tech news falls into nowadays:</p><ol><li>Artificial Intelligence</li><li>Blockchain</li><li>The Metaverse (specifically XR, AR/VR)</li></ol><p>It seems that nearly everything is either using or adding these and yet it’s insane how painful (and flatly impossible) it is to scale these services.</p><h3>We need more hardware</h3><p>The first and foremost shortcoming remains <strong>hardware. </strong>It’s no secret that the hardware requirements for tech are increasing exponentially year by year, both in price and specification. This is effectively becoming a blocking factor in widespread adoption, and in the case of blockchain it’s a killer.</p><p><strong>Let’s start with blockchain.</strong></p><p>Blockchain usually relies on “miners” that “mine” blocks and allow the network to execute the most basic of tasks (i.e. verifying transactions). The process of “mining” is something I won’t go into deeply, but it involves executing large cryptographic functions, a process known as “hashing.”</p><p>The problem with hashing is that it requires a significant amount of computing power; in fact, such a large amount that efficiently mining on the blockchain requires special hardware, either a GPU or an ASIC. To be even the smallest miner, you’ll need a high-spec GPU (let’s be real, Intel HD Graphics is consumer standard right now so basically every GPU above that is high-spec) or a dedicated ASIC. This racks up a pretty large bill, with most devices pricing in hundreds to thousands of dollars per unit, not counting the high electricity usage that’s considered an operating cost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*KPrO913KbpNEYw3o.png" /><figcaption>the great GPU shortage of 2018–2022</figcaption></figure><p>This has turned out to be a major, blocking factor in blockchain’s adoption. In fact, due to this very reason, <a href="https://www.theverge.com/2018/1/30/16949550/bitcoin-graphics-cards-pc-prices-surge">GPU prices skyrocketed and the stock all but disappeared, leaving prospective miners without any tools, crashing the PC gaming industry and inhibiting researchers from gaining access to high end hardware.</a> Even GPU manufacturers like Nvidia that <em>profited </em>from the rise have ultimately <a href="https://www.techspot.com/news/72854-nvidia-asking-graphics-card-retailers-prioritize-gamers-over.html">spoken out on the intense shortage</a> as a direct cause of blockchain.</p><p>A year later, most of these blockchain technologies have vanished and the prices of major cryptocurrencies have crashed to a quarter of their valuation. The most widely stated concern with blockchain is it’s failure to scale, and hardware remains (and will continue to be) a blocking factor in the scaling of blockchain technology.</p><p>This issue isn’t limited to blockchain, however. Artificial Intelligence relies on the same hardware (GPUs) and this is becoming a blocking factor in that field as well; most upcoming startups can’t afford to build an enormous GPU farm like Google’s, and buying computing power from cloud providers like AWS comes at a 4x price hike that rack up an enormous server bill.</p><p><strong>And what about the Metaverse?</strong></p><p>With XR, the issue is even more exasperated. In order to run these experiences, consumers have to render the visuals and the consumer’s device has to process sensory input. This puts the burden mostly on the consumer, which sounds great for businesses looking to get into the XR space but causes a major issue with scaling.</p><p>For example, let’s take Oculus VR. Oculus was one of the first VR headsets to launch and remains a popular name in the field, having been since acquired by Facebook and solidifying their place in the market.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MFKZ5Dgyf621DKxM.jpg" /><figcaption>the ridiculously high “minimum spec” for Oculus; USB 3.0, a good Nvidia GPU, 8GB of RAM… this is from their first model, in 2018 — in 2022 the minimum spec is exponentially larger and you’re going to need a dedicated rig</figcaption></figure><p>Unfortunately, Oculus almost exclusively operates on desktop PCs, moreover requiring high GPU specs. It faces the same pitfalls as blockchain and AI, but with much larger concerns: whereas with AI a business could just throw money at their infrastructure to temporarily resolve the issue, with XR the burden falls on consumers. The adoption cycle of AR/VR relies on consumer GPUs to become more widespread and higher specification, as well as waiting on prices to fall to levels that are affordable to the average user. It’s a process that could potentially take decades — while Nvidia CEO Jensen Huang proclaims that Moore’s law favors GPUs, the time required for the sufficient development needed to raise specs while lowering prices still amounts to several years. Case and point — it’s been 4 years and GPU availability has barely increased.</p><p><strong>Let’s go back to AI.</strong> My first draft of this article was written in 2018, 4 years ago today. Since then, AI has gotten aggressively compute-hungry. Nowadays all the buzz is about <strong>large language models</strong> that span hundreds of millions or, in 2022, hundreds of billions of parameters. A “parameter” is usually a numerical weight in the neural network; it’s common for these to be precision FP16, which is shorthand for a floating point number that’s 2 bytes long.</p><p>Let’s say you have a 1 billion parameter language model (that’s small btw — GPT-3’s popular form is 175x that size, with T5, PaLM, etc far exceeding that). Let’s say that you’re using FP16. Then, the memory required becomes:</p><p>1 billion * 2 bytes = 2 billion bytes = <strong>2 Gigabytes</strong></p><p>While you can optimize this to be much smaller during inference, you need to store all 2 billion bytes during training so you can backpropagate. Moreover, this is with a batch size of 1; if you want do a batch size of 16, 32, etc. you’ll need that many times more bytes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EMyf5XRzcJTUqmqb" /><figcaption>the storage space you need to house a GPU farm for an AI that writes very badly structured jokes in 2024</figcaption></figure><p>And let’s not forget that reality hits hard: the big language models you see in the news are hundreds of billions of parameters large (100+x bytes), and use FP32 (2x more bytes). Add that up and you have an absolutely ridiculous, mind boggling amount of memory usage <strong>just to store the model’s results</strong> (let alone other computations involved).</p><p>As an example, we built a medium-sized language model at <a href="https://aikomail.com/">Aiko Mail</a> that’s ~400 million parameters and shaped like a Transformer. All optimizations considered, it requires a whopping 24GB of memory just to train it with a batch size of 16 samples.</p><p>This is ludicrous for one simple reason: this memory is not RAM, but VRAM. This is GPU memory we’re talking about, and accessible GPUs do NOT have anywhere near this amount of RAM. Your GeForce 1060, 3070, 3080 etc have at most 8GB of VRAM, and will struggle just to run inference with large transformers.</p><p>To actually train even a mid-sized language model, you’re going to need 24GB of VRAM, commercially available only in the GeForce RTX 3090, which is <strong>literally the top of the line, best commercial GPU on the planet</strong> and will put you out a cool 3–4k$ per unit.</p><p>As a result, modern AI is largely inaccessible to most researchers and engineers. Compute costs to train in the cloud are astronomical, and estimates show it could cost millions to train a single large language model (+ the carbon emissions of tens of thousands of cars over their lifetimes).</p><p>There are companies trying to change this, like <a href="https://huggingface.co/">HuggingFace</a>, but we’re a long way from democratizing access to machine learning.</p><h3>Runtime</h3><p>When we talk about execution time in computing, we generally mean in <em>milliseconds</em>, as users have grown to expect instant interactions.</p><p>Unfortunately, this is not the case at all with all three of the above technologies.</p><p>With blockchain, mining a single block takes longer and longer as time goes on. It’s gotten to the point where it’s no longer profitable for individuals to mine large cryptocurrencies, and mining pools have started calling the shots on the network. For a “decentralized” technology, it favors an awful lot of centralization.</p><p>It also spells doom for blockchain’s (arguably) largest use case: cryptocurrency. Cryptocurrency thrives on fast transactions, without centralized authorities, across borders and without large fees.</p><figure><img alt="Image result for bitcoin fails to scale" src="https://cdn-images-1.medium.com/proxy/1*_uW9mMsLDNw3SitnM3F4LQ.png" /></figure><p>However, with the failure of blockchain to scale and the amount of time (time=electricity) required to mine a block, the fees associated with transactions rose to record highs; at its peak, Bitcoin reached a point where the fees associated with most small transactions were larger than the actual transaction cost, making it utterly useless and a total failure for everyday consumers. Transactions began to take hours, and regulation due to the runaway, rising price of Bitcoin made it difficult to move Bitcoin across borders.</p><p>And don’t even get me started on Ethereum. Holy sh*t, the only thing more expensive than gas for my car right now is the gas I need to put a picture of a monkey on a blockchain. The very idea that we need Layer 2 blockchains is insane — and with scaling blockchains like <a href="https://www.avax.network/">Avalanche</a> the issue becomes getting widespread adoption.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*TcI46y5629VZVaCd.jpg" /></figure><p><strong>Cryptocurrency became everything it promised to destroy.</strong></p><p>With artificial intelligence, there’s a very different issue arising. Inference time for many largescale models takes several seconds, which sounds like a small amount of time but begins to rack up and becomes a blocking figure when discussing userbases that amount in thousands.</p><p>Moreover, the figures quoted for inference time from most models are tricky — you’ve got to read between the lines, or rather to the next line which usually reads “as found on our XYZ GPU stack,” in which “XYZ GPU stack” costs several thousands of dollars and must be fully dedicated to that single inference task.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*glKnAym-oaMau4u3.jpg" /><figcaption>Google’s TPU stack that’s often used for training their models</figcaption></figure><p>Of course, research is being done here, but the research being conducted almost completely focuses on training time, which I would argue isn’t as important for businesses (but is important for research). For training, a week is not a big deal — any startup can spare a week to train a model that will become the cornerstone of their business.</p><p><strong>The larger problem lies in inference</strong>. At a root level, training requires inference — inference is usually quoted as the “forward” phase of a network, and this must occur in training before backprop is conducted. However, in training all the data that needs to be inferred is available at the beginning.</p><p>In other words, <strong>the entire batch can be processed at once</strong> (i.e. 100s of images at the same time), due to math scaling — multiplying a larger matrix of several matrices is more efficient than multiplying several matrices of 1 matrix (in other words, it’s more efficient to do more at once). <strong>This goes up to a certain point, similar to the idea of diminishing returns in economics</strong>, but it remains that batched training allows us to scale inference on large amounts of data. (given you have enough compute)</p><p><strong>Unfortunately, in practical sense, batch processing is almost never the case.</strong></p><p>It’s rare that a model has to run inference on 200 images all at once; it’s more likely that 200 images are submitted for inference in, say, a minute. There will be a hard bottleneck in inference time per image; even an inference time of say, a third of a second (which is insanely fast — those are the times boasted by the fastest of models, such as Gmail’s super-optimized sentence prediction model) will result in only 180 of those images being processed per minute. Even at the small load of 200/min, the model fails to scale, and in order to assess the growing backlog a second instance must be spawned to balance.</p><p>It’s a hard pill to swallow in a world where large loads are a given; in fact, Node’s Express has come under fire before for <em>only </em>supporting a couple thousand connections/second, similar to popular NoSQL databases being criticized for having a bottleneck at several thousand transactions/second.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jJui4JzvZWRDWcq_.jpg" /><figcaption>a dam that’s guaranteed to burst</figcaption></figure><p>It’s a figure <strong>unheard of </strong>in the world of AI, which runs into a hard bottleneck at a couple hundred per second in terms of separate inferences on a single instance of a model, <a href="https://towardsdatascience.com/scaling-ai-2be294368504">even with the optimizations you can make.</a></p><p>With XR, there’s a very, very different problem at hand. The issue lies in immersion — for immersion and avoiding the uncanny valley, interactions have to take place and the render has to adjust faster than humans are able to perceive. In other words, a couple hundred milliseconds <strong>isn’t fast enough.</strong></p><p><strong>With XR, we’re measuring things in small amounts of milliseconds. We’re measuring acceptable latency as below ~20ms</strong> (a figure that many gamers would disagree with, as 100+ fps and under 10 ping is considered normal for most rigs).</p><p>It’s a figure that remains a bottleneck and is one we’re nowhere near; while XR tech focuses intensively on this and numerous breakthroughs have been achieved in this field in terms of regular sensory input and rendering, we still face issues in terms of interaction; precisely, XR opens up a whole new, infinite range of possibilities for interaction that just can’t be processed very quickly by most engines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/579/1*_iwTUDvBJ-J9GdMJ02qFTw.gif" /><figcaption>ARCore’s Augmented Image engine as showcased at Google I/O 2018</figcaption></figure><p>Moreover, to run image augmentation with anything more complicated than homography, the latency is too high to be considered “acceptable” by most experiences. The fastest we’ve tested so far is Google’s ARCore Augmented Images, which uses homographies (there’s also an Augmented Faces demo which is a little slower and has some noticeable latency/lag).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/263/1*N-9r_ss0qCsYbumcAUoJFA.gif" /><figcaption>ARCore Augmented Faces demo</figcaption></figure><h3>It’s a big problem</h3><p>In the last two years alone, the number of advancements in these three fields are insurmountable.</p><p>In AI, we saw natural language generation make a leap forward with PaLM and GPT-3, models earning their reputation as the Imagenet of NLP. It’s opening a new world of possibilities with text processing. We’ve also seen convolutional networks and GANs grow to adopt 4k images, and we’re entering an era where AI can mimic HD images and generate content that seems believable to the human eye.</p><p>With blockchain, the field has become less muddied, with several cash grabs dying out and making more room for exposure with existing platforms. There’s been a lot of progress with the rise of NFTs and Web3, but currently ther market is in a total crash state so we’re waiting for recovery.</p><p>In XR, we’ve seen phone level XR and AR becoming a reality with many improvements to ARCore. It’s becoming possible to integrate AI into XR and develop experiences that allow for some level of human interaction (4 years ago when I drafted this, the highest level of interaction was AI-based: <a href="https://github.com/pshah123/FiddlerCore">Fiddler AR</a>, but now even physical touch is possible with reconstructed topologies).</p><p>These are all fields that bear a large, significant impact on the future of technology, and where human civilization is going as a whole. These are technologies that don’t impact the economy, they redefine it; they don’t impact society and interaction, they revitalize it. It makes an overhaul possible of our entire culture, and at a root level it’s being called another Industrial Revolution.</p><p><strong>For a movement that has such large bearing on the widespread public, it’s vital and paramount that it can scale to support the public with ease. </strong>Until this is implemented it impedes adoption and forms a hard-stop barrier to entry in terms of innovation outside of large corporations.</p><p>I hope to see more innovation in scaling these technologies. At Aiko Mail, we’re rolling out language models that out-perform GPT-3 and other large language models with a fraction (2/1000th right now) of the compute. It’s not <strong>impossible</strong> to scale — just very difficult.</p><p><a href="https://aikomail.com/">Aiko Mail - The First AI-Powered Inbox</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64b45c9fd502" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Someone is Spying on You]]></title>
            <link>https://medium.com/aiko-ai/someone-is-spying-on-you-a43503b1ff64?source=rss-29dfcc1e4a6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a43503b1ff64</guid>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[internet]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[privacy]]></category>
            <dc:creator><![CDATA[CUUUPID]]></dc:creator>
            <pubDate>Fri, 18 Jan 2019 04:17:48 GMT</pubDate>
            <atom:updated>2019-01-18T04:17:48.130Z</atom:updated>
            <content:encoded><![CDATA[<h3>Your Browser is Spying on You</h3><h4>It’s 2019 and websites are only getting more and more intrusive.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7Wj_j6OwxY0k3eSt" /></figure><p>It’s been a while since my last piece. A lot has changed. I’ve started working somewhere, I’ve begun new projects, I turned older…</p><p>But if I visited your site in the last few months, you probably already knew that.</p><p>As the web grows and companies begin to monetize their online presence in an increasingly aggressive manner, netizens like you and me have all but lost any sense of privacy as we browse the internet. Tracking tools from veterans like <a href="http://analytics.google.com/">Google Analytics</a> to the new kids on the block —<a href="http://hotjar.com/">Hotjar</a> and <a href="http://peekmap.com/">Peekmap</a> — have allowed websites to track every aspect of our visit, including where we came from, what we click, what our eyes are looking at, and where we’re going.</p><p><strong>It’s downright terrifying. </strong>How did we get here?</p><p>It started with bragging rights, evolved into Google Analytics, and brings us here today with “visitor recordings”, powered by an AI analytics tool that records your visitors and predicts exactly what they’re looking at, aiming to become the behemoth of user tracking.</p><h3>Bragging Rights</h3><p>It all started with bragging rights and the concept of social proof.</p><p>It’s a simple, psychologically driven idea, and works the same reason mob mentality does. If you visit a website and see 500 people liked it on Facebook, it has 2400 followers on Instagram, and has been tweeted 240,000 times, then you’re probably going to stay.</p><p>A more common earlier form of this was a page hit counter. This would basically be a big number on your page that represents the number of people currently viewing it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*3WpvXKRIq456zD7S.png" /></figure><p>Of course, if I refresh the page or my friend next to me opens it, I expect to see that number go up!</p><p>So how do we handle tracking visits? Enter sessions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/1*nIQgRy27thOtCD_s-q3Wgg.png" /><figcaption>High quality meme, anyone?</figcaption></figure><p>A session is basically one usage of the website/service. It starts when you enter the website and it counts when you visit a different page or click something. A session ends when you leave the site and close the tab. It’s a simple idea born out of necessity to count page views accurately.</p><p>So every time you visit the site, it counts your session! But this also needs to be tracked. We need a way of identifying each user accurately — enter <strong>fingerprinting</strong>. I won’t get too into it, but we can digitally fingerprint a user by tracking:</p><ul><li>IP address</li><li>MAC address</li><li>any hardware or internet signals we can get our hands on</li><li>your browser</li><li>your cookies</li><li>and a bunch of other creepy things that let me know <em>you’re you</em> no matter where you go, no matter when you go!</li></ul><p>This should already have been enough to raise some red flags, but it only escalates from there. What started as bragging rights/social proof quickly evolved into something much worse.</p><h3>Papers, please.</h3><p>When you go through an airport, you generally need a Passport. This nifty little book gets you through security checkpoints and allows airports to assert who you are and where you’re from. It also serves to track you as you go from country to country. When you land, normally you’re asked your port of origin (where you’re flying in from), your airline, and why you’re here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gBbkGoUSgBE7WXBD" /></figure><p>Websites started to check the same thing. They already check your IP, physical address, browser, etc. — your airline. But since websites fingerprint users from when they enter the site, it was a given that they would also take into account <em>where</em> the user was coming from — their online port of entry, if you will. This is known as the <em>referrer.</em></p><p>The referrer would go on to become <strong>tremendously important</strong> in identifying where users came from, what social channels performed the best, and what traffic was looking like via different sources. Tracking the referrer is an enormous boost to your website — the next time someone tweets you and gets a thousand likes, you’ll know the spike in traffic from your website is coming from Twitter and engage that tweet to milk it for what you can.</p><h3>Why are you here?</h3><p>Next came low level tracking tags. People started using services that would log clicks, impressions, conversions and other metrics that arise from user interaction on the website. Developers had been logging requests made to servers for decades at this point, it was a given that the next step would be to filter this and get some useful information.</p><p>So every time you click “Add to Cart” or “Buy Now,” the website tracks that and records that in your user file.</p><p>Now this isn’t that bad in itself, but what it turned into is the real monster. But first, we must meet the service that caused this mess.</p><h3>Age/Sex/Location?</h3><p>ASL — Age/Sex/Location — is a common slang in anonymous chatrooms in order to identify who someone is talking to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Tv3-V5wLABFizUe6" /></figure><p>Of course, your browser doesn’t need this information. But that doesn’t stop it from collecting it and giving it up!</p><p>Enter Google Analytics. Google Analytics launched as a webmaster tool — it’s only obvious that the search engine giant would have the best-selling SEO tool — and provided one of the tracking tags I just mentioned. You’d drop this tag into your site and Google would track referrers, sessions, and <strong>more</strong>, dropping this into a user profile marked with a user ID. You would then call a record event to GA every time an interaction you wanted to log occurred, and Google would amass the data into the correct user profile.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aK0nIvH2tdfK__7q" /><figcaption>Google Analytics, the cool hip…. monster</figcaption></figure><p>Of course, the “<strong>more</strong>” is the keyword above. Google, the data giant that it was/still is, began to collect information on user ages, locations, networks, and every piece of profiling data they could get their hands on, turning it all into a neat dashboard that anyone could browse. Google Analytics quickly turned from a logging tool into an intrusive marketing monster.</p><h3>It got worse.</h3><p>And then came <a href="http://hotjar.com/">Hotjar</a> and <a href="http://peekmap.com/">Peekmap</a>.</p><p>Heatmap/recording tools had been around for some time but they were always varying degrees of reliability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*i2JNcnAqllZ8zBb4" /></figure><p>When <a href="http://hotjar.com/"><strong>Hotjar</strong></a><strong> </strong>released its tool, it pretty much rocked the entire industry —<strong> easy visitor recordings for everyone.</strong> This was the beginning of one of the most intrusive practices in web analytics,<strong> and yet only the tip of the iceberg.</strong></p><p><a href="https://www.hotjar.com/">Hotjar: Website Heatmaps &amp; Behavior Analytics Tools</a></p><p>Combining mouse movement tracking with artificial intelligence, <a href="http://peekmap.com/"><strong>Peekmap</strong></a><strong> has managed to create a terrifying tool that tracks <em>everything imaginable.</em> </strong>Mouse clicks, movement, typing — it tracks everything and replays it in a nice little video that can be skimmed through. With AI, it then adds eye-tracking without the use of a webcam. It’s terrifying — <strong>imagine being able to pretty much <em>watch </em>how your users are using your website.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H4Iy0lrBfDqBAsl4NMF3KA.png" /><figcaption>Peekmap proudly touts its invasive eye-tracking</figcaption></figure><p>All of this put together tells a frightening narrative of being able to visualize exactly how visitors use your site — a miracle and the holy grail for webmasters, and a nightmare for users and data privacy.</p><p><a href="https://peekmap.com/">Peekmap | See What Your Users See</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eJ5wPVQ2G8sc84QD" /></figure><h3>It’s Time to Draw the Line</h3><p>With the passing of GDPR, a whole new wave of possibilities has surged in data privacy. What was once a futile proletariat fight against internet giants is now supported by governments across the world.</p><p>It’s time that we take a stand against intrusive “analytics” and prohibit tracking to the best of our ability.</p><p>It’s a problem that we may just solve with more tech. Take UBlock Origin — it’s a free and open source Chrome extensions that’ll block trackers and ads, keeping you and your data safe. Or <a href="http://helloaiko.com/">Aiko AI</a> — an AI email client that does all it can to block trackers in your emails. <strong>We can fight fire with fire.</strong></p><p>In the meantime, tread carefully. <strong>Because the next website you go on might just be spying on you.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a43503b1ff64" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aiko-ai/someone-is-spying-on-you-a43503b1ff64">Someone is Spying on You</a> was originally published in <a href="https://medium.com/aiko-ai">Aiko Mail</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What they don’t tell you about scaling AI]]></title>
            <link>https://medium.com/data-science/scaling-ai-2be294368504?source=rss-29dfcc1e4a6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/2be294368504</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[CUUUPID]]></dc:creator>
            <pubDate>Wed, 05 Dec 2018 21:15:05 GMT</pubDate>
            <atom:updated>2018-12-07T19:14:10.631Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VJKX1yppWAAtwpZRfWf37A.png" /></figure><p>There are tutorials for <em>everything</em> with AI nowadays. <em>How to do Object Detection, Image Classification, NLP, build a chatbot</em>, etc., the list goes on.</p><p>But when I looked for information on how to properly <em>scale </em>AI, I found very little content. Even more surprisingly, the few resources that did exist seemed to reiterate the same few points:</p><ul><li>build your model with a scalable framework like <strong>TensorFlow</strong></li><li>either package it into your client (TF.js, TF Lite, TF-slim, etc.) or deploy it as a microservice with containers</li></ul><p>I was more interested in the second portion as I had already developed a model, but I was surprised there was little detail provided on how to actually achieve this, and even fewer information regarding the shortcomings of each solutions. After researching for a few days and scaling AI at <a href="http://crane.ai/">Crane.ai</a>, I put together some more information <strong>regarding deploying, their drawbacks, and how to go about optimizing your TensorFlow model at a low level.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IY6K0dSlX22rrP-DFNbv-w.png" /></figure><h3>Packaging it into your client — it sucks!</h3><p>One of the most commonly used techniques is to package the AI into your client of choice using tools like TensorFlow.js, TF Lite, or TensorFlow Slim. I won’t go into too much details about how these frameworks operate, and instead focus on their drawbacks.</p><ul><li><strong>Computational Power. </strong>The issue with deploying many of these models is that they require an <strong>immense amount of memory </strong>(I’m talking mobile app or browser limits, i.e. &gt; 1–2GB RAM). Many mobile phones do not have this power, and desktop browsers will lag up the UI thread while also slowing down the user’s computer, heating it up, turning on the fan… etc.</li><li><strong>Inference Time. </strong>When you’re running models on a device with unknown computational power, the inference time is generally also unknown; however, these aren’t GPU-powered high RAM high CPU machines, they’re mobile phones, browsers and desktop apps running on average computers. Inference with some larger models can easily <strong>take over a minute</strong>, which is a huge <strong>NO</strong> from a user experience perspective.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/455/1*aDlEr5OL0Q6qFRqlVs6TAA.png" /><figcaption>Stolen from a <a href="https://www.reddit.com/r/ProgrammerHumor/comments/9cu51a/shamelessly_stolen_from_xkcd_credit_where_is_due/">Reddit parody</a> of <a href="https://xkcd.com/303/">XKCD 303</a></figcaption></figure><ul><li><strong>Large file. </strong>Unfortunately <strong>most models are stored in files that are quite large</strong> (we’re talking tens, hundreds of MB). As a result this will be slow and memory intensive to load, and increase your app bundle’s size by a large amount.</li><li><strong>Insecure.</strong> Unless you’re using an open source model, you’ll want to keep your AI and pretrained checkpoints relatively under wraps. Unfortunately, when you package your model with your application, not only will your inference code be vulnerable to decompilation, but <strong>your pretrained checkpoint will be inside the bundle and easily stolen</strong>.</li><li><strong>Harder to update. </strong>If you update your model, you have two choices in a client. Either the user is issued an update via a centralized manager i.e. Play Store, App Store, etc. which leads to <strong>frequent large updates</strong> (pretty annoying for the users and a process that could be interrupted or never started depending on their settings), or the application itself runs a fetch for the new model checkpoint and metadata. The latter sounds much better but this also means that <strong>you’ll have to download a 100MB+ file</strong> over the user’s perhaps shaky connection; it will take a while so your app will have to be open at least in the background for the process to finish, and you will incur pretty large internet-out costs (this depends on your cloud).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*suMHOOZBlZ-UB57N2wsfHQ.png" /></figure><ul><li><strong>Lack of trainability. </strong>Training models on new user data provides a level of personalization while improving its accuracy and building up a core, high signal dataset. <strong>Unfortunately most devices lack the computational power to train the model</strong>, and even if they did, it wouldn’t be possible to propagate the effects of training to your server or other devices running the application.</li></ul><p>These drawbacks make deploying and maintaining a large neural network close to impossible on clients, and so we’ll strike this one out as an option for scaling our model.</p><h3>Deploy it as a Cloud endpoint</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/0*McnoXUqaeBbvXdPg.png" /><figcaption><a href="https://xkcd.com/908/">XKCD 908</a>, and <a href="https://xkcd.com/1117/">1117</a> is also relevant</figcaption></figure><p>The <strong>cloud</strong> is a powerful tool for deploying your models at scale. You can spin up environments that are perfectly customized to your needs, containerize your application, and scale horizontally instantly while providing SLA and uptime that rivals big corporations.</p><p>For most TensorFlow models, the <strong>deployment cycle</strong> is the same:</p><ul><li>Freeze your graph into a Protobuf binary</li><li>Adjust your inference code to work with a frozen graph</li><li>Containerize your application</li><li>Add an API layer on top</li></ul><p>The first piece is relatively simple. “Freezing” your graph involves creating a protobuf binary with all the named nodes, weights, architecture, and metadata involved with your checkpoint. This can be done via a variety of tools, the most popular being TF’s own tool to <strong>freeze any graph given an output node name.</strong> You can find out more about this technique and how to go about completing it <a href="https://www.tensorflow.org/guide/extend/model_files#freezing">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/1*coCipR983gR425Dtiltymw.png" /></figure><p>Adjusting your inference code isn’t difficult either; in most cases, your feed_dict will stay the same and the main difference will be the addition of code to load the model and perhaps the specification of an output node.</p><p><strong>Containerization is also pretty trivial — just set up your environment in a Dockerfile (</strong><a href="https://www.tensorflow.org/install/docker"><strong>you can use a TF docker image as your base</strong></a><strong>).</strong> When we start to add an API layer though, things start to get messy. There are generally two ways to go about doing this:</p><ul><li><strong>Deploy scaling containers that run an inference script.</strong> These containers run a script against input, the script starts a Session and runs inference, and outputs something that is piped back to you as a result. <strong>This is extremely problematic;</strong> adding an API layer that manipulates containers and pipes in and out <strong>is not easy or simple</strong> to do with most cloud providers (e.g. AWS has API Gateway but it isn’t nearly as convenient as you would expect), and it is the <strong>least efficient</strong> method you could use. The issue here is that you lose valuable time in <strong>container startup, hardware allocation, session startup, and inference</strong>. If you leave the stdin open and keep piping output instead, you’ll speed up your script but <strong>lose scalability</strong> (you are now hooked to the STDIN of this container, and it also won’t be able to take multiple requests).</li><li><strong>Deploy scaling containers that run an API layer. </strong>This is much more efficient for several reasons although similar in architecture; by <strong>placing the API layer inside the containers</strong>, you mitigate most of the issues proposed earlier. While this takes a little more in resources, it is minimal and does not imply vertical scaling; it <strong>allows each container to stay running,</strong> and since the API in this case is decentralized there is no issue with hooking a specific stdin / stdout to your main request router. This means you <strong>get rid of startup time</strong>, and can <strong>maintain speed and horizontal scaling</strong> easily <strong>while serving multiple requests</strong>. You can centralize your containers with a <strong>load balancer,</strong> and use <a href="https://kubernetes.io/">Kubernetes</a> to guarantee nearly 100% uptime and manage your fleet. It’s simple and effective!</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TN5V3VRS2RrCx5bc0f4f0g.png" /><figcaption>Deploy your fleet!</figcaption></figure><p>The main drawback with decentralizing your API through a container fleet is that the cost can add up to a large sum relatively quickly. This is unfortunately unavoidable with AI, although there are ways to mitigate this a little.</p><ul><li><strong>Reuse your sessions. </strong>Your fleet grows and shrinks proportionate to the load, so your goal here is to minimize the time it takes to run inference so that the container can free up to handle another request. One way to do this is to<strong> reuse the </strong><strong>tf.Session and </strong><strong>tf.Graph by storing them once initialized and passing them around as global variables</strong>; this will remove the time it takes for TF to start up a session and build the graph, which will greatly speed up your inference tasks. This method is effective even on a single container and is widely used as a technique to <strong>minimize resource reallocation and maximize efficiency</strong>.</li><li><strong>Cache input and if possible, output. </strong>The dynamic programming paradigm is most important in AI; by caching input you save the time needed to preprocess it or fetch it from the remote, and by caching output you save the time needed to run inference. This can be done trivially in Python, <strong><em>although you should ask yourself if this is correct for your use case!</em></strong> Often times, your model will be getting better with time, and this would greatly affect your output caching mechanism. In my own systems I like to use what I call the <strong>“80–20” rule.</strong> When a model is below 80% accuracy, I <strong>don’t cache any output.</strong> Once it hits 80%, I start caching and <strong>set the cache to expire at a certain accuracy</strong> (instead of say, at a certain point in time). This way, the output changes as the model becomes more accurate, but there is less tradeoff between performance and speed in this 80–20 <em>mitigated</em> cache.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w_zf0higBNvjCEM1pVrc_A.png" /></figure><ul><li><strong>Use a task queue.</strong> There are often larger and smaller inference tasks that need to be run (in our case, larger and smaller, complex and simple images). It might be better for UX to use a heap queue here, and <strong>process smaller tasks with priority</strong>, so that a user running a trivial step only has to wait for that step rather than for the larger inference of another user to finish first. (If you’re thinking, <em>why don’t I just horizontally scale here?,</em> you could <strong>but it would increase cost</strong>)</li><li><strong>Train your models on a dedicated GPU with a task queue. </strong>Training is a long, difficult task that requires intensive resource usage and makes the model unusable during its duration. If you’re feeding each interaction back into your model for training, <strong>consider running this on a separate server,</strong> perhaps with a GPU. Once training is finished, you can deploy the model (in AWS you could centralize your model repository in S3) to your containers.</li></ul><h3>Conclusion</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ADcTgNpzb1UBVKzfKeYlAA.png" /></figure><p>After much deliberation we’ve come up with an efficient workflow for deploying AI at scale:</p><ul><li><strong>Freeze the graph</strong> and wrap inference under an <strong>API</strong></li><li><strong>Reuse</strong> the Session and Graph, and <strong>cache</strong> input and output</li><li><strong>Containerize</strong> the application using Docker (<strong>including the API layer</strong>)</li><li><strong>Deploy</strong> application at scale with Kubernetes on a cloud of your choice</li><li><strong>Isolate</strong> training from inference</li><li>Develop a task queue to <strong>prioritize</strong> smaller tasks</li></ul><p>Using these techniques, you should be able to deploy with minimal cost and overhead while maximizing speed and efficiency!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/64/0*kGtE8NfGDZwa65uq.png" /></figure><p><strong>About me</strong>: I’m an AI researcher at <a href="https://crane.ai/">Crane.ai </a>(we make apps with AI). I’ve been working with AI over the last couple years and spent a few months this year researching how to scale AI! I hope this was informational, and if you’ve got any questions please feel free to ask away.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2be294368504" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/scaling-ai-2be294368504">What they don’t tell you about scaling AI</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fund your Startup without Money (2018)]]></title>
            <link>https://medium.com/aiko-ai/fund-your-startup-without-money-pt-1-464ccf60613f?source=rss-29dfcc1e4a6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/464ccf60613f</guid>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[tools]]></category>
            <category><![CDATA[fundraising]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[bootstrap]]></category>
            <dc:creator><![CDATA[CUUUPID]]></dc:creator>
            <pubDate>Sat, 28 Jul 2018 15:01:01 GMT</pubDate>
            <atom:updated>2022-05-08T21:40:46.387Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RiCyrIds-sPFh_3ICN9Oug.png" /></figure><p>Over the last year I’ve been working on Aiko AI, an AI email assistant, between classes and in my spare time. <strong>We’ve found it more than difficult to raise money as we’re pre-revenue, but still have a lot of uses for cash.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*cQC_wGVOHLvoQ9nqrnyg1g.gif" /></figure><p>So, we decided to <strong>bootstrap</strong>! For those of you who haven’t heard of bootstrapping, it’s essentially when you <strong>start a venture with little to no capital</strong>. Over the last year we’ve grown a toolkit that has become essential to our operations while keeping our costs at $0, and I’ll try to cover as many of these special free tools as possible!</p><p><strong>The first part of this series is going to focus on team tools,</strong> as those tend to be a huge cash sink for many startups. I’ll go into technical costs in later parts so hit “follow” to stay tuned! Without further ado:</p><h3><strong>1. Slack — the communications king + integrations!</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZejK3D7oasNdljU_.png" /><figcaption>The workplace messaging app that turned into a household name</figcaption></figure><p>Most if not all of you will have heard of <a href="https://slack.com/"><strong>Slack</strong></a> before. It’s the king of workplace messaging <a href="https://www.forbes.com/sites/davidphelan/2018/07/27/slack-buys-hipchat-to-take-on-microsoft-teams-what-it-means/#7cf0cdd970f1">and stands its own in a ring with Microsoft</a>! Slack is definitely the most irreplaceable, valuable tool you can add to your workflow. It’s also FREE, with some limits you likely won’t hit before you can afford the paid version.</p><p>Communication, file sharing, and team management has never been easier — and the integrations add all the more spice! At <a href="https://helloaiko.com/">Aiko AI</a>, we use a few saucy integrations that have magically increased productivity:</p><ul><li><a href="https://slack.github.com/"><strong>Github for Slack</strong></a> — if you use GitHub this is an AMAZING tool that pings a channel each time someone pushes to Git. Makes commit tracking and communication <em>that</em> much easier. For those of you on Bitbucket, <a href="https://get.slack.help/hc/en-us/articles/115005265283-Bitbucket-for-Slack">there’s an integration for that too</a>, which is likely to grow much better now that Slack is buying from Atlassian.</li><li><a href="https://get.slack.help/hc/en-us/articles/205875058-Google-Drive-for-Slack"><strong>Google Drive for Slack</strong></a><strong> </strong>— we share much of our sheets and documents via Google Drive, and this was a no-brainer add to our workplace. It’ll automatically import Google Drive files and save you that extra click!</li><li><a href="https://www.polly.ai/slack-poll"><strong>Polly</strong></a> — polling tool that has grown in popularity. When making team decisions, just use a Polly poll which can be easily configured right in the channel and produces great analytics</li><li><a href="https://www.workast.io/to-do-bot.html"><strong>To-do</strong></a><strong> </strong>— simple todo tracker that can parse dates and works right from chat! Also pings you to make sure you get tasks done, think Asana but without the boards. There’s also Todoist which is great but lacked the simple element we were looking for.</li><li><strong>appear.in</strong> — I’ll cover this one in-depth later. The slack integration is 🔥 and we find it great. We were also recently recommended <a href="https://www.useloom.com/">Loom</a>, another great startup which is a good different (recordings!) and although we haven’t added that to our workflow yet I can see it becoming very useful!</li></ul><p>Scared Slack is spying on you? Try <a href="https://mattermost.com/">Mattermost</a> which is its self hosted sibling.</p><h3>2. Asana, 3. Haplen — task management made easy</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qMEDymrcLTE-HrCT.png" /><figcaption>🔥task management done right!</figcaption></figure><p>You may have also heard of Asana, or perhaps Flow. Asana is a great task management system that allows workplaces to be split into teams, organized into projects, and tasks to be created and assigned in Kanban-like boards. Asana also lets attachments be added, allows subtasks, and progress tracking both in-task and via a status page (great to show investors!).</p><p>As if this wasn’t enough, you can also talk directly inside Asana in “conversations,” and limiting tasks to be assigned to 1 person at a time may seem limiting at first but is a lifesaver when it comes to project ownership. Having a 1:1 ratio between task:owner means the progress of the task can always be traced back and managed by 1 person, which makes things much easier when you need to ask questions, fast!</p><p>The calendar view is great, and the UI is in general an extremely beautiful material-meets-cupertino blend. To top it off, it’s FREE! (with some limits that, again, you won’t hit until you can definitely afford it!)</p><p>There’s also Flow which is nice but only has a trial as a free option. At the top you’ll see I actually mentioned another startup, <a href="http://haplen.com/"><strong>Haplen</strong></a>. I just learned about Haplen a few hours ago and have already fallen in love with it! If you’re more of a Trello person but aren’t ready to shell out cash for it, try Haplen, which is a free alternative and looks awesome:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ys4my2JwdsZAuMGJ.gif" /><figcaption>Haplen beats Trello in our book, anyday!</figcaption></figure><p>It has milestone tracking, grouping, and is WAY easier and more powerful than Trello. Plus it’s totally FREE and has embraced Material Design, which gives it a sleek, startup professional look.</p><p>Haplen also caters to a more directly AGILE workflow than Asana. I’m in the process of moving our development team to Haplen, while the more general and business-side tasks will stay on Asana. Asana + Haplen = 🔥!</p><h3>4. Appear.in ❤️ — video conferencing done RIGHT!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DGsbI4uUsdad3rJQ.png" /></figure><p>You didn’t know you needed <a href="https://appear.in/"><strong>Appear.in</strong></a> until you saw it. But you do:</p><ul><li>getting into a video conference is a LONG process</li><li>Skype always sucks, Google Hangouts always sucks</li><li>bad quality, dropped connections, and trouble finding contacts always</li><li>screenshare options are nonexistent or hard to use, audio does not forward</li><li>incredible latency</li></ul><p><a href="https://appear.in/">Appear.in</a> solves all of the above. Simply share your room link (you don’t even need to sign up, think of a random room name e.g. cat-is-lame and append that to the url → appear.in/cat-is-lame), type that into your browser and you’re good to go!</p><p>No more long process, just drop them an appear.in link. No more sucky interface, appear.in is right in the web and works flawlessly (I think it uses WebRTC but I’m not sure, whatever is under the hood is AMAZING!).</p><p>It’s totally free for rooms up to 4 people, which is fine for most conferences. The paid version is dirt cheap too! In the free version you can “claim” 1 room which is GREAT for branding: we have aiko-ai and I just drop a slack message (I mentioned the integration above, just /appearin &lt;room-name&gt;) to start a meeting. Investors dig it, and it was a lifesaver for conducting remote interviews. It’s like Zoom but free, easier, faster and without an install.</p><p>Appear.in also has screenshare built in, and gloriously — you can filter to choose to share tabs, apps, or your whole screen. There’s no latency as far as I can tell, and it forwards audio so we can all jam out together or watch a video without blasting it through the speakers.</p><p>There’s also a chat on the right side which we mostly use to drop links or take quick notes. We’re hoping they open their API again so <a href="https://helloaiko.com/">Aiko</a> can integrate in!</p><h3>5. Cloud App, 6. Tiny Take — faster screen recordings</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/429/0*3B64oYhb1e2lrWi-.jpg" /></figure><p>When working with products that are constantly evolving, it can be crucial to get GIF or video content of it working. In this situation, TinyTake and CloudApp both come to the rescue. Both allow you to highlight portions of the screen which can then be recorded and saved to GIF or MP4, and even generate links to share!</p><p>I have found <a href="https://tinytake.com/"><strong>TinyTake</strong></a> to be higher resolution in some cases, while <a href="https://www.getcloudapp.com/"><strong>CloudApp</strong></a> is razor fast for recording quick interactions.</p><p>If you’re looking just to take a clip of your screen,<strong> please stop using Snipping Tool or Cmd+Shift+4. </strong>Whether you’re on Mac or Windows, your OS ships with tools that copy to clipboard, and almost every major tool (including everything on this list) supports pasting from clipboard! <strong>On Mac, use Cmd+Control+Shift+4, and on Windows it’s Win+Shift+S. </strong>It might seem like a small change, but you’ll find it makes your life way easier and you’ll never look back! As a bonus it’ll stop your Desktop or Pictures folder from being littered with “Screenshot-##############.png”.</p><p>If you’re trying to record a Terminal session, check out <a href="https://asciinema.org/">Asciinema</a>!</p><p>Both TinyTake and CloudApp have free tiers that are just fine for what they accomplish.</p><h3>7. Canva — Graphics for the rest of us</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/0*NOM2pnfAyl8miQ4H" /></figure><p>Alright so <a href="https://canva.com/"><strong>Canva</strong></a> isn’t totally free by any means. However, even the free features are totally worth it. For those of you who haven’t tried it before, it’s essentially WordPress for graphics!</p><p>Canva comes with hundreds if not thousands if not hundreds of thousands of fliers, covers, documents, templates, etc. — if you’ve got a graphics need, Canva definitely has a solution!</p><p>I use Canva to build my resumes/CVs, and we use it in-house as a design tool for making quick graphics for our affiliates. It saves us countless hours of hunting down Photoshop templates and editing them to suit our needs.</p><p>Canva also stores brand colors which is a great feature, and sports a simple UI that is speedy and beautiful!</p><p>I HIGHLY recommend it — if you haven’t got an in-house designer, Canva will be a godsend and save you hundreds of dollars in design work. (The header was made in exactly 34 seconds using Canva!)</p><p>That’s unfortunately all for Part 1! With the above you should be able to start up a team, set up task tracking and milestones, communicate effectively online and via video, share content with ease, and design graphics to make your business look 🔥!</p><p>These are just a few of my favorites. Here are some more cool tools that didn’t show up above (and may show up in later parts 😉):</p><ul><li><a href="http://www.shakelaw.com/"><strong>Shake</strong></a>, free legal agreements so you can avoid a lawyer <em>(we also have a Stripe Atlas invite link, email me for it!)</em></li><li><a href="https://icons8.com/lunacy"><strong>Lunacy</strong></a>, free alternative to Sketch, for Windows</li><li><a href="https://www.slidescarnival.com/"><strong>SlidesCarnival</strong></a>, not a tool as much as a collection of awesome Google Slides themes to make beautiful decks!</li><li><a href="http://freebbble.com/"><strong>Freebbble</strong></a>, free Dribbble designs!</li><li><a href="https://www.mockupworld.co/all-mockups/"><strong>MockupWorld</strong></a>, free mockups to showcase your designwork/application!</li><li><strong>TellForm</strong>, a free Typeform alternative — there’s a bit of an issue with the main repo right now but <a href="https://github.com/Aiko/aiko-tellform">we maintain an open source fork</a> which has less bugs and is more or less functional!</li><li>+ you can find way more on<strong> </strong><a href="http://startupstash.com/"><strong>Startup Stash</strong></a>!</li></ul><p><a href="https://helloaiko.com/"><strong>Aiko</strong></a><strong> </strong>will also be entering <strong>free</strong> private beta soon and will make your inbox management worries disappear!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=464ccf60613f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aiko-ai/fund-your-startup-without-money-pt-1-464ccf60613f">Fund your Startup without Money (2018)</a> was originally published in <a href="https://medium.com/aiko-ai">Aiko Mail</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Save Lives With 10 Lines of Code: Detecting Parkinson’s with XGBoost]]></title>
            <link>https://medium.com/data-science/detect-parkinsons-with-10-lines-of-code-intro-to-xgboost-51a4bf76b2e6?source=rss-29dfcc1e4a6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/51a4bf76b2e6</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[xgboost]]></category>
            <category><![CDATA[parkinson-disease]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[CUUUPID]]></dc:creator>
            <pubDate>Wed, 04 Apr 2018 05:35:28 GMT</pubDate>
            <atom:updated>2018-05-06T15:31:38.581Z</atom:updated>
            <content:encoded><![CDATA[<p>So you’ve dabbled in data science and have heard the term “XGBoost” thrown around a bit, but don’t know what it is. I’m a big fan of learning by <em>doing</em>, so let’s try using XGBoost on a real-life problem: diagnosing Parkinson’s.</p><p>XGBoost is a popular technique and a neat alternative to traditional regression/neural nets. It stands for E<strong>X</strong>treme <strong>G</strong>radient <strong>Boost</strong>ing, and basically builds something of a decision tree to compute gradients. Here’s a popular graphic from the <a href="http://xgboost.readthedocs.io/en/latest/model.html">XGBoost website</a> as an example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/0*fRF3aDCRtElR38sV.png" /><figcaption>Not so menacing now, huh?</figcaption></figure><p>This sounds simple in practice but can be extremely powerful. Take, for example, Parkinson’s detection: we have several metrics that we can analyze and ultimately we need to diagnose Parkinson’s (classification!). This is a perfect problem for XGBoost (<em>especially since there is a single output so we don’t need to use the MultiOutput wrapper — more on that later)</em>.</p><h4>Let’s Write 10 Lines of Code</h4><p>Let’s get started by gathering some data. By direction of my good friend <a href="https://medium.com/@shlok_85483">Shlok </a>I found an excellently formatted dataset: hop on over to UCI’s ML database and download the dataset for Parkinson’s called parkinsons.data (<a href="https://archive.ics.uci.edu/ml/machine-learning-databases/parkinsons/">here’s a link</a>) (and in case that disappears it’s in <a href="https://github.com/pshah123/parkinsons-AI">this repo</a>). They’re CSVs so we can parse them quick with Pandas:</p><pre>df = pd.read_csv(&#39;parkinsons.data&#39;)</pre><p>Next, we need to get features and labels. The columns are conveniently all numeric except for the first column (name), and the column with the labels is ‘status’ (already 0 or 1). Let’s ignore what they mean for now and blindly analyze them <em>(don’t do this in practice)</em>. This makes it rather convenient for us to quickly get the training data:</p><pre>features = df.loc[:, df.columns != &#39;status&#39;].values[:, 1:]<br>labels = df.loc[:, &#39;status&#39;].values</pre><p>Next, we’ll need to scale our features so they’re between -1 and 1 so they’re normalized. We can do this using sklearn’s brilliant MinMaxScaler:</p><pre>scaler = MinMaxScaler((-1, 1))<br>X = scaler.fit_transform(features)</pre><p>We’re at 5 lines so far. Next, let’s split it into training and testing data so we can protect against overfitting. There aren’t many datapoints so let’s split 14% off into test data, this time using sklearn’s train_test_split convenience function:</p><pre>X_r, X_s, Y_r, Y_s = train_test_split(X, labels, test_size=0.14)</pre><p>Then we use xgboost’s XGBClassifier which is already built for classification and provided through the xgboost module (pip install xgboost):</p><pre>model = XGBClassifier()<br>model.fit(X_r, Y_r)</pre><p>That should take a split second and then finish building the tree. <em>Isn’t it great that we can converge without spending hours training?</em></p><p>We’re at 8 lines now — cutting it close! Let’s wrap this deal up by evaluating our model against the test set from earlier, with the accuracy_score function from sklearn:</p><pre>Y_hat = [round(yhat) <strong>for</strong> yhat <strong>in</strong> model.predict(X_test)]<br>print(accuracy_score(Y_test, Y_hat))</pre><p>You should see an accuracy in the high 90’s (~96.42% on the test set!). This is already amazing since the original paper from 2007 cites classification accuracy of 91.8 ± 2.0%, and other papers from 2016 cite accuracies of <a href="https://www.sciencedirect.com/science/article/pii/S1386505616300326">96.4% (SVM)</a> and <a href="https://arxiv.org/pdf/1610.08250.pdf">97% with tuning (Boosted LogReg)</a>; with some tuning our model could far exceed state-of-the-art methods!</p><p>And that’s it! 10 lines of code and you’ve trained a full XGBoosting classifier for Parkinson’s. You can find the full source code for this alongside another model for the UDPRS data in the Train.ipynb Jupyter notebook <a href="https://github.com/pshah123/parkinsons-AI">here</a>.</p><h4>Afterthoughts</h4><p>XGBoosting is extremely powerful and can definitely be a useful tool for your next project! This goes much deeper though — for multiple outputs you’d need a MultiOutput model (SciKit Learn has a great wrapper for this), and for getting more accurate you’ll need to fine tune your XGBoost model. In the Jupyter notebook before (and in the repo linked below) I explore using Keras for the continuous data, but the MultiOutput wrapper from sklearn can serve almost as a drop-in replacement for the Keras model. If you’d like a more advanced tutorial on fine-tuning XGBoost in particular, check out <a href="https://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/">this great post from 2016</a>!</p><p><a href="https://github.com/pshah123/parkinsons-AI">pshah123/parkinsons-AI</a></p><p>Like this, and more? At <a href="https://helloaiko.com/">Aiko AI</a>, we love open source projects and exploring and visualizing data!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=51a4bf76b2e6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/detect-parkinsons-with-10-lines-of-code-intro-to-xgboost-51a4bf76b2e6">Save Lives With 10 Lines of Code: Detecting Parkinson’s with XGBoost</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Hack GitHub (kind of)]]></title>
            <link>https://medium.com/hackernoon/how-to-hack-github-kind-of-12b08a46d02e?source=rss-29dfcc1e4a6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/12b08a46d02e</guid>
            <category><![CDATA[github]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[hacking]]></category>
            <dc:creator><![CDATA[CUUUPID]]></dc:creator>
            <pubDate>Mon, 19 Mar 2018 17:41:00 GMT</pubDate>
            <atom:updated>2019-12-21T18:21:58.439Z</atom:updated>
            <content:encoded><![CDATA[<p>A commit a day keeps the bugs away! But we’ll be doing 100 commits a day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*cWchH1zz76wdYIg18vw_7w.png" /></figure><p>It’s no secret that GitHub’s contribution chart isn’t very secure. In fact, by <em>pushing a repository with pre-configured commit history</em>, you can gain the commits as contributions.</p><p>I love coding. I really do. And I really love open source. When I started out, open source was my favorite thing to do: build software, release it for everyone, and contribute to others’ projects.</p><p>It’s something I still love to do, but it’s something I’ve been forced to sacrifice. Recently, I started working fulltime on Aiko AI, a startup aiming to bring the benefits of advances in AI to the general public by integrating them into everyday products. We’re behind popular products like <a href="https://helloaiko.com/meet">Aiko Meet</a> and <a href="https://helloaiko.com/mail">Aiko Mail</a> and our recent success has meant that I have had less and less time to do open source work.</p><p><a href="https://helloaiko.com/">Aiko AI | The New Inbox, Powered by AI</a></p><p>Unfortunately, this means I had to choose between my passion for Aiko and my love for open source. I’ve continued to work on open source projects every now and then, but Aiko has taken priority, and that means I can’t commit with the frequency I used to.</p><p>And that means my contribution chart will dry up — unless we fill it up with commits 😉</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6HSqDdKTQY-pl4H_" /><figcaption>Sneak peek of my contribs chart in 2 months</figcaption></figure><p>For those of you who don’t frequent GitHub often, the contribution chart tracks your commits on a daily basis to see your coding activity/cointributions to open source over time. It is simultaneously the <strong>weirdest of all flexes </strong>and a nice little trophy to show for your open source efforts.</p><p>So much so that even some employers will look at your GitHub contribution chart to gain an idea of just how much of an avid coder you really are. Of course this isn’t really a reflection of your coding ability; you may not use GitHub often, or you may be developing mostly closed source. Perhaps you’re in a role where you commit infrequently but with large features/heavy loads. Or perhaps you look to this chart and feel a sense of insecurity as you think, <em>all my friends have more GitHub commits!</em></p><p>Whether it’s necessity, curiosity, or impostor syndrome that motivates you, there’s an easy fix to this — just get a bunch of open source commits!</p><p>Of course this would take time and you can’t go back and edit your history — or can you? I’m going to be showing you how to hack GitHub’s contribution chart, and get that green pasture you yearn for so much.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dlK97BQtyrTMUc1w" /><figcaption>Pictured: the hacker known as _____</figcaption></figure><p>Before I get started I’ll point you towards <a href="https://github.com/gelstudios/gitfiti">Gitfiti</a>, another project based on the same system that lets you draw art on your GitHub contributions.</p><p>Rather than art, we’ll focus on <em>commit history </em>to gain fraudulent commits! I’ll try my best to explain what I can about bash, but this assumes some familiarity with terminal and git— if you’re not comfortable with a CLI, I recommend you check out <a href="http://www.bash.academy/">Bash Academy</a> before diving into this.</p><p>Just want the script? <a href="https://gist.githubusercontent.com/pshah123/0513d98556021d738c0cb72be0a63615/raw/">Here it is.</a></p><p>Don’t want to code? Check out <a href="https://gitgardener.com/">Github Gardener</a>, a free to use website that’ll help you out!</p><h3>How It Works</h3><p>Exploiting <strong>commit history</strong> we can “author” commits using git’s commit timestamps.</p><p>Let’s start by authoring one commit at a date of our choice. To accomplish this we’ll need to change two environment variables that git uses to set the commit timestamp, GIT_COMMITTER_DATE and GIT_AUTHOR_DATE. These two variables hold standard YY-MM-DD HH:MM:SS ISO format timestamps. We’ll also commit on that timestamp to cover all the bases — this way we can be completely sure that there is no way to differentiate our fake commit from one actually at that point in time.</p><pre>export Y = 2000<br>export M = 12<br>export D = 20<br>export GIT_COMMITTER_DATE=&quot;$Y-$M-$D 12:00:00&quot;   <br>export GIT_AUTHOR_DATE=&quot;$Y-$M-$D 12:00:00&quot;   <br>git commit --date=&quot;$Y-$M-$D 12:00:00&quot; -m &quot;Committed on $M $D $Y&quot;</pre><p>This above snippet will “author” a commit from 12/20/2000. This can also be extended to “author” a commit from, say, 1/1/1970; way before you could have possibly coded this! As a result, your “git” history is vulnerable (and not, in fact, a “history”, it’s more a collection of pre-authored structures).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/990/0*EPM0QmUz0e8dD3WK.jpg" /><figcaption>Write your own history! Didn’t turn out so great for Churchill though…</figcaption></figure><p>We can then extend this to author a commit history every day from a set date. This is trivially achieved using a bash for loop, with structure:</p><pre>for VAR_NAME in {start..end}</pre><p>The indented block following will be able to use $VAR_NAME as a variable in its execution. As a result, bash for loops become ideal for us as we can iterate through years, months, and days, to author fake commit histories.</p><h3>Let’s Make A Script</h3><p><em>The final version of this script is available as a Gist </em><a href="https://gist.githubusercontent.com/pshah123/0513d98556021d738c0cb72be0a63615"><em>here</em></a><em>.</em></p><p>We can achieve this “exploit” by<em> looping through </em>years, months, and days in that order:</p><pre>for Y in {1999..2018}<br>    for M in {01..12}<br>        for D in {01..28}</pre><p>We loop through every year from 1999 to 2018 (I chose 1999 as a guess pre-2000). We then loop through each month, and finally each day. Note that we only loop from 1 to 28, as February can be a 28 day month! We could loop through every day if we wanted to, but it wouldn’t be valid for February; in this case you can use a conditional to select the days. Another approach I’ considered is using Python to set the year, month, and day to weekdays in order to choose only working days — whichever you pick is up to you! I’ll stick with 1–28 as this works the best from a speed perspective (it can take some time to author this many commits).</p><p>We also need to create a new file each time:</p><pre>echo &quot;Committed on $M/$D/$Y&quot; &gt; commit.md</pre><p>Here’s a completed script to just create files and commit them en masse:</p><pre>for Y in {1999..2018}<br>do<br>  mkdir $Y<br>  for M in {01..12}<br>  do<br>    cd $Y<br>    mkdir $M<br>    cd ../<br>    for D in {01..28}<br>    do<br>      cd $Y/$M<br>      mkdir $D<br>      cd $D<br>      echo &quot;Committed on $M/$D/$Y&quot; &gt; commit.md<br>      cd ../../../<br>      export GIT_COMMITTER_DATE=&quot;$Y-$M-$D 12:00:00&quot;<br>      export GIT_AUTHOR_DATE=&quot;$Y-$M-$D 12:00:00&quot;<br>      git add $Y/$M/$D/commit.md -f<br>      git commit --date=&quot;$Y-$M-$D 12:00:00&quot; -m &quot;$M $D $Y&quot;<br>      git push origin master<br>    done<br>  done<br>done</pre><p>This will then author commits from 1999 to 2018. You can execute this from any bash prompt--I recommend placing it in an executable bash file and running that.</p><p>But this is clearly extremely slow! How can we make this faster?</p><ol><li>First off, unless you have nested git repos, we<strong> don’t need to traverse back up </strong>the directory tree each time we commit.</li><li>It also turns out that the “push” to GitHub doesn’t take into account the timestamps present (hence allowing you to push your old local git repos to GitHub), so we can just <strong>push at the end</strong> of our script.</li><li>Lastly, while not really a speed enhancement, adding support for days 29 through 31 doesn’t really affect our performance anymore with the above improvements, and GitHub will sanitize extra days automatically.</li></ol><p>This increases the speed tenfold, and we can reach greater heights by pushing at the end.</p><pre>for Y in {1999..2018}<br>do<br>  mkdir $Y<br>  cd $Y<br>  ...<br>        git add commit.md -f<br>        git commit --date=&quot;$Y-$M-$D 12:$i:00&quot; -m &quot;$i on $M $D $Y&quot;<br>      done<br>      cd ../<br>    done<br>    cd ../<br>  done<br>  cd ../<br>done<br>git push origin master</pre><p>Implementing these changes, along with <strong>removing the folders at the end </strong>for cleanup, we arrive with the final script:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/370cf08307ef279a63e8a1c4d1c39e15/href">https://medium.com/media/370cf08307ef279a63e8a1c4d1c39e15/href</a></iframe><p>You’ll notice I’ve introduced another bash loop with variable i — this controls <strong>how many commits</strong> are made each day, and can be used with this much, much faster script. If you get an “invalid” date format error, please check you’re using a date that exists after computers (the earliest date you can do is January 1, 1970), and that your version of bash is new enough to support leading zeroes in for loops.</p><p>The neatest part about this is that it authors commits in the <strong>FUTURE. <br>This means that in the future you will have a commit every day!</strong></p><p>I’ve put the completed script <a href="https://gist.githubusercontent.com/pshah123/0513d98556021d738c0cb72be0a63615">here</a>. (shortlink: <a href="http://git.io/vxnon">git.io/vxnon</a>)</p><p>Example Run-through:</p><pre>priansh@laptop:~$ cd ~/Documents/github<br>priansh@laptop:~$ mkdir gitsploit # make the repo folder anywhere<br>priansh@laptop:~$ cd gitsploit &amp;&amp; git init<br>priansh@laptop:~$ # Next we&#39;ll set up tracking a GitHub repo<br>priansh@laptop:~$ git remote add origin &lt;YOUR GITHUB REPO LINK HERE&gt;<br>priansh@laptop:~$ echo &quot;Hello World&quot; &gt; README.md &amp;&amp; git add --all<br>priansh@laptop:~$ git commit -m &quot;init&quot; &amp;&amp; git push -u origin master<br>priansh@laptop:~$ wget https://git.io/vxnon -O a_million_commits.sh<br>priansh@laptop:~$ chmod a+x a_million_commits.sh # make executable<br>priansh@laptop:~$ ./a_million_commits.sh</pre><h3>Post-Mortem</h3><p>I’ll mention that GitHub can also be further exploited using bots (as they have no captcha!) and used for stars, follows, and more, although I won’t cover that in this quick article (nor do I encourage this as it borders on malicious).</p><p>Just a quick pic to prove this works:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/1*m8FH-hon6wWBMa42A5TY4g.png" /><figcaption>20k commits!</figcaption></figure><p>You can delete the repo you’re committing to in order to remove the fake history, or make it private and set contribution settings to public only, to accomplish the same effect as removing the fakes from your chart. (You can also overwrite the branch/commit history entirely.)</p><p>While we’re not exploiting GitHub in a malicious way, this is still somewhat significant as some companies check your contribution count in the hiring process — meaning this could (potentially) affect your chances of landing a job. It will <em>also</em> make you the coolest kid at recess. Happy “hacking!”</p><p><a href="https://github.com/pshah123">pshah123 (Priansh Shah)</a></p><p>Like this, and more? At <a href="https://helloaiko.com/">Aiko AI</a>, we love open source projects and exploring and visualizing data!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12b08a46d02e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/how-to-hack-github-kind-of-12b08a46d02e">How to Hack GitHub (kind of)</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Masquerade: Take the pain out of blogging.]]></title>
            <link>https://medium.com/aiko-ai/a-couple-weeks-ago-i-came-across-several-web-blogs-424e656c5197?source=rss-29dfcc1e4a6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/424e656c5197</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[blog]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[cms]]></category>
            <category><![CDATA[blogging]]></category>
            <dc:creator><![CDATA[CUUUPID]]></dc:creator>
            <pubDate>Sun, 19 Nov 2017 07:19:48 GMT</pubDate>
            <atom:updated>2020-01-14T17:23:19.637Z</atom:updated>
            <content:encoded><![CDATA[<h4>Simple Templating for Static Sites</h4><p>Here I’ll discuss how we built Masquerade, a really simple static template tool that we use as a low level CMS. Using Masquerade we can generate hundreds of pages detailing information, features, and our company blog, all without the pain of manually editing HTML files and large refactors on every minor update to our footer.</p><p>This article was originally written in 2017.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Uuvrxn92ng7ErJjv" /><figcaption>Photo by <a href="https://unsplash.com/@plushdesignstudio?utm_source=medium&amp;utm_medium=referral">Plush Design Studio</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A couple weeks ago, I came across several web blogs. They all looked pretty suave, with minimalistic structure and a very light reading theme. However, the bulk of modern blogs remain on CMS’s like Jekyll, WordPress, Wix, etc., all of which (in my opinion) SUCK.</p><p>And I understand this is an unpopular opinion--don’t get me wrong here, if I’ve got a simple website to design for a quick buck with a non-technical client, I’ll WordPress it in half an hour and be around to collect my check. But I can’t help but feel that something is very much lacking from these engines--or rather, three things are lacking.</p><ol><li><strong>Speed.</strong> The beauty of static sites usually lies in their speed, but content managers like WordPress rely on bulky interfaces sourced in PHP. As a result, these sites are extremely heavy loaders, pointing to several plugins and dependencies which point to other dependencies up and down the chart.</li><li><strong>Ease of use.</strong> For a quick and dirty setup, WordPress can actually be extremely painful. It can take hours to setup if something goes wrong and hours more to move or recover it. Jekyll itself has so many setup issues that I find myself wanting to punch my desk. On top of this, posting is ridden with a billion horrible plugins and themes all fighting with each other to show you dashboard notifications. Not to mention your site is constantly getting attacked by ukranian bots.</li><li><strong>Focus on content.</strong> All of these existing CMS’s fail on content BIG TIME. A focus on content would create a very light weight content editor with ease of use and simple formatting. Instead, blog customization has ushered in an age of unnecessary and annoying bulk, with content editors that don’t work for the user.</li></ol><p>So to address these three issues, I built an ultra light weight blog CMS, complete with a templating engine and full generation. I call it Masquerade, and to develop it I focused on three solutions:</p><ol><li><strong>Speed.</strong> To make my system faster, I integrated it with version control and setup a python script. Python is usually natively supported by most hosts, and can very easily and efficiently work as a templating engine. By generating fully static files, and replacing the bulky nonsense of plugins and themes with simple, shared CSS/JS, Masquerade achieves razor fast load speeds.</li><li><strong>Ease of use.</strong> To make Masquerade as easy as possible to use, I integrated it directly into version control with a pre-commit hook. I simply add content and then commit, and Masquerade handles the rest: generating an index, handling 404, creating blog content, rendering excerpts, you name it. And this brings me to our last focus,</li><li><strong>Focus on content.</strong> For this, I took a page out of GitHub and Jekyll’s book. Essentially, GitHub and most coder friendly tools use markdown so the coder can quickly develop documentation and instead focus on the actual documentation rather than how it looks. I integrated my own flavor of markdown, taking in the title, timestamp, tags, and excerpt in the beginning of the file (I call this the metadata), and then rendering simple markup (text formatting, images) in the coming body of text. This allows the blogger to focus on simply getting the content straight, while Masquerade handles all the heavy lifting.</li></ol><p>Essentially, Masquerade takes in simple blog files:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LLB_H4k4dJqPPmOZyJF8nw.png" /></figure><p>Then, once you commit, it begins to build your blog. It generates all the blog content pages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hz0C-9AdUEsVaQmkWazPkw.png" /></figure><p>And also generates the index file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HEnL2_e2iGJo_kNSYvAa2A.png" /></figure><p>In addition, Masquerade comes with a no-cache flag for development, but still operates quickly. Masquerade also integrated favicons and mobile compatibility. On mobile, the interface is actually accessible through reading list as well:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F4YGQXg3fvlFx7XRp-ZSWg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M4oR58yVfOBy4OYIZliSGw.png" /></figure><p>And thus, Masquerade was born. It has a few shortcomings (I haven’t finished creating prettier 404, comingsoon, etc pages, or building the tags page). In the future, I hope to integrate link detection and other Markdown features, while providing different themes in CSS format. Masquerade is, of course, entirely open source on my repo and hosted with GitHub pages. Check out the full demo <a href="http://masq.gq/">here</a>.</p><p><a href="http://www.github.com/pshah123/masquerade-web/">pshah123/masquerade-web</a></p><p>Like this, and more? At <a href="https://helloaiko.com/">Aiko AI</a>, we love open source projects and exploring and visualizing data!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=424e656c5197" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aiko-ai/a-couple-weeks-ago-i-came-across-several-web-blogs-424e656c5197">Masquerade: Take the pain out of blogging.</a> was originally published in <a href="https://medium.com/aiko-ai">Aiko Mail</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting College Acceptance with AI]]></title>
            <link>https://medium.com/data-science/predicting-college-acceptance-with-ai-6d8abd702385?source=rss-29dfcc1e4a6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d8abd702385</guid>
            <category><![CDATA[predictive-analytics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[education]]></category>
            <dc:creator><![CDATA[CUUUPID]]></dc:creator>
            <pubDate>Thu, 12 Oct 2017 17:46:01 GMT</pubDate>
            <atom:updated>2019-02-27T22:51:36.043Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*zOmMd0Yb0EkZrfQxXfOydw.jpeg" /></figure><p>Around this time last year, I was busy rattling away on my keyboard, applying to colleges around the globe. As I filled out the seemingly endless application in front of me, it occurred to me that this was wholly inefficient and risky.</p><p>Each year, over 2 million college applications are filed, and each of them comes with a certain element of <em>chance. </em>Even students of the highest academic caliber apply with an element of randomness, and often face difficult situations as a result — the intended meritocracy inherent in college admissions gives way to uncertainty, doubt, and anxiety.</p><p>Of course, many factors influence admission, but it’s no secret that two factors heavily weigh in on determining acceptance: GPA and SAT/ACT scores. While other factors are certainly taken into consideration, it is statistically and practically undeniable that these two metrics hold incredible importance to a student’s application — although no one truly <em>knows</em> how these are judged and filtered by colleges.</p><p>But at the end of the day, these two metrics are <em>metrics</em>: numbers that can be easily assessed using data science to analyze trends and relationships. And so, being extremely impatient and as anxious as I am, I decided to attempt to <strong>predict college acceptance</strong> rather than wait for decisions to roll out<strong>.</strong> And while there are thousands of statistical methods to analyze the trends between GPA/test scores and application outcome, I chose a recently popular predictive method: Machine Learning.</p><p>Machine Learning is a broad field with many solutions and applications, but I honed in particularly on Artificial Neural Networks. Seeing as my inputs were entirely numeric and my output could be one-hot encoded into a binary value (0 for rejected, 1 for accepted), an ANN worked out quite nicely.</p><p>After implementing an architecture in <a href="http://www.tensorflow.org/">Tensorflow</a>, I trained my network on a dataset gathered for Carnegie Mellon in particular (just an example, with some 90 datapoints). After approximately 150,000 iterations (~1 minute on a GeForce 1060 GPU), I was able to reach an accuracy between <strong>75–80%</strong>. While this may not seem that accurate, it is significant enough to draw some conclusions, and definitely functions better than arbitrary linear models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zA2h3O0nagpr_RmOGZEenw.png" /><figcaption>Decreasing Loss over 150,000 steps (faded blue line is cross validation, no overfitting + room for more training)</figcaption></figure><p>From the commandline I was able to gather some pretty decent results on predicting college acceptance:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*Yd9HCekZzIv-u3s9yCD9KA.png" /><figcaption>Using a randomly high GPA and high SAT.</figcaption></figure><p>(Beautiful logging courtesy of <a href="https://github.com/pshah123/console-logging">console-logging</a> built from source)</p><p>And for ease of use I designed a simple GUI with Bootstrap 3:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kT2IrWzPfdaQPCpPlzzOzg.png" /></figure><p>All in all, it proved somewhat useful in not only calming my anxiety but filtering my applications to limit them to which schools I was likely to get into. <em>Ironically, it even predicted my rejection from Carnegie Mellon!</em></p><p>The full source of the project is available on Github <a href="https://github.com/pshah123/ChanceyNN">in this repo.</a></p><p><a href="https://github.com/pshah123/ChanceyNN">pshah123/ChanceyNN</a></p><p>Like this, and more? At <a href="https://helloaiko.com/">Aiko AI</a>, we love open source projects and exploring and visualizing data!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d8abd702385" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/predicting-college-acceptance-with-ai-6d8abd702385">Predicting College Acceptance with AI</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>