<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by George Williams on Medium]]></title>
        <description><![CDATA[Stories by George Williams on Medium]]></description>
        <link>https://medium.com/@georgewilliams?source=rss-d62fa8861970------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*YKO3dA4c0pUBOHV4Uly9hA.jpeg</url>
            <title>Stories by George Williams on Medium</title>
            <link>https://medium.com/@georgewilliams?source=rss-d62fa8861970------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 16:50:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@georgewilliams/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Is Modular About to Crack Nvidia’s Software Moat?]]></title>
            <link>https://georgewilliams.medium.com/is-modular-about-to-crack-nvidias-software-moat-3a8048d335fd?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/3a8048d335fd</guid>
            <category><![CDATA[nvidia]]></category>
            <category><![CDATA[hardware]]></category>
            <category><![CDATA[cuda]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[gpu]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Thu, 13 Aug 2026 22:32:50 GMT</pubDate>
            <atom:updated>2026-08-13T22:32:50.503Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ugZM-TGLFgQ9DGF00Xgk3g.jpeg" /></figure><p>Nvidia’s dominance in AI has never really been about the chips. It’s about CUDA — the software layer that made Nvidia GPUs the path of least resistance for anyone building AI systems. That software moat is a big part of why Nvidia commands the margins it does, and why so much of the AI stack has been built, implicitly or explicitly, around one vendor’s hardware.</p><p>I think that moat is about to get its most serious test yet, and the company doing the testing is Modular.</p><p><strong>The pitch: a platform for heterogeneous hardware</strong></p><p>Modular doesn’t describe itself as an Nvidia alternative. It describes itself as a platform for <em>heterogeneous</em> hardware — software that lets teams write once and run efficiently across GPUs, CPUs, NPUs, and custom accelerators, without hand-tuning for each one. That framing matters, because I don’t think the next decade of the data center is going to be dominated by a single hardware type the way GPUs have dominated the last one.</p><p>Data centers are increasingly going to be host to a genuine mix: GPUs for training, purpose-built inference chips, NPUs at the edge, and specialized silicon for particular workloads. Whoever can orchestrate cleanly <em>across</em> that heterogeneity — rather than optimizing for just one corner of it — has a real shot at becoming the default software layer for AI infrastructure. That’s the seat Modular is trying to take.</p><p><strong>The Qualcomm acquisition changes the calculus</strong></p><p>Qualcomm completed its acquisition of Modular at the end of July 2026, in a deal reported to be worth somewhere in the $3–4 billion range depending on when you mark the valuation. Two things about this deal stand out to me:</p><p><strong>Funding and staying power.</strong> Modular now has the balance sheet of a company doing tens of billions in annual revenue behind it, rather than the runway of a venture-backed startup. That’s a meaningful edge if this is going to be a multi-year war for developer mindshare.</p><p><strong>Edge and physical AI.</strong> Qualcomm’s strength has always been efficient compute at the edge — mobile, automotive, industrial. Pairing that with Modular’s software gives the combined company a credible story not just for the data center, but for robotics and other physical-AI applications where inference has to happen on-device. That’s territory Nvidia’s data-center-first stack doesn’t cover nearly as well.</p><p>Worth noting, in fairness to skeptics: Modular is now owned by a chipmaker, and some in the developer community have already raised the obvious question — can a vendor-neutral platform stay vendor-neutral once it’s owned by a vendor? Qualcomm has an obvious incentive to keep Mojo and MAX running well everywhere, since a language that only targets Qualcomm silicon isn’t one anyone adopts. But it’s a real tension worth watching, not just a talking point. I recently attended Qualcomm’s AMA on their Discord developer channel and leadership guaranteed they wouldn’t change Modular’s original plans to be hardware agnostic. We’ll see I guess if that actual happens!</p><p><strong>The pedigree</strong></p><p>It also matters who’s behind this. Modular’s co-founder and CEO, Chris Lattner, created LLVM and Swift — two of the more consequential pieces of developer infrastructure of the last two decades. Building a compiler and runtime layer that has to work across wildly different hardware targets is exactly the kind of problem LLVM was built to solve, just applied to AI workloads instead of general-purpose compilation. That’s a rare skill set, and it’s arguably the reason Modular is a credible contender for this role at all rather than just another AI infra startup.</p><p><strong>Open source as the growth lever</strong></p><p>Modular has publicly committed to open-sourcing its compiler and toolchain sometime in 2026 (the standard library has already been open source for a while). That hasn’t happened yet, and I want to be careful not to overstate what’s confirmed — but the timing lines up in an interesting way: Modular’s ModCon developer conference is happening in San Francisco next week, with a livestream for anyone who can’t attend in person, and there’s real speculation that this is where a fuller open-source release could land.</p><p>If it does, I think it’s the single biggest lever Modular has to actually pull this off. An open compiler means the broader AI ecosystem — not just Modular’s own engineers — can extend support to new hardware targets. That’s how you out-scale a proprietary moat: not by out-building it feature for feature, but by turning the entire developer community into your hardware-support team.</p><p>I’m genuinely looking forward to the ModCon livestream next week to see how much of this actually materializes.</p><p><strong>A quick disclosure</strong></p><p>I don’t work for Modular or Qualcomm, and I have no financial relationship with either. I’m writing this as someone who’s been wanting a truly open, hardware-agnostic software layer for AI — one that works from edge devices to the data center to (eventually) space-based compute — for a while now. Modular might be the closest thing to that I’ve seen. I could be wrong about how this plays out, but it’s the development in AI infrastructure I’m most curious about right now. I’ll be writing more about how I will be contributing to Modular to unify the fractured and chaotic hardware ecosystem in aerospace for space computing and AI workloads, so watch out for upcoming blogs!</p><p><strong>References</strong></p><ol><li>Qualcomm Incorporated, <a href="https://investor.qualcomm.com/news-events/press-releases/news-details/2026/Qualcomm-to-Acquire-Modular/default.aspx">“Qualcomm to Acquire Modular”</a>, Investor Relations press release, June 24, 2026.</li><li>Qualcomm Incorporated / PR Newswire, <a href="https://www.prnewswire.com/news-releases/qualcomm-completes-acquisition-of-modular-302837286.html">“Qualcomm Completes Acquisition of Modular”</a>, July 29, 2026.</li><li>Bloomberg, <a href="https://www.bloomberg.com/news/articles/2026-06-22/qualcomm-is-said-to-near-deal-for-ai-chip-startup-modular">“Qualcomm Is Said to Near Deal for AI Software Firm Modular”</a>, June 22, 2026.</li><li>CNBC, <a href="https://www.cnbc.com/2026/06/24/qualcomm-ai-chip-modular-software.html">“Qualcomm inks deal for AI startup Modular to bolster software stack, data center build-out”</a>, June 24, 2026.</li><li>Manufacturing Dive, <a href="https://www.manufacturingdive.com/news/qualcomm-acquires-modular-AI-data-centers-semiconductors/823703/">“Qualcomm Technologies agrees to acquire Modular for $3.9B”</a>, June 24, 2026.</li><li>HPCwire / AIwire, <a href="https://www.hpcwire.com/aiwire/2026/06/25/qualcomm-strengthens-data-center-ai-push-with-modular-acquisition/">“Qualcomm Strengthens Data Center AI Push with Modular Acquisition”</a>, June 25, 2026.</li><li>TechTimes, <a href="https://www.techtimes.com/articles/324051/20260812/mojo-hits-10-ai-systems-language-locks-stable-api-ending-three-years-churn.htm">“Mojo Hits 1.0: AI Systems Language Locks In Stable API, Ending Three Years of Churn”</a>, August 12, 2026.</li><li>H2S Media, <a href="https://www.how2shout.com/news/mojo-1-0-release-modular-qualcomm.html">“Mojo Hits 1.0 — Two Weeks After Qualcomm Bought the Company Behind It”</a>, August 12, 2026.</li><li>Modular, <a href="https://www.modular.com/modcon">ModCon 2026: Compute Unlocked</a> — official conference page, San Francisco, August 18, 2026.</li><li>Modular, <a href="https://forum.modular.com/t/qualcomm-completes-acquisition-of-modular/3353">“Qualcomm Completes Acquisition of Modular”</a>, Official Announcements, Modular community forum.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3a8048d335fd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Space Is Freezing. So Why Is Cooling the Hardest Part?]]></title>
            <link>https://georgewilliams.medium.com/space-is-freezing-so-why-is-cooling-the-hardest-part-fe9d94cfa362?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe9d94cfa362</guid>
            <category><![CDATA[data-center]]></category>
            <category><![CDATA[thermal-management]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[future-of-computing]]></category>
            <category><![CDATA[space]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Tue, 11 Aug 2026 23:54:57 GMT</pubDate>
            <atom:updated>2026-08-11T23:54:57.880Z</atom:updated>
            <content:encoded><![CDATA[<h4>A deep dive into the first myth of data centers in space, and why “it’s cold up there, so cooling is free” gets the physics exactly backwards.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BrFwoSvrqN7fU4uJ3nbeeg.png" /></figure><p>In the first post in this series, <a href="https://medium.com/@georgewilliams/data-centers-in-space-a-reality-check-1aa93278626f">Data Centers in Space: A Reality Check</a>, we took a run at four comfortable assumptions behind the sudden rush to put data centers in orbit. We promised to come back and dig into each one properly. So welcome to the first deep dive, on the myth we find most delightfully wrong:</p><blockquote>“Space is cold, so cooling is basically free.”</blockquote><p>It’s the most intuitive thing in the world. Space is a few degrees above absolute zero. Your servers run hot. Put the servers in the cold place, problem solved. Right?</p><p>It is, in fact, the opposite. Cooling is arguably the <em>single hardest</em> engineering problem for an orbital data center. It’s harder than radiation, and harder than launch. Let’s walk through why, with numbers.</p><h3>First: why cooling is a make-or-break problem at all</h3><p>Every watt of electricity a chip consumes comes back out as heat. All of it. A processor is, thermodynamically, a very expensive space heater that happens to do math on the side. And modern AI chips consume a <em>lot</em> of watts in a <em>very</em> small area.</p><p>Consider NVIDIA’s current flagship. A single GB200 “superchip” dissipates up to about <strong>1,200 watts</strong>, and a full GB200 NVL72 rack draws on the order of <strong>120–140 kilowatts</strong>. That’s why NVIDIA doesn’t just recommend liquid cooling for it; it <em>requires</em> it. The reason is heat flux, meaning watts per unit area. A Blackwell GPU die runs at a punishing <strong>500–600 W/cm²</strong>. Old-fashioned air cooling tops out around <strong>5–15 W/cm²</strong>, and a fan-cooled rack maxes out near 8–25 kW before the physics simply gives up. AI blew past that ceiling years ago.</p><p>Why does any of this matter so much? Because silicon has a temperature limit. Push a junction much past ~90–100 °C and performance throttles, error rates climb, and lifetime collapses. Keeping the chip <em>cool enough to function</em> isn’t a nice-to-have; it’s the whole ballgame. It’s also expensive on the ground: historically, a third or more of a data center’s total energy has gone not to computing but to <em>cooling the computing</em>.</p><p><strong>Now scale that up.</strong> A one-gigawatt AI data center, the size everyone is racing toward, must continuously get rid of roughly <strong>one gigawatt of heat</strong>. Hold that number.</p><h3>How we do it on Earth (spoiler: we cheat)</h3><p>On the ground, we have a magnificent, free, invisible coolant surrounding everything: <strong>air</strong>. And behind the air, <strong>water</strong>.</p><p>The heat path in a typical facility looks like a relay race, and for most of the industry’s history every leg has run on <strong>air</strong>. Heat moves from the die into a metal heatsink, and a fan blows air across its fins, carrying the heat away as warm exhaust. The room’s air handlers (the big units lining a data hall) pull that hot air in and pass the heat to a building chiller loop, which finally dumps it outdoors, very often through a cooling tower that rejects heat by evaporating water. Every leg of that relay leans on <strong>convection</strong>: the bulk motion of a fluid, here mostly air, physically carrying heat away and replacing itself with something cooler. Convection is a workhorse. It’s why a desk fan works, why your car has a radiator, and why a data center can shed hundreds of megawatts. (Direct-to-chip liquid cooling is spreading fast for the densest AI racks, which we’ll get to shortly, but the overwhelming majority of servers running today are still cooled, at heart, by moving air.)</p><p>That entire strategy depends on being immersed in a fluid you didn’t have to bring with you. And that is precisely the thing you lose the moment you leave the planet.</p><h3>Why none of that works in space</h3><p>A vacuum has no air and no water. There is nothing to convect <em>into</em> and nothing to conduct <em>into</em>. Two of the three ways heat can move, conduction and convection, both need a material medium touching your hot object, and in orbit there isn’t one.</p><p>That leaves exactly one mechanism: <strong>radiation</strong>. Your spacecraft can only get rid of heat by glowing, emitting infrared light into the dark.</p><blockquote>A vacuum is one of the best insulators humanity has ever found. We literally build thermos flasks out of vacuum to stop heat from moving. An orbital data center is a server farm wrapped in a thermos.</blockquote><p>Here’s the punchline that surprises people: the “cold” of space barely helps you. The temperature of the environment matters far less than intuition suggests, because a vacuum can’t <em>carry</em> heat away. It can only <em>accept radiation</em> you manage to emit. And emitting enough radiation, as we’re about to see, is brutally hard.</p><h3>The black-body reality check</h3><p>How much heat can you actually radiate away? Physics gives a hard ceiling, the <strong>Stefan–Boltzmann law</strong>:</p><p><strong>P = ε σ A T⁴</strong></p><p>where <strong>P</strong> is the power radiated, <strong>A</strong> is the radiator’s area, <strong>T</strong> is its absolute temperature, <strong>ε</strong> is emissivity (how close to a perfect radiator the surface is, between 0 and 1), and <strong>σ</strong> is the Stefan–Boltzmann constant, 5.67 × 10⁻⁸ W/m²·K⁴.</p><p>Two features of that equation govern the entire problem.</p><p>First, radiated power scales with the <strong>fourth power of temperature</strong>. That sounds generous (double the temperature and you radiate sixteen times more), but you can’t just crank up the radiator temperature, because it’s fed by the coolant loop that touches chips which must stay under ~100 °C. In practice, radiators run somewhere in the 300–350 K range (room temperature to a bit above).</p><p>Second, and this is the killer: at those temperatures, a near-perfect radiator emits only about <strong>460 watts per square meter</strong> at 300 K. That’s it. That’s your budget.</p><p>Now do the arithmetic on our gigawatt facility. One billion watts, divided by ~460 W/m², is on the order of <strong>two million square meters</strong> of radiator, roughly <strong>one to two square kilometers of surface</strong>. And that’s the <em>ideal</em> case, ignoring that both sides don’t always face empty space. Scale it down to a single AI rack: about 130 kW ÷ 460 W/m² ≈ <strong>280 m², a tennis court’s worth of radiator, per rack.</strong></p><p>And it gets worse, because a good radiator is a good <em>absorber</em>. A surface engineered to dump infrared efficiently will happily soak up sunlight (the Sun delivers ~1,360 W/m² at Earth’s distance) and infrared glowing off the daylit Earth. So orbital radiators have to be aimed edge-on to the Sun, coated with special optical solar reflectors that reflect sunlight while emitting infrared, and designed to survive being baked and frozen on every ~90-minute orbit, cycling through thermal fatigue over and over, for years.</p><p>Here’s the comparison that makes it visceral. The International Space Station’s entire thermal-control system, which is six big deployed radiator wings plumbed with pumped ammonia, rejects about <strong>70 kilowatts</strong> of heat. <em>A single GB200 rack produces nearly twice that.</em> We would need the ISS’s whole radiator farm, roughly doubled, to cool <strong>one rack</strong> of an AI cluster that might contain thousands.</p><h3>“Just use water cooling”: yes, and also no</h3><p>A natural objection: fine, use liquid cooling like we do on Earth. And the answer is genuinely <em>yes</em>, but it doesn’t solve the problem you think it does.</p><p>Liquid loops in space are real and essential. The ISS doesn’t use water, though. Water freezes and bursts pipes in that environment, so it runs <strong>ammonia</strong> through its external loops. These loops are superb at the <em>transport</em> leg: moving heat from the equipment out to the radiators.</p><p>But transport isn’t rejection. A pumped loop carries heat to the radiator; the radiator still has to <em>radiate</em> it away, and we just saw how stingy radiation is. Liquid cooling doesn’t repeal the Stefan–Boltzmann law; it just delivers the heat to the place where that law does its slow, unforgiving work.</p><p>What about <em>open-loop</em> cooling, meaning boiling off water and venting the vapor, the way sweat cools you? It works thermodynamically, and it’s used for short-duration needs. But at data-center scale it means consuming coolant as a <em>consumable</em>: every kilogram of water you boil into space is a kilogram you had to launch, and you’d be launching it continuously, forever. At gigawatt scale that’s an endless freight train of water to orbit. It doesn’t close.</p><h3>What the researchers are actually working on</h3><p>None of this makes orbital cooling impossible. It makes it a serious engineering frontier. A few of the more interesting directions:</p><ul><li><strong>Two-phase pumped loops.</strong> Instead of just warming a liquid, you <em>boil</em> it, exploiting the large latent heat of a phase change to move far more energy per kilogram of coolant, and keeping the radiator closer to a single temperature so more of its area works hard. Modern designs even use controlled freezing of the ammonia to modulate output across huge ranges.</li><li><strong>Loop heat pipes.</strong> Passive, wick-driven two-phase devices with no moving parts, increasingly built by additive manufacturing and paired with deployable radiators to unlock more heat rejection from small satellites.</li><li><strong>Deployable and two-sided radiators.</strong> Since area is everything, you fold enormous radiators for launch and unfurl them in orbit; exposing <em>both</em> faces to space rather than one can boost rejection by roughly 50%.</li><li><strong>Liquid droplet radiators</strong>, the exotic one. Instead of a heavy solid panel, you spray a fine stream of hot droplets directly into space, let them radiate as they fly, and catch them downstream to recirculate. Because you’re not lugging kilograms of metal panel around (solid radiators run on the order of several kg per square meter), the mass savings at scale could be dramatic. It’s been studied for decades and is still maturing.</li><li><strong>Running hotter, on purpose.</strong> Because of that T⁴ term, a radiator that can safely operate at higher temperature sheds disproportionately more heat, which is pushing interest in high-temperature-tolerant electronics and coolants.</li></ul><p>All promising. All hard. And every one of them is ultimately still bound by the same equation.</p><h3>Takeaways</h3><ul><li><strong>Heat, not cold, is the problem.</strong> Every watt in becomes a watt of heat out, and AI chips concentrate enormous power into tiny areas.</li><li><strong>Space removes your best tools.</strong> No air, no water, no convection: only radiation, and a vacuum insulates rather than cools.</li><li><strong>The numbers are humbling.</strong> A gigawatt facility implies a square kilometer or more of radiator, and a single AI rack out-produces the entire ISS’s heat-rejection capacity.</li><li><strong>Liquid cooling helps transport heat, not reject it.</strong> The radiator, and Stefan–Boltzmann, always gets the final say.</li><li><strong>The frontier is real, not closed.</strong> Two-phase loops, deployable and droplet radiators, and hotter-running hardware are all chipping away at the problem.</li></ul><p>Space is cold. It just turns out that “cold” and “able to remove heat” are not the same thing, and orbit is generous with the former and miserly with the latter.</p><p><strong>Next in the series:</strong> we tackle the myth that feels the most liberating and is quietly the most wrong: <em>“nobody owns space, so there’s infinite room to grow.”</em> We’ll get into the tragedy of the orbital commons, the Kessler cascade, and why “unowned” turns out to mean “ungoverned,” not “unlimited.”</p><p><em>If this made the physics click, give it a clap or two and share it with the friend who keeps saying we’ll just move everything to the cold of space. Follow along so you catch the rest of the series, and tell us in the responses: did the tennis-court-per-rack number surprise you as much as it surprised us?</em></p><h3>Sources &amp; further reading</h3><ul><li><strong>AI chip and rack power / cooling:</strong> NVIDIA GB200 NVL72 product page (2024–25); HPE press release (Feb 2025); SemiAnalysis, “GB200 Hardware Architecture” (Oct 2025); TrendForce (Mar 2025); ToneCooling, “NVIDIA GB200 NVL72 Cooling Requirements” (Apr 2026).</li><li><strong>ISS thermal control (~70 kW via ammonia radiator loops):</strong> N. Iwata et al., “Thermal Performance Evaluation of Space Radiator for Single-Phase Mechanically Pumped Fluid Loop,” <em>Journal of Spacecraft and Rockets</em>, Vol. 59, №1, 2022.</li><li><strong>The physics of radiative cooling in orbit:</strong> the Stefan–Boltzmann law (J. Stefan, 1879; L. Boltzmann, 1884); Advanced Cooling Technologies, “Orbital Data Centers: The Challenge of Cooling Compute in Space” (Jun 2026); SatNews, “The ‘Physics Wall’: Orbiting Data Centers Face a Massive Cooling Challenge” (Mar 2026).</li><li><strong>Advanced space cooling research:</strong> NASA, “Thermal Management in Space” (NASA SP-509, 1992); “Additive Manufactured Loop Heat Pipe with Deployable Radiator,” Utah State University SmallSat Conference (2024); droplet- and two-phase-radiator studies in <em>Applied Thermal Engineering</em> (2023) and the <em>Journal of Engineering Physics and Thermophysics</em> (2024); “Thermal Management Technologies for Space Data Centers: Current Status and Prospects” (2026).</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe9d94cfa362" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Centers In Space: A Reality Check]]></title>
            <link>https://georgewilliams.medium.com/data-centers-in-space-a-reality-check-1aa93278626f?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/1aa93278626f</guid>
            <category><![CDATA[space]]></category>
            <category><![CDATA[future-of-computing]]></category>
            <category><![CDATA[data-center]]></category>
            <category><![CDATA[sustainability]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 05:29:09 GMT</pubDate>
            <atom:updated>2026-06-13T05:29:09.869Z</atom:updated>
            <content:encoded><![CDATA[<h4>The case for orbital computing is real. But before we get carried away, four comfortable myths need busting.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*46bpWBy-NJ6zNlyBsUriPw.png" /></figure><p>There’s a new dream sweeping through tech: if AI’s hunger for electricity is straining our power grids, draining our water, and turning data centers into the most unwelcome neighbors in America — why not just put the data centers in space?</p><p>Up there, the sun never sets. There’s no water bill, no angry town hall meeting, no transmission line to permit. Capture that endless solar energy, beam the results home, and let Earth off the hook. It sounds like science fiction finally growing up.</p><p>We’re optimists about this idea. We genuinely want it to work. But wanting something to work is exactly why it deserves a hard look rather than a hopeful one — and when you give the orbital-data-center pitch that look, you find four assumptions that get waved through far too easily. This is our attempt to bust them, in the friendliest possible way.</p><h4>An old dream with a brand-new engine</h4><p>Computing in space isn’t a new idea. Space-based solar power was popularized by Isaac Asimov back in the 1940s and formalized by Peter Glaser in 1968. The 1980s “Star Wars” missile-defense program imagined autonomous computers doing real work in orbit. More recently, HPE ran ordinary servers on the International Space Station for years, Microsoft sank a sealed data center onto the seafloor and found it *<em>more</em>* reliable than one on land, and a startup called Lonestar has been testing data storage on the Moon.</p><p>What’s different now is the <strong>motive</strong>. None of those older projects were about running hyperscale AI. This one is — and that changes the math entirely.</p><p>The numbers behind AI’s appetite are genuinely startling. The International Energy Agency expects data centers to roughly double their electricity use, from about 415 TWh in 2024 to around 945 TWh by 2030 — with the AI-specific portion tripling or quadrupling. The combined data-center spending of a handful of tech giants now rivals the entire world’s investment in oil and gas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qeRw3riZKOgCZ34mNV5GjQ.png" /><figcaption>Global data-center electricity demand is on track to roughly triple between 2024 and 2035 (IEA base case), with AI accounting for a fast-growing share.</figcaption></figure><p>And the costs land <strong>locally</strong> while the benefits spread out everywhere. Google’s data centers drank about 6.1 billion gallons of water in 2024. In Ireland, data centers already pull more than a fifth of the national grid. So it’s no surprise that communities are fighting back: roughly $64 billion of U.S. data-center projects were blocked or delayed by local opposition through 2025, and in 2026 Monterey Park, California became the first U.S. city to ban them by popular vote.</p><p>When your neighbors won’t have you, orbit starts to look awfully appealing. It’s the ultimate back yard — one with no neighbors to object.</p><h4>Why space, actually?</h4><p>The pull is real, and worth stating plainly. Park a satellite in a “dawn–dusk” sun-synchronous orbit and its solar panels sit in nearly perpetual, atmosphere-free sunlight — on the order of eight times the energy yield of the same panel on the ground. No municipal water. No fight over land or an aquifer. No new substation. Waste heat can in principle be radiated straight into the cold of space, and laser links between satellites can move data without touching a single mile of terrestrial fiber.</p><p>This is not a fringe fantasy anymore. In late 2025, <strong>Starcloud flew an NVIDIA H100 GPU in orbit</strong> and ran a model on it. <strong>Google’s Project Suncatcher</strong> plans to launch TPU-equipped prototype satellites in 2027. <strong>Axiom Space</strong> put its first orbital-data-center nodes up in early 2026. SpaceX, Blue Origin, and China’s Adaspace have all filed for big constellations. In about eighteen months, “data centers in space” went from curiosity to roadmap.</p><p>So let’s take the roadmap seriously — by stress-testing what’s underneath it.</p><h4>Myth 1: “Space is one giant refrigerator, so cooling is free”</h4><p>This is the most common one, and it’s exactly backwards.</p><p>The intuition is that space is near absolute zero, so a hot computer should shed heat effortlessly. In reality, <strong>getting rid of heat in space is harder than on Earth, not easier.</strong> Heat moves three ways — conduction, convection, and radiation — and in a vacuum the first two basically don’t exist. There’s no air and no water to carry heat away, so the workhorse of every data center on Earth (convection) is simply gone.</p><p>That leaves radiation as the <em>only</em> way out, and radiation is far less efficient, pound for pound, than blowing cold air or pumping water. Worse, a surface good at radiating heat away is also good at <em>absorbing</em> sunlight, so radiators have to be angled edge-on to the sun and survive being baked and frozen every orbit. As AI chips get denser and hotter, the radiator quietly becomes one of the biggest, heaviest parts of the whole spacecraft.</p><p><strong>Verdict: Busted.</strong> Space is cold, but a vacuum is one of the best insulators there is. Cooling isn’t a freebie — it’s arguably the single hardest problem.</p><h4>Myth 2: “It’s not deep space, so radiation is a non-issue”</h4><p>Engineers who build spacecraft have spent fifty years learning to fear radiation, and “we’re close to home” doesn’t get you off the hook.</p><p>Even in low-Earth orbit, electronics get hit two ways: a slow, cumulative dose that degrades parts over time, and sudden “single-event” hits — bit flips, lock-ups, even burnouts — from a single energetic particle. The sources are everywhere: the Van Allen belts, the South Atlantic Anomaly (where the radiation dips low enough to matter), cosmic rays, and the occasional solar storm that spikes the dose by orders of magnitude.</p><p>Here’s the catch for AI specifically: the chips that make this exciting — H100-class GPUs and Google’s TPUs — <strong>are not radiation-hardened.</strong> Rad-hardened parts exist, but they’re roughly twice as slow and far more expensive, which erases the whole point. Whether a TPU can even survive up there is literally the question Google’s Suncatcher mission exists to answer. And the Moon? With no magnetic field to shield it, it’s <em>worse</em> than low orbit, not better.</p><p><strong>Verdict: Busted.</strong> Radiation is a serious, expensive, well-understood threat — and it’s harsher at the Moon, not gentler.</p><h4>Myth 3: “Skip the rocket — just 3D-print the data center in orbit”</h4><p>The dream here is elegant: dodge the brutal cost of launch by manufacturing everything up there. Reality isn’t close.</p><p>In-space 3D printing is real, but today it’s mostly plastic prototypes and spare parts. Reliably printing <em>metal</em> parts in microgravity — never mind anything more complex — isn’t there yet, because without gravity, molten material and loose powder behave in inconvenient and sometimes hazardous ways.</p><p>But here’s the deeper problem: <strong>you can’t 3D-print a chip.</strong> Now, there’s genuinely promising research showing that microgravity grows better semiconductor <em>crystals</em> — bigger, more uniform, fewer defects. That’s real and exciting. But growing a nice crystal is the raw-material step. It is not the same as the staggeringly precise lithography — the patterning, etching, layering, doping — that turns a wafer into an actual processor. Tellingly, NASA’s in-space electronics work deliberately targets <em>simple memory chips</em> using printing methods precisely <em>because</em> normal chip fabrication doesn’t transfer to orbit. The valuable silicon still has to be built on Earth and launched.</p><p><strong>Verdict: Busted (for now).</strong> In-space manufacturing may someday help with bulk structure. It does not let you skip launching the parts that matter.</p><h4>Myth 4: “Nobody owns space, so there’s infinite room to grow</h4><p>This one mistakes <em>no ownership</em> for <em>no limits</em> — and it’s precisely backwards.</p><p>The 1967 Outer Space Treaty says no nation can claim territory in space. Sounds liberating, but it means there’s no such thing as an orbital property right. That makes orbit a textbook <strong>commons</strong> — and commons without rules get overgrazed. Because nobody can own a slice of orbit, nobody has any incentive to ration it, and every new satellite quietly imposes collision risk on everyone else.</p><p>The physical ceiling is the <strong>Kessler syndrome</strong>: debris from one collision triggers more collisions, cascading until a whole region becomes unusable. We’re filing for hundreds of thousands — even a million — satellites. Starlink alone made roughly 50,000 collision-avoidance maneuvers in just the first half of 2024. And the <em>best</em> orbits (those sun-drenched dawn–dusk paths) are exactly the ones everyone wants at once. The rules to manage all this? Still voluntary.</p><p><strong>Verdict: Busted.</strong> “Nobody owns it” is the problem, not the solution.</p><h4>So what would it actually take?</h4><p>Here’s the optimist’s turn: none of this is fatal. Every one of these is a research program and a budget line, not a brick wall.</p><p>The single biggest lever is <strong>launch cost.</strong> Today, getting a kilogram to orbit on a Falcon 9 runs about $3,600. Google’s own analysis says orbital computing only reaches price parity with the ground at roughly <strong>$200/kg — an 18× reduction</strong> — and even that hinges on SpaceX’s Starship becoming cheap, reusable, and flying constantly. Build a gigawatt data center in orbit today and you’re looking at around $42 billion, roughly triple its earthly twin.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*asqBK8duZktJsw2K_0hqNg.png" /><figcaption>Orbital compute needs roughly an 18× collapse in launch cost — from today’s ~$3,600/kg toward ~$200/kg — to reach price parity with the ground.</figcaption></figure><p><em>The launch-cost gap orbital computing has to close, per Google’s Project Suncatcher analysis.</em></p><p>A credible 25-year program looks something like this:</p><p>- <strong>Survive (2025–28):</strong> prove commercial AI chips can take the radiation, heat cycling, and vacuum of low orbit.</p><p>- <strong>Cool &amp; connect (2028–32):</strong> demonstrate megawatt-scale heat rejection and high-bandwidth optical links across a cluster.</p><p>- <strong>Scale launch (2030–35):</strong> drive launch toward ~$200/kg and mass-produce the satellites.</p><p>- <strong>Operate (2035–50):</strong> gigawatt constellations with real servicing, upgrades, and end-of-life de-orbit.</p><p>- <strong>Fabricate (the moonshot):</strong> in-space chip fabrication — the only thing that would ever let us stop launching the most valuable payload of all. Don’t hold your breath before 2050.</p><p>And technology is only half of it. The next 25 years also need treaties we simply don’t have yet: binding rules for space traffic and debris, a modern way to allocate the crowded good orbits, updated liability law, a commercial-era reading of the Outer Space Treaty, a coherent plan for the Moon, and — this one’s underrated — rules for <em>whose laws apply to data being processed in orbit.</em> Treaties move slowly, so that work has to start now.</p><p>The good news? The will is clearly there. SpaceX’s blockbuster 2026 IPO drew enormous demand from Wall Street and everyday investors alike. The money and the enthusiasm exist in abundance. <strong>The real constraint isn’t desire or dollars — it’s physics, engineering maturity, and governance.</strong></p><h4>The bottom line</h4><p>The dream is old, the engineering is real, and the reason behind it is legitimate: AI’s demand for power, water, and land is straining the planet right now. We’re not here to say it can’t or shouldn’t be done.</p><p>We’re here to say that four cozy assumptions don’t survive contact with physics and law. Cooling is hard. Radiation is harsh. The chips stay Earth-made. And an unowned sky invites a traffic jam, not infinite room.</p><p>None of that is a dead end. It’s a to-do list. The path to orbit runs through an 18× drop in launch cost, real answers to heat and radiation, and a rulebook for the sky — all achievable in twenty-five years <em>if</em> we start by retiring the myths.</p><p>Optimism and rigor aren’t opposites. The surest way to actually reach orbit is to be honest about how hard the climb really is.</p><p>— -</p><p><em>If you found this useful, a clap or a share helps. We’d love to hear which myth surprised you most — or which one you think we got wrong. We will deep dive into many of these topics in follow-up blogs.</em></p><p><strong>Sources &amp; further reading:</strong> International Energy Agency (<em>Energy and AI</em>, 2025–26); Carbon Brief; Data Center Watch; <em>Governing</em>; Google (<em>Project Suncatcher</em>); Scientific American; Advanced Cooling Technologies; <em>IEEE Spectrum</em>; NASA (thermal control, radiation lessons learned, in-space manufacturing/ODME); <em>npj Microgravity</em>; the Outer Space Treaty and analyses in <em>Business Law Today</em> and <em>EJIL: Talk!</em>; TechCrunch; Reuters/CNBC (SpaceX IPO).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1aa93278626f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Nvidia GPU Acceleration For Mac!]]></title>
            <link>https://georgewilliams.medium.com/nvidia-gpu-acceleration-for-your-mac-468779749825?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/468779749825</guid>
            <category><![CDATA[cuda]]></category>
            <category><![CDATA[nvidia-gpu]]></category>
            <category><![CDATA[apple]]></category>
            <category><![CDATA[deepseek]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Sun, 09 Mar 2025 17:03:42 GMT</pubDate>
            <atom:updated>2025-03-15T15:39:45.095Z</atom:updated>
            <content:encoded><![CDATA[<p>Impossible, right? After all, <a href="https://support.apple.com/en-us/102363">Nvidia GPUs don’t work with Macs</a>. Well actually <em>it is possible</em> using the latest innovation from<a href="https://www.juicelabs.co/"> Juice Labs</a>. In this blog, I’ll show you step-by-step how to run your CUDA-based app on Mac, accelerated on a remote Nvidia GPU — all of this without any code changes!</p><h4>Apple GPU, NVidia GPU, and CUDA</h4><p>So why would you want to run CUDA apps on Mac? The latest Macs have decent GPUs. In fact, many popular AI frameworks (<a href="https://huggingface.co/docs/accelerate/en/usage_guides/mps">Pytorch, HuggingFace’s Transformer library,</a> and <a href="https://docs.vllm.ai/en/stable/getting_started/installation/gpu/index.html">vLLM</a> to name a few) have started to support native Mac GPU acceleration, bypassing CUDA and instead leveraging <a href="https://developer.apple.com/metal/pytorch/">Apple’s Metal</a> or <a href="https://github.com/ml-explore/mlx">MLX</a> software libraries.</p><p>All that said, the best AI trains and runs in the cloud. Apple has little presence in the data center where Nvidia maintains its dominance. Nvidia is light years ahead of its competitors (<a href="https://www.pcgamer.com/hardware/graphics-cards/intel-nixes-its-next-gen-ai-gpu-but-still-has-plans-to-take-on-nvidia/">Intel</a>, <a href="https://www.amd.com/en/newsroom/press-releases/2025-2-28-amd-unveils-next-generation-amd-rdna-4-architectu.html">AMD</a>, <a href="https://cloud.google.com/blog/products/compute/introducing-trillium-6th-gen-tpus">Google</a>, <a href="https://azure.microsoft.com/en-us/blog/azure-maia-for-the-era-of-ai-from-silicon-to-software-to-systems/">Microsoft</a>, <a href="https://about.fb.com/news/2024/04/introducing-our-next-generation-infrastructure-for-ai/">Meta</a>, <a href="https://www.businessinsider.com/amazon-nvidia-aws-ai-chip-dominance-gpu-trainium-inferentia-2024-5">Amazon</a>, <a href="https://cerebras.ai/">Cerebras</a>, <a href="https://groq.com/why-ai-requires-a-new-chip-architecture/">Groq</a>, <a href="https://tenstorrent.com/en">Tenstorrent</a>, etc) not only because of its decade head start in hardware innovation, but also because of its CUDA software stack powers the lion share of AI applications that train and serve models.</p><p>So how do you connect-the-dots from CUDA apps running on Mac to Nvidia GPUs in the data center? <a href="https://www.juicelabs.co/">JuiceLabs</a> has introduced a unique GPU-over-IP solution that’s truly friction-free — it does not require re-architecting CUDA applications or changing a line of code. I’ll take you through how to use these tools yourself on your Mac. Alright, let’s get started!</p><h4>Get Started With JuiceLabs</h4><p>You will need the following:</p><ul><li>You need a Mac (obviously!)</li><li>You should know how to run CLI commands in Mac’s terminal console.</li><li>Hopefully you have some experience running CUDA apps on Linux or Windows machines with an attached Nvidia GPU.</li></ul><p>Next go to the<a href="https://www.juicelabs.co/"> JuiceLabs web site</a> and sign up for a free trial account. In the final step, you should download the Linux installation tar ball. The file is called <em>juice-gpu-linux.tar.gz. </em>Don’t do anything with that file yet, we’ll deal with it later.</p><h4>Set Up Your Mac</h4><p>You should probably update your Mac to the latest operating system. I tested with Sequoia 15.3.1 (I have an M2 MacBook Pro). Likely you have <strong>brew</strong> installed, but if not, follow these <a href="https://brew.sh/">instructions</a>.</p><p>You might have been wondering why I had you download the Linux installer. As some of you know, Linux apps usually run only on machines with X86 host processors (Intel and AMD) but recent Macs have ARM processors. In the next step, I’ll take you through installing X86 virtualization<em> </em>software called <a href="https://github.com/abiosoft/colima"><strong>colima</strong></a><strong> </strong>which lets you run Linux apps on Mac.</p><p>Open the terminal app on your Mac and run the commands shown in the gist below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7b5ca44175211c9ae0e1724ab0acf866/href">https://medium.com/media/7b5ca44175211c9ae0e1724ab0acf866/href</a></iframe><p>Those steps take you through first installing <a href="https://github.com/abiosoft/colima"><strong>colima</strong></a>, then creating an Ubuntu Linux virtual machine (named <em>juice), </em>and<em> </em>finally ensuring the virtual machine is running. Note the gist above is not showing the output of all commands, but you should make sure each one runs successfully.</p><h4><strong>Set Up Your Virtual Machine</strong></h4><p>The next commands take you through logging into the virtual machine you just created by using ssh, installing a few Linux system packages, downloading the latest Anaconda Python distribution, and finally installing Python 3.10, CUDA 12.1.1, and Pytorch 2.6.0.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d12e0e8b84848a857961c72d26269715/href">https://medium.com/media/d12e0e8b84848a857961c72d26269715/href</a></iframe><p>As before, I don’t show the output of each command in the gist above, so you need to make sure each command completes successfully.</p><h4>Pytorch-&gt;CUDA-&gt;NVidia GPU On Your Mac</h4><p>Now your system is ready to test a simple Pytorch script that leverages CUDA to communicate with an Nvidia GPU. The following gist shows a complete session. When you try it yourself, you will need to adjust a few of the command arguments. I’ve made a relevant comment near the command:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/50d5b9e601f35af0e25ba194e0bac472/href">https://medium.com/media/50d5b9e601f35af0e25ba194e0bac472/href</a></iframe><h4>Next Steps With DeepSeek Models</h4><p>Admittedly, running a couple of lines of Pytorch is not the same as running a real fully-featured CUDA app. In this blog, I just wanted to get your Mac setup properly and familiarize you with<a href="https://www.juicelabs.co/"> JuiceLabs</a>’ utilities. In the next blog, we’ll do something a bit more advanced. We’ll run a CUDA app that loads and runs <a href="https://github.com/sourcesync/Janus">a DeepSeek model </a>(for a sneak peek, checkout the youtube video below.). I’ll also show you some benchmarks I’ve run that compares “juiced” models vs running them natively on my Mac.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F9BpbeV-KvNI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D9BpbeV-KvNI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F9BpbeV-KvNI%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/26064693a1afe228da6698a1301505d9/href">https://medium.com/media/26064693a1afe228da6698a1301505d9/href</a></iframe><p>If you liked this blog, please hit the like button. And don’t forget to subscribe to my medium feed to get a notification of the next blog!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*GGpithfm9D_zv_jwEpQbfA.png" /></figure><p>Disclaimer: I am not currently affiliated with Apple, Nvidia, or JuiceLabs. I’m just a tech evangelist working at the intersection of hardware and artificial intelligence.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=468779749825" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Winning Chess Hackathons For Fun and Profit!]]></title>
            <link>https://georgewilliams.medium.com/winning-chess-hackathons-for-fun-and-profit-1928fccded39?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/1928fccded39</guid>
            <category><![CDATA[chess]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[hackathons]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Sun, 19 Jan 2025 18:24:24 GMT</pubDate>
            <atom:updated>2025-03-09T17:19:16.091Z</atom:updated>
            <content:encoded><![CDATA[<p>Part I: The Data and The Model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bdIvekSbepf9w2qKxjhmyw.jpeg" /></figure><h4>Introduction</h4><p>I recently participated in <a href="https://strongcompute.com/">Strong Compute</a>’s last AI Chess Hackathon. Honestly I really had no business getting involved. My last hackathon was years ago and, perhaps more importantly, I’m not a great chess player. That said, machine learning has become very powerful lately and I was curious if I could train a deep learning model based on convolutional neural networks or transformers that could play chess well. I’ve leveraged machine learning for all sorts of challenging problem domains — computer vision, natural language, drug discovery, bioinformatics, radio signal processing, sports analytics, motion capture, robotics- but never for games such as chess.</p><p>As it turns out, I ended up winning a trophy— beating all other newbie competitors in the final tournament. In addition, I also qualified to get a bunch of free credits to use Strong’s <a href="https://strongcompute.com/">massive GPU cluster</a>!</p><p>How did I do it you might be asking? Well that’s what this 3-part blog series is about. These blogs are targeted to data scientists, AI engineers, even software engineers who have a basic understand of machine learning (preferably who have some experience running and training models.) Some experience with chess will be useful, and, of course, you should have an interest in training a chess playing AI. Who knows, maybe my notes here could help you win a chess hackathon too!</p><p>Here is the outline of the series:</p><ul><li>In <em>Part 1: The Data and The Model </em>I’ll start with some basic background. We’ll get into the PGN chess data format, chess scoring, relevant deep learning primitives, and the model architecture that I trained.</li><li>In <em>Part 2: Training and Inference </em>I’ll get into the details of training the model in PyTorch and I’ll show the game results of the trained model playing against itself. This will give me a chance to discuss how chess players (either human or AI) are rated and ranked.</li><li>In <em>Part 3: Scaling Chess AI </em>I’ll dive into some of the latest bleeding chess AI research, including <a href="https://arxiv.org/abs/2402.04494">DeepMind’s release of the most comprehensive chess training se</a>t ever curated. I’ll show you how you can train state-of-the-art generative AI models leveraging <a href="https://strongcompute.com/">Strong Compute</a>’s easy-to-use distributed GPU tools and infrastructure.</li></ul><p>This will culminate in a final virtual tournament in which I’ll have all the models developed in this blog series compete against each other.</p><h4>A Very Brief History Of Chess And Computing</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*yZzn9-k-_OUZkTglBU8l5g.jpeg" /><figcaption>IBM’s Deep Blue in the Computer History Museum in Mountain View, CA.</figcaption></figure><p>Before we get into it, let’s take a step back. Since the beginning of computing in the 1940s, the development of competitive chess engines has been a singular goal for software/hardware engineers and computer scientists. Chess has also served as the backdrop for key milestones in computing, including:</p><ul><li><em>1948</em> Alan Turing developed the first algorithm and program for playing chess. Unfortunately no hardware powerful enough to run it was available until after Turing died in 1954! Turing developed the first methods to “score” potential chess moves (e.g., determine the best move to make given current state of the board.)</li><li><em>1985</em> Carnegie Melon University began work on ChipTest, the first chess engine designed from the ground up including custom silicon, micro-code, and a chess tree-based search algorithm that could score 50K moves per second.</li><li><em>1997</em> IBM’s Deep Blue beat Gary Kasparov becoming the first chess engine to beat the current (human) chess champion in a full match.</li><li><em>2017 </em>DeepMind released AlphaZero which leveraged neural networks in addition to search trees. All top chess engines now use neural networks and deep learning.</li></ul><h4>Chess Game Data</h4><p>OK, let’s start with the data. If you thought that leveraging previously played chess games would be useful to train a chess playing AI, you are on the right track. Fortunately, there’s a ton of openly available chess game play data out there.</p><p>One popular data format for chess games is called PGN, which stands for Portable Game Notation. It’s a compact way to track chess moves and board states for an entire chess game from beginning to end.</p><p>The following is an example of the PGN for an opening chess move:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*xOs51h0-DGluGp0DkM-38A.png" /><figcaption>In this board, “E4” indicates one of white’s pawns has moved forward two positions.</figcaption></figure><p>The move shown is encoded as the string “E4” which indicates the destination of a piece’s move. In this case, a white pawn has moved forward two positions in the board. The encoding uses the 2D coordinate system of a chess board (board on the right.) One axis is sequential letters and the other is sequential numbers. Simple, right?</p><p>The format also accommodates some unique scenarios:</p><ul><li>A chess move that overtakes an opponent’s piece is indicated with a preceding “x.”</li><li>If there is ambiguity because multiple pieces could be moved to the same spot on the board — this is indicated with a preceding letter that indicates the piece that moved, for example “Q” for the queen.</li></ul><p>A PGN string is a sequence of all the individual move encodings of a game. Check out the <a href="https://old.chesstempo.com/pgn-examples.html">following site</a> to acquire a visually intuitive and interactive understanding of a complete game using PGN notation.</p><p>If you are wondering how one might represent the state of the board at any point in the game sequence, all you need to do is replay the PGN game string from the beginning to any point in the game you want. We’ll need this because the model I trained takes a board state as input, and not a sequence of moves.</p><h4>Scoring Chess Boards</h4><p>In this hackathon, participants had to train a model that takes as input an arbitrary chess board state, returning a “score” as output. In the hackathon’s final tournament, whenever it was the model’s turn to move in a game — the hackathon’s game engine would send to the model all potential board states representing all currently available legal moves. The game engine picked the move with the highest score.</p><p>You may be wondering how ground truth scores were acquired for the training data. Fortunately, the hackathon provided various datasets for training, each with lots of pre-scored board states. I ended up using the dataset with scoring from one of the <a href="https://lczero.org/play/download/">Leela Chess Zero chess playing engines</a>. Leela Chess Zeros is a popular open source chess engine project and they train many of the best chess playing models that are openly available.</p><h4>Deep Learning Primitives for Chess AI</h4><p>Before getting into the model details, it’s worth taking a moment to review the common deep learning primitives that compose most modern day neural networks. The following sections summarize the primitives I ended up using for my model exploration. Likely you are familiar with many of these concepts and I won’t go into a ton of detail. Rather, I’ll highlight concepts that are potentially useful when building a neural network that can play chess.</p><p><em>Linear Layer</em></p><p>Every neural network has a linear layer somewhere in it, so let’s start here. The image below demonstrates a small linear layer. In this example, the layer takes a vector of 4 numbers on the left and outputs a vector of 8 numbers on the right. Notice that each input “neuron” connects to all output neurons via an edge connection. Each edge represents a constant “parameter” or “weight.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r6F7OJGRFqued2X7ow_EEQ.png" /><figcaption>A simple Linear Layer which transforms a vector of 4 elements into a new vector of 8 elements.</figcaption></figure><p>The value of each output neuron is the sum of all input neurons connected to it, where each connected input neuron is multiplied by the edge’s weight. Another way to think of the transformation from the input vector to the output vector as a matrix multiplication of the input vector by the implicit weight matrix.</p><p>These days, it’s not unusual for linear layers to be composed of hundreds or even thousands of both neurons and weights. That’s a lot of multiplications! In LLMs. for example, most of the weights (and compute workload) lie in the linear layers.</p><p><em>Convolutional Layer</em></p><p>Computer vision deals with multi-dimensional data instead of just 1D vectors. For example, you can think of a black-and-white image as 2D matrix of gray scale data. A convolutional operation is depicted in the animation below. The convolution is a “sliding window” operation applied across the input (the 2D matrix shown in blue.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/244/1*cFMF_uWgUFdVRMZAZ0Bfzg.gif" /><figcaption>A 3X3 convolution operation.</figcaption></figure><p>Notice here the final output of the completed operation is also 2D (in green.) The convolutional operator itself (depicted by the frustum) is actually just a small linear layer, in this case it has 3X3 input neurons and 1 output neuron.</p><p>There are many variants of the convolutional operator including 1D convolutions, 2D convolutions on multi-channel data (ie, consider a color image, each pixel composed of 3 elements), and higher dimensions convolutions like 3D and 4D. There are slightly different ways a convolution operation can traverse its input, for example by skipping pixels to reduce the computation requirements, or even skipping pixels of the input in each convolution operation as shown below (this is called “dilation.”)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/395/1*SVkgHoFoiMZkjy54zM_SUw.gif" /><figcaption>A dilated convolution skips pixels in each convolution.</figcaption></figure><p>A chess board is like an 8X8 image so its natural to include a convolutional layer within a chess playing neural network. The model I’ll describe later takes advantage of several convolutional layers.</p><p><em>Activation Functions</em></p><p>At this point, you’ve probably noticed that outputs of linear and convolutional layers are the result of lots of multiplications. As inputs “feed forward” through the neural network, there are more layers and therefore more successive multiplications. This means there is the danger of numeric overflow. Activation functions can mitigate this, as well other types of numerical instabilities, by restricting neuron values to a specific range. When present, these functions are applied to each individual output neurons of a layer. Consider the sigmoidal activation function below, a very common activation function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aL1OhnNrekauWZPnucrUvQ.png" /><figcaption>A sigmoidal activate function.</figcaption></figure><p>The range of possible values into this activation function is the full range of the x-axis but the function “squashes” the input to a range of values between 0 and 1 in a non-linear fashion. There are lot’s of different variants of activation functions and many of them perform this squashing. In addition to squashing and numeric stability, activation functions help the early stage of pre-training. The model I trained uses RELU (rectified linear unit) which is a simple function that passes positive values as-is, but clamps negative values to zero.</p><p><em>Normalization Layer</em></p><p>As you saw earlier, activation functions can have a normalizing effect on the data, but they don’t do this by capturing the statistics of the data itself. Batch normalization is a popular technique which performs this operation by looking at the range of data values for a subset of the training data at each step during training. And as opposed to activation functions, normalization layers can also have trainable parameters.</p><p><em>Residual Connection</em></p><p>A residual connection merely ties an input layer in one area of a neural network to an output layer somewhere else in the network. See the example below which shows the input layer “residually” connected to a layer further on down the network. What does this do? Mathematically it’s (typically) just a vector add operation — the input vector is just added element-wise to the destination’s vector. In the example there are no parameters associated with the connection at all (that’s what the “identity x” is emphasizing in the graphic.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*FBhxd4Qei0V0XvIaqGyB-A.png" /><figcaption>A residual connection from the input to the output of the second convolution layer.</figcaption></figure><p>Now, it doesn’t seem like a useful primitive, but it is in fact one of the most important primitives of deep learning. Residual connections single- handedly enabled neural network designers to go “deeper” and craft neural networks composed of hundreds of layers. If you’ve ever used the model called ResNet, the first convolutional neural network that leveraged residual connections, then you’ve used the deep learning model that ushered in the “Cambrian explosion” of deep learning in the early 2010s.</p><p><em>Drop-out Layer</em></p><p>You probably know that machine learning can “overfit” easily on the training data. An “overfit” model has memorized its training data to some degree. In other words, when this trained model is deployed, its predictions are only confident on model inputs that resemble closely the training samples. In order to help a model “generalize” instead of memorize, different techniques are available. One of those is to force some fraction of the model’s weights to be zero randomly during training. And that’s the role of a drop-out layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1O9DAxcnEOhstg2qlhK7bQ.png" /><figcaption>A 3 layer fully connected network in which dropout is applied at each layer.</figcaption></figure><p>It’s crazy how effective this method works and it’s a must for “parameter dense” layers such as linear layers that are prone to overfitting. It’s actually not unusual to see drop out rates of 50% in some models. Obviously we want to avoid training a chess playing model that memorizes moves (there are too many to memorize!), but rather generalizes the training data and forces it to learn something closer to strategy.</p><p><em>Embedding Layer</em></p><p>An embedding layer is the first layer of a language model, responsible for converting each language token into an embedding vector. Since a token can be mapped directly to an ordinal value, an embedding layer is often implemented as just a lookup table, mapping a token directly to a vector.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dtySLVoI-oJtgnRd5UWqHA.png" /><figcaption>An embedding layer is just a lookup table. The lookup integer results in selecting a specific set of model weights which is the resulting embedding of the layer.</figcaption></figure><p>In many ways, a chess piece is just like a token or symbol. But instead of a sentence, it’s positioned discretely on a chess board. I ended up using an embedding layer for my model for this reason.</p><p><em>Positional Encoding</em></p><p>You’ll also find positional encoding in language models in addition to the embedding layer. The idea is simple. Consider the same token appearing in different positions in a sentence or paragraph. There needs to be a way to adjust that token’s embeddings to reflect also its position. That’s the role of position encoder.</p><p>The graphic below shows a simple scheme which adjusts the tokens embedding by the value of a trig function that’s sensitive to the position.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*34AuqSZd1O2DdCLlCNCMWQ.png" /><figcaption>A position of the token produces a unique value of the trig function, which is added to the token embedding.</figcaption></figure><p>In a similar way, a chess piece (consider a pawn), can appear on different parts of a board. It could useful to encode a chess piece’s position on the board just like what language models do with the same token in a sequence of text.</p><p><em>Attention Layer</em></p><p>The combination of embedding layer and positional encoder gives language models a powerful front-end that embeds each token at each position into the same vector space. The attention layers that follow do the heavy lifting of developing mathematical relationships in that shared vector space. The classic example of the attention mechanism shows how different words in a sentence are associated with the same word that appears in multiple positions of a sentence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*3p7HA2azin47WimIczC_qg.png" /><figcaption>The word “the” appears twice in the sentence. Other words in the sentence “attend” to each appear of the word “the” in different ways.</figcaption></figure><p>In the example above, the word “the” appears twice, but each appearance of “the” relates to different words in the sentence. The first “the” with “sun”, and the second “the” with “eyes.”</p><p>We might think about leveraging this attention mechanism to help a chess playing model associate chess pieces across the board. Could this attention mechanism help the model predict the best next move?</p><h4>A Chess Playing Neural Network</h4><p>During the hackathon, I explored several different architectures using the building blocks described above. In addition, <a href="https://strongcompute.com/">Strong Compute</a> provided several code implementations in PyTorch. They also gave participants access to a massive GPU cluster which encouraged experimentation with different model architectures.</p><p>Although participants could train lots of different models, the hackathon rules forced you to choose just one for the final submission. I ended up choosing the specific model shown below, because this one had trained for the longest period of time:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/313/1*i3vsjIn5AeYVYw3caadaqg.png" /><figcaption>My winning hackathon chess playing model architecture!</figcaption></figure><p>A few things to note about this architecture:</p><ul><li><em>Model Input and Output</em> The model takes as input a board state (not the entire game history) and predicts a score for that board. During tournament play, the game engine invokes the model with all potential board states associated with legal moves available for its turn. The game engine then chose the move with the highest score.</li><li><em>Residual Blocks</em> The purple block of layers is repeated twice in this architecture. I tried adding more blocks, but as I did so the size of the network started to exceed the resource limit restrictions set by the hackathon rules.</li><li><em>Residual Connection</em> Each purple block of layers has one residual connection shown as the bold black line connecting the block’s input to the input of the final attention layer in the block. I enabled a model configuration setting which included learnable parameters in the residual connection, but I didn’t have enough time to determine if it was useful or not during the hackathon. I ended up doing some experiments after the hackathon.</li></ul><p>You can check out the model’s <a href="https://github.com/StrongResearch/chess-hackathon/blob/main/models/chessVision/hackathon-3-winner/model.py">PyTorch code here</a> and the <a href="https://github.com/StrongResearch/chess-hackathon/blob/main/models/chessVision/hackathon-3-winner/model_config.yaml">model configuration I used here</a>. I’ll get into more details of the model and the effect of different model configurations in the next blog.</p><h4>What’s Next?</h4><p>Stay tuned for Part II in this series:</p><ul><li>I’ll discuss details of training the model I described in this blog, including how I leveraged PyTorch’s DDP (e.g., Distributed Data Parallel) to speed up training over several GPUs.</li><li>I’ll dive into the impact of different model configuration on training time and chess play performance. For example, after the hackathon I performed various “ablation studies” in which I trained models with slightly different tweaks, such as removing the attention layer, adding dilation, and adding more training data. I’ll show the results of chess games in which I pitted these variants against each other.</li><li>I’ll get into the details of the Elo rating system, which is used for ranking human chess players and chess engines, based on their performance history.</li></ul><p>If you liked this blog please show your love and hit the “like” button. And follow me on Medium to get a notification when I publish the next blog!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1928fccded39" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Not All Face Recognition Is Created Equal: The Need For Liveness.]]></title>
            <link>https://medium.com/smileidentity/not-all-face-recognition-is-created-equal-the-need-for-liveness-b0acf5999a44?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0acf5999a44</guid>
            <category><![CDATA[kyc-verification]]></category>
            <category><![CDATA[identity-management]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[fraud-detection]]></category>
            <category><![CDATA[face-verification]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Mon, 18 Apr 2022 21:06:28 GMT</pubDate>
            <atom:updated>2022-04-19T15:58:59.134Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hKsZuXctSesjlc_UccG0KA.png" /></figure><p>Over the last two years of the covid-19 pandemic, the convenience that came with society’s digital transformation has unfortunately been matched by an increase in online fraud.</p><p>With password and data breaches becoming more common, many financial apps have turned to biometric face checks, matching consumers’ selfies against IDs or previously enrolled photos to ensure that only legitimate users are signing up for accounts or accessing services online.</p><p>The rise of “selfies” in the mid 2010’s and Apple’s introduction of FaceID in late 2017 made consumers more comfortable with face recognition as a medium.</p><p>Technology previously limited to high tech security services and science fiction movies was now available in everyone’s pocket. But the rapid adoption of face recognition tech has also resulted in new kinds of fraud. Cybercriminals are getting increasingly good at hacking face recognition.</p><p>In the simplest version of the attack, the criminal presents a target’s photo during a face check, fooling the app into unlocking access to the victim’s accounts and personal data.</p><p>Face recognition tech is vulnerable to abuse if it cannot ensure that a live person is in front of the camera. In this blog, we highlight the alarming rise in biometric crime, the emerging threat of “face spoofing”, and the role that “liveness detection” plays as an effective countermeasure.</p><h4>Face Recognition: From Novelty to Commodity</h4><p>Face recognition is now everywhere — in our laptops, our phones, at work, and at home. The technology has matured from niche status into a world-wide commodity in just the span of a decade.</p><p>Driven by consumers’ expectations of convenience when navigating their increasingly connected and digital lives, global demand for biometric technology like face recognition continues to grow at an exponential pace.</p><figure><img alt="Source: GrandView Research Biometric Technology Report" src="https://cdn-images-1.medium.com/max/1024/0*8lrXIoC-X7Q8pMHP" /><figcaption>Source: Grandview Research Biometrics Technology Report</figcaption></figure><p>In some areas of the world, biometrics has grown beyond convenience and into a necessity. In Africa users can<a href="https://platform.keesingtechnologies.com/biometrics-2/"> perform critical financial services like banking</a> using mobile apps that feature biometric authentication. In the US, the<a href="https://gizmodo.com/irs-will-require-facial-recognition-scans-to-access-you-1848387715"> Internal Revenue Service recently investigated requiring face checking</a> before allowing tax-payers access their on-line returns. And India has<a href="https://www.latimes.com/world/la-fg-india-database-2017-story.html"> nationalized biometrics</a> to provide all 1.3 billion of its citizens access to government services like voter registration with a fingerprint or face.</p><h4><strong>So What’s The Problem?</strong></h4><p>This aggressive expansion in biometrics has fueled global concerns over surveillance, data privacy, and ethical use. And as the laws mature surrounding biometrics, we will see<a href="https://www.michalsons.com/blog/biometrics-and-data-protection-law-around-the-world/42094"> varying regulatory approaches develop in different parts of the world</a>. These differences aside, the pervasiveness and wide applicability of face recognition has made it a popular target for hackers and criminals everywhere.</p><p>So, how easy is it to break vulnerable face recognition systems? Let’s look at a few of the clever techniques criminals have devised to fool face checking on devices and in apps.</p><p>Here, you see a device about to get unlocked with an image of the owner’s face presented from a different device:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/0*IX9aT5wFpZPX2S2z" /><figcaption>Source: <a href="https://www.youtube.com/watch?v=w-ehLXyJAqA">https://www.youtube.com/watch?v=w-ehLXyJAqA</a></figcaption></figure><p>Here someone is <a href="https://mashable.com/article/we-fooled-oneplus-6-face-unlock">unlocking a device using a print out</a> of their face:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*ZFW6WJRifOcwI4v2124xRA.jpeg" /><figcaption>Source: Mashable</figcaption></figure><p>It’s even possible to <a href="https://techcrunch.com/2018/12/16/3d-printed-heads-unlock-cops-hackers/">break face recognition</a> by wearing <a href="https://maskalike.com/">custom-made masks</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*SONbnkuBySZlmx3-MBnL1A.png" /><figcaption>Source:<a href="https://sites.google.com/qq.com/face-anti-spoofing/welcome/challengeiccv2021"> ICCV Antispoof Challenge</a></figcaption></figure><p>All of these so-called “cheap fakes” pale in sophistication to the latest breed of spoofs called “deep fakes,” which are convincing spoofs produced by specially trained AI. Generating high quality deep fakes is still beyond the means of most ordinary cybercriminals, but well within the scope of state-sponsored criminal organizations. Although there have been concerns that a well-timed <a href="http://v/">deep fake could spark conflict or war</a>, most of the fakes in 2021 were <a href="https://www.trtworld.com/magazine/deepfakes-and-cheap-fakes-the-biggest-threat-is-not-what-you-think-43046">predominately cheap ones</a>.</p><p>So, how does a criminal acquire an image of someone’s face? Besides searching popular sites like Google or LinkedIn, criminals can mine the images from social media posts. If criminals can’t locate any on public sites, they could purchase images of their victim’s faces on the dark web. The dark web’s data markets specialize in the illegal trade of private information, often extracted from hacked databases. In one of the worst data breaches in recent years, <a href="https://us.norton.com/internetsecurity-emerging-threats-biometric-data-breach-database-exposes-fingerprints-and-facial-recognition-data.html">criminals exposed 28 million records of private biometric data</a>, which included fingerprint and photos of faces.</p><h4>Liveness Detection To The Rescue</h4><p>Clearly, apps that leverage face recognition must also ensure that a real live person is present during a face check. Failure to do this leaves these apps vulnerable to face spoofing and subsequent fraud. Liveness algorithms complement and augment face recognition, resulting in robust face checks that users and app developers can trust.</p><p>So, how do these algorithms work? Many apps that leverage liveness detection capture a short video of the user during the face checking process. Alternatively, some apps grab a quick succession of shots while the user performs a specific motion or gesture in front of the camera. The liveness detection algorithm processes the image or video with specialized computer vision, acquiring mathematical certainty that a real human was present during capture rather than being the result of artificial production.</p><p>Some algorithms will also check for a “replay attack”, that is, a video recording of a real user presented from another device close by. There are tell-tale signs embedded in the captured video or images that reveal a criminal has presented photos of faces or pre-recorded videos.</p><p>A physical mask, even if it passes a human motion or gesture check, leaves evidence of its synthetic nature across the captured image’s color channels.</p><p>The best algorithms implement multiple levels of defense, and pick up on the subtle clues left behind by the spoof attacks.</p><h4>Experience Matters</h4><p>At SmileIdentity, we have been working on face recognition and identity management solutions since 2016. We have seen the rapid evolution of biometric crime, the constantly evolving sophistication of criminals, and we have evolved our custom liveness checking technology to stay ahead of the ongoing threat of face spoofing.</p><p>Our liveness check technology is called SmartSelfie™ and its powered by 6 AI-based anti-spoof models. SmartSelfie™ensures our customers that criminals can’t present fake or stolen images of faces or videos in order to commit fraud against their victims.</p><p>We constantly measure ourselves against our own privately curated database of face spoofs similar to the ones shown above, including selfie photos or printed posters. In our last evaluation, SmartSelfie™ detected spoof attempts with an accuracy of 98.8%.</p><p>And, we continually train our algorithms. As new data arrives, our human review team flags new spoof attempts, then we incrementally update our AI-based anti-spoof models. This ongoing learning and re-training keeps our service up to date with the latest types of spoof attacks as they start to surface.</p><p>In addition, we have optimized our face verification algorithm by training it on the ethnic variety of our user base in Africa. By doing so, we virtually eliminate face match failures that would likely be interpreted as fraud by other systems. At 99.8% accuracy, we beat Amazon Rekognition’s performance at 96%, measured on a public dataset comprising a balanced set of black and white faces.</p><h4>Conclusion</h4><p>Without a guarantee of proper liveness checks, such as SmileIdentity’s SmartSelfie™, an app’s face checks are vulnerable to face spoofing. If you are an app developer looking for an identity management solution that implements face verification with state-of-the-art liveness detection, please visit our <a href="https://www.smileidentity.com/">website </a>for more information, or reach out to our <a href="https://www.smileidentity.com/contact-us/">sales team for a demo</a>.</p><p>In addition to face spoofing we have seen firsthand the rise of other type of criminal activity. For example, we’ve noticed an uptick in fraudsters trying to use the same identity information to sign up for multiple accounts at the same time. In our next blog, we will talk about how we use a technique called “deduplication” to thwart this kind of costly fraud.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0acf5999a44" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smileidentity/not-all-face-recognition-is-created-equal-the-need-for-liveness-b0acf5999a44">Not All Face Recognition Is Created Equal: The Need For Liveness.</a> was originally published in <a href="https://medium.com/smileidentity">SmileIdentity</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Watts The Deal With Power ? Part I]]></title>
            <link>https://medium.com/big-ann-benchmarks/watts-the-deal-with-power-part-i-e86aef0dad5b?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/e86aef0dad5b</guid>
            <category><![CDATA[embedding]]></category>
            <category><![CDATA[power]]></category>
            <category><![CDATA[nearest-neighbor-search]]></category>
            <category><![CDATA[neurips]]></category>
            <category><![CDATA[data-center]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Thu, 10 Jun 2021 17:12:15 GMT</pubDate>
            <atom:updated>2021-06-11T15:30:24.117Z</atom:updated>
            <content:encoded><![CDATA[<h4>How We Implement Power Benchmarking In The Billion-Scale Approximate Nearest Neighbor Search Challenge</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*NdYAsxpZ9b69Rs8wysZDfA.jpeg" /></figure><h4>Introduction</h4><p>We announced in May that NeurIPS 2021 will host a unique data and algorithm challenge in billion-scale approximate nearest neighbor search (ANNS.) Participating teams will be evaluated across a set of challenging datasets, each with a billion records. We employ search accuracy ( measured as <em>recall</em> vs <em>throughput</em> ) as the defining ranking metric for the T1 and T2 competition tracks, which limit the size of RAM to 64GB within a standard server-grade system hosted in Microsoft’s Azure Cloud. However, in the T3 track, we don’t enforce the same hardware restrictions. T3 will also add two additional leaderboards, one that ranks participants relative to power usage and one related to hardware cost. In this blog, we will discuss the power leaderboard and we’ll get into some of the details around how we will collect and compute the power benchmarks.</p><h4>Why Do We Care About Power Consumption ?</h4><p>Let’s first ask ourselves why we even care about power consumption. For most of us, our direct relationship with a machine’s power consumption becomes most apparent when our laptops warn us the battery power is low. Gamers understand the need to purchase additional external power supplies required for the higher-end GPU cards when building a top-notch PC gaming workstation. Crypto miners who run their own on-prem hardware know all too well the importance of acquiring power efficient hardware to lower their power bill. For those of us who leverage the public cloud for day-to-day work, we are typically far removed (both literally and figuratively) from the power consumption of the cloud services we use and the underlying machines that run them (other than how it might be abstractly factored into the service’s cost.)</p><p>All that said, awareness surrounding the growing power demands of data centers has increased significantly over the past few years. We all generally agree that indeed “software is eating the world,” and implicitly we expect more and more machines are required to power this software-as-a-service, industrial revolution. It’s then not too surprising to learn what should be shocking facts about the growing demand for power at data centers:</p><ul><li>Global data centers consumed an estimated 205 terawatt-hours (TWh) in 2018, or 1 percent of global electricity use [1].</li><li>The amount of energy used by data centers doubles approximately every four years, meaning that data centers have the fastest-growing carbon footprint of any area within the IT sector [2].</li></ul><p>This will likely increase further as workloads become more data-intensive and AI-centric. In one highly recognized paper, researchers measured the cost of training a 200 million parameter transformer-based NLP neural network optimized with neural architecture search [3]. They found that model training and model optimization consumed enough energy to equate to 626K pounds of CO2 emissions. To put that into perspective: round trip air travel between SF and NY produces 2K pounds of CO2 emissions, and the average car produces 126K pounds of CO2 emissions in its lifetime. And that was reported in 2019. The latest NLP transformer models in 2021 are well into the billions of parameters.</p><h4>What’s Being Done?</h4><p>Fortunately, the major public cloud providers ( Microsoft’s Azure, Amazon’s AWS, and Google’s GCP ) have all already started major efforts to improve the power efficiency and to offset the carbon footprint of their data centers:</p><ul><li>Amazon AWS has recently announced they are providing low power ARM-based processors in their EC2 compute instance service. ARM-based chips are the CPUs that power most IOT devices and mobile devices [5].</li><li>In 2017, Google became the first large-scale public cloud provider to match 100% of its electricity consumption with renewable energy [6].</li></ul><p>Companies like Apple, Facebook, and Uber with similarly large-scale but private compute server footprints also have “green data center” initiatives underway. Industry wide collaborations like the Open Compute Project promote openly sharing ideas, specifications, and other intellectual property to maximize innovation in this space [7].</p><p>Clearly, new hardware and chipsets specifically designed for power efficiency will continue to be a major component of the design of future data centers. But software also has an equally important role to play. For example, compilers will need to be hardware-aware and will need to be able to produce compiled code that can leverage power efficient hardware features when they are available. Software engineers, data scientists, and algorithm developers must also be aware of their coding decisions and how those decisions can affect not just the typical metrics such as speed and accuracy, but also power consumption.</p><p>Creating this awareness was one of the goals of the T3 challenge, and that is why we are maintaining a separate leaderboard that ranks participants based on the measured power consumption of their algorithm. Before we get into those details, let’s take a step back and review some of the basic science around power and power consumption.</p><h4>The Science Of Power</h4><p>For many of us, our first encounter with the science of power occurred during our first lessons in physics and chemistry. We are taught very early that the classical notion of “energy” is conserved in any closed system, constantly being converted to and from its <em>kinetic</em> and <em>potential</em> forms. The symmetry of the energy conservation law of the universe is not only intuitively appealing, but is a fundamental underpinning of all the physical sciences and applied engineering fields, from astrophysics to biochemistry to civil engineering.</p><p>Often the concepts of energy and power are used synonymously, but there is a very subtle and important difference. A good example that demonstrates the difference- imagine lifting a box from the ground: it takes the same amount of energy to lift a box no matter how fast you lifted it. Power is another matter. Power defines how fast energy is consumed or transferred. Power is a function of energy and time. To lift the box faster, more power is required.</p><p>Power is usually reported as <em>watts</em>. To convert from power to energy over a certain period of time, you would just multiply the power measured over that time interval by the time interval ( assuming the power was constant throughout that time interval.). That is why you will typically see power consumption metrics like <em>kilowatt-seconds </em>or<em> kilowatt-hours. </em>Those quantities are reporting the total energy used over a time period.</p><h4>How Do We Measure Power Consumption In A Server?</h4><p>If you’ve seen a picture of a modern datacenter, you’ve likely noticed rows and rows of cabinets side-by-side. If you looked inside each cabinet (often called a rack) you would see multiple modules stacked vertically on top of each other. Typically each module is a rectangular chassis full of electronics, such as a motherboard with a CPU or two, add-on PCIe boards, network cards, hard drives, power supplies, and other electronics. The size of a module chassis is typically measured in integral units related to its height, starting with 1U (or 1 unit). Typical sizes are 1U, 2U, 3U. 4U module chassis are becoming more popular, for example, NVidia’s DGX system needs a 4U form factor to house all 8 of its GPUs boards.</p><p>A modern chassis is more than just a metal box that houses the electronics. The chassis itself will contain some light-weight, dedicated electronics that serve very specific purposes. The primary purpose is to support remote power management. Most chassis will host a small web server in which you access a simple web app which enables remote power down and power up to the system. Some support KVM, which is technology that allows you to remotely view the video output of the system and also control the mouse and keyboard input. These are tremendously useful tools if you need to manage a system remotely. Sometimes “turning it off, turning it back on” is the only resolution to a problem at a server in a datacenter!</p><p>These remote control capabilities are so useful that the server and datacenter industry has come up with a standard called IPMI (Intelligent Platform Management Interface.). Chassis manufactures that support the IPMI standard will benefit from all of the existing tools and services that data center engineers already use to manage their fleet of systems.</p><p>In addition to remote management, IPMI supports the notion of sensors. There are a broad array of sensors that a chassis manufacturer can build into their systems and this typically includes power monitoring.</p><p>The image below shows a listing of the sensor that are available for a chassis we use from the company called Advantech. This particular model is a 2U chassis system which houses a 2-CPU/56 core Intel Xeon chipset [8].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tyesK5aa2NsrqscI87q-qg.png" /><figcaption>Sensors available for monitoring in Advantech’s Sky 6200 2U Chassis.</figcaption></figure><p>The sensors which contain the sub-string “POWER_IN” are the sensors related to all the power supplies for the chassis and all the electronics the chassis contains. Notice there are two sensors related to two separate power supplies. It’s not unusual for a chassis to contain several independent power supplies. The total power to the system is simply the sum of all individual “POWER_IN” sensors.</p><p>We leverage these IPMI power sensors to assess the total power consumption of the participant algorithms in the T3 track of the NeurIPS Billion-Scale Approximate Nearest Neighbor Search Challenge.</p><h4>So How Much Power Do Different Server Workloads Consume?</h4><p>The following graphic below shows the power consumption of various workloads, all running for the same period of time (2 minutes). The workloads were run in Ubuntu Linux in a system with a 2-CPU/56 core Xeon chipset and VT100 GPU, which is one system we will be leveraging for the T3 track. We sampled all the power IPMI POWER_IN sensors at 1 second intervals and computed the total power consumption, reported as <em>kilowatt * seconds</em>. We took 3 different readings for each application to capture of any significant variance ( the red line is the median value ). Details of the workloads are outlined below the graphic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*18oOKu8xqWolFf7wlmnTSA.png" /></figure><p>Definition of the workloads A-F:</p><ul><li>For A, we ran no special workloads. So this is the power consumption of the system when it’s idle, for 2 minutes.</li><li>For B, we ran the linux utility called “stress” with the parameters “-cpu 1” which keeps 1 of the 56 CPU cores busy at 100%, for 2 minutes [9].</li><li>For C, we ran the linux utility called “stress” with the parameters “ -cpu 14” which keeps 14 of the 56 CPU cores busy at 100%, for 2 minutes.</li><li>For D, we ran the linux utility called “stress” with the parameters “-cpu 56” which keeps all 56 CPU cores busy at 100%, for 2 minutes.</li><li>For E, we ran the utility called “gpu-burn” which keeps all cores of the V100 GPU busy, for 2 minutes. None of the CPU cores were stressed during this time [10].</li><li>For F, we ran simultaneously the linux “stress” utility and the “gpu-burn” utility for 2 minutes. For stress, we used the parameters “-cpu 56 -vm 56 -io 56 -hdd 56” which keeps all 56 cores busy at 100% and applies load to virtual memory, RAM, and the hard drive. “gpu-stress” keeps all cores of the V100 GPU busy at 100%.</li></ul><h4>Whats Next?</h4><p>In Part 2 of this blog we get into the following details:</p><ul><li>How we convert the time series information from the IPMI power sensors to total power consumption and the software tools we use to do this.</li><li>Why there is small variance in the computed power consumption for the same workload and how we deal with that for the competition.</li><li>How we use the use power consumption to rank participants algorithms in the T3 track’s power leaderboard.</li></ul><h4>References</h4><ol><li><a href="https://energyinnovation.org/2020/03/17/how-much-energy-do-data-centers-really-use/">https://energyinnovation.org/2020/03/17/how-much-energy-do-data-centers-really-use/</a></li><li><a href="https://www.datacenterknowledge.com/industry-perspectives/data-center-dilemma-our-data-destroying-environment">https://www.datacenterknowledge.com/industry-perspectives/data-center-dilemma-our-data-destroying-environment</a></li><li><a href="https://arxiv.org/pdf/1906.02243.pdf">https://arxiv.org/pdf/1906.02243.pdf</a></li><li><a href="https://news.microsoft.com/2018/05/17/the-microsoft-cloud-can-save-customers-93-percent-and-more-in-energy-and-carbon-efficiency/">https://news.microsoft.com/2018/05/17/the-microsoft-cloud-can-save-customers-93-percent-and-more-in-energy-and-carbon-efficiency/</a></li><li><a href="https://aws.amazon.com/blogs/machine-learning/aws-and-nvidia-to-bring-arm-based-instances-with-gpus-to-the-cloud/">https://aws.amazon.com/blogs/machine-learning/aws-and-nvidia-to-bring-arm-based-instances-with-gpus-to-the-cloud/</a></li><li><a href="https://cloud.google.com/sustainability">https://cloud.google.com/sustainability</a></li><li><a href="https://www.opencompute.org/about">https://www.opencompute.org/about</a></li><li><a href="https://www.advantech.com/products/9fe67b0b-26e8-493e-9e73-2ab4eed2b38e/sky-8201/mod_56a399ee-e66a-48d6-adaf-9e90580e533f">https://www.advantech.com/products/9fe67b0b-26e8-493e-9e73-2ab4eed2b38e/sky-8201/mod_56a399ee-e66a-48d6-adaf-9e90580e533f</a></li><li><a href="https://linux.die.net/man/1/stress">https://linux.die.net/man/1/stress</a></li><li><a href="https://lambdalabs.com/blog/perform-gpu-and-cpu-stress-testing-on-linux/">https://lambdalabs.com/blog/perform-gpu-and-cpu-stress-testing-on-linux/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e86aef0dad5b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/big-ann-benchmarks/watts-the-deal-with-power-part-i-e86aef0dad5b">Watts The Deal With Power ? Part I</a> was originally published in <a href="https://medium.com/big-ann-benchmarks">Big-ANN-Benchmarks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NeurIPS 2021 Announcement: The Billion-Scale Approximate Nearest Neighbor Search Challenge]]></title>
            <link>https://medium.com/big-ann-benchmarks/neurips-2021-announcement-the-billion-scale-approximate-nearest-neighbor-search-challenge-72858f768f69?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/72858f768f69</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[vector-search-engine]]></category>
            <category><![CDATA[embedding]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[nearest-neighbor-search]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Tue, 18 May 2021 18:56:19 GMT</pubDate>
            <atom:updated>2021-05-20T22:19:40.035Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OtKDB5QyX1LGnyuA4azGrg.png" /></figure><p>We are excited to announce that this year’s NeurIPS 2021 Conference will host a first-of-its-kind competition in large scale approximate nearest neighbor search (ANNS). We will be inviting teams from the search community, both from academia and industry, to enter and submit their algorithm. Please see our <a href="http://big-ann-benchmarks.com/index.html#call">official call-for-participation</a>.</p><p>During the competition, participants will be evaluated across a set of challenging datasets, each containing at least 1 billion vector records. We have chosen benchmarks that will compare algorithms by hardware cost, accuracy, and performance and we’ll be posting the results and rankings on public leaderboards. Finally, at the NeurIPS 2021 Conference in December, we will announce the winning teams.</p><p>In this blog, we kick things off by providing some background in large-scale ANNS while touching upon our motivation behind organizing this competition. We’ll also discuss some of the competition logistics, the timeline of events leading up to NeurIPS 2021, and, importantly, how you can participate.</p><h4>What is Approximate Nearest Neighbor Search?</h4><p>Approximate Nearest Neighbor Search (ANNS) is a problem of fundamental importance to search, retrieval, and recommendation. In this problem, we are given a dataset <em>P</em> of points along with a pairwise distance function, typically the <em>d</em>-dimensional Euclidean metric or cosine similarity with <em>d</em> ranging from 50 to 1000. The goal is to design a data structure that, given a target <em>k</em> and a query point <em>q</em>, efficiently retrieves the <em>k</em> nearest neighbors of <em>q</em> in the dataset <em>P</em> according to the given distance function.</p><p>The challenge for ANNS algorithm designers is to craft a data structure that enables fast retrieval of the <em>k</em> nearest neighbors even as the database size grows. Several contemporary large-scale ANNS algorithms employ a class of data structure called an <em>index</em> which eliminates the need to compute a pair-wise distance between the search query and every record in the database. It can do this at the expense of a loss of accuracy, which is where the “approximate” term in ANNS comes into play. The trade-off between accuracy ( measured typically as <em>recall</em>) and search latency (measured typically as <em>query</em> <em>throughput</em>) is an important consideration when designing and evaluating ANNS performance.</p><p>In the past few years, we’ve seen a lot of new research and creative approaches tackling large-scale ANNS, including:</p><ul><li>Hash-based, partition-based, and graph-based indexing strategies ( as well as hybrid indexing approaches. )</li><li>Mixing RAM and SSD storage to efficiently store and process large datasets that exceed the size of RAM</li><li>Running on accelerator hardware such as GPUs, FPGAs, and other custom in-memory silicon.</li><li>Leveraging machine learning for dimensionality reduction of the original vectors.</li></ul><p>While the academic interest has seen an uptick, many of these algorithms have also made their way out of the research lab and into the real-world. Several implementations of large-scale ANNS now appear in production and high availability datacenter contexts- powering enterprise-grade, mission-critical, and web-scale search applications. In these deployment scenarios, benchmarks such as cost, preprocessing time, power consumption become just as important as the recall-vs-latency tradeoff.</p><p>The following are just some examples of the applications that leverage large-scale ANNS:</p><ul><li><strong>Document Ranking</strong> Retrieve the most appropriate document for a user query from a large corpus.</li><li><strong>Question Answering</strong> Highlight the sentence or paragraph from a large knowledge store (e.g. Wikipedia) that best answers a question.</li><li><strong>Entity Linking</strong> Match user query to a database of known entities.</li><li><strong>Ads Retrieval</strong> Match user query to a database of ad keywords.</li><li><strong>Image Match</strong> Find the best match to an image from a pool of known images.</li><li><strong>Text to Image Search</strong> Find the images that best match a given query.</li><li><strong>Map Match</strong> Find the best match to a picture from satellite data.</li><li><strong>Extreme Classification</strong> Find appropriate labels from amongst a billion for a datapoint (labels here can be text categories, relevant pages, users, etc.)</li><li><strong>Link prediction</strong> using graph embeddings.</li><li><strong>Contrastive Sampling</strong> Find hard negatives for training models using sparse data.</li><li><strong>Molecule Similarity Search</strong> Find structurally similar molecules to a virtual query molecule in computational drug discovery</li></ul><h4>Why Do We Need This Competition?</h4><p>Clearly, large-scale ANNS is a very important real-world problem cutting across application domains. Despite the broad range of algorithms and approaches for ANNS, most empirical evaluations of algorithms have focused on smaller datasets (1 million points). However, deploying recent algorithmic advances in ANNS techniques for search, recommendation and ranking at scale requires support at billion, trillion or larger scale. Barring a few recent papers, there is limited consensus on which algorithms are effective at this scale.</p><p>We believe that this challenge will be impactful in several ways:</p><ul><li>Provide a comparative understanding of algorithmic ideas and their application at scale.</li><li>Promotes the development of new techniques for the problem and demonstration of their value.</li><li>Provide a compilation of datasets to enable future development of algorithms.</li><li>Introduces of a standard benchmarking approach.</li></ul><p>We also believe that by unifying many of the interested parties in this problem (ranging from theory to possible applications), we will foster more cross-collaboration and collectively advance the field at a more rapid pace.</p><p>We hope to share what we learn during this competition in a research publication which will be publicly available.</p><h4>What Datasets Will The Competition Use ?</h4><p>We will evaluate participants on the following billion-scale datasets:</p><ul><li><strong>Microsoft-Turing-ANNS</strong> is a new dataset being released by the Microsoft Turing team for this competition.</li><li><strong>Microsoft-SPACEV1B</strong> is a new dataset being released by Microsoft Bing for this competition.</li><li><strong>Facebook-simsearchnet</strong> is a new dataset being released by Facebook for this competition.</li><li><strong>Text-to-Image search from Yandex</strong> is a new cross-model dataset, where database and query vectors can potentially have different distributions in a shared representation space.</li><li><strong>The BIGANN dataset</strong> which contains 1 billion SIFT descriptors extracted from a large image dataset.</li><li><strong>The Yandex DEEP1B dataset</strong> consisting of the outputs of a DNN for a billion images on the web.</li></ul><p>The datasets, as well as the ground truth nearest neighbors of a query set, will be made available for download prior to the start of the competition. Please note that datasets are subject to change. More details about the competition datasets can be found at our <a href="http://big-ann-benchmarks.com/">competition website.</a></p><h4>How Are Participants’ Algorithms Evaluated?</h4><p>Participants will submit their ANNS implementation as a docker container (the required format of the docker container will be provided to participants before competition begins.) During the evaluation phase of the competition, we will download the docker container onto a private machine and run the participant’s code on each of the aforementioned datasets. We will support up to 10 different search parameter sets for each dataset.</p><p>The competition will leverage datacenter grade machines that will be in complete control of the organizers. The exact hardware specification depends on the competition track. In the T1 track, we will constrain the size of RAM to 64GB. In the T2 track, in addition to 64GB of RAM, the machine will host a 1TB SSD drive.</p><p>In the T1 and T2 tracks, we will collect the following benchmarks:</p><ul><li>search time ( measured as <em>recall vs throughput</em> )</li><li>index build time</li></ul><p>Note that we will exclude participants that are unable to build an index within a certain period of days. This time limit, the exact details on how we collect and compute the benchmarks, how the benchmarks translate into leaderboards rankings, as well as specific machine and operating system details will be made public soon, before the competition begins. The latest updates can be found on the <a href="http://big-ann-benchmarks.com/">competition website.</a></p><h4>What If I Have Custom Hardware?</h4><p>The T3 track of the competition will support non-standard hardware such as GPUs, AI accelerators, FPGAs, and custom in-memory silicon. In this track, participants will either 1) send their hardware (such as PCIe boards) to us at their expense 2) or participants will give us remote access to their systems.</p><p>In the T3 track, we will collect the following benchmarks:</p><ul><li>search time<em> ( </em>measured as <em>recall vs throughput )</em></li><li>index build time</li><li>cost and power ( measured as <em>throughput/watt</em> and <em>MSRP/watt</em> )</li></ul><p>We will ask the participant to provide details and proof of their hardware’s cost in the form of manufactured suggested retail price (MSRP).</p><p>As in the T1/T2 track, we will exclude participants that are unable to build an index within a certain period of days. This time limit, the exact details on how we collect and compute the benchmarks, and how the benchmarks translate into leaderboards rankings will be made public on our <a href="http://big-ann-benchmarks.com/">competition website</a> before competition begins.</p><h4>What’s The Competition Timeline?</h4><p>We roughly aim for the following timeline:</p><ul><li><strong>May:</strong> release of data, guidelines, and a call for participation. The competition registration web site goes on-line.</li><li><strong>Mid-June:</strong> Baseline results, testing infrastructure and final ranking metrics released.</li><li><strong>June 30th:</strong> Participants in need of compute resources to submit an expression of interest.</li><li><strong>Mid-July:</strong> Allocation of compute resources.</li><li><strong>July 30th:</strong> Final deadline for participants to submit an expression of interest through CMT.</li><li><strong>October 22nd:</strong> End of competition period. Teams to release of code in a containerized form, and complete a pull request to the eval framework with code to run the algorithms.</li><li><strong>October 29th:</strong> Participants submit a brief report outlining their algorithm and results.</li><li><strong>Mid-November:</strong> Release of preliminary results on standardized machines. Review of code by organizers and participants. Participants can raise concerns about the evaluation.</li><li><strong>Early December:</strong> Final results published, and competition results archived (the competition will go on if interest continues).</li><li><strong>During NeurIPS:</strong> Organizers will provide an overview of the competition and the results. Organizers will also request the best entries (including leaderboard toppers, or promising new approaches) to present an overview for further discussion.</li></ul><p>For the latest updates on the timeline, please visit the <a href="http://big-ann-benchmarks.com/">competition website</a>.</p><p><strong>Who’s Running This Competition?</strong></p><p>The following people have generously donated their expertise, time and resources to organize and run this competition:</p><ul><li>Harsha Vardhan Simhadri, Microsoft Research India</li><li>George Williams, GSI Technology</li><li>Martin Aumüller, IT University of Copenhagen</li><li>Artem Babenko, Yandex</li><li>Dmitry Baranchuk, Yandex</li><li>Qi Chen, Microsoft Research Asia</li><li>Matthijs Douze, Facebook AI Research</li><li>Ravishankar Krishnaswamy, Microsoft Research India and IIT Madras</li><li>Gopal Srinivasa, Microsoft Research India</li><li>Suhas Jayaram Subramanya, Carnegie Mellon University</li><li>Jingdong Wang, Microsoft Research Asia</li></ul><p><strong>How Do I Enter The Competition?</strong></p><p>Registration is now open! Please visit the <a href="http://big-ann-benchmarks.com/index.html#call">official call-for-participation</a> for the details.</p><p><strong>What’s Next?</strong></p><p>We will be blogging through-out the competition on various topics, including:</p><ul><li>Registration instructions and more details regarding competition logistics and rules.</li><li>Detailed specification of the evaluation machine hardware and operating system.</li><li>Further details on how we collect and compute the competition benchmarks.</li><li>Descriptions of various kinds of ANNS algorithms and how they work.</li><li>Updates on leaderboard activity.</li><li>Other late-breaking news and announcements.</li></ul><p>So please <a href="https://medium.com/big-ann-benchmarks">stay tuned here</a> or get the latest updates at the <a href="http://big-ann-benchmarks.com/">official competition web-site</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72858f768f69" width="1" height="1" alt=""><hr><p><a href="https://medium.com/big-ann-benchmarks/neurips-2021-announcement-the-billion-scale-approximate-nearest-neighbor-search-challenge-72858f768f69">NeurIPS 2021 Announcement: The Billion-Scale Approximate Nearest Neighbor Search Challenge</a> was originally published in <a href="https://medium.com/big-ann-benchmarks">Big-ANN-Benchmarks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The New Face of Biometrics In The Era of Hacks, Fakes, Bans, and GANs]]></title>
            <link>https://medium.com/gsi-technology/the-new-face-of-biometrics-in-the-era-of-hacks-fakes-bans-and-gans-14b64d9c6332?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/14b64d9c6332</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[biometrics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[surveillance]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Thu, 24 Oct 2019 20:52:00 GMT</pubDate>
            <atom:updated>2020-02-05T23:12:09.025Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*WdU3is0bxV5QLKNVW2ANKw.png" /></figure><p><em>I have been hosting a series of panels with the same title in the Bay Area. The first panel was at ODSC West 2019, in Burlingame. Panelists included Michael Gormish (Clarifai), Mark Straub (Smile Identity), Hyrum Anderson (Endgame Security), and Giorgio Patrini (Deeptrace). Their bios are at the end of the article. It was definitely be an enlightening and spirited discussion! Follow me on medium here or twitter @cgeorgewilliams for future events and dates.</em></p><h4>Introduction</h4><p>“Hacks, Fakes, Bans, and GANs” were just four of the many topics I could have chosen for the title, but these particular ones put together exhibit a nice poetic quality. The word cloud at the top shows perhaps a more complete set, the full diversity of concerns that intersect contemporary biometrics— topics from algorithms, to cybersecurity, to policy, to artificial intelligence.</p><p>The technology behind biometrics is improving rapidly. The business of biometrics is booming. But it’s the social impact around biometric identification and use in surveillance that has captured many of the headlines lately. Trust, fairness, privacy, transparency, ethics- biometrics underlies many hot-button social issues, and this conversation is happening on a global scale.</p><h4>How Did We Get Here?</h4><p>Its likely we wouldn’t be talking about these social issues, if the tech behind biometrics itself hadn’t already surpassed a threshold of adoption and performance.</p><p>Mass adoption of biometric tech is already here, and growing rapidly:</p><ul><li>Many mobile device and laptop users already authenticate and unlock using some form of biometric such as fingerprint or face recognition.</li><li>Airports were early adopters, and its now routine to encounter biometrics at customs. The biometrics company CLEAR employs biometrics in its kiosks at many airports. Subscribers to the service bypass the long lines at x-ray security once they are authenticated.</li><li>China alone will deploy over 600 million surveillance cameras with face recognition by 2020, compared to 160 million in 2016[28].</li><li>Due to economy of scale, inexpensive fingerprint sensors will start to appear on credit cards ( called biometric or fingerprint cards. )</li></ul><p>Biometrics accuracy and performance have improved dramatically. Face recognition is a great example. The graphic below shows the results of various US Government sponsored efforts to quantify the error of face recognition systems over the last few decades. Between 1993 and 2006, the error rate decreased significantly. Early systems performed poorly, but by the mid-2000s they reached a decent level of accuracy [1].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*umeMk2t_uuqPbNW8QPzUlw.png" /><figcaption>The error rate of face recognition systems reduce dramatically over the last two decades [1].</figcaption></figure><p>The best algorithms during this time period were not based on deep learning. That’s because neural networks weren’t yet powerful enough to beat other algorithms such as PCA, support vector machines, among others. Since then, the Deep Learning revolution has occurred and now algorithms such as convolutional neural networks (CNN) are the state-of-the-art in face recognition.</p><p>The chart below shows the evolution of CNNs for each year between 2015 to 2017 (last column in the violin plot) [2]. Not only did the performance jump significantly year-by-year, but also CNN’s surpassed human participants measured on the same face recognition task. “A2017b” is a resnet based CNN with a specialized loss function [20].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RELQKDtuoBhoREKWVE8rbw.jpeg" /><figcaption>Comparing CNNs neural network algorithms with various human experts in face recognition task. “Examiners” and “Reviewers” are trained specialists in face forensics, often employed by law enforcement. Super-recognizers are un-trained, but exhibited a high degree of capability for the task. Students were randomly selected [2].</figcaption></figure><p>Humans often perform better at face recognition under many conditions that challenge the best algorithms, including recognizing faces under extremes of illumination and non-frontal poses. Also, SOTA algorithms continue to have higher error rates for darker-skinned subjects and females. Nevertheless, the recent technological progress inspired by modern day neural networks, hype or otherwise, is a stunning achievement.</p><h4>The Biometrics Market</h4><p>Face recognition is just one example of biometric technique that has undergone continuous technological advancement and rapid adoption. The biometrics market as a whole encompasses the other usual suspects, the so-called “hard biometric” techniques, including eye/iris recognition, voice/speaker recognition, finger-print recognition, and signature recognition.</p><p>The biometric market segments include government, defense, law enforcement, banking and finance, consumer electronics, healthcare, and transportation. The graphic below shows the past and the projected market size in the US for all of these segments. If these projections are correct, in the US alone the total biometrics market will increase six-fold by 2025 from 2014.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*3-eKZeOgWqKIkqIFYj5x5A.png" /><figcaption>Source: GrandView Research, Biometrics Market Report, 2019.</figcaption></figure><p>Globally, the trend is also striking. As the next graphic shows, the major international markets are the North America, China, and the UK. This is not too surprising, since the US, China, and England are leading the world in biometric deployments. Now, what might be surprising is the size and growth in developing parts of the world like Africa and the Middle East (both captured by “MEA”).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pfdkkhHHYiZ_k2hHbvP3zw.png" /><figcaption>Source: GrandView Research, Biometrics Market Report, 2019.</figcaption></figure><p>It’s worth pausing a moment to consider the level of broad adoption and commoditization around biometrics that we are seeing today. A decade ago, few would have predicted the scale, the global impact and the ubiquity of biometrics. Many of us now routinely authenticate to our devices using our fingerprint or face and don’t think twice about it. Biometrics have blended into our lives, so it’s easy underestimate the tech’s pervasiveness.</p><p>In a related study, people were asked to estimate how often they were recorded on surveillance cameras. The results of that study are shown below. Most people thought its 10 times or less.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*d58M-24biVORDPlmnVJcog.jpeg" /><figcaption>Source: Reolink.com [19]</figcaption></figure><p>It’s actually a lot more. Especially if you live in an urban area, such as dense city. For example, a Londoner will likely appear on 300 cameras per day, and in major cities in the US, its around 75 times a day [19].</p><p>Of course, not all of the cameras pointed at you are going to recognize you, but in many areas of the world- identifying and tracking people is already deeply intertwined with surveillance [28]. Countries like China and India are deploying mass surveillance and collecting the biometric data of all it’s inhabitants.</p><h4>Brave New World: Science Fiction No Longer</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/343/1*3pWubhB0oaNoTc8KxiW8qg.jpeg" /><figcaption>Source: “Minority Report,” Dir. Steven Spielberg, 20th Century Fox, 2002.</figcaption></figure><p>Science fiction has been warning us for decades. Consider the popular movie “Minority Report” (2002). It depicted a future where, for example, as soon as you walk into the mall, you are identified and instantly bombarded with personalized advertisements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/297/1*DqsVzHmQ5W9xTJgfR4NQCQ.png" /><figcaption>Source: “Demolition Man,” Dir. Marco Brambilla, Warner Bros, 1993.</figcaption></figure><p>In “Demolition Man” (1993) we saw a precursor to China’s social credit system. In that dystopian view of the future, the State continually listens to what you say. As soon as you utter any profanity, a nearby kiosk instantly spits out a ticket with a heavy fine.</p><p>These movies were inspired by science fiction stories that go all the way back to the 1930s. At the time the movies’ release, actual biometric technology wasn’t even close to the fictional version. So the depictions were farcical, and therefore comical. Now, the ideas don’t seem so far-fetched. This is the brave new world we live in.</p><p>It is said that technology is amoral, in the sense that it is not inherently either good or bad. It is really up to it’s creators to apply the tech for good purposes, or abuse it for malicious purposes. We are seeing this play out on a grand scale with biometrics.</p><p>I will now highlight a few things I see across that spectrum-starting with the good, then the bad, then the ugly.</p><h4>Biometrics: The Good</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*_13LoL09BcsG63r1UpAUDw.jpeg" /></figure><p><strong>Password Replacement at the Edge </strong>Biometrics is now often offered as a replacement for password-based authentication on most mobile devices and laptops, and for many apps like banking. There are a lot of issues with passwords — such as users choosing easy-to-guess passwords, or conversely, systems requiring complex passwords that are hard to remember. The FBI is recommending that all authentication systems incorporate biometrics because password-based and even multi-factor-based systems are vulnerable to attack [29].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/297/1*6wHtG4djGSxcSlQfw6BVqw.jpeg" /></figure><p><strong>Democratization of Algorithms</strong> The deep learning revolution was not started at a company, but rather in a few research labs. This means that many algorithms are in the public domain, detailed in papers that anyone can read and implement. This open endeavor continues to this day, and has amplified even further — now many research universities and industry labs contribute to the state-of-the-art. The best algorithms for face recognition, for example, were published and presented at conferences just in the last few years. I believe that rapid progress in this area could have never occurred if it were a pure commercial venture. It’s not clear how long this open-ness will last. Even Google has filed patents for fundamental techniques in machine learning[3].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*AUB8ZIHBPpUG6D35xHCA6A.png" /></figure><p><strong>Democratization of Software </strong>All of this open<strong> </strong>research is matched by a lot of open-source software activity. Many deep-learning research papers now also include a TensorFlow or PyTorch open-source software implementation, that is free to download and can be used by anyone [4,5]. This, combined now with now common-place public cloud infrastructure such as Azure, Google Cloud, and Amazon AWS, has created a burgeoning turnkey services eco-system (AI-As-A-Service, Machine-Learning-As-Service, Biometrics-As-A-Service, etc.). You do not need to hire a team of researchers, nor own any computing equipment, in order to deploy state-of-the-art biometric solutions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*nIIsqYmBtH5rHg7-MhDQcg.jpeg" /></figure><p><strong>Democracy Enabler </strong>Besides facilitating e-commerce transactions ( Face ID, Voice Pay, etc ), several governments around the world are considering biometrics as a way to allow citizens to vote. In many parts of the developing world, getting to a location to register or to vote can be extremely difficult. If done right, this could be a democracy game changer in many countries [7].</p><h4>Biometrics: The Bad</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/281/1*mbBA_iPHitzKMPYMPqRjTg.jpeg" /></figure><p><strong>Questionable Application of Biometrics By Law Enforcement </strong>Many users of biometric tech don’t understand its limitations. This can create a dangerous situation. Consider a law enforcement entity with the power to arrest and detain people based solely on suspicion or light evidence. It was reported that a major police department in the US had been searching databases of mug-shots in order to find a suspect in a crime. But, the search was not based on surveillance of the criminal caught in the act, but rather an eye-witness claiming the perp looked like a celebrity. A picture of the alleged look-like celebrity was used in the mugshot database face search [8].</p><p>In related news, Amazon, which sells it’s face-recognition-as-a-service (Rekognition) to law enforcement, underwent heavy scrutiny and criticism. In a study, the ACLU used Amazon’s tech to search a public mugshot database (25,000 people), using, as the query, the faces of all 535 members of Congress. The result of the search turned up 28 false matches [23]! What’s worse, 40% of the false matches were people of color. This is evidence of the racial bias in even state of the art face recognition algorithms, as well as the databases used to train them. Google has suspended the availability of its face recognition service for these reasons [26], but recently they released a new version of their personal home assistant which includes both face and voice recognition features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/265/1*TSVbvjwbVxYlVRTbTC9rHg.png" /></figure><p><strong>Biometric Data Security</strong> The so-called “hard biometrics” like your fingerprints, your voice, your face are “intrinsic.” They don’t change much even as you age, wear makeup, change hairstyles, change outfits, etc. This is why hard biometrics are a highly effective way to identify an individual. But what if the data is stolen from your mobile device or from a server in the cloud? That data could then be used to authenticate to services without your approval. You can’t change your biometrics, like you can change a password. Given the sheer number of high-profile data breaches we’ve seen recently, we should all be very concerned about how institutions are protecting our biometric data if they have it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*ZeucGJxsISjCTW1-JBLOag.jpeg" /></figure><p><strong>Biometric Data Sharing</strong> Facebook’s Cambridge Analytica fiasco revealed to everyone the level of sophistication and the scale of the data sharing economy that has been operating for many years, invisible to online users. Algorithms are currently being used to recommend products and services to you, based on previous purchases, views, and clicks. This data is bought and sold in the lucrative data market. But what about your biometric data? Has that also been sold to third parties without your consent? Certainly, if any company believes that your biometrics data can also predict your purchase preferences, your data becomes indeed as “valuable as oil.” Conversely, your biometric data could also be used to restrict you from opportunities, because you don’t fit a predictive model.</p><h4><strong>Biometrics: The Ugly</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*vf9mpa5jUOzFP_zfq-Fx_Q.png" /></figure><p><strong>Mass Surveillance and Tracking </strong>I think the issues around biometric data hacking and surreptitious data sharing stems from a basic fact: biometric data is very personal information and users are now starting to feel violated when that data is stolen, or when it’s being used without consent. Mass surveillance and tracking takes this to a whole new level, because the privacy violation is now being perpetrated by a government.</p><p>Many believe that mass surveillance, if used judiciously, is just another tool to protect citizens. Consider the case of the Boston Marathon bombers who were caught because the police were able to stitch a timeline of events based on surveillance footage. I have no doubt it can be an effective tool to spot terrorists, intruders, burglars, thieves, arsonists, and other criminals.</p><p>On the other hand, mass surveillance could be used to enforce certain behaviors that the state deems proper or improper, beyond just criminal activity. For example, consider the use of surveillance in China’s “social credit system.” There have already been reports that mass surveillance has been used as a suppression technique against a muslim minority class in certain parts of the world[6]. Fortunately, privacy advocates in the country are pushing to curtail the government’s use of mass surveillance tech [27].</p><p>Until we have the right regulation and consent framework in place, I believe that the adoption of mass surveillance is too much of a slippery slope for democratic countries like the US, countries that guarantee civil liberties and certain privacy rights.</p><p>This may actually be an opportunity for tech companies to be the ethics leaders in this space. One biometrics+surveillance company, in particular, Blink Identity, is notable in this regard. As the CEO has stated recently, “We believe the key is that [identity technology] solutions must be <em>transparent, </em>the vendor must be <em>accountable</em>, and participation must be <em>voluntary.” [21]</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/285/1*cl7FJqp6QWkDgI40zzHaUg.jpeg" /></figure><p><strong>AI Synthesized Media </strong>At this point, we’ve all seen humorous video clips of face swapping or augmented reality apps that place hats or masks on video participants in real-time. New deep learning-based techniques, such as generative adversarial networks, are helping to take this kind of video manipulation to the next level. Anyone can now craft artificial videos showing, for example, a politician say something they never actually said, or making it look like a CEO did something that they actually never did.</p><p>These are deepfakes. They are a problem now, they are easy to generate with open-source tools, and they are becoming more realistic and convincing every day. Fortunately, a deepfake has not yet been directly responsible for a missile launch or tanking a stock market. But perhaps just as troubling as possible global catastrophe, deep fakes have already been used as an instrument of slander and misogyny. Deeptrace has been tracking the dissemination of deepfakes on the internet and the problem is growing at an alarming rate. There are twice as many deepfakes now compared to just a year ago [21].</p><p>Recent research demonstrated how this GAN technique can be used to spoof face recognition systems. The researchers face swapped one user onto a different user, in a video. The latter video was presented to a face recognition system that was based on two different convolutional networks (VGG and FaceNet.) The system was not able to detect the fake and instead authenticated the original user [24].</p><h4>Where do we go from here?</h4><p>Fortunately, there is a lot of work already underway to deal with many of these challenges and issues. Some of them are technical, and some are related to policy and regulation. Here are just some examples of ongoing efforts:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uC63CCf18PRmYQrJRtebBg.jpeg" /></figure><p><strong>Liveness Detection </strong>This is an anti-spoofing technique to ensure that a real human is presenting themselves to a biometric system, and not, for example, an image of a fingerprint or face. There are many techniques that range in sophistication. For example, the authentication system could guide a user to move their head in a particular direction. More complex systems look for artifacts that reveal spoofing, such as differentiating between the texture of actual skin and that of the paper a spoof photo is printed on [17]. Both the IPhone and Google’s Pixel now use depth sensing technology to prevent image-based spoofing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0xmWEcNgaup-4aeh-BStOA.png" /></figure><p><strong>Secure Silicon </strong>A few mobile platforms like Apple have decided not to back up biometric data to the cloud. Rather the sensitive data remains in the device and is processed within a highly secure area in the device’s chipset, called the secure enclave. The secure enclave is extremely tamperproof [18]. ARM-based chips having something similar, called the TrustZone. And Google Pixel phones have an entirely separate tamperproof chip called the Titan M.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QKfAW2TY0tNcs21LYQDViA.png" /></figure><p><strong>Privacy Preserving Encryption</strong> In cases where biometric data must be sent off the device to the cloud, researchers have devised an encryption scheme that can maintain privacy. Sometimes called “private encryption,” the idea is that data is encrypted using a special technique called “homomorphic encryption.” The receiver of this biometric “ciphertext” can still perform operations directly on it, without having to decrypt it. The result of the “homomorphic” operation, such as biometric verification or identification, is returned back to the sender. The entire process never reveals the original biometric data because it was never decrypted. At least one company has already deployed a product based on this technique for voice/speaker verification [9].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*s5KeyAlPcsnvSuU5fLCP0A.png" /></figure><p><strong>Federated Learning</strong> Let’s say biometric data is needed to train a deep learning-based model. Instead of pooling all that biometric data in one place and then training on it, it’s possible to distribute the training to the edge devices in a decentralized way. In this case, the “learners” at the edge contribute their trained weights to the master neural network, at a central data center. This method of training a model is called federated learning, and it avoids the situation of having to store all that biometric data in a centralized database in cloud, which would be an attractive target for hackers [11].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/1*SERQYXN7EGDYVKFrzFSJ8Q.png" /></figure><p><strong>Ensembles of Soft Biometrics</strong> The so-called hard biometrics have evolved to have very low false positive errors. For example, Apple’s FaceID claims a one a million error rate [16]. But, there are times when traditional hard biometrics can not be used. Some researchers have demonstrated that the right collection of “soft biometrics” could be just as discriminating as 1 hard biometric. In cases where the hard biometric is not available or fails, the right fusion of soft biometrics could pick up the slack [11]. Soft biometrics include eye color, hair color, body art, and includes “behavioral biometrics” like walking style(gait), gesture style(body language), how one types at a keyboard, and many others.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pWsB5Xu8vV1lIgZ6z0ijWw.jpeg" /></figure><p><strong>Detecting Deep Fakes</strong> Soft biometrics have adopted another role recently, and that is in combatting deepfakes. The idea here is that it’s extremely difficult to incorporate many of the unique traits of a person in the production of that person’s fake video. For example, consider someone who has nervous tick, or a unique way they gesture with their hands when they speak. A face-swap alone isn’t going to be enough to complete replicate a person digitally [12].</p><p>Google announced recently a comprehensive dataset of deepfakes to help researchers develop and test methods to detect them [25]. Other industry groups, including Microsoft and are sponsoring data science challenges around detecting fakes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*7P_Fz9OA6Ix6p0Uz-ekUbQ.jpeg" /></figure><p><strong>Large-Scale Regulation</strong> The GDPR ( General Data Protection Regulation ) has already gone into effect in Europe, and it holds organizations and companies highly accountable for how consumer’s data is used and protected, and that may encompass biometric data soon [13]. California will have a similar policy in place, called the CCPA ( California Consumer Privacy Act ) [14]. The Illinois’ Biometric Information Privacy Act (BIPA) requires companies to get consent prior to collecting biometric data for commercial purposes. Recently, Facebook was fined millions of dollars due to violations of BIPA. The city of San Francisco has already banned the use of face recognition by law enforcement, and other cities are considering the same action. There are several proposed bills that are specifically targeting those who craft deepfakes for malicious purposes, both in California and on Capital Hill [15].</p><h4>ODSC West 2019: Panelists</h4><p>We will cover a lot of these topics and more at the ODSC West 2019 panel by the same name. Come by our session on Nov 1st, at 11am and hear from the following experts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*_Bb6ZWyUHTtxedqXeNmhuQ.jpeg" /></figure><p><strong>Michael Gormish</strong> is Head of Research at Clarifai. His team works on bringing image and language understanding to Clarifai’s platform. Recent projects have invovled custom face recognition, object detection, tracking, visual search, and image classification. Previously he was responsible for two search engines which connected images from mobile phones to AR experiences. Work at Ricoh included considerations of document and image authenticity including precursors to blockchain technologies. His research as a computer vision scientist included inventing algorithms used in video games, digital cinema, satellite and medical image acquisition. He led several aspects of the JPEG 2000 standardization and provided key inventions used in photocopiers, digital cameras, tablets and imaging services. Michael was named Ricoh Patent Master for being a co-inventor of over 100 US patents. He earned a PhD degree in Electrical Engineering dealing with image and data compression from Stanford University and has served the research community as an Associate Editor of the IEEE Signal Processing Magazine, and the Journal of Electronic Imaging. Currently he is interested changing the world via image understanding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/1*sEXYph-32K-zzAKTwb1-yg.jpeg" /></figure><p><strong>Mark Straub</strong> is the CEO and Co-founder of Smile Identity, a leading provider of digital authentication and KYC services across Africa. Smile’s facial recognition SDKs and ID verification APIs enable banks, telecoms and fintechs to confirm the true identity of any smartphone user with just a Smartselfie™. Previously, Mark led the Khosla Impact Fund, with investments in payments, solar, lending and ecommerce across Africa and India. He began his career in investment banking and venture capital in Silicon Valley at Bank of America and then Draper Fisher Jurvetson. He is passionate about the power of technology and entrepreneurship to transform emerging markets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*A59Ge4ZmfTNGmT1Ezj8-aw.jpeg" /></figure><p><strong>Hyrum Anderson</strong> is the Chief Scientist at Endgame, where he leads research on detecting adversaries and their tools using machine learning. Prior to joining Endgame he conducted information security and situational awareness research as a researcher at FireEye, Mandiant, Sandia National Laboratories and MIT Lincoln Laboratory. He received his PhD in Electrical Engineering (signal and image processing + machine learning) from the University of Washington and BS/MS degrees from BYU. Research interests include adversarial machine learning, large-scale malware classification, and early time-series classification.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*_85cIgZSiVufmo9Yq0fftg.jpeg" /></figure><p><strong>Giorgio Patrini</strong> is CEO and Chief Scientist at Deeptrace, an Amsterdam-based cybersecurity startup building deep learning technology for detecting and understanding fake videos. Previously, he was a postdoctoral researcher at the University of Amsterdam, working on deep generative models; and earlier at CSIRO Data61 in Sydney, Australia, building privacy-preserving learning systems with homomorphic encryption. He obtained his PhD in machine learning at the Australian National University. In 2012 he cofounded Waynaut, an Internet mobility startup acquired by <a href="http://lastminute.com/">lastminute.com</a> in 2017.</p><h4>References</h4><ol><li>“FVRT 2006 and ICE 2006 Large-Scale Results,” Jonathan Phillips, et.al., NIST, March 2007.</li><li>“Face Recognition Accuracy of Forensic Examiners, Super-Recognizers, and Face Recognition Algorithms,” P. Jonathon Phillips, et.al., PNAS, June 12, 2018.</li><li>“System and Method For Addressing Overfitting in a Neural Network,” Geoffrey Hinton, et.al., US9406017B2.</li><li>TensorFlow, <a href="http://www.tensorflow.org./">http://www.tensorflow.org.</a></li><li>PyTorch, <a href="http://www.pytorch.org./">http://www.pytorch.org.</a></li><li>“China’s Hi-Tech War on its Muslim Minority,” Darren Byler, TheGuardian, April 11, 2019.</li><li>“UN Commision Exec Calls for African Nations to Focus on Civil Registration for Development Gains,” Chris Burt, BiometricUpdate, Oct 21, 2019.</li><li>“NPYD used Woody Harrelson Photo to Find Lookalike Beer Thief,” Michael R. Sisak, AP, May 16, 2019.</li><li><a href="http://privatebiometrics.com/">http://privateBiometrics.com</a></li><li>“Bags of Soft Biometrics for Person Identification,” Antitza Dantcheva, et.al., Multimedia Tools and Applications, January 2011.</li><li>“Federated Learning: Collaborative Machine Learning without Centralized Training Data,” Brendan McMahan, et.al., April 6, 2017.</li><li>“Protecting World Leaders Against Deep Fakes,” S. Agarwal, H. Farid, Y. Gu, M. He, K. Nagano, and H. Li., Workshop on Media Forensics at CVPR, Long Beach, CA, 2019.</li><li><a href="http://eudgpr.org/">http://eudgpr.org</a></li><li><a href="https://oag.ca.gov/privacy/ccpa">https://oag.ca.gov/privacy/ccpa</a></li><li>“Malicious Deep Fake Prohibition Act of 2018,” S.3805.</li><li>“About Face ID Advanced Technology,” <a href="https://support.apple.com/en-us/HT208108">https://support.apple.com/en-us/HT208108</a></li><li>BioID Liveness Detection, <a href="https://www.bioid.com/liveness-detection/">https://www.bioid.com/liveness-detection/</a></li><li>“Security enclave processor for a system on a chip,” US8832465B2.</li><li><a href="https://reolink.com/how-many-times-you-caught-on-camera-per-day/">https://reolink.com/how-many-times-you-caught-on-camera-per-day/</a></li><li>“L2-Constrained Softmax Loss For Discriminative Face Verification,” Rajeev Ranjan, et.al., arXiv:1703.09507.</li><li><a href="https://deeptracelabs.com/resources/">https://deeptracelabs.com/resources/</a></li><li>BiometricUpdate.com interview with Mary Haskett, CEO of BlinkIdentity, <a href="https://www.biometricupdate.com/201910/blink-identity-ceo-on-preserving-privacy-in-a-facial-recognition-ticketing-system">https://www.biometricupdate.com/201910/blink-identity-ceo-on-preserving-privacy-in-a-facial-recognition-ticketing-system</a></li><li><a href="https://www.theverge.com/2019/4/3/18291995/amazon-facial-recognition-technology-rekognition-police-ai-researchers-ban-flawed#targetText=In%20an%20open%20letter%20published,darker%2Dskinned%20and%20female%20faces.&amp;targetText=As%20one%20of%20the%20leading,repeatedly%20scrutinized%20in%20this%20way.">https://www.theverge.com/2019/4/3/18291995/amazon-facial-recognition-technology-rekognition-police-ai-researchers-ban-flawed#targetText=In%20an%20open%20letter%20published,darker%2Dskinned%20and%20female%20faces.&amp;targetText=As%20one%20of%20the%20leading,repeatedly%20scrutinized%20in%20this%20way.</a></li><li>“Vulnerability of Face Recognition To Deep Morphing,” Pavel Korshunov, ICBB, 2019.</li><li><a href="https://ai.googleblog.com/2019/09/contributing-data-to-deepfake-detection.html">https://ai.googleblog.com/2019/09/contributing-data-to-deepfake-detection.html</a></li><li><a href="https://www.blog.google/around-the-globe/google-asia/ai-social-good-asia-pacific/">https://www.blog.google/around-the-globe/google-asia/ai-social-good-asia-pacific/</a></li><li><a href="https://www.scmp.com/news/china/politics/article/3008844/china-working-data-privacy-law-enforcement-stumbling-block">https://www.scmp.com/news/china/politics/article/3008844/china-working-data-privacy-law-enforcement-stumbling-block</a></li><li><a href="https://www.scmp.com/news/china/politics/article/3008844/china-working-data-privacy-law-enforcement-stumbling-block">https://www.scmp.com/news/china/politics/article/3008844/china-working-data-privacy-law-enforcement-stumbling-block</a></li><li><a href="https://info.publicintelligence.net/FBI-CircumventingMultiFactorAuthentication.pdf">https://info.publicintelligence.net/FBI-CircumventingMultiFactorAuthentication.pdf</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14b64d9c6332" width="1" height="1" alt=""><hr><p><a href="https://medium.com/gsi-technology/the-new-face-of-biometrics-in-the-era-of-hacks-fakes-bans-and-gans-14b64d9c6332">The New Face of Biometrics In The Era of Hacks, Fakes, Bans, and GANs</a> was originally published in <a href="https://medium.com/gsi-technology">GSI Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The APU: Novel Hardware For Accelerated Similarity Search]]></title>
            <link>https://medium.com/gsi-technology/the-apu-novel-hardware-for-accelerated-similarity-search-beaef9b19b97?source=rss-d62fa8861970------2</link>
            <guid isPermaLink="false">https://medium.com/p/beaef9b19b97</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[visual-search]]></category>
            <category><![CDATA[similarity-search]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[cheminformatics]]></category>
            <dc:creator><![CDATA[George Williams]]></dc:creator>
            <pubDate>Fri, 29 Mar 2019 18:39:31 GMT</pubDate>
            <atom:updated>2019-03-29T19:31:49.590Z</atom:updated>
            <content:encoded><![CDATA[<p>In this blog, we will describe our custom APU chip — an innovative and exciting new design that will accelerate similarity search for a broad range of applications.</p><p>But, before we get into the details of our chip, let’s take a step back and first talk about similarity search and how it is used today.</p><h4>What Is Similarity Search ?</h4><p>Similarity search is different than traditional database search. Objects in a similarity search database are represented numerically — either as a vector of floating point numbers, integers, or as a bits.</p><p>These numeric representations are sometimes known by other names, such as “embeddings”, “codes”, or “fingerprints.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0mOUNN-HrLpB544eD2FZtw.png" /><figcaption><strong>Molecule fingerprints as bit-vectors. Each bit represents the presence of a specific functional group.</strong></figcaption></figure><p>In similarity search, a query is performed against this numeric database by computing a “distance” between the query and the objects in the database. Objects that are closer to the query are ranked higher in the results.</p><h4>Nearest Neighbors</h4><p>Usually an application may only care about getting a few of the top results from a similarity search. These top results are called the “nearest neighbors.” The application will typically only need “K” of them, or it may need just the neighbors that fall within a distance threshold. The value of “K” or the threshold are search parameters for Nearest Neighbor Search (NNS).</p><h4>Why Is Similarity Search Useful ?</h4><p>Similarity search is integral to several application domains, such as computational drug discovery and visual search. In computational drug discovery, similarity search can be used to search large molecule databases for potentially beneficial biopharmaceutical drugs. These molecules can eventually turn into “seeds” for actual synthesis and clinical trials.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/1*vES2gETGd-XxtrxRnRLnZw.png" /><figcaption><strong>Similarity search is useful in computational drug discovery.</strong></figcaption></figure><p>Visual search is a popular application in e-commerce that leverages similarity search. Here, users take a picture of something they like with a mobile app and are then instantly matched with visually similar products in inventory. There are several great visual search apps from companies like eBay, Google, Amazon, Microsoft, Pinterest, Clarifai, Walmart Labs, Wayfair, and many others.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*BO02aSyFvJJuxYoJdqnF5A.png" /><figcaption><strong>An example of Wayfair’s visual search application. Searching for the right couch has never been easier!</strong></figcaption></figure><h4>Challenges In Similarity Search</h4><p>Similarity search databases can be very large. For example, there are several openly available molecule databases that contain tens of millions of molecules. On the e-commerce side, it’s now commonplace to encounter product inventories that number into the billions of items.</p><p>End users of similarity search applications, whether they are shoppers or cheminformaticians, want their search results as fast as possible. Unfortunately, performing a distance computation between a query and all the objects in a database can be slow, especially as the database gets larger and larger.</p><h4>Scaling Similarity Search</h4><p>Scaling similarity search often involves employing a few different tricks. Briefly, they can be categorized as follows:</p><ul><li><strong>Database Partitioning</strong> — Here, a large database is partitioned into smaller parts. Similarity search is constrained by looking only into the partitions where the nearest neighbors will likely reside. By doing this, a costly “exhaustive” search or “full scan” can be avoided.</li><li><strong>Approximate Search</strong> — For some applications, 100% accurate search results may not be needed. In these cases, Approximate Nearest Neighbor (ANN) algorithms may work just as well as “exact” search. ANN methods trade a small loss in accuracy for huge gains in performance. For example, most ANN algorithms provide guarantees about what fraction of the results are the true nearest neighbors, given a certain desired level of search latency.</li><li><strong>Faster Distance Computation</strong> — It takes fewer compute cycles to compute the hamming difference between two bit-vectors than the Euclidean distance between two floating point vectors. So, the choice of numeric representation is important. Specialized hardware can also accelerate these distance computations, especially if the distance function can be programmed close to or at the level of the silicon.</li><li><strong>Parallelization</strong> — If more computational hardware is available, then more pairwise distance computations can be performed simultaneously.</li><li><strong>Reducing Memory Transfer Overhead</strong> — Moving data around can be expensive both in terms of power and latency, so reducing the data transfer bottleneck is key to maintaining similarity search performance.</li></ul><p>These tricks are often combined — such as in our custom chip for similarity search, called the APU.</p><h4>APU: Hardware Accelerated Similarity Search</h4><p>We introduce the APU, the Associative Processing Unit, for hardware accelerated similarity search. The APU chip is composed of multiple banks of SRAM memory cells interleaved with computational bit-logic.</p><p>Let’s have a closer look at how similarity search can be optimized with the APU hardware and software.</p><h4>APU: In-Memory Architecture</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0qg74LdC_Hb39UO7mx8vhA.png" /><figcaption><strong>Bit-logic is interleaved throughout the chip memory, computing on the data “in-place”.</strong></figcaption></figure><p>A picture of the chip is shown above. If you think it looks a lot like standard computer memory, you are right! GSI has leveraged its twenty years of experience creating high performance DRAM and SRAM in designing the APU.</p><p>The key difference is the bit-logic. There are 2 million of these modules interleaved throughout the chip and each can be programmed to compute simple functions on the data “in-memory.” That’s a massive number of simultaneous computations! And by computing on the data where it lives, the APU can eliminate many of the memory transfer bottlenecks that exist in most standard computer architectures.</p><h4>APU: Programmable Distance Function</h4><p>A pure embedded approach might bake an algorithm directly into the silicon. The APU takes a different approach. The APU bit-logic can be programmed. This means it can accelerate the computation of several different kinds of distance functions. For example, in addition to hamming distance, the bit-logic can be programmed for the Tanimoto distance, a popular distance function in molecule search. In addition to bit-vector based distances, the APU can be programmed to compute the standard floating point-based distances as well, such as cosine and Euclidean distance.</p><h4>APU: Approximate Search</h4><p>The APU supports and accelerates many ANN techniques, including ones that are core to the popular FAISS open source similarity search library. We support Locality Sensitive Hashing (LSH) and Product Quantization-based algorithms, and we are actively investigating other ANN methods.</p><p>We’ve performed some preliminary benchmarks on the SIFT1M dataset. The graphic below shows how the APU can accelerate LSH-based hamming search (<strong>orange</strong> line). We’ve compared it to other popular FAISS algorithms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SSWIgppIgQF0XohYiNodgQ.jpeg" /><figcaption><strong>Comparing the APU LSH approximate nearest neighbor to other state-of-the-art ANN methods</strong>.</figcaption></figure><p>As you can see, we are competitive to other state-of-the-art ANN methods, including HNSW and IVF. But this is just the beginning, as we’ve only just started to scratch the surface accelerating ANN in the APU.</p><p>We are actively exploring other optimization tricks, including the following research topics:</p><ul><li>In the APU, we can retrieve more nearest neighbors (more “K”) with little to no additional cost. This is due to the APU’s highly parallelized architecture. What this means is that we can improve the recall of approximate search significantly, with little affect on the search latency.</li><li>K-means clustering is typically used on floating point vector as a means of partitioning large datasets. We are looking also into using K-means as a way to cluster bit-vectors. This will help scale hamming-based search up to several billions of objects, while maintaining low latency.</li></ul><h4>APU Boards</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WAPbNbCB4H0eL5vnv0Fe_Q.png" /><figcaption><strong>Four (4) APU boards fit into a 1U rack. That’s 8 APU chips working together accelerating very large scale similarity search.</strong></figcaption></figure><p>The APU chip has 96 megabits of fast L1 cache. We have put two chips into 1 PCIe board, which has 16 gigabytes of main memory. So, one board is enough for many similarity search applications. For example, the Weizmann Institute uses 1 APU board to accelerate molecule search on a database of 16 million molecule fingerprints (fingerprint size = 1024 bits).</p><p>For even larger databases, we support clusters of multiple boards. A 1U rack system can house 4 tightly clustered boards. We also support multiple racks working in unison, as well as sourcing data from Network Attached Storage (NAS) via Remote Direct Memory Access (RDMA).</p><h4>What’s Next ?</h4><p>We are actively working with our partners and collaborators, integrating the APU system into their similarity search applications. For example, we have integrated the APU into the Biovia Pipeline Pilot, a popular application used in the cheminformatics community.</p><p>Check out our similarity search demo for visual search below.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F326706504%3Fapp_id%3D122963&amp;dntp=1&amp;url=https%3A%2F%2Fvimeo.com%2F326706504&amp;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F770795544_1280.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1728" height="1080" frameborder="0" scrolling="no"><a href="https://medium.com/media/b66b4b7e377d42a350fbd4c0b36a9cfc/href">https://medium.com/media/b66b4b7e377d42a350fbd4c0b36a9cfc/href</a></iframe><p>The APU can accelerate similarity search in many kinds of applications, including large scale face recognition. The APU is also applicable to other domains, such as Natural Language Processing and cybersecurity. Watch for our upcoming blogs on these topics.</p><p>Can your application also benefit from accelerated similarity search ? Let’s discuss the possibilities. For more information and a customized demo with your data, please reach out to us at <a href="/service/mailto:associativecomputing@gsitechnology.com">associativecomputing@gsitechnology.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=beaef9b19b97" width="1" height="1" alt=""><hr><p><a href="https://medium.com/gsi-technology/the-apu-novel-hardware-for-accelerated-similarity-search-beaef9b19b97">The APU: Novel Hardware For Accelerated Similarity Search</a> was originally published in <a href="https://medium.com/gsi-technology">GSI Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>