<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dubo Dubon Duponey on Medium]]></title>
        <description><![CDATA[Stories by Dubo Dubon Duponey on Medium]]></description>
        <link>https://medium.com/@dubo-dubon-duponey?source=rss-c27455c54764------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*WgaavZw9dJa-TiZD7GftSg.jpeg</url>
            <title>Stories by Dubo Dubon Duponey on Medium</title>
            <link>https://medium.com/@dubo-dubon-duponey?source=rss-c27455c54764------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 01:23:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dubo-dubon-duponey/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Docker 20.10 on ARM/V7 Buster is… bust]]></title>
            <link>https://dubo-dubon-duponey.medium.com/docker-20-10-on-arm-v7-buster-is-bust-fbde9574cad8?source=rss-c27455c54764------2</link>
            <guid isPermaLink="false">https://medium.com/p/fbde9574cad8</guid>
            <category><![CDATA[seccomp]]></category>
            <category><![CDATA[dumpster-fire]]></category>
            <category><![CDATA[debian]]></category>
            <category><![CDATA[arm]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Dubo Dubon Duponey]]></dc:creator>
            <pubDate>Wed, 11 Aug 2021 03:57:01 GMT</pubDate>
            <atom:updated>2021-08-11T03:57:01.974Z</atom:updated>
            <content:encoded><![CDATA[<p>So, you have a RPI, happily running fully patched Docker 20.10.8.</p><p>Then…</p><p>Maybe you tried todocker run -ti debian:bullseye apt-get update and all hell broke loose with apt telling you the debian repo cannot be trusted:</p><pre>At least one invalid signature was encountered</pre><p>— while the same command works just fine if using the :buster tag</p><p>Or maybe a valid Golang binary seemingly refuses to start with some confusing mumbo jumbo about your (armhf) CPU lacking… hf...</p><pre>runtime: this CPU has no floating point hardware, so it cannot run<br>this GOARM=7 binary. Recompile using GOARM=5.</pre><p>… or any other mind-boggling, very hard-to-reason-about issue</p><h3>Be not afraid, for you are not alone</h3><p>Before you dive into the rabbit hole trying to figure out what’s wrong with setcap, or apt, or your network, or your cpu, or the internet, and spend countless hours deep down into a very dark rabbit hole…</p><p>Try this one magic trick:</p><pre>docker run -ti --security-opt seccomp=unconfined debian:bullseye apt-get update</pre><p>If things <em>now</em> work, congrats, as you have fallen victim of the great seccomp dumpster fire of twenty-twen^H errrr, sometime ago.</p><p>Now, try this too:</p><pre>dpkg -l | grep seccomp</pre><p>Version says anything smaller than 2.4.2? That’s it.</p><p>seccomp in buster is broken in a (not so) subtle way.</p><h3>“Solutions”</h3><p>Pretty much, either:</p><ul><li>embrace the wild side and upgrade your RPIs to bullseye</li><li>just install libseccomp from bullseye while holding the rest of your system back</li><li>run all your containers with --security-opt seccomp=unconfined</li><li>run dockerd with a tweaked seccomp profile (passing to dockerd something in the line of: --seccomp-profile=default.json) — you can get the original <a href="https://github.com/moby/moby/blob/master/profiles/seccomp/default.json">from here</a>, then sed /”defaultAction”: “SCMP_ACT_ERRNO”/”defaultAction”: “SCMP_ACT_TRACE”/</li></ul><p>There is no good solution. It’s pretty much about either downgrading your security, rushing a system upgrade to an unreleased Debian version, or mangling your system with a newer version of a critical package.</p><h3>References</h3><ul><li><a href="https://github.com/moby/moby/blob/master/profiles/seccomp/default.json">moby/profiles/seccomp/default.json at master · moby/moby</a></li><li><a href="https://github.com/moby/moby/issues/40734">Seccomp blocks install of &quot;libc6&quot; in Ubuntu 20.04 &quot;focal&quot; image on armhf (arm32v7) · Issue #40734 · moby/moby</a></li><li><a href="https://github.com/docker/for-linux/issues/1145">gettimeofday() is broken on an arm32v7 program running in a an arm64v8 image on an arm64v8 host · Issue #1145 · docker/for-linux</a></li><li><a href="https://github.com/debuerreotype/docker-debian-artifacts/issues/106">arm32v7/debian:bullseye-slim: Clock not working · Issue #106 · debuerreotype/docker-debian-artifacts</a></li><li><a href="https://docs.docker.com/engine/security/seccomp/">Seccomp security profiles for Docker</a></li><li><a href="https://github.com/itzg/docker-minecraft-server/issues/755#issuecomment-781615497">After Update Docker starts only with privileged: true · Issue #755 · itzg/docker-minecraft-server</a></li></ul><p>Prior art on this:</p><p><a href="https://blog.samcater.com/fix-workaround-rpi4-docker-libseccomp2-docker-20/">Fix/Workaround - libseccomp2 and Alpine 3.13 - Installing Raspbian Docker 19.04+ on Raspberry Pi 4 Buster</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fbde9574cad8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A beginner’s guide to cross-compiling static CGO PIE binaries (golang 1.16),]]></title>
            <link>https://dubo-dubon-duponey.medium.com/a-beginners-guide-to-cross-compiling-static-cgo-pie-binaries-golang-1-16-792eea92d5aa?source=rss-c27455c54764------2</link>
            <guid isPermaLink="false">https://medium.com/p/792eea92d5aa</guid>
            <category><![CDATA[pie]]></category>
            <category><![CDATA[build]]></category>
            <category><![CDATA[static]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[cgo]]></category>
            <dc:creator><![CDATA[Dubo Dubon Duponey]]></dc:creator>
            <pubDate>Fri, 30 Jul 2021 05:56:32 GMT</pubDate>
            <atom:updated>2021-07-30T05:56:32.412Z</atom:updated>
            <content:encoded><![CDATA[<h3>A beginner’s guide to cross-compiling static CGO PIE binaries (golang 1.16), and other ways to hurt yourself purposedly</h3><blockquote>TL;DR? Scroll down to the bottom</blockquote><h3>What kind of PIE? Apple pie?</h3><p><a href="https://en.wikipedia.org/wiki/Position-independent_code">Position Independent Code</a> (PIC) describes code that has been built in a way that allows it to run properly regardless of location in memory. This is especially useful for shared libraries, so that they can be loaded in each app memory space.</p><p>PI<em>E</em> specifically describes <em>executables</em> that have been built entirely this way.</p><p>PIE further enables <a href="https://en.wikipedia.org/wiki/Address_space_layout_randomization">address space layout randomization</a>, thwarting classes of exploits related to memory corruption issues (or at least making it significantly harder to exploit said vulnerabilities).</p><p>Major OS-es and distributions (macOS, iOS, Android, FreeBSD, Ubuntu, Debian, Fedora) have all enabled by default or started supporting PIE during the past decade, and a recent increase in interest for it (or more accurately, concerns over memory safety issues) have prompted gradual phasing out of non-PIE binaries.</p><p>As far as golang is concerned, building PIE has been supported since <a href="https://golang.org/doc/go1.6">go 1.6</a>, has been made the default on Windows with <a href="https://golang.org/doc/go1.15">go 1.16</a>, and is supported on an increasing number of platforms and architectures.</p><p>Savvy readers would evidently question the relevance of PIE in the go world. After all, isn’t golang managed memory model actually preventing the problems that ASLR claims to make harder to exploit?</p><p><a href="https://groups.google.com/g/golang-nuts/c/cXhRsmNsMwo">This is true to some extent</a>: if you build <em>without CGO</em>, it is going to require a lot of effort to produce go code with that kind of issue.</p><p>Now, if you do use CGO, you are linking in C code (whether third-party or libc, through netcgo for example, which, unfortunately, there are still <a href="https://github.com/golang/go/issues/12524">a number of reasons</a> to use), and it is still painfully obvious in 2021 that memory issues in C code are and will always be <a href="https://nvd.nist.gov/vuln/detail/CVE-2021-33574">a major effing PITA</a>, because, as has been repeatedly demonstrated, even the (self-described) best developers cannot be trusted to <em>never</em> make any mistake while holding a loaded gun standing on one foot on a stool on a banana peel wearing a pink pajama (read: writing convoluted C code managing memory manually).</p><p>In simple cases, to enable PIE when building your go app, all you have to do is pass `-buildmode=pie` in your `GOFLAGS`. But of course, the simple cases are also the cases where building with PIE does not matter much…</p><h3>On static builds</h3><p>The advent of containerization probably contributed to a renewed interest in building static binaries.</p><p>After all, the point of shared libraries was largely to reduce footprint on systems running many programs sharing code, and to ease maintenance and patching-in-place of said shared dependencies, for non-transient systems.</p><p>Of course, dynamic libraries (and binaries that use them) suffer from many issues stemming into the fact that said shared libraries may change out of band and break applications in (possibly not so) subtle ways (case in point: <a href="https://en.wikipedia.org/wiki/Dependency_hell">dependency hell</a>). When shipping a binary that link dynamically to some library, there is nothing you can do as a developer but pray that it will be run with a truly compatible version of an untampered library it was meant to run with. After all, “<a href="https://en.wikipedia.org/wiki/Mozilla_software_rebranded_by_Debian#Iceweasel">promises [of backward compatibility] only bind those who believe in them</a>” (&lt;- yes, libpng, looking at you).</p><p>Now, in a world of (pretend) micro-services that (supposedly) run just one thing, where systems are treated increasingly as cattle rather than pets, of continuous deployment and constant rebuilding instead of patching-in-place, and of rapidly increasing storage and raw computing power, the downsides of shared libraries significantly out-weight their (questionable) benefits. Furthermore, out of concerns over attack surface, and optimization at scale, interest in running bare-bones workloads rather than full-blown operating systems (eg: single binaries in a “FROM scratch” container for example — or even <a href="https://en.wikipedia.org/wiki/Unikernel">Unikernels</a>), have made building statically even more attractive.</p><p>While, without CGO, golang <em>does</em> produce static binaries by default, building statically <em>with</em> CGO <a href="https://github.com/golang/go/issues/26492">has always been a little bit arcane</a>. For some time, `-ldflags ‘-extldflags “-static”’ -tags ‘static_build’` was the 50% tribal way to go (* see below about the tribal part), but things got messier with go 1.15, when the linker used by go was <a href="https://golang.org/doc/go1.15#linker">switched to default to the go internal linker </a><a href="https://golang.org/doc/go1.15#linker"><em>when using PIE</em></a>, but only <a href="https://golang.org/doc/go1.15#linker"><em>on certain platforms</em></a>. This has tripped even <a href="https://github.com/golang/go/issues/40711">seasoned developers</a>. Put otherwise: magic is impressive for as long as you do not change the outcome of the trick overnight, depending on the rabbit’s fur color. If you do, it’s no longer magic, it’s just a garden variety mess.</p><p>To top it all, the `file` binary, that many rely on to check if a binary is static or not, has a notoriously bad history at operating properly with PIE binaries, and will seemingly randomly report PIE static binaries as either “shared object”, or “dynamic”, depending on `file` version (`ldd` is not always perfect either btw).</p><p>Finally, things get even messier with/without CGO, and if you add gccgo to the mix… it’s increasingly confusing for newcomers and veterans alike which compiler / linker is being used in what circumstances, and what kind of rabbit you will get out of it. And, of course, <a href="https://stackoverflow.com/questions/61553723/whats-the-difference-between-statically-linked-and-not-a-dynamic-executable/61554975#61554975">it’s not just go</a>…</p><p>(*) many projects make use of the `static_build` tag. So, to build them statically, you do <em>also</em> have to pass this tag along (on top of linker flags). It’s not standard (<a href="https://github.com/golang/go/issues/26492">yet</a>): any random project could very well use a `static` tag instead, or something else entirely.</p><h3>On cross-compiling (cgo)</h3><p>Cross-compilation has historically been the privilege of a handful of people, and used to require significant efforts and intricate low-level knowledge (rolling your own cross compilation toolchain, to boot) — not to mention the fact that a large fraction of existing software was not portable at all to start with, or had (borked) build systems deeply tied to the host native toolchain.</p><p>Over the years, <a href="https://mxe.cc/">laudable efforts</a> started to show-up, mostly out of the desire to be able to cross-compile for Windows (or even OSX, dare you) without having to touch the darn thing with a barge pole.</p><p>The advent of ARM further propelled cross-compilation into mainstream, up to the point even web developers started jonesing for it, and came-up with their own cross-compi^H <a href="https://scotch.io/tutorials/javascript-transpilers-what-they-are-why-we-need-them">trans-pilation</a> so they too could solve the <em>one</em> problem they did not have in the first place — in a less comp-gendered way evidently.</p><p>Finally, Apple changing architecture every-time they feel like getting in a romp with their supply chain probably furthered this even more (culminating with the M1).</p><p>In 2021, good linux distributions do provide pre-packaged cross-compilation toolchains, and modern (compiled) languages have recognized the necessity to ship by default with a toolchain that can target different architectures. Rust — and golang, elegantly came-up with fantastic cross-compilation off the boot. Just set GOOS, GOARCH, and… magic time!</p><p>Mostly… Unless…</p><p>You use PIE… in that case, things work out of the box <em>just</em> on arm64 and amd64 — other platforms require an external <em>linker</em> from a cross-compilation toolchain that you have to setup yourself outside of go.</p><p>Or… unless…</p><p>You need CGO. In that case, you do need an external cross-compilation toolchain either way (compiler and linker), and PIE (finally really useful in that case) will just make things… significantly thornier…</p><h3>On libc(s)</h3><p>To compile statically with PIE, with CGO, you do of course <em>also</em> need the appropriate c runtime library (`rcrt1.o`).</p><p>Support for that was added in <a href="https://lists.gnu.org/archive/html/info-gnu/2018-02/msg00000.html">glibc 2.27, in 2018</a>, but only for a few platforms. Even in June 2021, Debian Sid only ships it for amd64 and arm64.</p><p>It seems you are pretty much out of luck if you want to support armhf for example, and it is unclear why glibc would not add more platforms after over three years.</p><p>Furthermore, glibc has a rather long history of <a href="https://lwn.net/Articles/488847/">being not specifically (static) friendly</a>, nor that interested in even acknowledging the relevance of <a href="https://sourceware.org/bugzilla/show_bug.cgi?id=5070#c5">other (architectures</a>).</p><p>Alternatively, you can go with <a href="https://musl.libc.org/">musl</a>. Comparing libc implementations is out of scope here, though you might want to start reading <a href="https://wiki.debian.org/musl">this</a>, <a href="https://www.etalabs.net/compare_libcs.html">this</a>, and/or <a href="https://en.wikipedia.org/wiki/Musl">this</a>. While musl is not as widely used as glibc evidently, and while there may be issues in building certain type of apps against it (mono was problematic last I checked, and more generally anything that relies on non-standard extensions provided by glibc), it is still a very reasonable choice, and a popular one amongst go nuts and other people looking for a modern, correct, static friendly libc.</p><p>Unfortunately though, musl does not use NSS, so, if your reason to use netcgo is <a href="https://wiki.musl-libc.org/future-ideas.html">mDNS, you are out of luck right now</a>.</p><p>Finally note that if you build <em>statically</em> against glibc, and hope to use <em>NSS via netcgo</em>, <a href="https://sourceware.org/bugzilla/show_bug.cgi?id=19341">just… stop… you are loosing your time</a>.</p><h3>TL;DR</h3><ol><li><strong>If you do <em>not</em> need CGO:</strong></li></ol><ul><li>and do <em>NOT care about PIE</em></li><li>or <em>if you want PIE</em> but <em>only care about arm64 and amd64</em></li></ul><p>Do yourself a huge favor and keep it simple. Overall, remember that CGO is a great piece of tech, but also something you should really try to avoid. Just use go tools:</p><pre>GOARCH=X go build<br>GOARCH=X go build -buildmode=pie</pre><p>The PIE binary will be dynamic, the other one static.</p><p><strong>2. If you really need <em>CGO, but not netcgo</em></strong></p><p>If you do not care about the distro you are building on, do yourself a favor here too and go with musl. Really.</p><p>If you cannot, glibc will still work, but mind the rabbit hole when things will go haywire (they will).</p><p>Typically, you can get a cross-compiled non-PIE static binary with:</p><pre>dpkg --add-architecture X<br>apt-get update<br>apt-get install crossbuild-essential-X libc6-dev:X</pre><pre>export GOARCH=X<br>eval &quot;$(dpkg-architecture -A &quot;$(echo <strong>&quot;</strong>$GOARCH<strong>&quot; </strong>| sed <strong>-</strong>e &quot;s/arm$/armhf/&quot; -e &quot;s/ppc64le/ppc64el/&quot; -e &quot;s/386/i386/&quot;)&quot;)&quot;<br>export CC=${DEB_TARGET_GNU_TYPE}-gcc<br>export CGO_ENABLED=1</pre><pre>go build <strong>-</strong>tags &quot;cgo netgo osusergo static_build&quot;<strong> </strong>-ldflags &quot;-linkmode=external -extldflags -static&quot;</pre><p>Or a static PIE one — only on arm64 and amd64:</p><pre># Use the same preamble as above ^ then:<br>go build -buildmode=pie <strong>-</strong>tags &quot;cgo netgo osusergo static_build&quot;<strong> </strong>-ldflags &quot;-linkmode=external -extldflags -static-pie&quot;</pre><p>Other platforms have to choose between static (above), or PIE:</p><pre># Use the same preamble as above ^ then:<br>go build -buildmode=pie <strong>-</strong>tags &quot;cgo netgo osusergo&quot;</pre><p>Finally, since you are doing CGO, you should <em>really</em> take a serious look at how to harden your binary — linker options (relro, now, defs, noexecstack) and compiler / preprocessor options (-fstack-protector-strong, -fstack-clash-protection, -D_GLIBCXX_ASSERTION, -D_FORTIFY_SOURCE=2).</p><p>3. <strong>If you do really need <em>netcgo</em></strong></p><p>Probably because you really need <a href="https://en.wikipedia.org/wiki/Name_Service_Switch">NSS</a>.</p><p>You are stuck with glibc (as previously mentioned, musl does not do NSS — although, an idea would be to run a localhost patched coreDNS that would provide the resolution you need).</p><p>And there is no known way to get a static binary in that case (PIE or otherwise).</p><p><a href="https://sourceware.org/bugzilla/show_bug.cgi?id=19341">This here</a> is still a problem six years in (or should we say, twenty years in). It has been <a href="https://github.com/golang/go/issues/13470">rehashed</a>, over and over again. The resulting binary of compiling statically with netcgo against glibc <em>will</em> SIGSEGV.</p><p>What you can do, though, is to compile statically <em>everything but</em> <em>glibc</em>:</p><pre>go build <strong>-</strong>tags &quot;cgo netcgo osusergo&quot;<strong> </strong>-ldflags &quot;-linkmode=external -extldflags &#39;-static-libgcc -static-libstdc++&#39;&quot;</pre><p>This last approach is probably the best trade-off, since, anyway, even with a fully static binary (that would work), you would still need to ship alongside your binary the dynamic (exact same) version of glibc to use NSS… Mind blowing, right?</p><h3>Take-away</h3><p>NSS is a venerable (as in: old enough to buy property in Florida) piece of… tech. It’s of course controversial, and there are good reasons why “obviously musl does not have (or want) NSS”. But then, no out-of-the-box support for mDNS for example is hard to defend.</p><p>glibc is even harder to defend nowadays though. Of course, it has been instrumental in the past twenty something years. But then, a wet glitter-stripped doormat, in its place, would have been instrumental just the same. The glory was in the function, not the essence.</p><p>There are clear, blatant reasons why modern stacks (like go) chose to do away with it (pretty much the same reasons they do away with openssl).</p><p>Then, there are still important pieces of tech written in C that have yet to be ported over to something that does not ^$!$`+/ù (and doing so will be costly). In that context, the ability to use CGO is both god-send and a curse. And one thing should be said about the go toolchain overall: it’s <em>way</em> too easy to produce broken binaries. I’m sure they are working on that, but right now, if you are poking the CGO-bear — be sure to watch your six. Hope that helps.</p><h3>Notes, further reading, errata</h3><h4>Interesting reads</h4><ul><li><a href="https://www.leviathansecurity.com/blog/aslr-protection-for-statically-linked-executables">https://www.leviathansecurity.com/blog/aslr-protection-for-statically-linked-executables</a></li><li><a href="http://dominik.honnef.co/posts/2015/06/statically_compiled_go_programs__always__even_with_cgo__using_musl/">http://dominik.honnef.co/posts/2015/06/statically_compiled_go_programs__always__even_with_cgo__using_musl/</a></li><li><a href="https://dave.cheney.net/2016/01/18/cgo-is-not-go">https://dave.cheney.net/2016/01/18/cgo-is-not-go</a></li><li><a href="https://deepu.tech/memory-management-in-golang/">https://deepu.tech/memory-management-in-golang/</a></li></ul><h4>Credit trailer: auditing binaries</h4><p>Use readelf — from package binutils — specifically -p .interp and -d.</p><p>File can still provide some information (if the version you have is not garbage), though you can get the same in a more reliable way out of readelf.</p><p>ldd <em>will</em> let you down.</p><p>Also have a look at hardening-check (from devscripts).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=792eea92d5aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Advanced ARG and ENV Dockerfile tricks]]></title>
            <link>https://dubo-dubon-duponey.medium.com/advanced-arg-and-env-dockerfile-tricks-6b1eae68d02a?source=rss-c27455c54764------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b1eae68d02a</guid>
            <category><![CDATA[env]]></category>
            <category><![CDATA[environment]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[dockerfiles]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Dubo Dubon Duponey]]></dc:creator>
            <pubDate>Wed, 28 Jul 2021 20:15:25 GMT</pubDate>
            <atom:updated>2021-07-30T06:07:50.803Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Or how to (not) hang yourself manipulating a container environment at build-time, runtime, across stages and across images boundaries</blockquote><blockquote>Jump directly to section “Advanced Tricks” if you think you do not need to read the refresher and pitfalls.</blockquote><h3>Refresher on ARG and ENV basic concepts</h3><h4>ENV</h4><p>In a Dockerfile, ENV lets you declare environment variables that will be available during <em>both</em> the build phase and at runtime.</p><p>Here is an example:</p><pre>FROM debian<br># Declare the variable<br>ENV THIS_IS_ENV=default_env_value<br># Echo it&#39;s value at build time<br>RUN echo &quot;I&#39;m inside the image at build time and THIS_IS_ENV value is ${THIS_IS_ENV:-}&quot;<br># This will be executed at runtime<br>CMD [&quot;bash&quot;, &quot;-c&quot;, &quot;echo At runtime and THIS_IS_ENV value is ${THIS_IS_ENV:-}&quot;]</pre><p>At build-time, THIS_IS_ENV will <em>always</em> be set to the value being declared in the Dockerfile, but, at runtime, its value <em>may</em> be overridden by passing --env THIS_IS_ENV=overridden to your docker run call.</p><p>Tester:</p><pre># Build<br>docker build -f Dockerfile --progress=plain --no-cache -t tester .<br># Run it as-is<br>docker run tester<br># Run it overriding the value of the env variable<br>docker run --env THIS_IS_ENV=overridden tester</pre><h4>ARG</h4><p>On the other hand, ARG allows you to define an environment variable that will <em>only</em> be available at build-time, and that <em>may</em> be overridden <em>when building</em>.</p><pre>FROM debian as base<br># Declare the arg<br>ARG THIS_IS_ARG=default_arg_value<br># Echo its value at build time<br>RUN echo &quot;I&#39;m in the base image at build time and THIS_IS_ARG value is ${THIS_IS_ARG:-}&quot;<br># Notice that at runtime, it&#39;s undeclared<br>CMD [&quot;bash&quot;, &quot;-c&quot;, &quot;echo At runtime and THIS_IS_ARG value is ${THIS_IS_ARG:-}&quot;]</pre><p>Tester:</p><pre>docker build -f Dockerfile --progress=plain --no-cache -t tester .<br>docker build -f Dockerfile --progress=plain --no-cache -t tester --build-arg THIS_IS_ARG=overridden .<br>docker run --rm tester</pre><h4>Multi-stage builds</h4><p>With multi-stage builds (where a stage starts from another), you inherit all ENV and ARG defined in <em>any previous stage</em>.</p><p>If you do override an ARG at build-time, all stages will get the modified value.</p><p>Evidently, ENV still cannot be altered at build-time, but only at runtime.</p><p>For example:</p><pre># First stage<br>FROM debian as base<br>ENV THIS_IS_ENV=default_env_value<br>ARG THIS_IS_ARG=default_arg_value<br>RUN echo &quot;I&#39;m in the base image at build time and THIS_IS_ENV value is ${THIS_IS_ENV:-}&quot;<br>RUN echo &quot;I&#39;m in the base image at build time and THIS_IS_ARG value is ${THIS_IS_ARG:-}&quot;<br># Second stage<br>FROM base<br>RUN echo &quot;I&#39;m in the second stage image at build time and THIS_IS_ENV value is ${THIS_IS_ENV:-}&quot;<br>RUN echo &quot;I&#39;m in the second stage image at build time and THIS_IS_ARG value is ${THIS_IS_ARG:-}&quot;<br>CMD [&quot;bash&quot;, &quot;-c&quot;, &quot;echo At runtime and THIS_IS_ENV value is ${THIS_IS_ENV:-}, THIS_IS_ARG value is ${THIS_IS_ARG:-}&quot;]</pre><p>Tester:</p><pre>docker build -f Dockerfile --progress=plain --no-cache -t tester .<br>docker build -f Dockerfile --build-arg THIS_IS_ARG=overridden_arg --progress=plain --no-cache -t tester .<br>docker run --env THIS_IS_ENV=overridden_env tester</pre><h4>Pre-FROM args</h4><p>If you would like to parameterize your FROM image, Docker neatly allows you to declare ARG <em>before</em> your first FROM at the top of your Dockerfile.</p><p>For example:</p><pre>ARG FROM_IMAGE=debian<br>FROM $FROM_IMAGE</pre><p>This allows you to parameterize and control your base image url at build time without changing anything in your Dockerfile, using --build-arg FROM_IMAGE=something_else .</p><p>This is of course very neat if you would like to pull your base image from a local registry instead of from, say, Docker Hub (assuming you have local copies of course), or if you would like to test quickly a newer/different version of your base image.</p><h4>ONBUILD</h4><p>Another neat Docker mechanism is the concept of ONBUILD that lets you preemptively declare ARG and ENV in a stage, that will actually <em>only</em> be available at any subsequent stage.</p><p>Here is a concrete example:</p><pre># First stage<br>FROM debian as base<br>ENV THIS_IS_ENV=default_env_value<br>ARG THIS_IS_ARG=default_arg_value<br>ONBUILD ENV THIS_IS_ONBUILD_ENV=default_onbuild_env_value<br>ONBUILD ARG THIS_IS_ONBUILD_ARG=default_onbuild_arg_value<br>RUN echo &quot;I&#39;m in the base image at build time&quot;; env | grep THIS_<br># Second stage<br>FROM base<br>RUN echo &quot;I&#39;m in the second stage image at build time&quot;; env | grep THIS_<br>CMD [&quot;bash&quot;, &quot;-c&quot;, &quot;echo At runtime; env | grep THIS_&quot;]</pre><h4>ARGs and ENVs are position dependent</h4><p>This may sound obvious, but is better said: the point at which you declare an ARG or an ENV inside your Dockerfile makes it available in the environment for any instruction that <em>follows</em>.</p><h3>Common pitfalls</h3><h4>#1 ENVs cannot be declared before FROM, at the top</h4><p>Unlike args, envs have to declared <em>after </em>the first FROM.</p><h4>#2 overriding an ARG with -build-arg does override all occurrences of that arg in all stages</h4><p>This sounds rather obvious.</p><h4>#3 (ONBUILD) ARGs behave differently if defined in your base FROM image (as compared to using multi-stage)</h4><p>What has been said above with regard to multi-stage images does <em>not</em> apply when you build FROM another, already built image.</p><p>An easy way to think about this is:</p><ul><li>in a multi-stage build, you are building everything</li><li>in a FROM scenario, your base image has been <em>already</em> built and will <em>not</em> be built again</li></ul><p>Specifically:</p><ul><li>ARGs declared inside your base image are <em>not</em> available in your inheriting image</li><li>an ONBUILD ARG declared in your base image is only made available if it has been declared in the <em>last stage</em> of the base image</li></ul><h4>#4 you cannot use variables inside the --from segment of a COPY instruction (unlike in a FROM)</h4><p>See trick below for a workaround.</p><h4>#5 ARGs declared before your FROM are not available to the rest of your Dockerfile</h4><p>See trick below for a workaround.</p><h4>#6 if the same variable is defined as both an ARG and an ENV in prior stages, the ENV value will override the ARG one in subsequent stages (but not in the one where the ARG first appears in)</h4><p>This one is a very convenient way to hurt yourself. Think twice before you use the same variable name for both an ENV and an ARG. While this is a powerful trick (see below), it is also a mind-bender.</p><h3>Advanced Tricks</h3><h4>Parameter expansion</h4><p>Both ENV and ARG (in ONBUILD as well), can reference other environment variables and support <a href="https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html">some parameter expansion</a>.</p><p>This is pretty neat and allows you for example to derive computed values out of another variable:</p><pre>FROM debian</pre><pre>ARG SET_OR_UNSET<br>ENV THIS_ENV=${SET_OR_UNSET:+value_if_arg_is_set}</pre><p>In this scenario, THIS_ENV will:</p><ul><li>be empty if SET_OR_UNSET is not set</li><li>will have the value value_if_arg_is_set if SET_OR_UNSET has been set</li></ul><p>Note that FROM <em>also</em> support parameter expansion</p><h4>Using repeat ENV at build time</h4><p>The usefulness of repeating the same ARG at build time is rather limited. If it is overridden, all instances will end-up with the same value.</p><p>Similarly, repeating the same ENV has no impact at all at runtime.</p><p>But now, repeating ENV at build-time will allow for more advanced environment manipulation when combined with parameter expansion.</p><p>For example:</p><pre>FROM debian<br>ARG SET_OR_UNSET<br>ENV THIS_ENV=${SET_OR_UNSET:+value_if_this_is_set}<br>ENV THIS_ENV=${THIS_ENV:-value_if_arg_is_not_set}</pre><p>You have now successfully implemented a ternary.</p><h4>Shadowing at build time</h4><p>Declaring the same variable with ENV and ARG will allow the same variable to have a different value at build time than at runtime, letting the operator control <em>both</em> independently.</p><pre>FROM debian<br>ENV THIS_VAR=value_at_runtime<br>ARG THIS_VAR=value_at_buildtime</pre><h4>Shadowing at build time, for just one stage</h4><p>In a base image:</p><pre>FROM debian<br>ENV THIS_VAR=&quot;always available to all stages&quot;<br>ONBUILD ARG THIS_VAR=&quot;will override THIS_VAR *just* for the next first stage&quot;</pre><p>And in your inheriting image:</p><pre>FROM my_image_built_from_above as first_stage</pre><pre>RUN echo &quot;THIS_VAR value is that of the ONBUILD ARG $THIS_VAR&quot;</pre><pre>FROM first_stage as second_stage</pre><pre>RUN echo &quot;THIS_VAR value is that of the ENV: $THIS_VAR&quot;</pre><h4>Workaround: access to pre-FROM ARGs</h4><p>Easy. Declare your ARG twice.</p><pre>ARG BASE_IMAGE=debian<br>FROM $BASE_IMAGE<br>ARG BASE_IMAGE<br>RUN echo &quot;$BASE_IMAGE&quot;</pre><p>Note that you do not have to set the same default value. If left as-is, it will get the default value set before.</p><h4>Workaround: parameterized COPY --from with image staging</h4><p>As stated above, you unfortunately cannot use COPY --from=$PARAM</p><p>The easy way to workaround that is to leverage FROM instead, declare a named extra stage and use that fixed name in your COPY.</p><p>Concretely:</p><pre>ARG COPY_SOURCE_IMAGE=golang<br>FROM $COPY_SOURCE_IMAGE as image_staging<br>FROM debian<br>COPY --from=image_staging /thing /destination</pre><h3>Concrete use-cases</h3><blockquote>A few useful scenarios</blockquote><h4>Easy registry selection, with “scratch” fallback</h4><pre>ARG FROM_REGISTRY=ghcr.io<br>ARG FROM_IMAGE=${FROM_REGISTRY:+$FROM_REGISTRY/me/my_image}<br>FROM ${FROM_IMAGE:-scratch}</pre><p>The FROM will default to “ghcr.io/me/my_image”.</p><p>Overriding the build-arg FROM_REGISTRY will allow the builder to control from which registry this image is being sourced.</p><p>Overriding the build-arg FROM_IMAGE will further allow the builder to control the FQN of the image.</p><p>If FROM_REGISTRY is overridden to the empty string, the image will instead be scratch.</p><h4>Controlling app specific behavior in a single subsequent stage through shadowing</h4><p>In a base image:</p><pre>FROM golang<br>ENV GOPROXY=off<br>ONBUILD ARG GOPROXY=https://go-proxy.local</pre><p>Inheriting image</p><pre># Only this stage allows go to retrieve content<br>FROM my_base_trick as fetch_stage<br>RUN git clone whatever<br>RUN go mod download<br># This stage and all subsequent will have GOPROXY=off<br>FROM fetch_stage<br>RUN echo &quot;GOPROXY is: $GOPROXY&quot;</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b1eae68d02a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sonic is just as full of shit as Comcast]]></title>
            <link>https://dubo-dubon-duponey.medium.com/sonic-is-just-as-full-of-shit-as-comcast-846fb08bc804?source=rss-c27455c54764------2</link>
            <guid isPermaLink="false">https://medium.com/p/846fb08bc804</guid>
            <category><![CDATA[sonic]]></category>
            <dc:creator><![CDATA[Dubo Dubon Duponey]]></dc:creator>
            <pubDate>Thu, 15 Jul 2021 23:38:52 GMT</pubDate>
            <atom:updated>2021-07-15T23:38:52.420Z</atom:updated>
            <content:encoded><![CDATA[<p>For about $80 a month, I got a quite bad DSL service (since fiber was not available on Page St, San Francisco…): seemingly random freezes and huge drops in bandwidth, that Sonic Technical Support was never able to provide an explanation for, claiming that “everything is fine on the line, must be your equipment after the router”.</p><p>I still stuck with them for about a year, for a number of reasons — one of them being my reluctance to go with Comcast which was the only alternative.</p><p>Eventually, I recently (April 2021) moved the contract over to a new place, where I was hoping the service would be better.</p><p>It was not.</p><p>Sometimes early June, I finally self-diagnosed the problem: whenever you upload <em>anything</em>, download speed will drop <em>abysmally</em>.</p><p>Told Sonic technical support about that.</p><p>Response: “That’s not a bug. We have documentation about that, and that’s because of network congestion. When you reach your maximum upload speed <em>[ndlr: somewhere below 1MB/s, mind you]</em>, you <em>have no download anymore</em>. The solution is for you to throttle the upload speed for every application that uploads something — to around 250kb/s”.</p><p>Of course, this sounds like a bad joke. It is not.</p><p>At that point, I was still trying to find excuses for them, as I really thought Sonic was a nice company — good people, investment in better infrastructure, commitment to net neutrality — and I was willing to give them a pass on that one, since they actually just resell AT&amp;T.</p><p>So, I just wanted to cancel, stay friends and move on to something else — no hard feelings.</p><p>When I did pull the plug early July, I was told there is a $170 “early termination fee” — that I should have been aware that moving my contract over to a new address resets the clock (the “early” part), and that I had an opportunity to cancel the service during the first month instead of foolishly giving them a chance to diagnose / fix things.</p><p>Here is the take-away: Sonic will sell you absolute garbage when they cannot provide fiber, seemingly just to make a buck. The fact that they are just reselling someone else garbage (AT&amp;T DSL) does not change anything. If you are patient with them, and unwilling to go away immediately, hoping that things will get better, you will get fucked, and they will explain to you that it’s your fault for not paying attention to the fine print, or the fact that they are actually selling garbage.</p><p>Good job Sonic. You are just the new Comcast. Except Comcast at least provides a service that somewhat works.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=846fb08bc804" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GOPROXY(ies) on authentication]]></title>
            <link>https://dubo-dubon-duponey.medium.com/goproxy-ies-on-authentication-4d02db9e7cc5?source=rss-c27455c54764------2</link>
            <guid isPermaLink="false">https://medium.com/p/4d02db9e7cc5</guid>
            <category><![CDATA[go]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[tls]]></category>
            <category><![CDATA[authentication]]></category>
            <dc:creator><![CDATA[Dubo Dubon Duponey]]></dc:creator>
            <pubDate>Thu, 01 Jul 2021 03:20:40 GMT</pubDate>
            <atom:updated>2021-07-01T03:20:40.154Z</atom:updated>
            <content:encoded><![CDATA[<p>If you do run your own Athens at any significant scale, there is a good chance you want to provide encryption and/or authentication for it.</p><p>The APT world (or more generally the HTTPs world) after all does provide the means to implement mTLS and a a great deal of options for authentication.</p><p>Alas, go mod is not yet at that level of maturity, and your options are limited.</p><p>Here is what you can, and cannot do.</p><h3>Using TLS for your server: YES</h3><p>If you sign with Let’s Encrypt (or any other reputable CA), things will (evidently?) just work.</p><p>So called self-signed certificates though, require a little bit more work (as usual).</p><p>Here are your options:</p><h4>Install your CA or server certificate at the OS level</h4><p>Out of scope here, this is evidently highly dependent on your operating system.</p><p>On Debian, you likely want to leverage the ca-certificates package.</p><p>On mac, you should be able to get away with it using a machete:</p><pre>openssl s_client -showcerts -servername go.local -connect go.local:443 &lt;/dev/null 2&gt;/dev/null | awk &#39;/BEGIN/,/END/{ if(/BEGIN/){a++}; print}&#39; &gt; go.local.ca.crt<br>security add-trusted-cert -d -r trustRoot -k ~/Library/Keychains/login.keychain go.local.ca.crt</pre><p>While feasible, we usually try our best to avoid having to use this approach, specifically because our main use case is in the context of building docker images — for a series of reasons:</p><ul><li>we certainly do not want our internal CA to ship inside our final images (as there is no reason for the runtime image to trust it), which means you have to be extra careful in how and when you mount it in your image, or how you rely on system tools to handle it (looking at you update-ca-certificates)</li><li>a system-wide install is likely to have unintended side-effects, specifically… making <em>everything</em> trust this CA — or even trust <em>just</em> this CA, depending on how you do it</li></ul><h4>Install in a specific location and point to it</h4><p>That works, for the most part.</p><p>Thanks to <a href="https://golang.org/src/crypto/x509/root_unix.go">https://golang.org/src/crypto/x509/root_unix.go</a>, using the SSL_CERT_FILE environment variable allows you to override the system store with your own.</p><p>Practically, in a Dockerfile:</p><pre><strong>ARG           </strong>GOPROXY=https://go.local<br><strong>RUN           --</strong>mount=type=secret,id=CA \<br>              SSL_CERT_FILE=<strong>/</strong>run<strong>/</strong>secrets<strong>/</strong>CA go mod download</pre><p>… will now let you use a build secret for your CA, which will not interfere with anything else in your build, will not ship in your final image regardless of how the image is being structured (not everybody does multi-stage unfortunately), is system-independent (*), and requires a minimum amount of boilerplate that does not rely on any system tools.</p><p>(*) This will not work natively on macOS: <a href="https://github.com/golang/go/issues/37907">https://github.com/golang/go/issues/37907</a></p><h3>Using client TLS certificates / mTLS: NOPE</h3><p>We use mTLS generally in our internal stack, relying on Caddy to provide TLS termination and TLS client certificate verification for all our services.</p><p>Unfortunately, there does not seem to be a way to use client TLS with go tools right now: <a href="https://github.com/golang/go/issues/30119">https://github.com/golang/go/issues/30119</a></p><h3>Using basic authentication: YES</h3><p>Short of being able to use mTLS, you possibly want to fallback on credentials based authentication.</p><p>go tools do support netrc files to pass credentials for specific servers.</p><p>The format of netrc is widely documented (if tribally). In a shell:</p><pre>machine go.local login dubo-dubon-duponey password XXXXXXXXXX</pre><p>… which you can use in a similar fashion as above in a Dockerfile (be SURE to use a secret for this one, evidently):</p><pre><strong>ARG           </strong>GOPROXY=https://go.local<br><strong>RUN           --</strong>mount=type=secret,id=CA \<br>     <strong>         --</strong>mount=type=secret,id=NETRC \<br>              NETRC=<strong>/</strong>run<strong>/</strong>secrets<strong>/</strong>NETRC SSL_CERT_FILE=<strong>/</strong>run<strong>/</strong>secrets<strong>/</strong>CA go mod download</pre><p>This works fine, for the most part, but…</p><h3>Using OAuth: NOPE</h3><p>Unfortunately, and unlike libcurl, other means of authentication besides basic (eg: OAuth) cannot be used currently: <a href="https://github.com/golang/go/issues/40215">https://github.com/golang/go/issues/40215</a></p><p>Indeed, empty passwords or logins are not handled at all: <a href="https://github.com/golang/go/blob/fa98f46741f818913a8c11b877520a548715131f/src/cmd/go/internal/auth/netrc.go#L65">https://github.com/golang/go/blob/fa98f46741f818913a8c11b877520a548715131f/src/cmd/go/internal/auth/netrc.go#L65</a></p><h3>Take-away</h3><p>GOPROXY and modules are still fairly new in the go world, and obviously require more work to provide the flexibility that other (older) systems offer (apt).</p><p>Still, you <em>can</em> use (self-signed or not) TLS for your proxy, and basic authentication.</p><p>Though, and short of relying on more arcane stuff (eg: mitmdump), you are out of luck for mTLS unfortunately. Or any form of <em>not-basic</em> authentication.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d02db9e7cc5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GOPROXY(ies) on performance]]></title>
            <link>https://dubo-dubon-duponey.medium.com/goproxy-ies-on-performance-f61406812bb7?source=rss-c27455c54764------2</link>
            <guid isPermaLink="false">https://medium.com/p/f61406812bb7</guid>
            <category><![CDATA[go]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Dubo Dubon Duponey]]></dc:creator>
            <pubDate>Tue, 29 Jun 2021 22:09:38 GMT</pubDate>
            <atom:updated>2021-06-29T22:56:51.286Z</atom:updated>
            <content:encoded><![CDATA[<p>Go modules <a href="https://github.com/golang/go/wiki/Modules">have been available since July 2018</a>, and are nowadays the officially sanctioned way to manage go dependencies.</p><p>With go 1.13, modules are downloaded by default from <a href="https://proxy.golang.org/">https://proxy.golang.org</a> (and <a href="https://sum.golang.org/">https://sum.golang.org</a>).</p><p>A number of other modules proxy <a href="https://github.com/golang/go/wiki/Modules#are-there-always-on-module-repositories-and-enterprise-proxies">are available in the wild</a>, for example <a href="https://athens.azurefd.net/">https://athens.azurefd.net</a>, and its underlying open source implementation, the <a href="https://github.com/gomods/athens">Athens Project</a>.</p><p>There are <a href="https://docs.gomods.io/intro/why/">several significant upsides</a> in using a proxy for go modules, and performance is listed prominently as one of these.</p><p>The following present numerical results from a test on <a href="https://github.com/moby/buildkit">the buildkit project</a>, establishing a baseline comparison between not using any proxy and the official, google maintained proxy, the Athens public server, and a privately operated instance of Athens.</p><h3>Test setup</h3><p>The gist of our test is very simple, and defined inside a Dockerfile as follows:</p><pre>FROM ghcr.io/dubo-dubon-duponey/base:builder-latest</pre><pre>ARG GOPROXY=direct</pre><pre>RUN go get github.com/moby/buildkit@v0.8.3; exit 1</pre><p>The base image we use contains golang 1.16, and can be replaced by any similar image providing it.</p><p>Our self-hosted Athens server is built from: <a href="https://github.com/dubo-dubon-duponey/docker-goproxy/blob/work/Dockerfile">https://github.com/dubo-dubon-duponey/docker-goproxy/blob/work/Dockerfile</a></p><p>Athens runs inside a container, uses the default configuration, with <a href="https://github.com/dubo-dubon-duponey/docker-goproxy/blob/work/context/runtime/boot/entrypoint.sh#L68">in-memory index and filesystem storage</a>, and is fronted by a Caddy server providing TLS termination and authentication.</p><p>We are running these tests on a self-hosted NUC server, wired in a decent (1Gb/s) residential line. Our Athens server runs on the same machine.</p><p>After a first cold run, each test is ran 10 times and results are averaged.</p><h3>Direct</h3><p>Not using any proxy yields (as expected) very slow results, in the range of 45 minutes.</p><p>This is painstakingly slow, and is literally a deal-killer for any significant project.</p><h3>Golang Proxy</h3><p>Using Google maintained proxy consistently clocks in at about 1 minute 50 seconds.</p><p>This is obviously the reference.</p><h3>Athens public server</h3><p>After a first cold run at 8 minutes, Athens public server hovers around 2 minutes, with some runs falling down to up to 6 minutes.</p><p>Results are much more erratic than with the official proxy, which has more likely to do with the supporting infrastructure and network backend rather than with the software itself. It’s certainly hard to compete with google resources and availability.</p><h3>Athens self-hosted</h3><p>Given Athens is configured to use direct for its backend requests, cold start ranges in the same ballpark as mentioned above.</p><p>Subsequent runs clock in around 1 minutes 20 seconds on average.</p><h3>Caveats</h3><p>None of this is scientifically rigorous. This is meant to provide a sense of how things are performing, and what you can expect from running your own proxy, using google infrastructure, or embracing direct.</p><p>We have not tested <a href="https://github.com/goproxy/goproxy">GoProxy</a>, neither <a href="https://github.com/thumbai/thumbai">Thumbai</a>, as neither projects seems to be actively developed, and it seems that Athens is the only serious kid in town at this point.</p><p>UPDATE: <a href="https://github.com/goproxyio/goproxy">https://github.com/goproxyio/goproxy</a> may be worth a try.</p><h3>Take-away</h3><p>It is hard to rationalize using “direct”.</p><p>From the perspective of having critical build-systems not relying on (yet) another third-party piece of infrastructure, “direct” turns out to be actually much worse, since it will make your build dependent on a (likely large) number of VCS that are typically cached / hidden by a module proxy. Furthermore, direct does not provide any of the guarantees of content availability &amp; immutability that proxies (to some extent) advertise.</p><p>Athens public server is not meant for production use, as clearly outlined by the project, and while its results are not bad from a performance perspective, it’s unlikely there is any upside in using it instead of the golang official proxy.</p><p>Should you self-host Athens then?</p><p>You will likely not get a massive performance increase by doing so for simple use-cases: cold-start are likely going to be (much more) costly, compared to relying on golang proxy, and subsequent runs only deliver a marginal upside (but on humongous projects).</p><p>Cold starts issues may be alleviated if Athens would itself rely on golang proxy instead of using direct, though that would clearly take-away the benefit of not depending on their availability (but then again, using direct makes you depend on VCS).</p><p>That being said, hosting such a service <em>on-premise</em> (ie: <em>as close as possible to where you build</em>) would yield a clear upside if you have many concurrent builds (collocated large teams, or a busy build server), or networking limitations that would benefit from slashing down overall traffic.</p><p>These benefits would obviously have to be weighted against the cost of maintaining such a service.</p><p>If performance is the only angle, astute caching (for eg retrieving dependencies separately from building, or using go mod vendor) would likely give more bang for the buck — while running Athens alongside buildkitd would add a nice additional layer of resiliency and some additional (if marginal) speedup.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f61406812bb7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>