<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ricardo Erikson on Medium]]></title>
        <description><![CDATA[Stories by Ricardo Erikson on Medium]]></description>
        <link>https://medium.com/@ricardoerikson?source=rss-2984f4b1820d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*sLwb9l5FjytYTPpYbU25bw.jpeg</url>
            <title>Stories by Ricardo Erikson on Medium</title>
            <link>https://medium.com/@ricardoerikson?source=rss-2984f4b1820d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 17:12:24 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ricardoerikson/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Memory Allocation and Performance in Golang maps]]></title>
            <link>https://levelup.gitconnected.com/memory-allocation-and-performance-in-golang-maps-b267b5ad9217?source=rss-2984f4b1820d------2</link>
            <guid isPermaLink="false">https://medium.com/p/b267b5ad9217</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[maps]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[memory-allocation]]></category>
            <category><![CDATA[empty-struct]]></category>
            <dc:creator><![CDATA[Ricardo Erikson]]></dc:creator>
            <pubDate>Mon, 25 Jan 2021 04:28:28 GMT</pubDate>
            <atom:updated>2023-08-31T21:49:31.118Z</atom:updated>
            <content:encoded><![CDATA[<h4>GOLANG</h4><h3>Memory Allocation and Performance in Golang Maps</h3><h4>Golang maps can be tricky. Understanding how they work can help you improve the performance of your code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pv0FXQ6YKQat0gblywA2xg.jpeg" /><figcaption>(source: <a href="https://www.pexels.com/">pexels.com</a>)</figcaption></figure><p>Recently, I found an interesting <a href="https://stackoverflow.com/questions/65258003/memory-allocation-of-mapintinterface-vs-mapintstruct/65866774">question on Stack Overflow</a>. It is related to memory allocation in Golang maps when we use an interface as the value (map[int]interface{})<em> vs. </em>an empty struct as the value(map[int]struct{}). The OP set up two benchmark tests to compare the two map types and found some weird results. The benchmarks are described as follows:</p><p>In the code below, we have a function for each map type. Each function basically creates a map and then repeats a zero-value assignment a fixed number of times. That is, by the end of the execution, the map will have a fixed number of entries, and each entry will receive the zero value of the type (nil for interfaces and struct{}{} for empty structs).</p><pre>package main<br><br>func main() {}<br><br>func MapWithInterface() {<br>    m := map[int]interface{}{}<br>    for i := 1; i &lt;= 100; i++ {<br>        m[i] = nil<br>    }<br>}<br><br>func MapWithEmptyStruct() {<br>    m := map[int]struct{}{}<br>    for i := 1; i &lt;= 100; i++ {<br>        m[i] = struct{}{}<br>    }<br>}</pre><p>Benchmarks:</p><pre>package main<br><br>import &quot;testing&quot;<br><br>func Benchmark_Interface(b *testing.B) {<br>    for i := 0; i &lt; b.N; i++ {<br>        MapWithInterface()<br>    }<br>}<br><br>func Benchmark_EmptyStruct(b *testing.B) {<br>    for i := 0; i &lt; b.N; i++ {<br>        MapWithEmptyStruct()<br>    }<br>}</pre><p>The results of the benchmark execution are as follows:</p><pre>goos: darwin<br>goarch: amd64<br>pkg: awesomeProject1<br>Benchmark_Interface-8         130419          8949 ns/op        7824 B/op          7 allocs/op<br>Benchmark_EmptyStruct-8       165147          6964 ns/op        3070 B/op         17 allocs/op<br>PASS<br>ok      awesomeProject1 3.122s</pre><p>The entries of both maps are assigned to zero values (nil and struct{}{} , i.e., values that don’t require allocation). However, we see that the empty struct version runs faster and uses less memory, but it does more allocations. <strong>What’s happening here? Why do these benchmarks that are so similar have such different results?</strong></p><h3>TL;DR</h3><p>The internal design of maps in Golang is highly optimized for performance and memory management. Maps keep track of keys and values that can hold pointers. If the entries in a bucket can’t hold pointers, maps just create overflow buckets to avoid unnecessary overhead with garbage collection (GC), which results in more allocations and better performance (the case of map[int]struct{}).</p><h3>Long Answer</h3><p>Before we get to the answers, we need to understand map initialization and map structure. We’ll go through these topics first, and then we will analyze some benchmarks to see the performance of the two map types we are trying to understand here. I created a <a href="https://github.com/ricardoerikson/benchmark-golang-maps">repository</a> that contains some tests to help in the understanding of this answer. So, if you see some reference to a test or benchmark in the text, it’s probably in the repository.</p><h4>Map Initialization</h4><p>There are two methods for map initialization:</p><ul><li>make(map[int]string): when we have no idea about how many entries will be added to a map.</li><li>make(map[int]string, hint): when we have an idea about how many entries will be added. hint is an estimate of the initial capacity of a map.</li></ul><p>Maps are mutable and they will grow on-demand, no matter which initialization method we choose. However, the second method pre-allocates memory for at least hint entries, which results in increased performance.</p><h4>Map Structure</h4><p>A map in Go is a hash table that stores its key/value pairs into buckets. Each bucket is an array that holds up to 8 entries. The default number of buckets is 1. Once the number of entries across each bucket reaches an average load of buckets (aka load factor), the map gets bigger by doubling the number of buckets. Every time a map grows, it allocates memory for the new-coming entries. In practice, every time the load of the buckets reaches 6.5 or more, the map grows. This value is hardcoded and was chosen to optimize memory usage.</p><p>Behind the scenes, a map is a pointer to the hmap struct (<a href="https://github.com/golang/go/blob/cd99385ff4a4b7534c71bb92420da6f462c5598e/src/runtime/map.go#L115">map.go, line 115</a>). There is also the maptype struct (<a href="https://github.com/golang/go/blob/b634f5d97a6e65f19057c00ed2095a1a872c7fa8/src/runtime/type.go#L372">type.go, line 372</a>), which holds some information about the type of the map. The source code for Golang maps can be found here:</p><p><a href="https://github.com/golang/go/blob/master/src/runtime/map.go">https://github.com/golang/go/blob/master/src/runtime/map.go</a></p><p>As you can see in the link above, the map&#39;s internal implementation is complex. You can find some insights on how to hack a map type in the links below. <a href="https://hackernoon.com/u/avk-ai">Aleksandr Kochetkov</a> did a terrific job showing some details of map internals.</p><p><a href="https://hackernoon.com/some-insights-on-maps-in-golang-rm5v3ywh">https://hackernoon.com/some-insights-on-maps-in-golang-rm5v3ywh</a></p><p><a href="https://play.golang.org/p/NaoC8fkmy9x">https://play.golang.org/p/NaoC8fkmy9x</a></p><p><strong>One important thing to note is that maps keep track of keys and values that can hold pointers. If the entries in a bucket can’t hold any pointers, the bucket is marked as containing no pointers, and maps just create overflow buckets (which means more memory allocations). This avoids unnecessary overhead with GC. See this </strong><a href="https://github.com/golang/go/blob/682a1d2176b02337460aeede0ff9e49429525195/src/runtime/map.go#L132"><strong>comment in the </strong></a><a href="https://github.com/golang/go/blob/682a1d2176b02337460aeede0ff9e49429525195/src/runtime/map.go#L132"><strong>mapextra struct (map.go, line 132)</strong></a><strong> and this </strong><a href="https://www.komu.engineer/blogs/go-gc-maps"><strong>post</strong></a><strong> for reference.</strong></p><h4>Benchmarks Analysis</h4><p>An empty struct struct{} has no fields and cannot hold any pointers. As a result, the buckets in the empty struct case will be marked as <em>containing no pointers</em>. This will avoid unnecessary scanning through the map, and we can expect improved execution speed. In addition, we can also expect more memory allocations for a map of type map[int]struct{} since it creates more overflow buckets as it grows.</p><p>An interface{} type, differently from the empty struct, can hold ANY values, including pointers. To comprehend how this impacts map performance, we need to understand that map buckets keep track of the size of the memory prefix holding all pointers (<a href="https://github.com/golang/go/blob/cd99385ff4a4b7534c71bb92420da6f462c5598e/src/runtime/type.go#L33">ptrdata field, line 33</a>). Map types that can hold pointers don’t allocate additional overflow buckets. However, these types require scanning by GC.</p><p>In the map internal implementation, we can see how the ptrdata field is used to decide whether more overflow buckets will be created or not (<a href="https://github.com/golang/go/blob/b634f5d97a6e65f19057c00ed2095a1a872c7fa8/src/runtime/map.go#L265">map.go, line 265</a>). Refer to this <a href="https://play.golang.org/p/_-QKWu1GBnr">link</a> to see the size of the memory prefix holding all pointers for map[int]struct{} and map[int]interface{}.</p><p>The difference between the two benchmarks (Benchmark_EmptyStruct and Benchmark_Interface) is clear when we see the CPU profile. Benchmark_Interface does not have the method (*hmap)createOverflow that results in an additional memory allocation flow.</p><h4>Benchmark_EmptyStruct CPU profile</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*yMiIJkGEnfYgQ13MkIBE0Q.png" /><figcaption><em>Benchmark_EmptyStruct CPU profile (</em><a href="https://raw.githubusercontent.com/ricardoerikson/benchmark-golang-maps/main/map_empty_struct_cpu_profile.png"><em>png</em></a><em>, </em><a href="https://raw.githubusercontent.com/ricardoerikson/benchmark-golang-maps/main/map_empty_struct_cpu_profile.svg"><em>svg</em></a><em>)</em></figcaption></figure><h4>Benchmark_Interface CPU profile</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*1yyZ6uldnwtlBao3pQlxTQ.png" /><figcaption><em>Benchmark_Interface CPU profile (</em><a href="https://raw.githubusercontent.com/ricardoerikson/benchmark-golang-maps/main/map_interface_cpu_profile.png"><em>png</em></a><em>, </em><a href="https://raw.githubusercontent.com/ricardoerikson/benchmark-golang-maps/main/map_interface_cpu_profile.svg"><em>svg</em></a><em>)</em></figcaption></figure><h4>Benchmarks Results</h4><p>I customized the benchmark tests to pass the number of entries and the map’s initial capacity (hint). Here are the results of the executions. Results are basically the same when there are few entries or when the initial capacity is greater than the number of entries. If you have many entries and an initial capacity of 0, you will get quite a different number for allocations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*1SNbOKMhW-QGhcJbdiwnOw.png" /></figure><p>As the number of entries gets bigger we clearly see a difference in speed when we compare Interface and EmptyStruct. Interface benchmarks are much slower than EmptyStruct benchmarks. We can also see that when the initial capacity of the map is bigger than the number of entries, maps don’t require too many allocations because there’s a pre-allocated space for all entries.</p><h4>Conclusion</h4><p>The different results that lead to the question on Stack Overflow are related to map optimizations for performance and memory management. Maps of type map[int]interface{} are slower because they suffer performance degradation when GC scans the buckets that can hold pointers. Maps of type map[int]struct{} use less memory because they, in fact, use less memory 😄. There’s no need to allocate space for an empty struct (the Test_EmptyStructValueSize shows that a struct{}{} is zero-sized). Despite nil being the zero value for aninterface{}, the interface{} type requires some space to store ANY values (the TestNilInterfaceValueSize test shows the size of an interface{} type holding a nil value is not zero). Finally, the empty struct benchmark allocations are higher because the map[int]struct{} type requires more overflow buckets (for performance optimization) since its buckets don’t hold any pointers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b267b5ad9217" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/memory-allocation-and-performance-in-golang-maps-b267b5ad9217">Memory Allocation and Performance in Golang maps</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>