<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[software under the hood - Medium]]></title>
        <description><![CDATA[Driver gets the best performance out of a race car, when they knows how it works. Works in harmony with your machine!… Deep dive into software, performance, observability and mechanically sympathetic programing - Medium]]></description>
        <link>https://medium.com/software-under-the-hood?source=rss----44e0597a33ff---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>software under the hood - Medium</title>
            <link>https://medium.com/software-under-the-hood?source=rss----44e0597a33ff---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 07:41:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/software-under-the-hood" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Under the hood system calls : Debugging, INT, SIGTRAP]]></title>
            <link>https://medium.com/software-under-the-hood/under-the-hood-system-calls-debugging-int-sigtrap-d211176970dd?source=rss----44e0597a33ff---4</link>
            <guid isPermaLink="false">https://medium.com/p/d211176970dd</guid>
            <category><![CDATA[interrupt]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[debugging]]></category>
            <dc:creator><![CDATA[aarti gupta]]></dc:creator>
            <pubDate>Mon, 24 Dec 2018 00:54:22 GMT</pubDate>
            <atom:updated>2018-12-24T00:54:22.618Z</atom:updated>
            <content:encoded><![CDATA[<h3>Under the hood system calls : Debugging, INT, SIGTRAP</h3><p>Ever wondered how debuggers work?? Gdb, strace, …etc…Here is more about them and why you should not use them in production facing systems!</p><h3><em>SIGTRAP</em></h3><pre>Signal         Value   Action   Comment<br>SIGTRAP        5        Core    Trace/breakpoint trap</pre><p>This signal (as the manpage suggests) is primarily used by debugging tools like gdb or strace, respectively the ptrace(2) system call. If one process is tracing another process&#39;s activity through ptrace(2), SIGTRAP will be heavily used to »interrupt« the child, even though the child in most cases won&#39;t effectively know this happened (a process monitored through ptrace(2)does not necessarily notice which signal had been sent to it, because of the tracer being able to intercept and filter delivered signals and SIGTRAP primarily serves to inform the tracer that something noticeable happened).</p><h3>INT</h3><p><strong>INT</strong> is an <a href="https://en.wikipedia.org/wiki/Assembly_language">assembly language</a> instruction for <a href="https://en.wikipedia.org/wiki/X86">x86</a> <a href="https://en.wikipedia.org/wiki/Central_processing_unit">processors</a> that generates a <a href="https://en.wikipedia.org/wiki/Software_interrupt">software interrupt</a>. It takes the interrupt number formatted as a <a href="https://en.wikipedia.org/wiki/Byte">byte</a> value.<a href="https://en.wikipedia.org/wiki/INT_(x86_instruction)#cite_note-1">[1]</a></p><p>When written in assembly language, the instruction is written like this:</p><p>INT <em>X</em></p><p>where <em>X</em> is the software interrupt that should be generated (0-255).</p><p>Depending on the context, <a href="https://en.wikipedia.org/wiki/Compiler">compiler</a>, or <a href="https://en.wikipedia.org/wiki/Assembly_language">assembler</a>, a software interrupt number is often given as a <a href="https://en.wikipedia.org/wiki/Hexadecimal">hexadecimal</a> value, sometimes with a prefix <em>0x</em> or the suffix <em>h</em>. For example, INT 21H will generate the software interrupt 0x21 (33 in decimal), causing the function pointed to by the 34th vector in the interrupt table to be executed, which is typically an <a href="https://en.wikipedia.org/wiki/MS-DOS_API">MS-DOS API</a> call.</p><h3>INT 3</h3><p>The INT 3 instruction is defined for use by <a href="https://en.wikipedia.org/wiki/Debugger">debuggers</a> to temporarily replace an instruction in a running program in order to set a <a href="https://en.wikipedia.org/wiki/Breakpoint">breakpoint</a>. Other INT instructions are encoded using two bytes. This makes them unsuitable for use in patching instructions (which can be one byte long); see <a href="https://en.wikipedia.org/wiki/SIGTRAP">SIGTRAP</a>.</p><p>The opcode for INT 3 is 0xCC, as opposed to the opcode for INT immediate, which is 0xCD imm8. Since the dedicated 0xCC opcode has some desired special properties for debugging, which are not shared by the <em>normal</em> two-byte opcode for an INT 3, assemblers do not normally generate the generic 0xCD 0x03 opcode from mnemonics.<a href="https://en.wikipedia.org/wiki/INT_(x86_instruction)#cite_note-Intel_2013_Arch-Ref-3">[3]</a></p><h3>Real Mode</h3><p>When generating a software interrupt, the processor calls one of the 256 functions pointed to by the interrupt address table, which is located in the first 1024 bytes of memory while in <a href="https://en.wikipedia.org/wiki/Real_mode">real mode</a> (See <a href="https://en.wikipedia.org/wiki/Interrupt_vector">Interrupt vector</a>). It is therefore entirely possible to use a far-call instruction to start the interrupt-function manually after pushing the flag register.</p><p>One of the most useful DOS software interrupts was interrupt 0x21. By calling it with different parameters in the registers (mostly ah and al) you could access various IO operations, string output and more.<a href="https://en.wikipedia.org/wiki/INT_(x86_instruction)#cite_note-2">[2]</a></p><p>Most <a href="https://en.wikipedia.org/wiki/Unix">Unix</a> systems and derivatives do not use software interrupts, with the exception of <strong>interrupt 0x80, used to make </strong><a href="https://en.wikipedia.org/wiki/System_call"><strong>system calls</strong></a><strong>. </strong>This is accomplished by entering a 32-bit value corresponding to a kernel function into the EAX register of the processor and then executing INT 0x80.</p><h3>References:</h3><p><a href="https://en.wikipedia.org/wiki/INT_(x86_instruction)">https://en.wikipedia.org/wiki/INT_(x86_instruction)</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d211176970dd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-under-the-hood/under-the-hood-system-calls-debugging-int-sigtrap-d211176970dd">Under the hood system calls : Debugging, INT, SIGTRAP</a> was originally published in <a href="https://medium.com/software-under-the-hood">software under the hood</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Under the hood memory: Know your memory(and caches!)]]></title>
            <link>https://medium.com/software-under-the-hood/under-the-hood-memory-know-your-memory-and-caches-103c1a81c3b6?source=rss----44e0597a33ff---4</link>
            <guid isPermaLink="false">https://medium.com/p/103c1a81c3b6</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[memory-management]]></category>
            <category><![CDATA[computer-architecture]]></category>
            <category><![CDATA[cache]]></category>
            <dc:creator><![CDATA[aarti gupta]]></dc:creator>
            <pubDate>Thu, 08 Feb 2018 19:22:46 GMT</pubDate>
            <atom:updated>2018-02-08T22:03:15.324Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*07BY2SmkkHYle812QSyeKg.jpeg" /></figure><h3>Random Access Memory(RAM)</h3><p>A memory unit is designated as Random Access Memory (RAM) if any location can be accessed in some fixed amount of time that is independent of the location address.</p><h3>Static Random-Access Memory( SRAM)</h3><p><strong>Static random-access memory</strong> (<strong>static RAM</strong> or <strong>SRAM</strong>) is a type of <a href="https://en.wikipedia.org/wiki/Semiconductor">semiconductor</a> memory that uses <a href="https://en.wikipedia.org/wiki/Multivibrator">bistable</a> <a href="https://en.wikipedia.org/wiki/Flip-flop_(electronics)">latching circuitry (flip-flop)</a> to store each bit. SRAM exhibits <a href="https://en.wikipedia.org/wiki/Data_remanence">data remanence</a>,<a href="https://en.wikipedia.org/wiki/Static_random-access_memory#cite_note-skorobogatov-1">[1]</a> but it is still <a href="https://en.wikipedia.org/wiki/Volatile_memory"><em>volatile</em></a> in the conventional sense that data is eventually lost when the memory is not powered.</p><p>The term <em>static</em> differentiates SRAM from <a href="https://en.wikipedia.org/wiki/Dynamic_random-access_memory">DRAM</a> (<em>dynamic</em> random-access memory) which must be periodically <a href="https://en.wikipedia.org/wiki/Memory_refresh">refreshed</a>. SRAM is faster and more expensive than DRAM; it is typically used for <a href="https://en.wikipedia.org/wiki/CPU_cache">CPU cache</a> while DRAM is used for a computer’s <a href="https://en.wikipedia.org/wiki/Main_memory">main memory</a>.</p><h3><strong>Dynamic random-access memory</strong> (<strong>DRAM</strong>)</h3><p><strong>Dynamic random-access memory</strong> (<strong>DRAM</strong>) is a type of <a href="https://en.wikipedia.org/wiki/Random-access_memory">random access</a> <a href="https://en.wikipedia.org/wiki/Semiconductor_memory">semiconductor memory</a> that stores each <a href="https://en.wikipedia.org/wiki/Bit">bit</a> of data in a separate tiny <a href="https://en.wikipedia.org/wiki/Capacitor">capacitor</a> within an <a href="https://en.wikipedia.org/wiki/Integrated_circuit">integrated circuit</a>. The capacitor can either be charged or discharged; these two states are taken to represent the two values of a bit, conventionally called 0 and 1. The <a href="https://en.wikipedia.org/wiki/Electric_charge">electric charge</a> on the capacitors slowly leaks off, so without intervention the data on the chip would soon be lost. To prevent this, DRAM requires an external <a href="https://en.wikipedia.org/wiki/Memory_refresh"><em>memory refresh</em></a> circuit which periodically rewrites the data in the capacitors, restoring them to their original charge. Because of this refresh requirement, it is a <em>dynamic</em> memory as opposed to <a href="https://en.wikipedia.org/wiki/Static_random-access_memory">static random-access memory</a> (SRAM) which does not require data to be refreshed</p><p>DRAM is widely used in digital electronics where low-cost and high-capacity memory is required. One of the largest applications for DRAM is the <a href="https://en.wikipedia.org/wiki/Main_memory"><em>main memory</em></a> (colloquially called the “RAM”) in modern <a href="https://en.wikipedia.org/wiki/Computer">computers</a> and <a href="https://en.wikipedia.org/wiki/Graphics_card">graphics cards</a> (where the “main memory” is called the <em>graphics memory</em>). It is also used in many portable devices and <a href="https://en.wikipedia.org/wiki/Video_game">video game</a> consoles. In contrast, SRAM, which is faster and more expensive than DRAM, is typically used where speed is of greater concern than cost, such as the <a href="https://en.wikipedia.org/wiki/CPU_cache">cache memories</a> in <a href="https://en.wikipedia.org/wiki/Central_processing_unit">processors</a>.</p><h3>Variations in DRAM</h3><p><strong>Synchronous dynamic random-access memory</strong> (<strong>SDRAM</strong>) is any <a href="https://en.wikipedia.org/wiki/Dynamic_random-access_memory">dynamic random-access memory</a> (DRAM) where the operation of its external pin interface is coordinated by an externally supplied <a href="https://en.wikipedia.org/wiki/Clock_signal">clock signal</a>.<strong> DRAM</strong> used an <strong><em>asynchronous</em> interface</strong>, in which input control signals have a direct effect on internal functions only delayed by the trip across its semiconductor pathways. <strong>SDRAM has a <em>synchronous</em> interface</strong>, whereby changes on control inputs are recognised after a rising edge of its clock input. In SDRAM families, the clock signal controls the stepping of an internal <a href="https://en.wikipedia.org/wiki/Finite_state_machine">finite state machine</a> that responds to incoming commands. These commands can be pipelined to improve performance, with previously started operations completing while new commands are received, allowing the device to operate on a memory access command in each bank simultaneously and speed up access in an <a href="https://en.wikipedia.org/wiki/Interleaved_memory">interleaved</a> fashion. This allows SDRAMs to achieve greater concurrency and higher data transfer rates than asynchronous DRAMs could. <a href="https://en.wikipedia.org/wiki/Pipeline_(computing)">Pipelining</a> means that the chip can accept a new command before it has finished processing the previous one. For a pipelined write, the write command can be immediately followed by another command without waiting for the data to be written into the memory array. For a pipelined read, the requested data appears a fixed number of clock cycles (latency) after the read command, during which additional commands can be sent.</p><p><strong>Double data rate type SDRAM (DDR3 SDRAM)</strong> ,<strong> &lt;also Interleaving&gt;</strong></p><p>is a type of <strong>s</strong>ynchronous <strong>d</strong>ynamic <strong>r</strong>andom-<strong>a</strong>ccess <strong>m</strong>emory (SDRAM) with a high bandwidth (“double data rate”) interface, and has been in use since 2007. It is the higher-speed successor to DDR and DDR2 and predecessor to <strong>DDR4 synchronous dynamic </strong><a href="https://en.wikipedia.org/wiki/Synchronous_dynamic_random-access_memory"><strong>synchronous dynamic random-access memory</strong></a><strong>.</strong></p><p>The modifications to the access circuit give it the capacity to<strong> transmit twice by clock cycle: one in the beginning of the cycle and another in its end</strong>. Like this, a DDR module that operates at 133 MHz, for instance, possesses an equivalent performance to that would be reached by a SDRAM module of 266 MHz. To make possible the double data access per clock cycle, <strong>the cell array is organized in two memory banks, each of which can be accessed separately. </strong>Additionally, consecutive words of a given block are stored in different banks what leads to the <strong>interleaving concep</strong>t. Is this interleaving of words in different memory banks that allows <strong>simultaneous access to two words that are transferred on successive clock edges</strong></p><p><strong>Fast Page Mode DRAM (FPM DRAM)&lt;also burst mode&gt;</strong></p><p>FPM DRAM implements an improvement on conventional DRAM in which the rowaddress is held constant while data from multiple columns is read from MDR using several column-addresses. The data held in the MDR form an open page that can be accessed relatively quickly, what speeds up successive accesses. This mechanism is known as burst mode access, and permits a block of (typically) four sequential words to be transferred from/to the memory bank. The first word takes the same time as conventional DRAM, however the subsequent three words are transferred faster. This can be modelled with the time access form: x-y-y-y. FPM DRAM usually has an access time of 5–3–3–3 (5 cycles to the first access and the subsequent three take 3 cycles each), with speeds of 60 to 80 ns (for the first access) and a maximum bus rate of 66 MHz</p><p>Extended Data Out DRAM</p><p>Addition of a buffer on the Data Out bus drivers, leading to the Extended Data Out DRAM configuration. This stores the output data (in SRAM) and keeps it stable for the time required for it to be read through the bus. Thus, as the data is stored in these extra buffers, the chip can overlap the next read access with the previous one — i.e., the next column address can be processed while the previous data is being read. This enhancement allows the access times of the second, third and fourth data accesses in a burst mode access to be overlapped and therefore accelerated.</p><p>Rambus DRAM (RDRAM)</p><p>The strategy used to develop this memory is based on its division into a larger number of memory banks. This supports a simultaneously data transfer from/to several memory banks, obtaining higher operational frequencies. Due to effects of electromagnetic interference, resulting from the high frequencies of the data transfers, the width of the data bus had to be reduced to allow a larger isolation, achieved with the increase of the space among wires.</p><h3>Memory Bank</h3><p><strong>memory bank</strong> is a logical unit of storage in electronics, which is <a href="https://en.wikipedia.org/wiki/Computer_hardware">hardware</a>-dependent. In a <a href="https://en.wikipedia.org/wiki/Computer">computer</a>, the <a href="https://en.wikipedia.org/wiki/Computer_memory">memory</a> bank may be determined by the <a href="https://en.wikipedia.org/wiki/Memory_controller">memory controller</a> along with physical organization of the hardware memory slots. A bank consists of multiple rows and columns of storage units, and is usually spread out across several chips. In a single read or write operation, only one bank is accessed, therefore the number of bits in a column or a row, per bank and per chip, equals the memory bus width in bits (single channel). <strong>The size of a bank is further determined by the number of bits in a column and a row, per chip, multiplied by the number of chips in a bank.</strong></p><h3>Data Bus</h3><p><a href="https://en.wikipedia.org/wiki/Bus_(computing)">Bus (computing)</a>, a communication system that transfers data between different components in a computer or between different computers</p><p><a href="https://en.wikipedia.org/wiki/Memory_bus">memory bus</a>, a bus between the computer and the memory</p><h3>Memory Centric Architectures</h3><p>Prediction is that memory behaviour would be preponderant over the global performance of the computational system.</p><p>Smart memories, intelligent memories, intelligent RAM (IRAM), merged DRAM/Logic (MDL), processor in memory (PIM), etc</p><p>Memory chips have huge internal bandwidth .The connection pins are responsible for the external degradation of the bandwidth, thousands of times slower than internal bandwidth. Eliminating the connections not only improves the bandwidth, but also improves the latency, as logic and storage are closer to each other. To increase the amount of integrated storage space, most of the smart memories proposals use DRAM instead of SRAM</p><h4>Processor In Memory(PIM)</h4><p>Large array of simple computation elements (typically over 1,000) were built into the DRAM arrays. These processing elements are usually integrated at the output of the sense amplifiers and are controlled by a single control unit, as an SIMD processor. This design strategy can explore the massive on-chip bandwidth of the DRAM, as the computation elements are integrated directly into the DRAM outputs</p><h4>Vector DRAM</h4><p>This strategy integrates a complete vector processor on a DRAM chip. Unlike the PIM strategy, the computation is out of the DRAM array, what reduces the peak throughput. However, fewer but more powerful processing elements can be used, as the spatial limitations are not so crucial. Consequently it is possible to achieve performance improvements on a larger set of applications</p><h4><strong>Multiprocessor-on-a-Chip</strong></h4><p>The single-chip multiprocessor PPRAM design<strong> tries to avoid the central control unit bottleneck that been a problem of other architectures. It does it by integrating several relatively simple, but fully independent, cached RISC processors, each with a reasonable amount of local memory</strong> (= 8 MB — DRAM), what is called a processing element or PPRAM node. These nodes are connected by a very high-bandwidth interface (PPRAM-Link), and can be programmed using standard shared-memory or message-passing parallel algorithms. Because of the high-level programmability of these designs, they are more easily programmed for maximum parallelism than the other smart memory designs. However, the large amount of resources necessary for each node limits the total number of parallel nodes far short from the parallel processing elements of the previously described PIM design</p><h3>Caches</h3><p>CPU caches are small pools of memory that store information the CPU is most likely to need next. Which information is loaded into cache depends on sophisticated algorithms and certain assumptions about programming code. The goal of the cache system is to ensure that the CPU has the next bit of data it will need already loaded into cache by the time it goes looking for it (also called a cache hit).</p><h3>Cache Miss</h3><p>A cache miss, on the other hand, means the CPU has to go scampering off to find the data elsewhere. This is where the L2 cache comes into play — while it’s slower, it’s also much larger. Some processors use an inclusive cache design (meaning data stored in the L1 cache is also duplicated in the L2 cache) while others are exclusive (meaning the two caches never share data). If data can’t be found in the L2 cache, the CPU continues down the chain to L3 (typically still on-die), then L4 (if it exists) and main memory (DRAM).</p><h3>Hit Rate</h3><p>The percentage of accesses that result in <strong>cache hits</strong> is known as the <strong>hit rate</strong> or <strong>hit</strong> ratio of the <strong>cache</strong>. The alternative situation, when the <strong>cache</strong> is consulted and found not to contain data with the desired tag, has become known as a <strong>cache</strong> miss.</p><h3>Tag RAM and Cache Associativity</h3><p>The tag RAM is a record of all the memory locations that can map to any given block of cache. If a cache is fully associative, it means that any block of RAM data can be stored in any block of cache. The advantage of such a system is that the hit rate is high, but the search time is extremely long — the CPU has to look through its entire cache to find out if the data is present before searching main memory.</p><p>At the opposite end of the spectrum we have direct-mapped caches. A direct-mapped cache is a cache where each cache block can contain one and only one block of main memory. This type of cache can be searched extremely quickly, but since it maps 1:1 to memory locations, it has a low hit rate. In between these two extremes are <em>n-</em>way associative caches. A 2-way associative cache (Piledriver’s L1 is 2-way) means that each main memory block can map to one of two cache blocks. An eight-way associative cache means that each block of main memory could be in one of eight cache blocks.</p><h3>Cache Design and CPU Performance , How cache design impacts performance</h3><p>The performance impact of adding a CPU cache is directly related to its efficiency or hit rate; repeated cache misses can have a catastrophic impact on CPU performance. The following example is vastly simplified but should serve to illustrate the point.</p><p>Imagine that a CPU has to load data from the L1 cache 100 times in a row. The L1 cache has a 1ns access latency and a 100% hit rate. It therefore takes our CPU 100 nanoseconds to perform this operation.</p><p>Now, assume the cache has a 99 percent hit rate, but the data the CPU actually needs for its 100th access is sitting in L2, with a 10-cycle (10ns) access latency. That means it takes the CPU 99 nanoseconds to perform the first 99 reads and 10 nanoseconds to perform the 100th. A 1 percent reduction in hit rate has just slowed the CPU down by 10 percent.</p><p>In the real world, an L1 cache typically has a hit rate between 95 and 97 percent, but the <em>performance</em> impact of those two values in our simple example isn’t 2 percent — it’s 14 percent. Keep in mind, we’re assuming the missed data is always sitting in the L2 cache. If the data has been evicted from the cache and is sitting in main memory, with an access latency of 80–120ns, the performance difference between a 95 and 97 percent hit rate could nearly double the total time needed to execute the code.</p><h3>Memory hierarchy</h3><p>In a memory hierarchy, a processor is connected to hierarchical set of memories, each of which is larger, slower, and cheaper (per byte) than the memories closer to the processor</p><h3>CPU DRAM gap</h3><p>The performance of the processor-memory interface is characterized by two parameters: the <strong>latency and the bandwidth</strong>. <strong>The latency is the time between the initiation of a memory request, by the processor, and its completion</strong>. In fact the problem of the increasing divergence between the memory and processor speeds is a latency growing trouble. <strong>The bandwidth is the rate at which information can be transferred to or from the memory system.</strong></p><p>There are two major classes of techniques to reduce the impact of long memory latencies: latency reduction and latency tolerance. Latency reduction decreases the time between the issue of a memory request and the return of the needed operand. Latency tolerance involves performing other computation while a memory request is being serviced, so that the memory latency for that request is partially or completely hidden. This will be covered in the next section. The use and success of these techniques expose the bandwidth limitations [4], since they speedup the instructions rate, and consequently the necessity of operands also grows; they require more items than are effectively needed increasing the absolute amount of memory traffic. The bandwidth incapability slows the response times to the processor requests, i.e. increases the latency. Given the complex interactions between memory latency and bandwidth, however, it is difficult to determine whether memory-related processor<strong> degradation is due to raw memory latency or from insufficient bandwidth (which also increases the latency).</strong></p><p>So far, the larger effort to decrease the performance gap between processor and physical memory has been concentrated on efficient implementations of a memory hierarchy. Particular techniques have been developed to reduce miss rate, miss penalty and hit time:</p><p>—<strong> reducing miss rate: increasing the dimensions of a cache and/or their blocks, higher associativity, insertion of victim and/or pseudo-associative caches, hardware pre-fetch, compiler controlled prefetching, compiler reduce misses;</strong></p><p><strong>— reducing miss penalty: read priority over write on miss, sub-block placement, early restart and critical word first on miss, lockup-free caches, multi-level caches;</strong></p><p><strong>— reducing hit time: simple and small caches, pipelined write.</strong></p><p>CPU DRAM gap levelled out around 2005 with 3 GHz processors, and since then processors have scaled using more cores and hyperthreads, plus multi-socket configurations, all putting more demand on the memory subsystem. Processor manufacturers have tried to reduce this memory bottleneck with larger and smarter CPU caches, and faster memory busses and interconnects. But we’re still usually stalled.</p><h3>References</h3><p><a href="https://pdfs.semanticscholar.org/6ebe/c8701893a6770eb0e19a0d4a732852c86256.pdf">https://pdfs.semanticscholar.org/6ebe/c8701893a6770eb0e19a0d4a732852c86256.pdf</a></p><ul><li><a href="https://www.extremetech.com/extreme/188776-how-l1-and-l2-cpu-caches-work-and-why-theyre-an-essential-part-of-modern-chips">How L1 and L2 CPU Caches Work, and Why They&#39;re an Essential Part of Modern Chips - ExtremeTech</a></li><li><a href="https://en.wikipedia.org/wiki/DDR3_SDRAM">DDR3 SDRAM - Wikipedia</a></li><li><a href="https://en.wikipedia.org/wiki/Dynamic_random-access_memory">Dynamic random-access memory - Wikipedia</a></li><li><a href="https://en.wikipedia.org/wiki/Static_random-access_memory">Static random-access memory - Wikipedia</a></li></ul><p><a href="https://en.wikipedia.org/wiki/Memory_bank">https://en.wikipedia.org/wiki/Memory_bank</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=103c1a81c3b6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-under-the-hood/under-the-hood-memory-know-your-memory-and-caches-103c1a81c3b6">Under the hood memory: Know your memory(and caches!)</a> was originally published in <a href="https://medium.com/software-under-the-hood">software under the hood</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Under the hood CPU Utilization, CPI and Load Averages]]></title>
            <link>https://medium.com/software-under-the-hood/under-the-hood-cpu-utilization-cpi-and-load-averages-8614baa43192?source=rss----44e0597a33ff---4</link>
            <guid isPermaLink="false">https://medium.com/p/8614baa43192</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[cpu]]></category>
            <dc:creator><![CDATA[aarti gupta]]></dc:creator>
            <pubDate>Thu, 08 Feb 2018 03:06:37 GMT</pubDate>
            <atom:updated>2018-02-08T03:06:37.494Z</atom:updated>
            <content:encoded><![CDATA[<h3>What really is CPU Utilization?</h3><p>The metric we call CPU utilization is really “non-idle time”: the time the CPU was not running the idle thread. Your operating system kernel (whatever it is) usually tracks this during context switch. If a non-idle thread begins running, then stops 100 milliseconds later, the kernel considers that CPU utilized that entire time.</p><p>%CPU can be broken into two components</p><p>instruction-retired cycles and stalled cycles, eg, %INS and %STL.</p><p>IOWAIT !=Stalling</p><p>It is wrong to interpret high %CPU to mean that the processing unit is the bottleneck.</p><p>With hyperthreads, however, those stalled cycles can now be used by another thread, so %CPU may count cycles as utilized that are in fact available.</p><p>CPU Utilization is often waiting on bus traffic (loading caches, loading ram, loading instructions, decoding instructions) only rarely is the CPU _doing_ useful work.</p><p>The context of what you are measuring depends if this is useful work or not. The initial access of a buffer almost universally stalls (unless you prefetched 100+ instructions ago). But starting to stream this data into L1 is useful work.</p><p>Aiming for 100%+ IPC is _beyond_ difficult even for simple algorithms and critical hot path functions. You not only require assembler cooperation (to assure decoder alignment), but you need to know _what_ processor you are running on to know the constraints of its decoder, uOP cache, and uOP cache alignment.</p><p>The idea here is that it appears that you are limited by the CPU and want to make things faster.</p><h3>instructions/cycle (IPC)(note the total IPC = no of cores!)</h3><p>Low level performance can be accurately measured using IPC, instructions per cycle. IPC shows on average how many instructions we were completed for each CPU clock cycle. The higher, the better (a simplification). The above example of 0.78 sounds not bad (78% busy?) until you realize that this processor’s top speed is an IPC of 4.0. This is also known as <em>4-wide</em>, referring to the instruction fetch/decode path. Which means, the CPU can retire (complete) four instructions with every clock cycle.So an IPC of 0.78 on a 4-wide system, means the CPUs are running at 19.5% their top speed. Newer Intel processors may move to 5-wide.</p><p>If your <strong>IPC is &lt; 1.0</strong>, you are likely memory stalled, and software tuning strategies include reducing memory I/O, and improving CPU caching and memory locality, especially on NUMA systems. Hardware tuning includes using processors with larger CPU caches, and faster memory, busses, and interconnects.</p><p>If your <strong>IPC is &gt; 1.0</strong>, you are likely instruction bound. Look for ways to reduce code execution: eliminate unnecessary work, cache operations, etc. <a href="http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html">CPU flame graphs</a> are a great tool for this investigation. For hardware tuning, try a faster clock rate, and more cores/hyperthreads.</p><p>When doing optimized code, the question “should I optimize for memory or for computing?” comes up often. Should I cache results? Should I use a more complex data structure in order to save memory or improve locality?</p><p>IPC is a good indicator on how you should tackle the problem. High IPC means you are may be doing too many calculations, while low IPC means that you should look at your memory usage. BTW, most of the time, memory is the problem.</p><h3>Linux Load averages</h3><p>Load average — is the average system load calculated over a given period of time of 1, 5 and 15 minutes.</p><p>You probably have a system with multiple CPUs or a multi-core CPU. The load average numbers work a bit differently on such a system. For example, if you have a load average of 2 on a single-CPU system, this means your system was overloaded by 100 percent — the entire period of time, one process was using the CPU while one other process was waiting. On a system with two CPUs, this would be complete usage — two different processes were using two different CPUs the entire time. On a system with four CPUs, this would be half usage — two processes were using two CPUs, while two CPUs were sitting idle.</p><p>To understand the load average number, you need to know how many CPUs your system has. A load average of 6.03 would indicate a system with a single CPU was massively overloaded, but it would be fine on a computer with 8 CPUs.</p><h3>Deeper metrics</h3><p>When Linux load averages increase, you know you have higher demand for resources (CPUs, disks, and some locks), but you aren’t sure which. You can use other metrics for clarification. For example, for CPUs:</p><ul><li><strong>per-CPU utilization</strong>: eg, using mpstat -P ALL 1</li><li><strong>per-process CPU utilization</strong>: eg, top, pidstat 1, etc.</li><li><strong>per-thread run queue (scheduler) latency</strong>: eg, in /proc/PID/schedstats, delaystats, perf sched</li><li><strong>CPU run queue latency</strong>: eg, in /proc/schedstat, perf sched, my <a href="http://www.brendangregg.com/blog/2016-10-08/linux-bcc-runqlat.html">runqlat</a> <a href="https://github.com/iovisor/bcc">bcc</a> tool.</li><li><strong>CPU run queue length</strong>: eg, using vmstat 1 and the ‘r’ column, or my runqlen bcc tool.</li></ul><p>The first two are utilization metrics, the last three are saturation metrics. Utilization metrics are useful for workload characterization, and saturation metrics useful for identifying a performance problem. The best CPU saturation metrics are measures of run queue (or scheduler) latency: the time a task/thread was in a runnable state, but had to wait its turn. These allow you to calculate the magnitude of a performance problem, eg, the percent of time a thread spent in scheduler latency. Measuring the run queue length instead can suggest that there is a problem, but it’s more difficult to estimate the magnitude</p><p>— -</p><h3>References</h3><p><a href="http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html">http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html</a></p><p><a href="http://www.brendangregg.com/blog/2017-05-09/cpu-utilization-is-wrong.html">http://www.brendangregg.com/blog/2017-05-09/cpu-utilization-is-wrong.html</a></p><p><a href="https://www.howtogeek.com/194642/understanding-the-load-average-on-linux-and-other-unix-like-systems/">https://www.howtogeek.com/194642/understanding-the-load-average-on-linux-and-other-unix-like-systems/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8614baa43192" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-under-the-hood/under-the-hood-cpu-utilization-cpi-and-load-averages-8614baa43192">Under the hood CPU Utilization, CPI and Load Averages</a> was originally published in <a href="https://medium.com/software-under-the-hood">software under the hood</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Under the hood OS: Demand Paging, Page Faults and Working Set]]></title>
            <link>https://medium.com/software-under-the-hood/under-the-hood-os-demand-paging-page-faults-and-working-set-82849bb6b404?source=rss----44e0597a33ff---4</link>
            <guid isPermaLink="false">https://medium.com/p/82849bb6b404</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[operating-systems]]></category>
            <category><![CDATA[memory-management]]></category>
            <dc:creator><![CDATA[aarti gupta]]></dc:creator>
            <pubDate>Tue, 06 Feb 2018 03:03:15 GMT</pubDate>
            <atom:updated>2018-02-06T03:03:15.129Z</atom:updated>
            <content:encoded><![CDATA[<p>Linux is an on demand memory system. The malloc() call immediately returns virtual memory, which is only promoted to real memory later when it is used via a page fault.</p><h3><strong>Demand Paging</strong></h3><p>In <a href="https://en.wikipedia.org/wiki/Computer">computer</a> <a href="https://en.wikipedia.org/wiki/Operating_systems">operating systems</a>, <strong>demand paging</strong> (as opposed to <a href="https://en.wikipedia.org/wiki/Paging#Page_replacement_techniques">anticipatory</a> paging) is a method of <a href="https://en.wikipedia.org/wiki/Virtual_memory">virtual memory</a> management. In a system that uses demand paging, the operating system copies a disk <a href="https://en.wikipedia.org/wiki/Paging">page</a> into physical memory only if an attempt is made to access it and that page is not already in memory (<em>i.e.</em>, if a <a href="https://en.wikipedia.org/wiki/Page_fault">page fault</a> occurs). It follows that a <a href="https://en.wikipedia.org/wiki/Process_(computing)">process</a> begins execution with none of its pages in physical memory, and many page faults will occur until most of a process’s <a href="https://en.wikipedia.org/wiki/Working_set">working set</a> of pages is located in physical memory. This is an example of a <a href="https://en.wikipedia.org/wiki/Lazy_loading">lazy loading</a> technique.</p><h3>Page Fault</h3><p>A <strong>page fault</strong> (sometimes called <strong>#PF</strong>, <strong>PF</strong> or <strong>hard fault</strong><a href="https://en.wikipedia.org/wiki/Page_fault#cite_note-1">[a]</a>) is a type of <a href="https://en.wikipedia.org/wiki/Exception_handling">exception</a> raised by computer hardware when a running program accesses a <a href="https://en.wikipedia.org/wiki/Memory_page">memory page</a> that is not currently mapped by the <a href="https://en.wikipedia.org/wiki/Memory_management_unit">memory management unit</a> (MMU) into the <a href="https://en.wikipedia.org/wiki/Virtual_address_space">virtual address space</a>of a process. Logically, the page may be accessible to the process, but requires a mapping to be added to the process page tables, and may additionally require the actual page contents to be loaded from a backing store such as a <a href="https://en.wikipedia.org/wiki/Disk_storage">disk</a>. The processor’s MMU detects the page fault, while the <a href="https://en.wikipedia.org/wiki/Exception_handling">exception handling</a> software that handles page faults is generally a part of the operating system <a href="https://en.wikipedia.org/wiki/Kernel_(operating_system)">kernel</a>. When handling a page fault, the <a href="https://en.wikipedia.org/wiki/Operating_system">operating system</a> generally tries to make the required page accessible at the location in physical memory, or terminates the program in case of an illegal memory access.</p><p>Contrary to what “fault” might suggest, valid page faults are not errors, and are common and necessary to increase the amount of memory available to programs in any operating system that utilizes <a href="https://en.wikipedia.org/wiki/Virtual_memory">virtual memory</a>.</p><h3><strong>Page Maps</strong></h3><pre>pagemap is a new (as of 2.6.25) set of interfaces in the kernel that allow<br>userspace programs to examine the page tables and related information by<br>reading files in /proc.<br><br>There are four components to pagemap:<br><br> * /proc/pid/pagemap.  This file lets a userspace process find out which<br>   physical frame each virtual page is mapped to.  It contains one 64-bit<br>   value for each virtual page, containing the following data (from<br>   fs/proc/task_mmu.c, above pagemap_read):<br><br>    * Bits 0-54  page frame number (PFN) if present<br>    * Bits 0-4   swap type if swapped<br>    * Bits 5-54  swap offset if swapped<br>    * Bit  55    pte is soft-dirty (see Documentation/vm/soft-dirty.txt)<br>    * Bit  56    page exclusively mapped (since 4.2)<br>    * Bits 57-60 zero<br>    * Bit  61    page is file-page or shared-anon (since 3.5)<br>    * Bit  62    page swapped<br>    * Bit  63    page present</pre><h3>Idle Page Flags</h3><h3>Working Set Size</h3><p>The Working Set Size (WSS) is how much memory an application needs to keep working. Your application may have 100 Gbytes of main memory allocated and page mapped, but it is only touching 50 Mbytes each second to do its job. That’s the working set size: the “hot” memory that is frequently used. It is useful to know for capacity planning and scalability analysis.</p><h3>References</h3><ul><li><a href="https://en.wikipedia.org/wiki/Demand_paging">Demand paging - Wikipedia</a></li><li><a href="https://en.wikipedia.org/wiki/Page_fault">Page fault - Wikipedia</a></li><li><a href="http://www.brendangregg.com/wss.html">Working Set Size Estimation</a></li></ul><p><a href="https://www.kernel.org/doc/Documentation/vm/pagemap.txt">https://www.kernel.org/doc/Documentation/vm/pagemap.txt</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=82849bb6b404" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-under-the-hood/under-the-hood-os-demand-paging-page-faults-and-working-set-82849bb6b404">Under the hood OS: Demand Paging, Page Faults and Working Set</a> was originally published in <a href="https://medium.com/software-under-the-hood">software under the hood</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Under the hood attacks: Spectre and Meltdown]]></title>
            <link>https://medium.com/software-under-the-hood/under-the-hood-attacks-spectre-and-meltdown-4198093e9dd7?source=rss----44e0597a33ff---4</link>
            <guid isPermaLink="false">https://medium.com/p/4198093e9dd7</guid>
            <category><![CDATA[cpu]]></category>
            <category><![CDATA[respect]]></category>
            <category><![CDATA[attack]]></category>
            <category><![CDATA[meltdown]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[aarti gupta]]></dc:creator>
            <pubDate>Fri, 05 Jan 2018 17:19:06 GMT</pubDate>
            <atom:updated>2018-01-22T06:43:19.710Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fr8xWwg_YBomW1PB0Ac3Zw.jpeg" /></figure><blockquote>taken with permission of the author from <a href="https://twitter.com/gsuberland/status/948907452786933762">https://twitter.com/gsuberland/status/948907452786933762</a> as this is the best explanation I have found !</blockquote><p>When a processor reaches a conditional branch in code (e.g. an ‘if’ clause), it tries to predict which branch will be taken before it actually knows the result. It executes that branch ahead of time — a feature called “speculative execution”.</p><p>The idea is that if it gets the prediction right (which modern processors are quite good at) it’ll already have executed the next bit of code by the time the actually-selected branch is known. If it gets it wrong, execution unwinds back and the correct branch is executed instead.</p><p>What makes the processor so good at branch prediction is that it stores details about previous branch operations, in what’s called the Branch History Buffer (BHB). If a particular branch instruction took path A before, it’ll probably take path A again, rather than path B.</p><p>What makes this interesting is that code is executed *speculatively*, before the result of a conditional statement has completed. That conditional statement could be security-critical. Thankfully the processor is (mostly) smart enough to roll back any side-effects of execution.</p><p>There are two important exclusions to the rollback of side-effects: cache and branch prediction history. These generally aren’t rolled back because speculative execution is a performance feature, and rolling back cache and BHB contents would generally hurt performance.</p><p>There are three ways to exploit this behaviour. The Spectre paper describes the first two exploits, with the following results:</p><p><strong>1. Kernel memory disclosure from userspace on bare metal.</strong></p><p>The first exploit works by getting the kernel to execute some carefully written attacker-specified code which contains an array bounds check followed by an array read, where the read index is controlled by an attacker. This sounds like a big ask, but it’s not thanks to JIT.</p><p><strong>On Linux, Extended Berkley Packet Filter (eBPF) allows users to write socket filters from usermode which get JIT compiled by the kernel in order to efficiently filter packets on a socket. The details aren’t important, but it means an attacker can get the kernel to execute code.</strong></p><p><strong>The exploit involves writing eBPF code which compiles to the following steps: 1. Allocate two fixed-size arrays 2. Bounds-check the user-provided index 3. If ok, read from the array1 at that index 4. Compute another index from 1 bit of the result 5. Read from array2 at that index</strong></p><p>In terms of “real” execution, this code always terminates at step 2 when the user passes an out-of-bounds index for array1. But if the processor’s branch predictor assumes that check will succeed, it’ll speculatively execute the out-of-bounds read in step 3, and continue to 5.</p><p>Here’s the clever bit. In step 4 we take the value we got from the out-of-bounds read (which we wouldn’t normally have access to) and use one bit from it to select a particular memory address (array index) to read. If b=0 it reads index 0x200; if b=1 it reads index 0x300.</p><p>This ensures that the memory at either index 0x200 or index 0x300 is now cached. The CPU then realises that the bounds check in step 2 failed, so it unwinds back to that branch. However, the data from step 5 is still cached!</p><p>We can then go in and read the data at 0x200 and 0x300 and see which is cached by measuring how quick the read is. Once we know which index was cached we can directly infer one bit of kernel memory, based on the index selection from step 4.</p><p>There are some details as to how the cache needs to be primed before this attack, but it is possible to do this whole process in a loop and dump kernel memory from unprivileged userspace.</p><p>The second attack described in the Spectre paper involves poisoning the branch prediction history to trick the processor into speculatively executing code at an attacker-specified address, leading to further cache attacks as described above.</p><p>By performing a carefully selected sequence of indirect jumps, an attacker can fill up the branch prediction history in a way that allows the attacker to select which branch will be speculatively executed when performing an indirect jump.</p><p>This can be very powerful. If I know there’s a piece of code in kernel space that exhibits similar behaviour to our eBPF example from before, and I know what the address of that code is, I can indirectly jump to that code and the CPU will speculatively execute it</p><p>If you’ve done exploitation before, you’ll probably recognise this as being similar to a ROP gadget. We’re looking for a sequence of code in kernel space that happens to have the right sequence of instructions to leak information via cache.</p><p>Keep in mind that the execution is speculative only — the processor will later realise that I didn’t have the privilege to jump to that code and throw an exception. So the target code has to leak kernel data via cache side-channels like before</p><p>You’ll also notice that we need to know address of the target kernel code. With KASLR this isn’t so easy. Project Zero’s writeup explains how KASLR can be defeated using branch prediction and caching as side-channels, so I won’t go into the details here. <a href="https://t.co/6yLboeoukG">https://googleprojectzero.blogspot.co.uk/2018/01/reading-privileged-memory-with-side.html …</a></p><p><strong>2. Kernel memory disclosure of the VM host/hypervisor from kernelspace in a VM.</strong></p><p>What makes this extra powerful is that it works across VM boundaries too. Instead of a traditional indirect jump (e.g. jmp eax), we can use the vmcall instruction to speculatively execute code within the VM host’s kernel in the same way we would our VM’s kernel.</p><p>Finally, there’s the third approach. This involves a flush+reload cache attack against kernel memory, similar to the first variant of the attack but without requiring kernel code execution — it can all be done from usermode.</p><p>The idea is that we try to read kernelspace memory using a mov instruction, then perform a secondary memory read with an address based on the value that was read. If you’re thinking the first mov will fail because we’re in usermode and can’t read kernel addresses, you’re right.</p><p>The trick is that the microarchitectural implementation of mov contains the memory page privilege level check, which itself is a branch instruction. The processor may speculatively execute that branch like any other.</p><p>So, if you can outrun the interrupt, you can speculatively execute some other instruction that loads data into cache based on the value read from kernelspace. This then becomes a cache attack like the previous tricks.</p><p><a href="https://twitter.com/gsuberland">Graham Sutherland [Polynomial^DSS] (@gsuberland) | Twitter</a></p><p><a href="https://t.co/6QOUv6FvR8">https://spectreattack.com/spectre.pdf </a><a href="https://t.co/F9i3MbgK2W">https://meltdownattack.com/meltdown.pdf</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4198093e9dd7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-under-the-hood/under-the-hood-attacks-spectre-and-meltdown-4198093e9dd7">Under the hood attacks: Spectre and Meltdown</a> was originally published in <a href="https://medium.com/software-under-the-hood">software under the hood</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Under the hood definitions: Memory model and memory barriers]]></title>
            <link>https://medium.com/software-under-the-hood/under-the-hood-definitions-memory-model-f41cd3b47757?source=rss----44e0597a33ff---4</link>
            <guid isPermaLink="false">https://medium.com/p/f41cd3b47757</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[definition]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[aarti gupta]]></dc:creator>
            <pubDate>Wed, 03 Jan 2018 18:36:08 GMT</pubDate>
            <atom:updated>2018-05-03T17:15:56.742Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*8ZnhPu9ZX7JZsht_HHnxtw.jpeg" /></figure><h3>What is a memory model</h3><p><em>c</em><a href="http://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html"><em>ould not find a better explanation till date, quoting with full references</em></a></p><blockquote><em>In multiprocessor systems, processors generally have </em><strong><em>one or more layers of memory cache</em></strong><em>, which improves performance both by speeding access to data (because the data is closer to the processor) and reducing traffic on the shared memory bus (because many memory operations can be satisfied by local caches.) Memory caches can improve performance tremendously, but they present a host of new challenges. What, for example, happens when two processors examine the same memory location at the same time? Under what conditions will they see the same value?</em></blockquote><blockquote><em>At the processor level, </em><strong><em>a memory model defines necessary and sufficient conditions for knowing that writes to memory by other processors are visible to the current processor, and writes by the current processor are visible to other processors</em></strong><em>. Some processors exhibit a </em><strong><em>strong memory model, where all processors see exactly the same value for any given memory location at all times</em></strong><em>. Other processors exhibit a weaker memory model, where special instructions, called </em><strong><em>memory barriers, are required to flush or invalidate the local processor cache in order to see writes made by other processors or make writes by this processor visible to others. These memory barriers are usually performed when lock and unlock actions</em></strong><em> are taken; they are invisible to programmers in a high level language.</em></blockquote><p>A properly placed <strong>memory barrier</strong> prevents this problem by forcing the processor to serialize pending memory operations.<br>Memory barriers are not directly exposed by the JVM; instead they are inserted into the instruction sequence by the JVM in order to uphold the semantics of language level concurrency primitives.<br>The keyword volatile is an example It establishes a happens before relationship. The compiler cannot re-order two volatile write operations and if necessary it must forbid the processor from doing so with a memory barrier.<br>Memory barriers can be hardware specific. For the volatile read ordering and volatile write ordering,<strong> Java inserts assembly instructions such as load acquire (ld1.acq) and store release (st1.rel) respectively.</strong><br>Note these are also unidirectional barriers (applicable to either reads or writes respectively)<br><strong>bidirectional memory barriers such as memory fence (mf) instruction can also be inserted by the compiler where necessary.</strong></p><blockquote><em>It can sometimes be easier to write programs for strong memory models, because of the reduced need for memory barriers. However, even on some of the strongest memory models, memory barriers are often necessary; quite frequently their placement is counterintuitive. Recent trends in processor design have encouraged weaker memory models, because the </em><strong><em>relaxations they make for cache consistency allow for greater scalability across multiple processors and larger amounts of memory.</em></strong></blockquote><blockquote><em>The issue of when a write becomes visible to another thread is compounded by the compiler’s reordering of code. For example, the compiler might decide that it is more efficient to move a write operation later in the program; as long as this code motion does not change the program’s semantics, it is free to do so. If a compiler defers an operation, another thread will not see it until it is performed; this mirrors the effect of caching.</em></blockquote><blockquote><em>Moreover, writes to memory can be moved earlier in a program; in this case, other threads might see a write before it actually “occurs” in the program. All of this flexibility is by design — by giving the compiler, runtime, or hardware the flexibility to execute operations in the optimal order, within the bounds of the memory model, we can achieve higher performance.</em></blockquote><blockquote><em>A simple example of this can be seen in the following code:</em></blockquote><blockquote><em>Class Reordering {<br> int x = 0, y = 0;<br> public void writer() {<br> x = 1;<br> y = 2;<br> }</em></blockquote><blockquote><em>public void reader() {<br> int r1 = y;<br> int r2 = x;<br> }<br>}</em></blockquote><blockquote><em>Let’s say that this code is executed in two threads concurrently, and the read of y sees the value 2. Because this write came after the write to x, the programmer might assume that the read of x must see the value 1. However, the writes may have been reordered. If this takes place, then the write to y could happen, the reads of both variables could follow, and then the write to x could take place. The result would be that r1 has the value 2, but r2 has the value 0.</em></blockquote><blockquote><strong><em>The Java Memory Model describes what behaviors are legal in multithreaded code, and how threads may interact through memory. It describes the relationship between variables in a program and the low-level details of storing and retrieving them to and from memory or registers in a real computer system.</em></strong><em> It does this in a way that can be implemented correctly using a wide variety of hardware and a wide variety of compiler optimizations.</em></blockquote><blockquote><em>Java includes several language constructs, including </em><strong><em>volatile, final, and synchronized</em></strong><em>, which are intended to help the programmer describe a program’s concurrency requirements to the compiler. The Java Memory Model defines the behavior of volatile and synchronized, and, more importantly, ensures that a correctly synchronized Java program runs correctly on all processor architectures.</em></blockquote><blockquote><a href="http://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html">http://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f41cd3b47757" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-under-the-hood/under-the-hood-definitions-memory-model-f41cd3b47757">Under the hood definitions: Memory model and memory barriers</a> was originally published in <a href="https://medium.com/software-under-the-hood">software under the hood</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Under the hood JVM: Safepoints]]></title>
            <link>https://medium.com/software-under-the-hood/under-the-hood-java-peak-safepoints-dd45af07d766?source=rss----44e0597a33ff---4</link>
            <guid isPermaLink="false">https://medium.com/p/dd45af07d766</guid>
            <category><![CDATA[jvm]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[aarti gupta]]></dc:creator>
            <pubDate>Sun, 31 Dec 2017 23:02:35 GMT</pubDate>
            <atom:updated>2017-12-31T23:59:57.724Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/416/1*mDFrDEXLJiGs5hUlwRHSJg.jpeg" /></figure><ul><li>A <strong>safepoint</strong> is a range of execution where the state of the executing thread is well described. Safepoints are a common JVM implementation detail</li><li><strong>At a safepoint</strong> the <strong>mutator</strong> thread is at a known and well defined point in it’s interaction with the heap. This means that all the references on the stack are mapped (at known locations) and the JVM can account for all of them. As long as the thread remains <strong>at a safepoint</strong> we can safely manipulate the heap + stack such that the thread’s view of the world remains consistent when it leaves the safepoint.</li><li>All current JVMs have some requirement for <strong>global safepoints</strong></li><li>A Java thread is <strong>at a safepoint</strong> if it is blocked on a lock or synchronized block, waiting on a monitor, parked, or blocked on blocking IO. Essentially these all qualify as orderly de-scheduling events for the Java thread and as part of tidying up before put on hold the thread is brought to a safepoint.</li><li>A Java thread is <strong>at a safepoint</strong> while executing JNI code. Before crossing the native call boundary the stack is left in a consistent state before handing off to the native code. This means that the thread can still run while <strong>at a safepoint</strong>.</li><li>A Java thread which is executing bytecode <strong>is NOT at a safepoint </strong>(or at least the JVM cannot assume that it is at a safepoint).</li><li>A Java thread which is interrupted (by the OS) while not at a safepoint is not brought to a safepoint before being de-scheduled.</li><li>The JVM and your running Java threads have the following relationship around safepoints:</li><li>The JVM cannot force any thread into a safepoint state.</li><li>The JVM can stop threads from leaving a safepoint state.</li></ul><p>So how can the JVM bring all threads into a safepoint state? The problem is suspending a thread at a known state, not just interrupting it. To achieve this goal JVMs have the Java threads suspend themselves at convenient spots if they observe a ‘safepoint flag’.</p><h3>Bringing a Java Thread to a Safepoint</h3><p>Java threads poll a ‘safepoint flag’ (global or thread level) at ‘reasonable’ intervals and transition into a safepoint state (thread is blocked at a safepoint) when they observe a ‘Go to safepoint’ flag. This is simple enough, but since we don’t want to spend all of our time checking if we need to stop the C1/C2 compilers (-client/-server JIT compilers) try and keep safepoint polls to a minimum. On top of the cost of the flag check itself, maintaining a ‘known state’ adds significant complexity to the implementation of certain optimizations and so keeping safepoints further apart opens up a wider scope for optimization. These considerations combined lead to the following locations for <strong>safepoint polls:</strong></p><ul><li>Between any 2 bytecodes while running in the interpreter (effectively)</li><li>On ‘non-counted’ loop back edge in C1/C2 compiled code</li><li>Method entry/exit (entry for Zing, exit for OpenJDK) in C1/C2 compiled code. Note that the compiler will remove these safepoint polls when methods are inlined.</li></ul><p>If you are the sort of person who looks at assembly for fun (or profit, or both) you’ll find safepoint polls in the -XX:+PrintAssembly output by looking for:</p><ul><li><strong>‘{poll}’</strong> or ‘<strong>{poll return}’ </strong>on OpenJDK, this will be in the instructions comments</li></ul><p>From the mechanical sympathy mailing thread</p><blockquote><em>1. A thread can be at a safepoint or not be at a safepoint. When at a safepoint, the thread’s representation of it’s Java machine state is well described, and can be safely manipulated and observed by other threads in the JVM. When not at a safepoint, the thread’s representation of the java machine state will NOT be manipulated by other threads in the JVM. [Note that other threads do not manipulate a thread’s actual logical machine state, just it’s representation of that state. A simple example of changing the representation of machine state is changing the virtual addresss that a java reference stack variable points to as a result of relocating that object. The logical state of the reference variable is not affected by this change, as the reference still refers to the same object, and two references variable referring to the same object will still be logically equal to each other even if they temporarily point to different virtual addresses].</em></blockquote><blockquote><em>2. “Being at a safepoint” does not mean “being blocked” (e.g. JNI code runs at a safepoint), but “being blocked” always happens at a safepoint.</em></blockquote><blockquote><em>3. The JVM may choose to reach a global safepoint (aka Stop-The-World), where all threads are at a safepoint and can’t leave the safe point until the JVM decides to let it do so. This is useful for doing all sorts of work (like certain GC operations, deoptimization during class loading, etc.) that require ALL threads to be at a well described state.</em></blockquote><blockquote><em>4. Some JVMs can bring individual threads to safepoint without requiring a global safepoint. E.g. Zing uses the term Checkpoint (first published in [1]) to describe a JVM mechanism that individually passes threads through thread-specidfic safepoints to perform certain very short operations on individual thread state without requiring a Stop-The-Wolrd pause.</em></blockquote><blockquote><em>5. When you write Unsafe java code, you must assume that a safepoint MAY occur between any two bytecodes.</em></blockquote><blockquote><em>6. Unsafe calls are not required to have safepoints within them (and many/most don’t), but they MAY include one or more safepoints. E.g. an unsafe memoryCopy MAY include a periodic safepoint opportunities (e.g. take a safepoint every 16KB). Zing sure does, as we do a lot of under the hood work to keep TTSP in check.</em></blockquote><blockquote><em>7. All [practical] JVMs apply some highly efficient mechanism for frequently crossing safepoint opportunities, where the thread does not actually enter a safepoint unless someone else indicates the need to do so. E.g. most call sites and loop backedges in generated code will include some sort of safepoint polling sequence that amounts to “do I need to go to a safepoint now?”. Many HotSpot variants (OpenJDK and Oracle JDK) currently use a simple global “go to safepoint” indicator in the form of a page that is protected when a safepoint is needed, and unprotected otherwise. The safepoint polling for this mechanism amounts to a load from a fixed address in that page. If the load traps with a SEGV, the thread knows it needs to go to enter a safepoint. Zing uses a different, per-thread go-to-safepoint indicator of similar efficiency.</em></blockquote><blockquote><em>8. All JNI code executes at a safepoint. No Java machine state of the executing thread can be changed or observed by it’s JNI code while at a safepoint. Any manipulation or observation of Java state done by JNI code is achieved via JNI API calls that leave the safepoint for the duration of the API call, and then enter a safepoint again before returning to the calling JNI code. This “crossing of the JNI line” is where most of the JNI call and API overhead lies, but it’s fairly quick (entering and leaving safepoint normally amounts to some CAS operations).</em></blockquote><h4><strong>Examples of Operations that may encounter safepoints</strong></h4><ul><li>Deoptimization</li><li>PrintThreads</li><li>PrintJNI</li><li>FindDeadlock</li><li>ThreadDump</li><li>EnableBiasLocking</li><li>RevokeBias</li><li>HeapDumper</li><li>GetAllStackTrace</li></ul><p><strong><em>References</em></strong></p><p><a href="http://psy-lob-saw.blogspot.com/2015/12/safepoints.html">http://psy-lob-saw.blogspot.com/2015/12/safepoints.html</a></p><p><a href="http://chriskirk.blogspot.com/2013/09/what-is-java-safepoint.html">http://chriskirk.blogspot.com/2013/09/what-is-java-safepoint.html</a></p><p><a href="http://jpbempel.blogspot.com/2013/03/safety-first-safepoints.html">http://jpbempel.blogspot.com/2013/03/safety-first-safepoints.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd45af07d766" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-under-the-hood/under-the-hood-java-peak-safepoints-dd45af07d766">Under the hood JVM: Safepoints</a> was originally published in <a href="https://medium.com/software-under-the-hood">software under the hood</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Under the hood CPU: Know your CPU]]></title>
            <link>https://medium.com/software-under-the-hood/under-the-hood-experiment-1-know-your-cpu-be26a2435c6b?source=rss----44e0597a33ff---4</link>
            <guid isPermaLink="false">https://medium.com/p/be26a2435c6b</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[computer-architecture]]></category>
            <category><![CDATA[scale]]></category>
            <category><![CDATA[cpu]]></category>
            <category><![CDATA[high-performance]]></category>
            <dc:creator><![CDATA[aarti gupta]]></dc:creator>
            <pubDate>Wed, 27 Dec 2017 01:19:09 GMT</pubDate>
            <atom:updated>2018-01-03T14:56:11.371Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong>know the mechanics of your race car, part 1: CPU</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p3WhAs3Dl35ig3eur01Buw.jpeg" /></figure><h3>Terminology used</h3><p><strong><em>CPU or Central Processing Unit</em></strong> executes instructions of a computer program by performing the basic arithmetic, logical, control and input/output (I/O) operations specified by the instructions</p><p><strong>CPU cores: </strong>A core is usually the basic computation unit of the CPU — it can run a single program context (or multiple ones if it supports hardware threads such as hyperthreading on Intel CPUs).A CPU may have one or more cores to perform tasks at a given time.</p><p><strong>CPU clock speed, or clock rate,</strong> is measured in Hertz — generally in gigahertz, or GHz. A CPU’s clock speed rate is a measure of how many clock cycles a CPU can perform per second. For example, a CPU with a clock rate of 1.8 GHz can perform 1,800,000,000 clock cycles per second.</p><p><strong>Hardware threads:</strong> Generic term that refers to multithreading achieved mostly by duplicating thread state and sharing most everything else in a processing core. Multithreading achieved by duplicating most everything, the whole “core,” is what multicore and many-core designs are all about. An OS may have many threads to run, but the CPU can only run X such tasks at a given time, where X = number cores * number of hardware threads per core. The rest would have to wait for the OS to schedule them whether by preempting currently running tasks or any other means.</p><p><strong>Hyperthreading:</strong> The CPU pretends it has more cores than it does, and it uses its own logic to speed up program execution. Hyper-threading allows the two logical CPU cores to share physical execution resources. This can speed things up somewhat — if one virtual CPU is stalled and waiting, the other virtual CPU can borrow its execution resources. Hyper-threading can help speed your system up, but it’s nowhere near as good as having actual additional cores. In other words, the operating system is tricked into seeing two CPUs for each actual CPU core. Useful for tasks like video editing, 3D rendering, and <strong>heavy multi</strong>-tasking. Similarly, Hyper Threading can help a CPU push light tasks like background applications or browser windows to one <strong>processor</strong>, while <strong>heavy</strong> applications like games or <strong>full</strong>-screen video goes to another</p><p><strong>Difference in hardware threads and hyperthreading:</strong> “Hyperthreading” is a very specific form of implementing a “hardware thread” that is only found on dynamic (a.k.a. out-of-order) execution engines.</p><p><strong>Dynamic Execution/Out of order execution engines:</strong> Dynamic reordering of instructions lets the CPU hide memory latencies, allowing for even higher clock speeds. For every cache miss, a Pentium 4 3.6GHz has to wait around 230 clock cycles to get data from main memory, which is a lot of idle time in the eyes of the CPU.<strong> Incremental increase in instruction level parallelism</strong> — by reordering instructions on the fly, out-of-order architectures can improve ILP as best as possible in areas where the compiler fails to.</p><blockquote><strong>Components of the CPU</strong></blockquote><blockquote><strong>1. Execution Units</strong></blockquote><blockquote><strong>Control unit</strong></blockquote><p>The control unit of the CPU contains circuitry that uses electrical signals to direct the entire computer system to carry out stored program instructions. The control unit does not execute program instructions; rather, it directs other parts of the system to do so. The control unit communicates with both the ALU and memory.</p><blockquote><strong>Arithmetic logic unit</strong></blockquote><p>The arithmetic logic unit (ALU) is a digital circuit within the processor that performs integer arithmetic and bitwise logic operations. The inputs to the ALU are the data words to be operated on (called operands), status information from previous operations, and a code from the control unit indicating which operation to perform. Depending on the instruction being executed, the operands may come from internal CPU registers or external memory, or they may be constants generated by the ALU itself.</p><blockquote><strong>Memory management unit</strong></blockquote><p>Most high-end microprocessors (in desktop, laptop, server computers) have a memory management unit, translating logical addresses into physical RAM addresses, providing memory protection and paging abilities, useful for virtual memory. Simpler processors, especially <a href="https://en.wikipedia.org/wiki/Microcontroller">microcontrollers</a>, usually don’t include an MMU.</p><blockquote><strong>2. Registers</strong></blockquote><p>In computer architecture, a <strong>processor register</strong> is a quickly accessible location available to a computer’s central processing unit (CPU).</p><p>Registers are normally measured by the number of bits they can hold, for example, an “<a href="https://en.wikipedia.org/wiki/8-bit">8-bit</a> register” or a “<a href="https://en.wikipedia.org/wiki/32-bit">32-bit</a> register”. A processor often contains several kinds of registers, that can be classified according to their content or instructions that operate on them:</p><blockquote><strong>CPU Operations and Instruction Cycle</strong></blockquote><p>The fundamental operation of most CPUs, regardless of the physical form they take, is to execute a sequence of stored <a href="https://en.wikipedia.org/wiki/Instruction_(computing)">instructions</a> that is called a program. The instructions to be executed are kept in some kind of <a href="https://en.wikipedia.org/wiki/Memory_(computers)">computer memory</a>. Nearly all CPUs follow the<strong> fetch, decode and execute</strong> steps in their operation, which are collectively known as the <a href="https://en.wikipedia.org/wiki/Instruction_cycle">instruction cycle</a>.</p><p>After the execution of an instruction, the entire process repeats, with the next instruction cycle normally fetching the next-in-sequence instruction because of the incremented value in the <a href="https://en.wikipedia.org/wiki/Program_counter">program counter</a>. If a jump instruction was executed, the program counter will be modified to contain the address of the instruction that was jumped to and program execution continues normally. In more complex CPUs, multiple instructions can be fetched, decoded, and executed simultaneously.</p><blockquote><strong>SIMP: Single Instructions Multiple Data</strong></blockquote><p>SIMD instructions allow the CPU to perform the same operation on multiple values simultaneously. For example we would like to perform four multiplications on eight values:</p><pre>z1 = x1 * y1<br>z2 = x2 * y2<br>z3 = x3 * y3<br>z4 = x4 * y4</pre><p>Normally that would require eight instructions to load values from memory into registers and four multiplication instructions. Using SIMD instructions, the CPU can load all four x values into the xmm0 with a single MOVUPS instruction,, another MOVUPS to load the four y values into the xmm1 register and a single MULPS instruction to multiply them</p><pre>+-------+-------+-------+-------+<br>|   x3  |   x2  |   x1  |   x0  | xmm0<br>+-------+-------+-------+-------+<br>    *       *       *       *<br>+-------+-------+-------+-------+<br>|   y3  |   y2  |   y1  |   y0  | xmm1<br>+-------+-------+-------+-------+<br>    =       =       =       =<br>+-------+-------+-------+-------+<br>| x3*y3 | x2*y2 | x1*y1 | x0*y0 | xmm0<br>+-------+-------+-------+-------+</pre><p>The key feature here is that this multiplication will be performed simultaneously on all four values, which will be four times faster. SIMD instructions are often called vectorized instructions, because you can think of them as operating on vectors of values.</p><blockquote><strong>Determining equivalent for your computer</strong></blockquote><blockquote>cat /proc/cpuinfo<br>processor : 0<br>vendor_id : GenuineIntel<br>cpu family : 6<br>model : 63<br>model name : Intel(R) Xeon(R) CPU E5–2676 v3 @ 2.40GHz<br>stepping : 2<br>microcode : 0x3a<br>cpu MHz : 2400.223<br>cache size : 30720 KB<br>physical id : 0<br>siblings : 1<br>core id : 0<br>cpu cores : 1<br>apicid : 0<br>initial apicid : 0<br>fpu : yes<br>fpu_exception : yes<br>cpuid level : 13<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt<br>bugs :<br>bogomips : 4800.10<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 46 bits physical, 48 bits virtual<br>power management:</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be26a2435c6b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-under-the-hood/under-the-hood-experiment-1-know-your-cpu-be26a2435c6b">Under the hood CPU: Know your CPU</a> was originally published in <a href="https://medium.com/software-under-the-hood">software under the hood</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>