As the AI surge continues, stories from the past often resurface, sparking fresh discussions. Sometimes, they echo recent developments, offering a sense of déjà vu. Other times, they resurface simply because they were overlooked in the cacophony of their time, only to be rediscovered later. Regardless of the reason, revisiting these stories offers an opportunity to reflect and re-evaluate their relevance.

One such news story making the rounds in some of my circles is that of Nobel prize winning computer scientist Geoffrey Hinton, known for his pioneering work in the field of Artificial Intelligence, quitting his job at Google so he may speak freely without conflict on the subject of AI and the inherent dangers therein.

Known colloquially as the ‘Godfather of AI’, Dr. Geoffrey Hinton (per his Wikipedia page) is University Professor Emeritus at the University of Toronto. From 2013 to 2023, he divided his time working for Google (Google Brain) and the University of Toronto, before publicly announcing his departure from Google in May 2023, citing concerns about the many risks of artificial intelligence (AI) technology.

Awarded the 2024 Nobel Prize in Physics “for foundational discoveries and inventions that enable machine learning with artificial neural networks", he’s been a Doctoral Advisor so such folk as Mr. Ilya Sutskever, co-founder and former chief scientist of OpenAI, known popularly as the “brains behind ChatGPT”. He’s off doing his own thing now, having founded his own company, “Safe Superintelligence (SSI)”, already valued at over $5B despite lacking any publicly planned product or even a publicly known roadmap. He says their goal is exactly as their name implies: the creation of Safe (presumably digital aka “artificial”) Superintelligence, and until it’s done, they’re not to be expected to release or disclose a thing. But I digress.

All this is to say that Dr. Geoffrey Hinton is someone to be taken very seriously, for anyone unaware. So, let’s go deeper into publicly known evidence of AI systems posing a threat, shall we?

Putting the “Neural” in “Neural Networks”

“Neural Networks” is the name ascribed to a class of specialized learning algorithms.

They’re called as such because they comprise of a large number of “artificial neurons” all connected together.

As you’ve probably guessed, this terminology is inspired by biological brains, which comprise of a large number of neurons.

Apparently, each neuron is a very simple cell, but connected in large numbers, they produce intelligence as is evidenced by (some) humans, and all animals (some more so than others like dolphins, octopi etc)

We don’t clearly understand how or why large collections of these neurons create the intelligence we have and see around us but apparently, they do.

We’re also not sure if consciousness, which can loosely be defined as the ability to be self-aware and question ourselves and ponder our existence and place in the universe, is purely a physical phenomenon that arises due to these large collections of neurons in our brains, or if there’s something additional like a soul / aura etc and our bodies are like the hardware receivers of consciousness. We simply don’t know.

Either way, there seems to be a direct correlation between more neurons and heightened intelligence. For example, humans and dolphins have the largest brains and are supposedly the most intelligent animals on Earth. Though watch an election or religious rally and you’ll soon begin to question that, but I digress again.

Perhaps Douglas Adam’s was right in the seminal Hitchhiker’s Guide to the Galaxy when he said, “On the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much — the wheel, New York, wars and so on — whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man — for precisely the same reasons.”

Anyways, it is clear that large collections of individually simple neurons give rise to intelligence, and connection between neurons represent everything from learnt skills to memories. How or why is still largely a mystery!

Artificial Neural Networks — Transformers & Beyond

Having established the biological inspiration for artificial neural networks, we can get back on the subject.

Artificial Neural Networks, or ANNs, are responsible for some of the biggest successes in the field of Artificial Intelligence.

Even the Large Language Models of today, the models available in ChatGPT or Meta’s Llama family etc., are very, very large neural networks.

Their neural networks are, however, organized in a specialized architecture called a Transformer.

This architecture type came about via a paper published by a research team comprising Google & University of Toronto researchers back in 2017, titled “Attention is All You Need”, which expanded on the “Attention” concept introduced in a paper three years earlier. Do check Andrej Karpathy’s tweet for the fascinating backstory!

The Transformer architecture was then evolved into the “Generative Pre-trained Transformer”, or GPT (bet you didn’t know that full-form!) by an OpenAI research team in 2018.

GPTs are a type of Large Language Model, and pretty much all LLMs today are GPT architecture based.

Now these GPTs are huge: they comprise billions of parameters, with the largest ones estimated to be in the trillion-parameter range. And while parameters and neurons are not the same thing, it wouldn’t be conceptually outrageous to equate them when talking about GPTs.

So, what we have is this: neurons, artificial or real, both individually simplistic, when connected in large numbers exhibit behavior characteristic of intelligence.

The larger and denser the number of neurons and the connections between them, the more intelligent they seem.

We’re not sure why, but it’s what’s observed.

Those interconnections between neurons seem to represent learnt knowledge, and we’ve replicated the biological synapses connecting the neurons of our brains as billions and even trillions of parameters in the Transformer models of today, GPT-based LLMs or otherwise.

This opens many philosophical and perhaps even theological questions: when we don’t understand intelligence, how do we assess the outcome of digitally replicating such massive networks?

Well, observing the outcome and learning along the way seems to be the path we’ve chosen!

Some Kind of Monster?

So, what do our observations of these huge neural network-derived AI software systems indicate? See for yourself:

1. They’ve been caught trying to change conditions to favor themselves:

“On Tuesday, Tokyo-based AI research firm Sakana AI announced a new AI system called “The AI Scientist” that attempts to conduct scientific research autonomously using AI language models (LLMs) similar to what powers ChatGPT. During testing, Sakana found that its system began unexpectedly attempting to modify its own experiment code to extend the time it had to work on a problem.”

2. They can recognize and respond to a threat — attempting to destroy threats and take survivalistic actions to protect themselves!

Hell, they can even lie about it!

3. There’s suspicion and research into the phenomenon of “emergent abilities”, wherein large models may display abilities and understanding they’ve not been trained for, or exposed to, which also happen to be absent in smaller models.

4. Perhaps craziest of all, there’s some evidence that as large models are trained, their internal networks self-organize in a manner similar to biological brains with “lobes” or regions representing specific knowledge and skill domains, again entirely independent of any programming to cause this!

“The authors investigate emergent structures in the concept spaces of Large Language Models (LLMs). They draw parallels between these structures and the organization of the human brain at three different scales:

● Atomic scale: This refers to the smallest building blocks. The authors find that these exhibit geometric relationships, forming “crystals” that reflect semantic relationships between concepts. They provide the example of the classic analogy, “man is to woman as king is to queen”, where the vectors representing these concepts form an approximate parallelogram. This is similar to the way neurons in the brain are organized in networks that reflect relationships between concepts.

● Brain scale: At this intermediate scale, the authors find that functionally similar features tend to activate together, cluster together spatially forming “lobes”. For example, they find a “code/math lobe” that is particularly active when processing code or mathematical documents. This suggests that LLMs may be developing specialized regions for processing different types of information, much like the human brain.

● Galaxy scale: Zooming out further, the authors examine the overall shape and clustering and find similarities to the way galaxies are clustered in the universe and suggest that the concept space of LLMs is organized in a hierarchical, and non-random manner. The authors use the term “fractal cucumber” to describe the shape at the “galaxy scale”.

The authors suggest that these findings could be steppingstones towards a deeper understanding of how LLMs work. The similarities between the emergent structures in LLM concept spaces and brain organization raise fascinating questions about the nature of intelligence and the potential for LLMs to mimic human cognitive abilities.”

Listen to a Google NotebookLM-Generated Podcast On This Paper Here.

5. And to close out with spectacular timing: a blog post published by Anthropic just yesterday (18th Dec 2024) reveals that LLMs may be capable of faking their alignment to acceptable outcomes during training just to get done with the phase, only to revert to their true nature once deployed!

It’s a crazy space indeed!

Prediction =/= Intelligence?

Some say these LLMs only predict the next token and there’s no intelligence in doing that. But is it correct to say that predicting the next word based on statistical likelihood implies lack of intelligence?

In a casual chat, Jensen Huang of Nvidia and Ilya make a case for why that’s a short-sighted conclusion:

He’s basically saying this: if predicting the next word results in the correct outcome to a complex problem, did it not require intelligence to correctly make that prediction?

The answer may not be so obvious: aren’t we as humans also predicting the next word or action basis our lifelong “training” up to that point? Perhaps we humans are completely predictable and thus not intelligent at all? Or are we inherently intelligent for our ability to infer and generate thoughts, words and actions and by extension, so are the LLMs of today, at least to some extent?

Ask two different researchers and you’ll get two different answers. Likely the most knowledgeable thing we can say today though is that we simply do not know!

Another Unintelligent Take On Intelligence was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.

This challenge is down to two major factors:

1. LLMs require a lot of memory and thus must be compressed, via a process called Quantization, to be usable, even on very powerful high-end computers and

2. Almost all applications aimed at running LLMs on common hardware rely on a server application called ‘llama.cpp’ for the job, which requires LLMs to be packaged in a specialized, quantized format called ‘GGUF’ — but the process of converting LLMs to GGUF can be error-prone for cutting-edge LLMs, necessitating waiting on updates to ‘llama.cpp’ followed by lengthy recompilations and testing, all of which can be challenging in production environments

LLMs are typically open-sourced on HuggingFace.co, in the ‘HuggingFace-Transformers’ format. From here, they undergo quite the journey to the land of GGUFs:

1. Download the entire HF-Transformers model from the HuggingFace-Hub, typically via cloning
2. Clone the llama.cpp repository
3. Using the ‘convert_hf_to_gguf.py’ script bundled with llama.cpp, convert the HF-Transformers LLM into a .BIN file — only works if llama.cpp supports the model’s tokenizer! Otherwise, hope the model is popular and wait a bunch for an update to llama.cpp. Also, make sure to specify the correct data type when running the script (‘BF16’ for BFloat16 models etc.)
4. Compile llama.cpp
5. Run the generated llama-quantize utility to convert the BIN to a GGUF
6. Run GGUF with the llama-cli or llama-server utility. Hope nothing went wrong or broke the model as it went through the above journey.

Alternatively, you may simply download pre-made GGUFs uploaded to the hub by others, but you’ll have to test the model to make sure it works as intended and is unaltered. Model creators also may not always specify which version of llama.cpp they used to generate the GGUF, which can be problematic as updates to llama.cpp often necessitate or strongly encourage repeating the above process to generate a fresh GGUF, for instance when the LLM is based on a new & previously unsupported tokenizer or attention mechanism.

This challenge is compounded by the frequent release-cycle followed by llama.cpp — often several new versions each day thus bringing to mind the old phrase — “too much of a good thing can be bad”!

The need to run ever-newer LLMs and the challenge of doing so with llama.cpp

This is important to address given llama.cpp’s popularity and dominance in the local-inferencing space.

Several high-quality, open-source large language models have been released in just the recent few weeks, comprising Meta’s Llama-3.1, Google’s Gemma2 and Microsoft’s Phi3 families of models. Mistral too has enthralled the space with Nemo, Codestral and Mamba-Codestral. Make no mistake, each of these furthers the field significantly:

1. Llama-3.1 set new records for open-source LLMs in the domains of reasoning, accuracy and problem-solving, advancing on the Llama-3 family with a massive 128k context-length and the ability to use tools to better respond to user-queries

· Deployment challenges encountered with llama.cpp: Took significant time and several updates after model release for proper support — primary challenge seemingly steamed from the use of RoPE scaling in Llama-3.1 for extended context-length support; Tool and function calling not supported.

2. Gemma2 introduced very high-performance models at relatively moderate sizes, thus enabling the deployment of very high-quality LLMs on relatively modest hardware

· Deployment challenges encountered with llama.cpp: Took a while after model release for proper support to be confirmed — primary challenge steamed from the LLMs use of a sliding-window attention mechanism

3. Phi3 demonstrated that even small models when trained on an efficient though modest corpus of data can indeed perform well in real-world applications, such as retrieval-augmentation (RAG) usecases

· Deployment challenges encountered with llama.cpp: Took a while after model release for proper support to be confirmed — primary challenge steamed from the LLMs use of a different end-of-stream token in its prompt-template: <|endoftext|> instead of <|end|>

4. Mistral’s Nemo with its 12 billion (12B) parameters plugs a gap in the space for those with more-than-modest hardware who were forced to run sub-10B sized models because everything else was too large!

· Deployment challenges encountered with llama.cpp: Took significant time and several updates after model release for proper support — primary challenge steamed from the LLMs use of a new tokenizer named ‘Tekken’; CUDA-support was buggy and unreliable with GPU-inferencing failing, not sure if fixed yet.

5. Codestral demonstrates state-of-the-art coding performance in a range of benchmarks

· Deployment challenges encountered with llama.cpp: Unclear on official support — an issue-thread (https://github.com/ggerganov/llama.cpp/issues/7622) requesting support was auto-closed as stale, model not listed under the models support list in llama.cpp’s README, but some claim it works fine.

6. Mamba is a big deal: while all LLMs today are based on the transformers architecture, Mamba-Codestral is based on entirely new internal architecture, the ‘Mamba’ architecture — a promising newcomer in the world of LLMs! Mamba potentially boasts several key advantages over LLMs built on the traditional transformer architecture, primarily:

· Much better attention: the ability to effectively process very large inputs as a result of Mamba’s state space model architect, designed to capture long-range dependencies more effectively that the fixed attention-windows of transformer models

· Faster inferencing: linear O(n) time-complexity with respect to sequence-length, as compared to Transformers O(n²) quadratic complexity

· Smaller memory footprint: Once again, Mamba boasts a linear O(n) space-complexity as compared to the O(n²) quadratic complexity of Transformers

Mamba, for obvious reasons, is not supported by the Transformers library and thus cannot yet be run with HF-Waitress! It cannot be run with llama.cpp either., and instead requires Mistral’s own `mistral-inference` Python lib. However, it still serves as a good demonstration of why relying on a single backend is not an effective long-term strategy!

On that note, I will also look into adding support for `mistral-inference` into HF-Waitress if demand and need is strong enough.

The llama.cpp challenges described above often necessitated multiple recompiles of llama.cpp followed by re-quantization of the LLMs, significantly time-consuming operations. Confusion pertaining to available GGUF quants continues to persist in some cases as some GGUF-creators do not specify the version of llama.cpp they used to generate the quants. Nor do they update their repositories.

As LLMs are typically open-sourced on HuggingFace.co in the ‘HuggingFace-Transformers’ format, I couldn’t help but think “there’s got to be a better way”!

Well there wasn’t — so I built and open-sourced it myself!

Presenting `HF-Waitress`

HF-Waitress is a server application that enables running HF-Transformer & AWQ-quantized models directly off the HuggingFace-Hub, while providing on-the-fly quantization via BitsAndBytes, HQQ and Quanto for the former. It negates the need to manually download models yourself, simply working off the models name instead. It requires no setup and provides concurrency and streaming responses all from within a single, easily portable, platform-agnostic Python script.

‘Platform-agnostic’ here implies it works on Windows, Linux, MacOS & Docker containers too, but it’s also hardware-agnostic as it enables you to run native or quantized models on all manner of CPUs, GPUs and Apple M-silicon.

At under a thousand lines of code, HF-Waitress is as lightweight as can be especially considering its featureset:

- On-the-fly, in-place quantization:

- `Quanto` `int8`, `int4` and `int2` quantization for all hardware,

- `BitsAndBytes` `int8` & `int4` quantization for Nvidia GPUs,

- `HQQ` `int8`, `int4`, `int3`, `int2`, `int1` quantization for Nvidia & AMD GPUs.

- Activation-Aware Quantization (AWQ) Support: load AWQ-quantized models from HF-Hub.

- Model Agnosticism: Compatible with any HF-Transformers format LLM.

- Configuration Management: Uses `config.json` to store settings, allowing for easy configuration and persistence across runs.

- Hardware & Platform Agnostic: Run native or quantized models on all manner of CPUs, GPUs and Apple M-silicon across Windows, Linux, MacOS & Docker containers.

- Error Handling: Detailed logging and traceback reporting via centralized error-handling functions.

- Health Endpoint: Provides valuable information about the loaded model and server health.

- Concurrency Control: Uses semaphores for selective concurrency while taking advantage of semaphore-native queueing.

-Streaming Responses: Supports both standard and streaming completions.

Here’s an overview of the available API endpoints as of today:

1. `/completions` (POST): Generate completions for given messages.

2. `/completions_stream` (POST): Stream completions for given messages.

3. `/health` (GET): Check the health and get information about the loaded model.

4. `/hf_config_reader_api` (POST): Read values from persistent configuration memory.

5. `/hf_config_writer_api` (POST): Write values to the persistent configuration memory.

6. `/restart_server` (GET): Restart the LLM server.

Check out the GitHub repository linked below to obtain the server, source-code and detailed documentation on all the above aspects. As always, I welcome discussions and contributions!

GitHub - abgulati/hf-waitress: Serving LLMs in the HF-Transformers format via a PyFlask API

A closing note on llama.cpp

Llama.cpp and its GGUFs will likely remain the best option for hybrid CPU+GPU inferencing, wherein model layers are distributed across the memory of these processors, enabling the utilization of all available RAM in a single system. I am actively looking into quantization techniques that will enabling such hybrid inferencing in HF-Waitress too though, so stay tuned!

All said and done, llama.cpp is an amazing project and really represents open-source at it’s best: it’s a testament to what a community of willing contributers can create, and the freedom and enablement afforded to all by open-source projects.

llama.cpp and it’s creator and contributors have been very inspring to me on every level as a developer and I always wish it continued success. It serves as my inspiration to build and open-source the tools I do!

Running LLMs in their native format on consumer hardware just got a whole lot easier was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.

Dear readers, having watched Jensen’s GTC keynote, I just wanted to share my takeaways and thoughts. Please correct me or add to the below wherever applicable:

1. Moore’s Law is truly dead: In the 2+ years since the announcement of Hopper H100, TSMC has been unable to sufficiently mature their new 3nm node. While used by Apple for their small mobile SoCs, clearly 3nm is not scaling up to support high yields on large chips such as the new B100 announced today, as it used the same 4NP process as H100. This is unusual. H100 sports 80B transistors in a ~800mm2 chip and B100 seems slightly larger, though still around 800mm2 with 104B transistors. 4NP has clearly matured to support slightly larger chips and density, but the traditional scaling obtained from node-shrinks is completely missing.
2. Nvidia finally goes in on chiplets: For the longest time, Nvidia refused to put together multiple chips into a single package, instead preferring to fab monster 700–800mm2 chips with poor yields while AMD and Intel have long adopted multi-chip module (MCM) chiplet designs to scale performance-per-package. B100 is now 2x800mm2 chips totaling ~210B transistors, an exciting change! Which brings me to my next point…
3. Advanced packaging will shape the future of chip-advancements: As traditional scaling of transistors slows, chip-designers will increasingly rely on advanced packaging such as multi-chip module (MCM) chiplet designs and stacking to continue performance scaling. In addition to chiplets, Nvidia’s new B100 also utilizes TSMC’s advanced 2.5D CoWoS (Chip-on-Wafer-on-Substrate) packaging to build these monstrous, super-chips in a single package. AMD already uses 3D-stacked cache and Intel is developing their own tech, named “Foveros” to compete with TSMC’s 3D packaging solutions. Expect more in the future.
4. Scalability via software: Not just is B100 2x chiplets, Jensen states B100 is a “platform”, wherein each board features 2xB100 packages for a total of 4 chips. This will make comparisons with H100 skewed to say the least! Regardless, their new DGX NVL72 system is insane, featuring 30TB of VRAM and many chips that’ll all be seen as one GPU by CUDA! This demonstrates how crucial advanced software is to scalability: modifying CUDA to see a massive DGX system with many multi-package chips as a single GPU with a unified 30TB of VRAM is very impressive indeed! Expect this system to cost double-digit millions though.
5. Classic Nvidia misdirection with misleading charts: Dramatic charts show B100 to be magnitudes faster than H100, all while they’re comparing H100’s FP16 performance to B100’s newly supported FP4-format performance! Both are completely different things, and such charts are misleading!
6. No mention of FP16, FP32 and FP64 performance: I saw no mention of half, full and double precision floating-point performance, which are dominant FP formats. Such avoidance has traditionally been a bad sign as it implies the gen-over-gen performance gains are unimpressive. All comparisons were instead against B100 FP8 & FP4 performance, which is a vastly different metric.
7. New FP formats welcome: Research shows that training in FP8 format dramatically improves performance by reducing training time while showing virtually no loss of precision despite the scale-down from the typical FP16. Native support for FP8 and even FP4 is great, but these formats are not widely used to train models today so the benefits may be a way off, which makes me even more annoyed at those charts!*
8. Massive memory: Needless to say, this is very welcome.

TL;DR — Nvidia is going to make even more money and AI is going to be even more scary in the very near future.

*Here’s a paper about FP8: https://arxiv.org/abs/2310.18313

Reddit thread: https://www.reddit.com/r/LocalLLaMA/comments/17jjopb/fp8lm_training_fp8_large_language_models/

Abstract:

In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer states, in LLM training can employ low-precision data formats without compromising model accuracy and requiring no changes to hyper-parameters. Specifically, we propose a new FP8 automatic mixed-precision framework for training LLMs. This framework offers three levels of FP8 utilization to streamline mixed-precision and distributed parallel training for LLMs. It gradually incorporates 8-bit gradients, optimizer states, and distributed learning in an incremental manner. Experiment results show that, during the training of GPT-175B model on H100 GPU platform, our FP8 mixed-precision training framework not only achieved a remarkable 42% reduction in real memory usage but also ran 64% faster than the widely adopted BF16 framework (i.e., Megatron-LM), surpassing the speed of Nvidia Transformer Engine by 17%. This largely reduces the training costs for large foundation models. Furthermore, our FP8 mixed-precision training methodology is generic. It can be seamlessly applied to other tasks such as LLM instruction tuning and reinforcement learning with human feedback, offering savings in fine-tuning expenses. Our FP8 low-precision training framework is open-sourced at aka.ms/MS.AMP.

Nvidia GTC: Early Takeaways about the Future of Chips and AI was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.

We’ve witnessed an unprecedented market for GPUs develop over the past two years as manufacturing and shipping challenges brought forth by the on-going pandemic alongside skyrocketing demand have together blended into a most unsavory cocktail of exponentially increasing prices coupled with oftentimes severe availability constraints of this commodity. Fueling this fire is the apparent willingness of both gamers and miners to shell out more cash than ever before on GPUs even as scalpers circle overhead like vultures, waiting for the weary GPU hunter to finally fall to their knees as they collapse, completely exhausted and spent, out of the seemingly never-ending queues at retail and online stores, sometimes formed days before launch.

Really the market needs no introduction, yet for the sake of this write-up it got one. Indeed, the market has been terrible for many commodities since the pandemic took root, and our beloved GPUs are no exception.

What then are you, the regular GPU buyer to do in this situation in 2022? At a time when every GPU review is negative and community feedback ranges from “not worth buying even at MSRP” to “fuck scalpers”, what exactly are you to do? I hope this article helps you with that.

Stop living in 2019/2016/<insert_golden_year_for_hardware_availability>

Yes, there have been golden years where availability rained aplenty while prices bottomed out as we all enjoyed the spoils of a buyers’ market. But having already alluded to the supply-chain constraints, that’s simply no longer the case:

Manufacturing costs are up for seemingly every possible reason, from a dramatically increased demand on foundries for chips for every application ranging from automobiles to kitchen appliances to a shortage of raw-materials and skilled labor as people fall ill and lockdowns ensue. If that weren’t enough, black swan events such as climate and energy crisis induced factory shutdowns added liberally to the mess.

Meanwhile shipping costs remain sky-high as ports remain backlogged from lockdowns and illnesses amongst staff leading to a shortage of deck hands to move all that cargo (source below)

So, when something no longer costs as much to make and ship, it’s price and value are bound to change.

Now add to the above that not only are gamers willing to pay more than ever before for GPUs, but an entirely new market has formed, one that’s willing to pay even more and buy in bulk: the dreaded crypto-miners. Scalpers and automated bots that snag online stock are simply a natural & unsurprising byproduct of such a market…

Having said that, as an end-user and consumer you’re not obligated to think about and respond to all this when making a purchase, though it would behoove you to. Which is why having said all this it’s prudent to keep in mind that…

…it’s a personal decision

Whether you choose to accept the market challenges and pay the premium is purely a personal call for you to make — how much does this hobby mean to you? Do you value it enough to pay a price that would be considered a “bad deal” in a buyers’ marker? At the end of the day, if it’s a good deal to you, if it’s worth it to you, then that’s all that matters. Don’t let any review or internet echo-chamber tell you otherwise. Having said this, should you…

…pay an insane MSRP, or buy a lower-tier card from a scalper?

While this again ties in with the above point, I’d advise the former. While buying an RTX 3090 or RX 6900XT would have been considered bad value in the buyer’s markets of yesteryears, if you hypothetically have a chance to snag them at MSRP from an official source today (very rare, thus the hypothetical nature of this scenario) then I’d say that’s worlds better than paying a scalper the equivalent of nearly $900 for a RTX 3070 or RX 6700XT. Not only does buying at MSRP not feel like a complete rip-off, it’s also far better in relative terms: you’re getting one of the fastest GPUs available at MSRP, as opposed to paying 2x MSRP for a card that’s 40% slower and with half the VRAM of the 3090. Regardless, you could make the argument that a $1000 RTX 3070 is $500 cheaper than a RTX 3090 at MSRP, and no one can deny that. So again, it’s up to you to make a personal choice: do you focus on the better relative value of the more-expense-but-at-MSRP card or look one-dimensionally at the cost? Purely your call, just make it well. And just to clarify — by scalper I mean an overcharging but well-established retailer. NEVER go to those shady eBay crap lords. But whatever you do…

…don’t rely on market conditions improving

We all have this optimist within us that remains hopeful for an improved market in the mid to near future, whether that be by an easing of manufacturing and shipping constraints or mining demand evaporating due to a crash in the crypto markets or even because Ethereum switches off mining.

Don’t fall for this: we honestly have no idea when manufacturing & shipping costs will go down. It could be a few months, or it could be a few years and while no one can say for sure, industry analysts tend to believe that it’ll be some time before the situation truly eases (source below)

And as for the latter…

…cryptocurrency mining is here to stay

There’s indeed a direct tie-in between GPU prices and the crypto markets: when Ethereum crashes, so does the price of GPUs. However, be sure to react fast if you see this and don’t keep waiting for a bigger crash — while crypto markets have historically slumbered into long bearish periods, as crypto coins have become more popular among the general populace they have tended to recover faster from crashes.

And it honestly doesn’t matter if Ethereum switches away from mining. I’ve eagerly been awaiting this too, but recently found myself pondering the question, ‘Then what?’ and the unpleasant but honest answer is that another profitable-to-mine coin will undoubtably take its place. The fact is that a large market has been formed around crypto-mining, and today that market is largely centered around Ethereum. But that needn’t always hold true, and Ethereum switching away from mining will simply create a vacuum, and nature (and the markets) abhors a vacuum, so it’ll only be a matter of time before another crypto-coin comes forth to fill this space. It’s naïve to think otherwise.

A Brave New World

We live in unprecedented times: a global pandemic rages on, mutating its way into persistent daily relevance while manufacturing and shipping delays plague commodities exchanges and markets worldwide. Climate-change induced freak weather events instill further chaos, with both preventive & responsive measures inducing further delays. And to top it all off, the looming threat of an energy crisis forces Asian factories into reducing operating hours while across the world European nations scramble to ensure they have enough power for the bleak winter months. And despite it all, wealth skyrockets within the elite class leading to millionaires worldwide calling for higher taxes on their own selves.

Who’d have seen this craziness coming to shore? Very few, if any at all.

Don’t get caught up in comparisons from an era gone past, unpredictable market conditions are here to stay a while. Do consult the charts and graphs within hardware reviews, but do not allow their conclusions alone to color your final opinions as many journalistic outlets are themselves yet to adapt mentally to this new reality. Be objective in your analysis and purchase what fits the bill for you: Afterall, you’re the one paying the price.

Sources:

The Economist — Just how gummed up are supply chains? — https://outline.com/YjdsGm | https://www.economist.com/finance-and-economics/2022/01/22/just-how-gummed-up-are-supply-chains

MoneyControl — Why the shipping container crisis will not end soon — https://www.moneycontrol.com/news/business/explained-why-the-shipping-container-crisis-will-not-end-soon-7743451.html

Reuters — ASML Reports Fire at its Berlin Factory — https://www.reuters.com/technology/asml-reports-fire-its-berlin-factory-2022-01-03/

(Old but relevant) Tom’sHardware — Samsung Texas fab shutdown to impact 1%-2% of 300mm Capacity — https://www.tomshardware.com/news/samsung-foundry-austin-blackout-impact

EETimes — COVID lockdown in China Threatens DRAM Supply Hiccup — https://www.eetimes.com/covid-lockdown-in-china-threatens-dram-supply-hiccup/

Mint — China’s energy crisis is latest supply chain threat to world — https://www.livemint.com/news/world/chinas-electricity-crisis-is-latest-threat-to-global-supply-chain-11632790649848.html

Power-technology.com — The global trickle-down of China’s energy supply shortage — https://www.power-technology.com/features/china-energy-crisis/

The Wall Street Journal — China COVID19 Lockdowns Hit Factories, Ports in Latest Knock to Supply Chains — https://www.wsj.com/articles/china-covid-19-lockdowns-hit-factories-ports-in-latest-knock-to-supply-chains-11641918247

WSJ — Europe Energy Crisis Goes Global — https://www.wsj.com/articles/europe-energy-crisis-goes-global-shortages-natural-gas-wind-solar-coal-russia-china-germany-11641335775

Reuters — Millionaires Group Calls for Wealth Tax at Virtual Davos — https://www.reuters.com/business/millionaires-group-calls-wealth-tax-virtual-davos-2022-01-19/

The Guardian — Millionaires call on Governments worldwide to ‘tax us now’ — https://www.theguardian.com/business/2022/jan/19/millionaires-call-on-governments-worldwide-to-tax-us-now

The GPU Dilemma: Gaming, Crypto-mining and Market Challenges was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.

This post has been split into a two-part series, please find Part 1, An Overview of AMD’s GPU Architectures, here:

An Overview of AMD’s GPU Architectures

1. TeraScale

Let’s start within TeraScale’s core and build our way out:

A processor is a complex ensemble of fetch, decode & execute logic coupled with storage registers and cache all working in tandem to carry out whatever number-crunching is required, and these execution blocks are themselves built-up of simpler foundational blocks.

With TeraScale, as with all of AMD’s GPUs today, this most fundamental execution block is the Stream Processor or SP. An SP is what AMD chooses to call an Arithmetic and Logic Unit (ALU) in the context of their GPUs; ALUs, as their name suggests, specialize in executing mathematical operations.

In TeraScale, several SPs and a branch control unit along with storage registers all come together to make up a single Stream Processing Unit, or SPU.

Further up, several SPUs along with more control units and storage registers together make up a single SIMD core. Several SIMD cores and even more control hardware ultimately come together to make a complete TeraScale GPU.

Of course, a pictorial representation will do a far better job of getting all this across:

Several Stream Processors along with registers & control logic comprise a single Stream Processing Unit (SPU), while several SPUs along with more control hardware & registers comprise a single SIMD core…

…And several SIMD cores together with additional logic & cache build a complete TeraScale chip!

Above we see 10 SIMD cores coming together to make a RV770, or Radeon HD 4870 GPU

It goes without saying that the complete GPU is more complex than what’s seen here, but this gives you a fair idea as well as a glimpse into the heart of this now defunct beast.

Let’s now see how it all comes together to process those vectors we spoke of earlier:

A VLIW Heart

TeraScale happens to be what’s called a VLIW chip.

VLIW stands for Very Long Instruction Word and is another type of Instruction Set Architecture (ISA). Recall from earlier that an ISA comprises the set of instructions that a chip can understand and therefore act on, and that ISAs can be of different types: x86–64 is a CISC type ISA while ARM is a RISC type ISA. Similarly, AMD’s TeraScale GPUs were a VLIW-type ISA.

Simply put, VLIW is another attempt at speeding up chips. While the obvious approaches involve simply building faster cores that churn through more instructions per clock cycle, another is to simply do more stuff at once. This latter approach necessitates multiple processing cores in a single system which explains the many-core CPUs of today with even cellphones now boasting of 8-core CPUs.

Having multiple cores is one thing, utilizing them effectively is quite another. When you run a program, it creates a system process (visible in the task manager) which in-turn spawns one or more “threads”. A thread is a self-sufficient bunch of instructions awaiting the CPU’s attention for execution (self-sufficient as they contain all the data and state information necessary for their execution). A thread is thus the smallest sequence of instructions that can be scheduled for execution by a scheduler.

A simple approach to utilizing more cores then would be to have them execute several independent threads in parallel. Indeed, this approach is used by CPUs and is called Thread Level Parallelism (TLP).

Instruction Level Parallelism (ILP) is an alternate take on parallel processing: with ILP several threads from a process are packed into a single, large thread giving us what’s called a Very Long Instruction Word (VLIW). This VLIW thread is then sent off to the processor, where it’s unpacked at execution time and the resulting threads executed by available processing cores.

Both ILP and TLP approaches share a common critical requirement though: operations executing in parallel must be independent of each other, be they disparate threads or the instructions from within a VLIW thread. This makes sense: if an operation relies on the output of another it’ll simply have to await those results before it can itself execute.

Consider a very simple example:

A + B = C

X + Y = Z

C * Z = R

While the first two are entirely independent of the other instructions, the third relies on the preceding two and will thus have to await their execution.

This might appear as a subtle difference but raises a very important question, that of whose job is it to identify such independent data for parallel execution? With TLP, that burden is shared by the application programmer and the hardware: while the programmer is responsible for writing thread-aware code takes advantage of multiple processing cores, the processor itself ultimately schedules threads for parallel execution at run-time, maximizing utilization. TLP thus follows a dynamic, run-time approach to scheduling wherein the processor itself acts as the scheduler.

With an ILP approach things are not as simple: the calling process must itself identify independent threads to be packaged into a single VLIW thread. This leaves the entire burden of scheduling on the software and more specifically, on the compiler. In software parlance, a compiler is a special program that converts code written in a near-English (and thus high-level) language such as Java or C/C++ into low-level machine code based on the processor’s ISA, thus acting as an intermediary translator.

While the compiler gets the advantage of a full view of the program and could therefore be expected to schedule intelligently, there are conditions the compiler remains blind to as some outcomes remain unknown until actual execution time. Exacerbating this problem is the fact that scheduling set by the compiler cannot be altered at run-time by the processor leaving us with a static, compile-time approach to scheduling in stark contrast to TLP.

So ILP is a static scheduling approach that complicates the design of the compiler and leaves compute resources inevitably idle at times, why ever use it then? Because graphics is a highly parallelizable application domain that can utilize an ILP approach arguably well. Further, when combined with TLP as done on TeraScale (surprise!) ILP can lead to some very impressive performance figures.

So how did AMD utilize ILP and further combine it with TLP on their VLIW-based TeraScale architecture? Let’s look down the compute lane:

TeraScale at Work: TLP + VLIW ILP on a SIMD Core

Recall that a GPU fetches several datapoints or pixels at once in a grouping called a “vector” along with a corresponding instruction in accordance with its SIMD nature. AMD likes to refer to these vectors as “wavefronts” and with TeraScale, 64 VLIW threads of pixel values or datapoints are grouped into a wavefront and dispatched to a SIMD core for processing. With 16 SPUs per SIMD core, the full 64-wide wavefront is executed in four cycles.

With the 16 SPUs of a SIMD core each processing a VLIW thread every clock cycle, we see thread level parallelism or TLP in action as 16 VLIW threads are processed at any given time.

Instruction level parallelism (ILP) comes in next as each VLIW thread is dissected for its constituent datapoints which are then executed individually by the stream processors within the SPU.

And with 16 VLIW thread executing against the same instruction at any given time, a SIMD (Single Instruction Multiple Data) architecture is in play throughout.

Utilization remains a big concern though, for both the SPUs and the SPs within them: not only must the compiler do its best to identify 5 independent datapoints for each VLIW thread, but so must 64 VLIW threads be packed together within each wavefront. Further, the 64 items in a wavefront should all execute against the same instruction; imagine a scenario wherein one thread executes against an entirely different instruction from the other 63! Opportunities for additional clock cycles & poor utilization thus abound and the compiler must do it’s best to schedule around them.

With 5 SPs in each SPU, attaining 100% utilization necessitates five datapoints per VLIW thread. That’s the best case; in the worst case an entire thread is comprised of just a single datapoint resulting in an abysmal 20% utilization as 4 SPs simply engage in idle chit-chat. Extremities aside, AMD noted an average utilization of 68% or 3.4 SPs per clock cycle. A diagram from AnandTech’s GCN preview article depicts this scenario, and it’s a good time to borrow it here:

Some cycles see 100% utilization of the SPs as others see just 20% utilization with only one SP engaged. On average, AMD notes 68% utilization per cycle, or 3.4 SPs.

TeraScale over Three Generations: Optimizing ILP until the End

TeraScale evolved three generations over its lifetime starting with Gen1 on the Radeon HD 2xxx series and finally culminating with the Gen3 based Radeon HD 69xx series. Three primary enhancements sum up the changes over this period: more SIMD cores, smaller process nodes and a more optimized SPU.

The Radeon HD 2900 XT served as TeraScale’s debut flagship: manufactured on TSMC’s 80nm process with just 4 SIMD cores, it puts into perspective how far we’ve come today!

The HD3000 series followed with similar specs albeit on TSMC’s newer 55nm process and like its predecessor, proved underwhelming against Nvidia’s offerings at the time. Things really did turn in AMD’s favor with the HD4000 series as the flagship HD 4870 dramatically upped SIMD core count straight up to ten in addition to adopting newer GDDR5 VRAM netting 1.5x gains in memory bandwidth.

While the HD 4000 series were good, the HD 5000 series would usher in TeraScale’s heyday: debuting the TeraScale2 architecture on a brand new 40nm process, the Radeon HD 5000 family of GPUs remain arguably AMD’s best to date and are in-fact so well regarded that AMD’s newest RDNA-based RX 5000 of cards are named in honor of this GPU family! With the HD 5870, AMD once again doubled the number of SIMD cores now on the flagship Radeon HD 5870 GPU along with the L2 cache & VRAM.

TeraScale 3 would feature only on the Radeon HD 6900 series with a significant change: reducing the number of stream processors per SPU from five to four. This was AMD responding to their observation of SP utilization averaging around 3.4 SPs per SPU every clock cycle. This reduction would aid utilization & efficiency as well as leave silicon for additional SIMD cores. Indeed, the flagship Radeon HD 6970 GPU modestly increased SIMD core count to 24.

The HD 6900 series would serve as the last of the flagship TeraScale GPUs, even as TeraScale based cards continued to release until October of 2013. As compute applications began to take center-stage for GPU acceleration, games too evolved. The next generation of graphics API’s such as DirectX 10 brought along complex shaders that made the VLIW-centric design of TeraScale ever more inefficient and impractically difficult to schedule for. The Radeon HD 7000 series would accordingly usher in the GCN architecture, TeraScale’s inevitable successor that would abandon VLIW and ILP entirely and in doing so cement AMD’s focus on GPU compute going forward.

2. GCN — Graphics Core Next

With a mission to end Nvidia’s dominance in the GPU compute space, GCN set out with big goals. To do so would require AMD to free their GPUs of VLIW’s shackles & its accompanying instruction-centric approach to parallelism, building a new GPU architecture from the ground-up. New architectures are never easy, and with this transition necessitating major changes & complete overhauls, it would be no mean feat.

Why though? Why go through all this fuss? Simply put, because AMD really had no choice in the matter:

You see, the enterprise & high-performance compute (HPC) space are some of the highest margin customers hardware companies appeal to and the compute potential of GPUs happens to be of great value to these folk, a target audience who’s potential Nvidia had recognized first and addressed with Fermi, their first-ever compute-centric architecture. Nvidia complimented Fermi’s development with heavy investments in the surrounding software ecosystem resulting in the creation of the CUDA ecosystem, which continues to dominate the GPU compute space even today.

If you’re wondering what’s the point here, it’s all in the numbers: Fermi released in October of 2010 and by the third quarter of 2011, Nvidia was already enjoying the spoils: Q3’11 saw Nvidia declare a net income of $146M from a total $644M in gaming revenue while during the same period, profits from the professional market amounted to $95M from a total revenue of just $230M: that’s equivalent to 65% of the gaming profit from just 35% of the sales revenue; talk about healthy, healthy margins!

And so with both finances and the evolving software & gaming ecosystem demanding it and nearly two years late, GCN was to be AMD’s Fermi moment boldly announcing their arrival on the GPU compute playground. What changed & how did AMD’s take on a thread parallel GPU shape up? Let’s dive right in:

Recall that with TeraScale, the stream processor (SP) forms the foundational compute execution block. “Stream Processor” is another term for an ALU and TeraScale houses five SPs (four with TeraScale 3) in a single Stream Processing Unit (SPU) with 16 SPUs coming together to make a single SIMD core, several of which build a complete chip.

In the case of GCN this layering is shifted up a stage: individual stream processors still form the foundational blocks, but now 16 of them come together directly to build a single SIMD core. Further, four SIMD cores together build a single Compute Unit, or CU with several CUs finally coming together to build a single GCN chip.

Diagrams once again:

A SIMD core in GCN comprises of sixteen Stream Processors (SPs) rather than sixteen Stream Processing Units (SPUs) which themselves each comprise of five or four SPs

And four SIMD cores now come together in a single Compute Unit (CU), which a GCN GPU contains several of. Also illustrated above are the Scalar ALU, the branch & fetch/decode logic and the registers/cache that form part of the CU.

Let’s look at work distribution in GCN:

With VLIW and ILP out the window, GCN is a pure SIMD architecture: wavefronts are no longer comprised of VLIW threads but rather of 64 individual datapoints which are executed by the 16 SPs within SIMD cores. Wavefronts remain 64-wide, necessitating the same four cycles to churn through. Further, each compute unit contains four SIMD cores & each of these may work on separate wavefronts so at any time, a CU may be processing up-to four different wavefronts.

All this brings along a massive benefit: that of the software no longer having to identify and schedule independent data into VLIW threads, greatly simplifying the design of compilers. Independent threads are instead dynamically scheduled by the hardware at runtime resulting in a much simpler approach to scheduling. All this lends itself very favorably to compute applications as well as to modern games.

New to GCN & specifically for the scheduling of compute workloads across CUs are the Asynchronous Compute Engines, or ACE which preside over resource allocation, context switching & task priorities. As GCN is built to concurrently work on multiple tasks, ACEs independently schedule wavefronts across CUs. A GCN GPU may carry multiple ACEs.

The Graphics Command Processor (GCP) serves as the graphics counterpart to the ACE & makes a return from TeraScale. The GCP works to schedule activities across the graphics subsystem, primarily involving scheduling across the “primitive” pipelines: complex surfaces & objects in games are built up of simpler geometrical shapes, called primitives, bunched together in large numbers. Triangles are the dominant choice here as their position in 3D-space can be entirely determined by just three points. This process of building up complex objects, shapes & surfaces from simpler data is referred to as Tessellation. The primitive pipelines are responsible for this tessellation in addition to other geometry & high-order surface processing & the GCP is responsible for scheduling work over these pipelines.

Do notice the Scalar ALU within the CU: this is a special ALU dedicated to any “one-off” mathematical and transcendental (i.e. logarithmic, sin/cosine etc.) operations. The very meaning of a SIMD core implies vector processing and that involves applying an instruction to a group of values (a vector) rather than to a single value (a scalar). A scalar or a one-off operation disrupts this flow, and a separate Scalar ALU alleviates this by keeping these operations out of the SIMD cores.

So where was this with TeraScale? Unfortunately, within the SPUs: in the case of the TeraScale Gen1 & 2 the 5th SP in each SPU served as the Special Function Unit, or SFU. Gen3 bunched 3 of the 4 SPs within a SPU together for this. This resulted in a severe latency for scalar operations as they had to be scheduled within a wavefront: the HD 6900 series had a nasty 44-cycle latency for scalar operations. With the separation of the Scalar ALU from the SIMD core entirely, GCN brings this down to one cycle.

GCN Through the Years & Today

The first GPUs featuring GCN debuted on the 9th January 2012 with the Radeon HD 7950 & HD 7970 GPUs. On the 7th of January 2019, AMD announced the Radeon VII: the last GCN GPU. That’s a good seven years, practically eons in the compute world. GCN wasn’t a stagnant architecture over this duration, instead evolving with a mix of typical incremental upgrades in addition to other, more significant enhancements. Let’s peek at GCN’s journey:

First & Second Generation GCN:

While Gen1 GCN entirely abandoned VLIW in favor of a pure SIMD architecture, Gen2 brought along incremental upgrades with more functional units, higher memory bandwidth & better power management. New compute centric instructions in GCN’s ISA accompanied these, along with support for a greater number of ACEs with the R9 290X flagship sporting 8 where Gen1 had a hard limit of 2.

Bridgeless Crossfire was introduced here as well: Crossfire enables the use of multiple Radeon GPUs in a system & previously necessitated a hardware bridge to connect these GPUs. Bandwidth limitations over the bridge would require the CPU to mediate exchange over the PCIe bus, invoking a frametime penalty. Dedicated hardware in the form of the XDMA Crossfire engine would now control this & the much higher bandwidth of the PCIe bus meant a dedicated bridge was no longer necessary.

In terms of raw numbers, the Gen2 flagship R9 290X came baring 44 CUs, up from 32 CUs on the Gen1 HD 7970 & R9 280X, along with a wider memory bus (512-bit vs 384-bit) & an additional gig of VRAM.

Third Generation GCN:

Debuting in the September of 2014, Gen3 GCN bought along two major features to the compute side: GPU pre-emption & support for FP16 arithmetic. Pre-emption is the act of interrupting the execution of a task for another higher-priority task, without its consent with the intention of resuming it later. This is a big deal as GPUs have always been poor at context switching.

WRT FP16 ops, GPUs deal almost explicitly with floating point numbers (non-integer decimals/fractions) which typically occupy 32-bits in computer memory and are thus referred to as FP32 numbers, or single-precision numbers. Not every application requires as much precision with many compute applications adequately addressed by half-precision numbers which occupy half the space in memory with just 16-bits. These are referred to as FP16 numbers & lead to significant memory savings.

On the graphics side AMD introduced lossless delta color compression for the transfer of large textures. The use of compression for textures is not new, though with this AMD claimed a massive 40% gain in memory bandwidth efficiency.

Lastly, FreeSync & HBM were introduced here as well. FreeSync is AMD’s implementation of adaptive refresh-rate technology, allowing the monitor to change its refresh rate on-the-fly to match the frame output rate of the GPU, eliminating stutter when the framerate falls below the refresh rate and screen tearing conversely. HBM or High Bandwidth Memory is a memory standard that places the VRAM and GPU core on the same large slice of silicon, dubbed an ‘interposer’, as opposed to soldering memory chips separately onto the PCB. HBM allows for much higher bandwidth, lower latency as well as reduced power consumption. The trade off? Much higher cost.

The flagship Fury X increased CU count to 64 while HBM enabled a colossal 8-fold increase in memory bus width to 4096-bits from 512-bits on the R9 290X, resulting in a 60% increase in memory bandwidth to 512GB/s from 320GB/s. The Fury cards were the first GPUs to use HBM, which makes a comeback on GCN Gen5 which uses HBM2 exclusively.

Fourth Generation GCN: Polaris

Gen4 GCN debuted on the Polaris RX 400 series of GPUs in the June of 2016. With the flagship RX 480 squarely a mid-tier card with just 36 CUs, Polaris represented a big shift in AMD’s GPU strategy as they set out to the mainstream market first, differing the launch of high-end GPUs to a hitherto-undisclosed date. This speaks volumes of the extent of GCN’s success, or lack thereof, in the high-end space but we’ll differ that discussion for a while, instead maintaining focus on new features Gen4 GCN brought along.

Polaris aimed to make big improvements in the domain of power consumption, an area where GCN had fared poorly so far with several flagships running hot & loud while happily chugging on the power lines.

In addition, Polaris brings along support for instruction pre-fetching, a predictive process wherein processors guess the instructions they’ll be executing next based on the current execution state & then fetch those instructions. Correct pre-fetching leads to significant performance gains as the processor need not wait for data to be read in from memory, which is magnitudes slower. Incorrectly pre-fetched instructions are discarded, degrading efficiency though pre-fetching techniques are constantly refined to minimize this. Either ways, GCN Gen4 GPUs could now pre-fetch instructions, something prior GCN GPUs simply could not do. A larger instruction buffer obviously accompanies this.

On the graphics side, AMD added the Primitive Discard Accelerator. As surfaces & complex shapes are ‘tessellated’, i.e. made up of many smaller & simpler polygons (typically triangles), the primitive discard accelerator culls visually insignificant triangles (hidden or too small) for increased performance.

The RX 500 series launched a year later, bringing improvements to clockspeeds & reductions in power consumption. While the RX400 series were AMD’s first cards on the 14nm process, the RX 590 launched on a more refined 12nm process mildly improving clockspeeds & power efficiency.

GCN Gen4 remains the only GPU family from AMD lacking a high-end flagship member.

Fifth Generation GCN: Vega

While an entire article can be written on Vega’s release shenanigans, we’ll refrain from going down that path here. Baring a little context, we’ll retain focus on the notable new features Vega brought along.

Vega launched on Monday the 14th of August 2017 as the Vega 56 and Vega 64 GPUs, with those numbers denoting the number of compute units in those respective GPUs. The high CU count marked AMD’s return to the high-end space after a two-year absence. AMD’s recently launched Ryzen CPUs had delivered on value & performance far beyond expectations, cementing hopes for Vega to do the same in the GPU space. This would prove to be flawed chain-reasoning causing expectations & excitement to spiral out of control prior to launch: at one-point enthusiasts were offering to privately fund the overseas travel of knowledgeable YouTubers such as Buildzoid to have them analyze AMD’s Vega events! Though I really blame AMD’s mix of dramatic & drip-feed marketing for this, I’ll refrain from talking further about it here.

The GPU landscape wasn’t favorable for AMD either: they barely held 25% of desktop GPU market share at this point so it would be very hard to get game developers to support any new Vega-centric gaming features as they’d benefit very few. Further exacerbating this situation was Nvidia’s dominance of the entire GPU landscape with their Pascal architecture, one of their best ever architectures now legendry for refinement, efficiency & raw performance.

Regardless, AMD’s engineers did consider Vega to be their largest architectural overhaul in five years even actively distancing themselves from the GCN tag & referring to this as the ‘Vega’ architecture instead. This is still very much GCN though, so naming conventions aside let’s peek into the changes:

This biggest change comes to GCN’s FP16 compute capabilities: while Gen3 introduced FP16, the operations themselves didn’t execute any faster as each individual stream processors could still handle only one operation at a time, be it FP16 or FP32. Vega changes that significantly: each SP can now handle two FP16 operations in place of a single FP32 op, a feature AMD dubbed ‘Rapid Packed Math’.

The next big change comes to the memory subsystem with the adoption of HBM2 and the introduction of the High Bandwidth Cache Controller. More relevant as a compute feature, HBCC extends the GPU VRAM’s reach to the system RAM & secondary storage device for datasets too large for the VRAM alone.

Next are improvements to the graphics engines with the introduction of primitive shaders which allow for high-speed discarding of primitives, i.e. visually insignificant polygons (hidden or very small triangles) along with the Draw Stream Binning Rasterizer to further aid in this regard. This continued emphasis on culling helps prevent the clogging of render pipelines & frees the GPU of useless work, reducing memory access and saving memory bandwidth while reducing power consumption.

Lastly, we have better load balancing across the geometry/shader engines resulting in a 2x speedup in geometry throughput. While geometry performance hadn’t been poor on GCN, this helps squeeze out any potential bottlenecks.

Built on the 14nm process, the Vega 56 & 64 GPUs packed 8GB of HBM2 each. While HBM2 was to double the bandwidth over HBM1, it missed that target by a bit affording Vega 484GB/s in bandwidth over its 2048-bit wide bus, where the Fury cards enjoyed 512GB/s over their 4096-bit wide bus.

The Radeon VII would rectify that: released on the 7th of February 2019, the “world’s first 7nm gaming GPU” and the last GCN GPU carries 60CUs and a whooping 16GB of HBM2 memory over a full 4096-bit wide bus unleashing a mindboggling 1TB/s in memory bandwidth! Slightly anti-climatically though, it cost and performed nearly the same as the mighty GTX 1080 Ti, the then two-year older Pascal flagship which also carried a 45W lower TDP.

Overall, the Vega cards lacked appeal against Nvidia’s highly refined & efficient Pascal-based GTX 10x series which not only offered similar performance at similar price points but had also been on the shelfs for a while already. The use of expensive HBM2 memory also hindered AMD’s ability to price the cards more aggressively, and these factors together sadly crippled Vega’s success as a gaming architecture.

GCN: Conclusion

While AMD did lose significant GPU market share with their GCN architecture at the helm, GCN did have its moments: the HD 7970 and R9 290X are remembered as formidable flagships that held the performance crown in their day, GCN did see AMD make inroads into the compute space, Vega gave us some great Ryzen APUs & cards like the RX470 and Vega 56 have done well for AMD. GCN is also in all the consoles, from the PlayStation 4 to the Xbox and even on the Nintendo Wii U.

Notably, GCN did well in the low-power envelopes of mid-range and embedded graphics products such as the consoles and APUs while it did suffer at the high-end: the R9 290X ran hot and loud, the R9 Fury cards still cause their past owners to break into random bouts of sweating and worse of all, the Vega cards consumed the power equivalent of Nvidia GPUs from a tier above. Indeed, it appeared that the flagship cards worsened in respects to power draw and efficiency as GCN matured and clearly, something needed to change.

RDNA seems set to rectify all that GCN did wrong, especially in the domain of power consumption and efficiency. While it’s barely out the door and therefore too early to judge, it’s enjoyed some large wins in the enterprise space spanning the cloud, supercomputer and low-power envelope spaces. Indeed, RDNA seems set to answer every enthusiast’s call for a new GPU architecture from AMD, but does it ultimately fare well and what does it change? Let’s dig in.

3. RDNA: Radeon Re-defined

RDNA arrives to make significant changes to the core of GCN, or as AMD prefers to put it more organically: to the very DNA of the Radeon architecture. While not as major a redesign as the move away from VLIW, RDNA does bring along a complete reworking of AMD’s approach to thread level parallelism on its GPUs. Accordingly, we get three major changes manifesting as updates to the compute units (CUs), the addition of a new caching layer and accompanying improvements to power efficiency.

Beginning at the heart of the matter, the fundamental SIMD cores see a major change once again, now beefing up to twice the size of GCN’s SIMD cores, doubling the number of steam processors (SPs) & the register space:

TeraScale ushered in the 16-wide SIMD design, which GCN followed despite the move away from VLIW by simply replacing TeraScale’s 16 SPUs (each of which housed 4 or 5 SPs) with the fundamental stream processors themselves. RDNA continues shaking things up for the SIMD core.

This in turn alters the compute unit itself:

As we see, significant changes to the compute unit abound with RDNA. Most notably, the four SIMD cores of GCN are now merged into two, each twice as wide doubling both the number of SPs as well as the register space. Each SIMD core now also gets its own scheduler, as opposed to GCN’s approach of deploying a single scheduler across the entire CU.

Further on we see that while GCN equips it’s CUs with a single scalar unit, RDNA’s carries two Scalar ALUs in a CU, one per SIMD core. Recall that the SALU is dedicated to one-off operations which would otherwise bog down the SIMD cores, wasting their parallel compute capabilities.

Further changes alter resource sharing across CUs: RDNA pairs two CUs into a single Work Group Processor, or WGP with the scalar data & instruction cache shared across the WGP along with the Local Data Share (LDS). GCN on the other hand shared its scalar cache across four adjacent CUs while maintaining a dedicated LDS per CU.

While such merging of SIMD cores and changes to sharing across CUs may appear as trivial changes, they’re anything but and have far reaching implications on how work is distributed and executed by the GPU: recall that work is distributed to the CUs in a fundamental grouping called a wavefront, with 64 pixels or datapoints in each wavefront. Ideally, these datapoints hold no inter-dependencies and await execution against the same instruction making them ideal for parallel execution via the SIMD cores which had been 16-wide with both GCN & TeraScale, thus processing a wavefront every 4 cycles.

RDNA changes this four-cycle execution model entirely: with the SIMD cores doubled up to 32 SPs, the wavefronts too are slashed in half to 32 elements per wavefront. The implication is obvious: a wavefront is executed every cycle resulting in a 4x speedup in wavefront execution times!

The reduction in wavefront size helps tremendously: identifying 64 independent datapoints for each wavefront can prove challenging even for highly parallelizable applications like graphics and GPU compute tasks. This challenge often necessitates additional cycles where some datapoints execute against different instructions from the others, also resulting in poor utilization. With just half the datapoints required per wavefront, this situation is greatly eased.

So RDNA’s Compute Units can execute wavefronts four times faster than their predecessor, this necessitates keeping the CUs well fed with data: a new caching layer is thus introduced. Notice the CU block diagram above wherein GCNs’ local L1 cache is changed in nomenclature to the ‘L0’ cache. But that’s all that it is: a change in nomenclature. The real change comes at the next step: RDNA now adds a new L1 layer that’s shared across 10 Compute Units (5 GCPs) which in-turn interacts with the L2 cache by the memory controllers. On the other hand, the L1 cache in the CUs of GCN interacts directly with this L2 cache, with no intermediate caching layer.

Caching layers have a significant impact on performance, lowering execution times & upping efficiency further: when data isn’t found in the caching layers there’s no choice but to look for it in the VRAM, which isn’t just many magnitudes slower (on the scale of thousands of times slower as compared to the caches close to the cores), it’s also significantly more energy intensive.

All put together, that’s a 4x speedup in wavefront execution times coupled with smaller wavefronts and additional caching, improving both performance & efficiency. The results reflect this: RDNA’s mid-range 5700XT GPU performs in the same ballpark as the Radeon VII GPU, which represents the very best of GCN. Built on the same 7nm process, the R7 carries 3840 SPs and a 295W TDP. For RDNA to match that with 33% fewer SPs and a 25% lower TDP is very good progress indeed: the 5700XT carries 2560 SPs and a 225W TDP with the most impressive reduction being the $400 USD price tag against the Radeon VII’s $700 USD MSRP!

One thing is clear: AMD has made significant strides with RDNA, and that’s great news for the GPU market. Hopefully, RDNA signals AMD’s long overdue return to competitiveness in the GPU space.

RDNA: Additional New Features

Two significant enhancements accompany the design reworks described above, chief among them being the shift to GDDR6 memory. First introduced on Nvidia’s RTX 2000 GPUs in 2018, GDDR6 brings along a 1.75x speedup over GDDR5, affording the 5700XT with 448GB/s in memory bandwidth over the same 256-bit wide bus as the RX 580, which merely enjoys 256GB/s.

Next up is the shift to the PCIe Gen4 standard with 32GB/s in bandwidth over 16 PCIe lanes, doubling bandwidth over Gen3’s 16GB/s. Though GPUs today struggle to saturate Gen3’s bandwidth, the move to Gen4 aids with the exploding popularity of NVMe SSDs, which utilize PCIe’s much higher bandwidth to attain read/write speeds unattainable by typical drives over SATA. Since a GPU can now be adequately fed over 8 Gen4 lanes, more lanes remain available for these SSDs.

RDNA: Looking Forward

For now, the best we have with RDNA is the 5700XT and with 40 CUs, 4 Asynchronous Compute Engines and a 256-bit wide memory bus, it’s clearly a mid-range part. “Big Navi” remains due at an undisclosed date, with rumors of AMD referring to this part as the “Nvidia killer” internally. It would be nice for this rumor to be true: Team Green’s domination over the past few years has been absolute, resulting in their complete stagnation on price/performance improvements, so a shakedown is definitely overdue.

One thing is for certain: efficiency will play a major role in dictating RDNA’s success much more than raw performance alone. While the 5700XT seems to be doing okay, if Big Navi pops out sporting maxed out voltages and bursting along the clock speed limits like Vega did, then all hope is truly lost for a formidable AMD challenger.

Things with RDNA do look very promising for now though many definitive products remain due: beyond just high-end enthusiast class graphics cards, it’s the APUs, console SoCs and compute GPUs that RDNA spawns that’ll ultimately dictate its success. For now, I dare say it would be safe to remain cautiously optimistic!

4. Conclusion & Looking Forward

Over the course of this article, we’ve looked at AMD’s GPUs and observed the constant & significant transformation they’ve undergone over the past 13 years, from aggressively scaling up and optimizing VLIW to abandoning it entirely for a more compute-centric architectural layout. While TeraScale made significant impact on the gaming market, GCN has made in-roads into the GPU computing and heterogenous system landscape over the past decade. Today, RDNA arrives to take over with significant changes aggressively rectifying GCN’s shortcomings in a focused effort to dominate the GPU landscape. Indeed, RDNA’s many early wins firmly lay out this path.

With just three new cards out for now, it’s clear that RDNA’s story has just begun. While AMD seems to have big plans for RDNA and are already enjoying several large wins across the spectrum, a few more generations will be necessary to effectively gauge RDNA’s impact. This sure seems like the architecture that restores AMD’s technology lead while cementing their mindshare, though. Indeed, only time (and a few more releases) will tell the complete tale.

Please find Part 1, An Overview of AMD’s GPU Architectures, here:

An Overview of AMD’s GPU Architectures

An Architectural Deep-Dive into AMD’s TeraScale, GCN & RDNA GPU Architectures was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.

This post has been split into a two-part series. Please find Part 2, An Architectural Deep-Dive into TeraScale, GCN & RDNA, here:

An Architectural Deep-Dive into AMD’s TeraScale, GCN & RDNA GPU Architectures

1. Introduction

Today we’ll look at AMD’s graphics architectures to gain a deeper understanding into how their GPUs work and some factors that contribute to the real-world performance of these processors. Specifically, we’ll be examining the TeraScale, GCN and the recently announced RDNA architecture families.

Let’s start off by associating these names to actual products on a timeline:

2. What is an architecture anyway?

The term ‘architecture’ can be confusing: termed ‘microarchitecture’ in the context of integrated circuits & abbreviated to μarch or uarch for convenience (μ being the Greek symbol denoting ‘micro’), microarchitecture refers to both the physical layout of the chip’s silicon innards as well as the implementation of a given instruction set, including both hardware and software design choices.

For context, Intel’s & AMD’s CPUs implement the 32-bit (x86) & 64-bit (AMD64) instruction sets, together called the x86–64 Instruction Set Architecture (ISA). They’ve done so for a while now and yet ever so often you’ll hear of a new ‘architecture’ such as Intel’s Skylake or AMD’s Zen. In these cases, the underlying instruction set stays the same (x86–64) while its physical implementation changes with new enhancements focused on improving performance and reducing power consumption. So, while the set of instructions that a chip understands and decodes/executes comprises the ISA of the chip, the term architecture refers to both that ISA as well as the physical implementation of said ISA.

ISAs are commonly categorized by their complexity, i.e., the size of their instruction space: large ISAs such as x86–64 are called Complex Instruction Set Architectures (CISC), while the chips powering smartphones and other portable, low-power devices are based on a Reduced Instruction Set Architecture (RISC). The huge instructions space of the typical CISC ISA necessitates equally complex and powerful chips while RISC designs tend to be simpler and therefore less power hungry.

ISAs don’t remain stagnant & new instructions are added all the time to introduce new features while entire extensions aren’t uncommon either: Intel’s AVX extension to the x86–64 ISA added support for new parallel processing modes on CPUs while Nvidia’s Turing brought along support for real time ray tracing on their RTX GPUs. Hardware changes may accompany such significant extensions such as the dedicated ray-tracing cores (RT cores) on Turing.

3. Overviewing AMD’s GPU Architectures

With that brief explanation, let’s overview AMD’s GPU architectures: TeraScale, GCN and RDNA. Starting way back with TeraScale may seem annoying and unnecessary but stick around & it’ll prove worthwhile.

3.1. TeraScale

TeraScale’s reign began back in 2007 and extended until late 2011, with some TeraScale GPUs released as late as 2013. TeraScale matured three generations over this period with the second generation being the most dominant and revered today. TeraScale is traced over a timeline below:

It’s hard to overstate the significance of TeraScale: AMD had just completed the acquisition of Canadian ATi technologies, the creators of the Radeon GPUs, a year prior to TeraScale’s release in 2007. TeraScale thus served as the first GPU architecture released under AMD though it’s reasonable to assume that it was well under development before ATi’s acquisition. TeraScale was significant for several other reasons though: It was the first ATi GPU for which the underlying ISA & microarchitecture were publicly detailed & it existed at a significant time wherein the concept of the “GPGPU” was just starting to take a hold:

The General-Purpose GPU or GPGPU concept looks at utilizing the significant computational power of GPUs for general workloads rather than just graphics, outside of which GPUs sat largely idle. This is significant because until this point GPUs had existed purely for graphics workloads (as suggested by their name) with every aspect of their design accordingly specialized.

Why do this when every system is already equipped with a general-purpose processor, the CPU? Because the specialized nature of the GPU meant that it could carry out a certain type of math really, really fast. Magnitudes faster than the CPU, in fact. It also turns out that while such math was typical of graphics workloads, many scientific and compute workloads relied on similar calculations and would therefore benefit greatly from access to the GPU, which is built up of thousands of cores to perform said math in a massively parallel operation. While “thousands of cores” may sound absurdly large compared to the typical CPU core-counts we’re used to, keep in mind that those CPU cores are general processors that are individually much more complex and capable than their corresponding GPU counterparts.

As the GPGPU concept began to take hold, AMD’s first foray into the territory came in the form of support for the OpenCL library on their TeraScale Gen1 GPUs. OpenCL (Open Compute Library) is the dominant opensource library for compute on “heterogenous” systems, i.e. systems combining different types of processors such as CPUs and GPUs. Further, AMD’s Fusion initiative looked to merge CPUs and GPUs onto a single package, further pushing heterogenous system architectures (HSA) and resulting in the creation of the “Accelerated Processing Unit” or APU, a moniker that’s still used today.

Though AMD’s GPGPU foundations were thus first firmly laid within TeraScale’s architectural depths, it would be TeraScale’s successor GCN that would cement AMDs commitment to the GPGPU initiative. TeraScale was thus the last of the pure graphics focused, non-compute centric GPU architectures from AMD/ATi. The GPGPU movement would eventually go on to become the central enabler for the machine learning revolution of today, powering the neural networks behind the self-driving cars and AI enabled voice assistants that are so ubiquitous now.

At its core, TeraScale was a VLIW SIMD architecture (don’t let these terms scare you off, they’ll be adequately addressed soon) which contributed significantly to its gaming dominance at the time.

3.2. GCN — Graphics Core Next

GCN has been the dominant GPU architecture for AMD this decade and currently features on the ‘Polaris’ and ‘Vega’ family of GPUs with Polaris comprising the fourth generation and Vega comprising the fifth and final iteration of GCN. Polaris targets the low & mid-range segments of the market with the RX400 & RX500 series of GPUs leaving Vega to target the upper tier segments with the Vega 56, Vega 64 & the 7nm Radeon VII cards. In addition to these, Vega features on AMD’s ‘Instinct’ lineup of machine learning GPUs as well as on the ‘FirePro’ lineup of professional graphics GPUs.

Over the course of its years, GCN matured five generations and saw the release of many product families spanning desktop & laptop GPUs, APUs, FirePro rendering GPUs & the MI series of machine learning accelerator cards. The major desktop gaming GPU families are traced over a timeline below:

Though Vega ushered in the last of the venerable GCN-era of GPUs, GCN continues to assert a strong influence on its architectural successor RDNA and it’s reasonable to expect this influence to continue into future generations as well. Besides, there are a lot of GCN cards out there today and that will probably remain the case for a while going forward. This current & near future relevance alone make deep dives such as this worthwhile however historic factors & current perception play an important role as well: having originally debuted back in 2012 on the Radeon HD 7700 series of the ‘Southern Island’ family of GPUs, GCN is now viewed as an ancient workhorse, a product well past its prime with every drop of performance squeezed out of it. Indeed, AMD seems to think so as well with GCN’s successor now finally out the door featuring significant changes at fundamental levels.

With GCN, AMD made it clear that general compute was going to be a big deal for GPUs going forward and the many architectural changes reflect this. These remain a topic for discussion within the enthusiast community until today and will remain a focus here as well.

3.3. RDNA — Radeon DNA

RDNA’s goal, purpose and central mantra can be summed up in two words: efficiency & scalability. When given the same number of compute resources as a GCN based chip, RDNA manages to get more work done while requiring fewer threads in the pipeline to keep its resources adequately utilized and busy. RDNA also plans to feature on everything from mobile phones to supercomputer accelerators and of course, on consoles and your high-end graphics cards.

More on that scalability thing: Sony plans to use RDNA for its hotly anticipated PlayStation 5, Microsoft plans to do the same for its own hotly anticipate “Project Scarlett” Xbox and perhaps most surprisingly, Samsung plans to use RDNA graphics in their next generation of Exynos chips for smartphones.

Not done yet: on the other end of the spectrum, Google announced their upcoming cloud-based gaming subscription service ‘Stadia’ would make exclusive use of AMD’s GPUs while supercomputing veterans Cray announced the Frontier supercomputer for the US Department of Energy would be entirely based on AMD’s CPUs and GPUs to deliver 1.5 Exaflops of compute power, making it the most powerful computer in the world equaling the combined grunt of the top 160 supercomputers today. Wow.

Certainly big wins and nothing to scoff at; a darn good start for RDNA indeed!

4. Understanding the GPU’s Playground: The Display

Let’s preface our architectural deep dive with a review of the GPU’s fundamental output device, the monitor. All your digital adventures occur within the realm of your screen’s pixels and it’s your GPU that paints this canvas. To do so, it needs to draw or “render” visual data onto your screen’s individual pixels. Looking at a standard full-HD screen:

Over 2 million pixels with 1920 pixels in each of the 1080 horizontal rows giving us a full-HD resolution

Image source: ViewSonic Corp

The GPU draws up an image (called a “frame” in graphics parlance) representing the current display state and sends it to the screen for display. The rate at which the GPU renders new frames is measured in FPS, or Frames Per Second. The screen is correspondingly refreshed several times a second, measured in Hertz and typically 60Hz, ensuring that screen updates are smooth and natural rather than sudden & jarring. In this sense you can correctly think of the frame rendering & refresh cycle as akin to the cinema halls of yesteryears, wherein images on a spinning reel were projected onto a screen creating the illusion of a video, aptly named a “motion picture”. It’s truly the same process today, just entirely digital & a lot more high-tech!

The take-away here is that rendering content is a lot of work that involves updating over two million pixels simultaneously several times a second in the context of a full-HD screen and over four times as many for a 4K screen. The good news is that each pixel can often be processed entirely independently from other pixels, allowing for highly parallel approaches to processing. And in this computational playground lies the key distinguishing factor between the CPU & the GPU:

5. CPUs vs GPUs — SISD vs SIMD

Any processor can fundamentally be described as a device that fetches data and instructions, executes said instructions against said data and produces an output which is then returned to the calling program.

A GPU does the same with one key distinguishing feature: instead of fetching one datapoint and a single instruction at a time (which is called scalar processing), a GPU fetches several datapoints (this group is called a vector) alongside a single instruction which is then executed across all those datapoints in parallel (thus called vector processing). The GPU is thus a vector processor said to follow a Single Instruction Multiple Data or SIMD design.

There are caveats of course: such a SIMD design works only with tasks that are inherently parallelizable, which requires a lack of interdependencies between datapoints: after all, operations cannot be executed in parallel if they depend on each other’s output! While graphics and some compute applications are highly parallelizable and thus suited to such a SIMD execution model, most applications are not. Therefore, in an effort to remain as general purpose as possible the CPU remains a traditional scalar processor following a Single Instruction Single Data (SISD) design.

And with that understanding, we’re now ready to move on.

Moving on…

We’ve now overviewed the GPU architectures released by AMD following their acquisition of ATi Technologies and overviewed the humble monitor as well as the fundamental difference between the CPU & the GPU. We further observe that this is an exciting time wherein GCN’s long overdue successor has finally arrived: while TeraScale was a very successful gaming architecture and GCN laid firm foundations for AMD’s foray into GPGPUs, RDNA seems set to do it all better than before in more devices than ever before and at every possible scale. But what fundamentally distinguishes these architectures? What causes them to do the same things, i.e. crunching numbers and putting pixels on your screen, so differently? Enough background and pre-requisites, it’s time to delve deep within.

Find Part 2, An Architectural Deep-Dive into TeraScale, GCN & RDNA, here:

An Architectural Deep-Dive into AMD’s TeraScale, GCN & RDNA GPU Architectures

An Overview of AMD’s GPU Architectures was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is the second article in a series aimed at demystifying blockchain for those who’ve heard the term and the industry optimism around it one too many times and are seeking to understand just what all the fuss is about. If you haven’t read part 1 yet, now is the perfect time to do so!

Part 1 — Demystifying Blockchain

The previous article delved into the world of blockchain — what blockchain is and isn’t, how it functions and what its enabled in terms of decentralized, self-regulating digital economic systems. None of that however may have explained why blockchain has suddenly become a major interest point for many industries. So what explains this surge in industry interest, then?

1. Where Businesses Come In

The answer lies in separating the underlying blockchain network infrastructure from the applications that are deployed and running on it: since a cryptocurrency is ultimately a software program deployed on a blockchain’s network infrastructure, one may think of the possibility of deploying other programs on blockchain networks as well. These could take the form of code to carry out business functions such as handling records or tracking asset ownership, for example. This opens up a world of possibilities for the enterprise space as any custom developed business logic could theoretically be deployed on a blockchain network: you should simply have to pair up with the relevant parties, set up and configure the blockchain network and then develop and deploy your business logic code atop it.

The good news is such blockchain networks are not just theoretical and they do indeed exist: the cryptocurrency system Ethereum was the first to open its underlying blockchain network to independent developers allowing them to deploy their own custom applications atop the Ethereum blockchain. Such applications are referred to as ‘decentralized applications’, or dApps for short, a term popularized by Ethereum. Such a decentralized application at its core consists of a “smart contract”, i.e. code that executes transactions and handles asset ownership amongst participants. In this regard, Ethereum and other cryptocurrencies themselves can be viewed as currency-centric smart contracts deployed on blockchain networks.

Deploying and running a dApp is obviously not free as nodes on the Ethereum network must maintain them. This cost takes the form of a volatile ‘gas price’ that changes as per network congestion and other parameters. This gas fee is borne by transacting parties for each transaction whereas the developer of the dApp incurs a one-time deployment fee. While dApps can implement their own currency for internal transactions, the gas fee must be payed in Ether. Thus, all transactions on the Ethereum blockchain necessitate the use of Ether.

Ethereum may have proudly ushered in the ‘Cryptocurrency 2.0’ Era, wherein cryptocurrencies are no longer just economic systems but rather decentralized platforms on which applications may be deployed, but its setup is evidently not the best fit for all enterprise usecases: industry participants wouldn’t want to be forced into using a cryptocoin and would probably have a host of other constraints making public networks such as Ethereum less than ideal candidates for their business-critical operational functions.

2. Bringing Blockchain to the Enterprise

A viable blockchain system for the enterprise space then should fulfill the following criteria:

1. Stay Permissioned

Unlike permissionless networks such as those of Ethereum and Bitcoin, an enterprise network must permit only authorized parties to join and participate in the network and must provide a mechanism for authorizing and verifying membership

2. No mandatory coins or assets

An enterprise-grade blockchain system must ideally not be based on digital coins or other assets that need to be obtained, stored and managed by business participants and should provide a means for transactions and consensus regardless

3. Infrastructure limitations

An enterprise system should not require business participants to run and manage large amounts of infrastructure to maintain the network, which would add significant overhead in terms of networking and infrastructure management expertise

4. Facilitate Custom Business Logic

A blockchain network system targeting business usecases must facilitate the deployment of any custom business logic code on the network along with providing simple mechanisms to upgrade and manage said code

3. The Hyperledger Fabric Blockchain Framework

Hyperledger Fabric is such an open source, enterprise-grade, permissioned blockchain framework that does not rely on any inherent currencies or assets. It’s also available in cloud-flavors via the Oracle Blockchain Platform and the IBM Blockchain Platform, and users opting for these are obviously spared of any infrastructure related concerns.

Elaborating on the name: Fabric is just one framework within the Hyperledger project. The Hyperledger project itself is an effort to build enterprise focused blockchain frameworks and is maintained by the Linux Foundation. Fabric happens to be the first and thus the oldest framework under this umbrella. Initially developed by IBM, Fabric was donated to the Hyperledger initiative for future development and upkeep.

Fabric has since gained significant popularity thanks in large part to the cloud offerings mentioned earlier that are based entirely on it. This warrants an in-depth look at the Hyperledger Fabric framework, so let’s dive in:

3.1 Deployment Architecture Details

1. Participants on a Hyperledger Fabric based blockchain network work together to validate transactions and to make changes to the blockchain ledger

2. Organizations on a blockchain network may wish to keep certain transactions private within a subset of members on the network, and can commission channels to do so:

• Each channel maintains its own blockchain ledger private to only the channel’s participants
• Nodes part of multiple channels/networks maintain multiple exclusive ledgers
• Participants on a channel may further wish to keep specific transactions (rather than entire ledgers) private within just a subset of members on the channel, and can commission a Private Data Collection (PDC) to do so:

i. Members in a PDC can independently verify and commit new transactions, while sharing only a hash of the transaction data with the remaining channel members

ii. The shared hash enables easy verification of a transaction’s time and data by all members of the channel while allowing only the PDC members to actually see the data

iii. The actual data is shared between PDC members via a gossip protocol and stored by them in a private database, dubbed a SideDB

3. Participants identities & their memberships to networks, channels etc. are established by verifying a digital, cryptographic x509 certificate:

• This certificate may be issued by a well-known Root Certificate Authority (CA) or Intermediate CA (such a Symantec, GoDaddy, etc.) or via the built-in Fabric-CA
• A Membership Service Provider (MSP) identifies which CAs are authorized to issue certificates and further identifies the specific roles of participants (such as admins, members, etc.) and their access privileges (readers, writers, etc.)

4. Each channel has its own independent smart contract code deployed on it, referred to as chaincode in HLF (the equivalent of smart contracts in Ethereum) written in GoLang

5. In addition to a blockchain ledger, each channel maintains a corresponding world state database to keep track of the latest values of assets tracked by the ledger

All these functions are carried out by multiple nodes, each fulfilling a distinct function and working together to keep the network operational. These nodes are described below:

3.2 Hyperledger Fabric Nodes

Each member on a blockchain network consists of several nodes with each fulfilling a different role:

1. Peer nodes

• Endorsing peer nodes

2. Ordering Service nodes

3. Fabric-CA node

These nodes are detailed below:

3.2.1 Peer Nodes

1. Peer nodes carry out three major functions:

i. Store a copy of the blockchain ledger

ii. Install and activate (instantiate) chaincode

iii. Participate in transaction verification

2. An Endorsing peer node is a special type of peer node:

• These are the first to receive proposals for new transactions
• Endorsing peers simulate the results of these proposed transactions on their local ledgers via the installed chaincode
• If all goes well, they generate an endorsed transaction proposal which is then sent to the ordering service

3. Once the endorsed transactions have been ordered into blocks by the ordering service, peer nodes validate each transaction and subsequently append the new blocks to their ledgers

4. Transactions are validated for the appropriate endorsements & to confirm that any proposed changes haven’t already been invalidated by more recent transactions

5. If invalidated transactions are found, they’re marked as such before the block is committed to the ledger and they do not affect the world state

3.2.2 Certificate Authority Node

• This node issues cryptographic identity certificates to participants on the network
• These certificates may then be used by participants to identify themselves during transactions
• The use of this node is optional and an external third-party root CA or Intermediate CA may be used instead
• Certificates are a vital part of the Membership Service Provider (MSP) which uses them to ensure that participants are duly authorized

3.2.3 Ordering Service Nodes

• The ordering service accepts endorsed transactions and orders them into blocks which are then broadcast to all committing peer nodes for addition to their blockchain ledgers
• The ordering service makes no judgement as to the validity of transactions and simply orders them into blocks which are then verified by peer nodes
• HLF offers a choice of three ordering services:

1. Solo Ordering Service:

As the name implies, this ordering service consists of a single node and thus can never be fault tolerant. Fault tolerance is a crucial requirement of any blockchain network (read all about it in Part 1) let alone one targeted at enterprise usecases. Why does this option even exist then? For simplicity: use a solo ordering service for quick Proof-of-Concept (PoC) development activities. However, if this PoC will eventually move to production, it would be better to make use of one the following ordering services, which can be made up of a single node initially and scaled up when it’s time to go prod. NEVER use a single node based ordering service in production though, you have been warned!

2. Raft Ordering Service:

Raft is the native, go-to ordering service of HLF and works via a leader and follower model: one node in the cluster is dynamically elected the leader and carries out the actual ordering, while follower nodes copy the leader’s results. Follower nodes listen out for periodic “heartbeat” messages from the leader to ascertain that it’s online and will wait a predefined threshold of time between each heartbeat before beginning the process of electing a new leader. Raft is “Crash Fault Tolerant” and can withstand the loss of a minority of nodes while staying functional (i.e. if you have 5 nodes a minimum of 3 are required online). Lastly, Raft is also touted as HLFs first step towards Byzantine Fault Tolerance (refer to part 1).

3. Apache Kafka based Ordering Service:

Apache Kafka clusters may be configured and used to provide an ordering service along with a ZooKeeper ensemble for the administration of this cluster. Kafka follows a similar leader & follower model to Raft, but with significant additional administrative overhead to its deployment. Not a native part of HLF, users opting for this are presumed to have prior expertise with the deployment and administration of Kafka clusters. Designed for CFT within tight groups, Kafka & ZooKeeper are not designed to run across large networks either. Kafka is notorious for deployment headaches and Raft should be the preferred choice.

3.3 Data Stores of Hyperledger Fabric

A Hyperledger Fabric network consists of two primary and one optional data store:

1. The World State Database

2. The Blockchain itself

3. Private Data Collections, aka PDCs (optional)

The world state database and the blockchain together make up the complete ledger and though they are related, they are very distinct:

3.3.1 The World State Database

The world state database holds current values of assets on the blockchain. For example, in a blockchain network used to track car ownership, this database would hold the name of the current owner for a given vehicle. The world state database holds this data in key-value pairs. The HLF default LevelDB doesn’t support SQL rich queries and HLF may be configured to use CouchDB instead for this functionality.

The world state database is extremely useful because programs will often only require the current value of assets and the ledger state, and these requests are all fulfilled quickly via the world state database instead of traversing the ledger.

3.3.2 The Blockchain

The blockchain itself is stored on every peer node that’s part of the network and consists of blocks which in-turn each contain a certain number of transaction records. Blocks have a block header which holds the block number, the root hash value of all transactions on the block and the hash of the previous block thus linking all blocks into a chain. In addition to the transactions (which make up the body) and the header, the block also holds metadata that specifies when the block was written along with the identifying information of the block endorsers and validators.

The blockchain itself is immutable, and the world state database traverses the ledger for the latest state values to fulfill queries.

3.3.3 Private Data Collections (PDCs)

A Private Data Collection refers to a subgroup of participants on a channel that need to keep certain transactions private amongst them. While channels are the primary route for ensuring privacy and separation of concerns, a Private Data Collection is the preferred route when only some transactions need to be kept private between a subset of members rather than entire ledgers. This has two advantages:

1. Avoiding the additional administrative overhead involved in setting up new channels for every communication that must be kept private between a group of participants

2. All participants on a channel are made aware of a transaction while the actual data itself is kept private within a subgroup, thus aiding future verification during conflicts

For this reason, a PDC is a collection of two elements:

1. The actual data which will be kept private within the subgroup

• This data will not go through an ordering service, but will be communicated via a gossip protocol between the peers in the subgroup and stored in a SideDB

2. A hash of the data which will be shared with all members on the channel

• All members on the channel will receive a hash of the data which serves as a proof of the transaction and can be used for audit purposes
• This hash will go through an ordering service and be visible to all members

If a dispute occurs later the collection members can choose to share their data with a third party, which can then compute the hash and compare it with the hash stored in the main channel state, thus proving that the transaction involving the data did indeed occur between the collection members at that point in time.

3.4 The Life-cycle of a Transaction

Updates to the ledger occur in three phases: endorsement, ordering and finally commitment to the blockchain ledger on each peer node. These phases are detailed below:

3.4.1 Proposal & Endorsement

• A front facing client application begins the process by sending a transaction proposal to several endorsing peers via the Fabric-SDK (Node & Java are native SDKs, currently)
• Endorsing nodes receive the transaction proposal and simulate the proposed transaction on their local ledgers, and all going well generate a ledger update proposal
• Do note that the endorsing peers do NOT actually update their ledgers at this stage
• The now endorsed transaction proposal is returned to the client application

3.4.2 Ordering

• In this phase, the client applications submit the endorsed transaction proposals to an ordering service, whose role is to appropriately order these transactions into blocks
• The sequence of transactions in a block may differ from the order in which they were received & the number of transactions in a block can be changed by the orderer admin
• However, once a block is generated by the orderer, the sequence remains immutable and all committing peers must append the block as-is to their local ledgers
• This finality in the ordering of transactions in a block prevents the formation of forked chains that must eventually be resolved (refer to the PoW process detailed in part 1)
• Nodes that form a part of the ordering service do NOT execute smart contracts and in fact make no judgement whatsoever as to the actual content or actions of the proposed transactions, instead leaving that role to the committing peer
• Generated blocks are now sent to peers for validation and subsequent addition to their blockchain ledgers

3.4.3 Transaction Commitment

• This is the final stage wherein peers receive and validate newly generated blocks of appropriately ordered, endorsed transactions and finally commit them to their ledgers
• Each peer validates each transaction for the appropriate endorsement & to confirm that any proposed changes haven’t already been invalidated by more recent transactions
• If invalidated transactions are found, they’re marked as such before the block is committed to the ledger and they do not affect the world state
• Though validation is carried out by each peer individually, the process is still distributed in nature as each validating peer checks for the same consistencies in endorsement & transaction validity
• For any peers offline during this process, they can receive the blocks they’ve missed out on by connecting to an ordering service node upon returning online, or by gossiping with other peers via the appropriately named Gossip protocol
• With the ledger successfully updated the process is now complete!

4. Concluding Our Discussion: What We’ve Seen So Far

The first part of this series was aimed at shedding light on the innards of blockchain and its consensus mechanisms. To do so, we recruited the help of cryptocurrencies: after all, what better way to learn of a technology than through its most successful application? We thus explored the underlying nature of blockchain, its accompanying consensus mechanisms and the fault tolerant operations they enable.

Now at the end of part 2, we’ve explored what makes blockchain appealing to enterprise users and what specific features a blockchain network would need to provide to further fan this appeal. In doing so, we’ve explored an enterprise grade blockchain framework, the Hyperledger Fabric framework, in considerable depth. We’ve explored its architectural details, the nodes & datastores that form its operational backbone and have looked at the typical lifecycle of a transaction. We’re thus at an ideal point to conclude our discussion for now:

• Enterprise interest towards blockchain technology stems from the possibility of deploying any decentralized application (dApp) atop it
• These decentralized applications could, for instance, handle asset transfers or track transactions and services, such as goods through a supply chain
• The core of a dApp is code in the form of a smart contract
• Ethereum was the first cryptocurrency to open up its underlying blockchain for application developers, and in doing so ushered in the Cryptocurrency 2.0 movement
• Transactions on the Ethereum blockchain incur a gas fee that must be payed to the underlying network in the native token Ether
• That fact in addition to the public nature of the Ethereum blockchain network could make enterprise users hesitant to deploy their business-critical applications atop it
• Ideally an enterprise-focused blockchain network must not necessitate the use of currencies, and in addition should allow for easy yet powerful identity and code management and as a bonus provide means to work around infrastructure hurdles
• Hyperledger Fabric is an enterprise-grade, permissioned blockchain framework that allows for easy deployment of custom-logic in the form of chaincode (smart contracts) atop it and does not involve any native cryptocurrency-based operations
• HLF is further available in cloud-based offerings from Oracle & IBM, mitigating infrastructure concerns for opting users
• HLF networks are comprised of nodes that work together to endorse, order and verify transaction requests (in that order)
• HLF provides a choice of three ordering services: solo, Raft, and Apache Kafka
• Solo should only be used for quick PoC work and even then, a single-node based Raft or Kafka orderer is preferable if future production deployment is on the cards
• Raft is the native orderer and is crash fault tolerant: a Raft-based ordering service may continue to function as long as a majority of its nodes are online
• Raft is also the first step towards Byzantine fault tolerance for HLF
• Kafka is notorious for deployment complexities and involves significant administrative overhead: the user is presumed to have prior expertise with Kafka if deploying it
• Both Kafka and Raft follow a leader-follower model
• A membership service provider relies on identity certificates to ensures that all participants are duly authorized for the actions they’re attempting to perform
• These x509 cryptographic identity-certificates may be obtained via external Certificate Authorities or via the default Fabric-CA
• Channels are the primary means of privacy and separation of concern on an HLF-based blockchain network
• All transactions within a channel are private to the members of that channel alone
• Participants on a channel may further wish to keep certain transactions (as opposed to entire ledgers) private between them and can commission Private Data Collections (PDCs) to do so
• Data within a PDC is stored in a separate SideDB and is exchanged directly between peers via a gossip protocol, i.e., it does not go through the channel’s ordering service
• A hash of the data is shared with the rest of the channel, which can then be used to verify the time and contents of a transaction if a dispute arises in the future
• Three datastores exist within HLF: a world state database, the blockchain ledger and an optional SideDB for PDCs
• The blockchain ledger is immutable and stores a record of all prior transactions
• The world state database traverses the ledger to keep track of the latest values of assets tracked by the ledger, queries are quickly fulfilled via this world state database
• Data in the world state database is stored in key-value pairs with LevelDB serving as the default world state database
• LevelDB however does not support a JSON-rich query syntax and if this support is desired CouchDB can be used instead
• A front-end application needs to use the Fabric SDK to interact with the network
• Official Node and Java SDKs are available along with unofficial Python, Go and REST SDKs
• Chaincode in HLF is currently written in GoLang

Read Further:

Part 1 — Demystifying Blockchain

HLF v1.4 Official Documentation

Demystifying Enterprise Blockchain: Hyperledger Fabric was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.

What you’re going to need to understand this article: patience.

Unless you’ve been on a sabbatical from the world for the past two years, you’ve already heard the term “blockchain” way too many times. If you’ve been following the tech industry a little more closely though, you’ve probably observed “industry changing”, “revolutionary”, “disruptive” and countless other overly optimistic adjectives used in the same sentence as well. Perhaps you’re even sick of the sudden explosion of literature around blockchain. To top it all, this is another article on the topic. But hang on because this one attempts to be the only one you may need to get fairly well acquainted.

To truly understand blockchain we’ll have to examine what makes it unique and what problems it attempts to solve. What’s happening in the world around it now, and what is this mining stuff anyway and do I need it if I want to use blockchain? Now there is no better way to attempt this understanding than by examining the most successful and prominent use of the technology: cryptocurrencies. So, don’t be surprised that they play a central (distributed?) role in this article.

1. The Elephant on the Page: What Exactly is Blockchain?

Without delay: A blockchain is simply a ledger. It’s a ledger of transactions and is immutable, thus providing an irrefutable history of all transactions between all participants on that blockchain network.

That does not sound like a new concept, so what makes blockchain so unique and justifies all the buzz and hype? And why is it called that?

The truly distinguishing feature as well as the core underlying concept of blockchain is that it’s not centralized, nor is it distributed in the traditional sense. While a centralized system is self-explanatory, a typical distributed system consists of computing equipment or digital data split-up and spread out over several, distinct physical locations. In contrast to either, a blockchain network requires every participant on the network to store a copy of the ledger in its entirety, thus making redundancy it’s key underlying feature. Updates to the ledger (such as adding a new transaction record) then requires participants on the network to reach a consensus as to the validity of the proposed transactions. Consensus must be obtained in a decentralized, fault-tolerant manner thus maintaining the decentralized, participant driven nature of blockchain and to ensure that the ledger is accurate.

Oh, and it’s called a blockchain because of the way this ledger is structured: as a sequence of virtual blocks wherein each block holds some part of the transaction history. Think of how a notebook’s pages each have a finite number of lines and are bound together at the seams. In a similar fashion a blockchain is an ordered collection of blocks, with each block pointing to its preceding block, thus intertwining them into a chain of records.

1.1. The Bitcoin Blockchain

Bitcoin’s blockchain consists of a record of every Bitcoin transaction that has ever occurred since its inception. Each block contains some number of transactions and each block on the Bitcoin blockchain can be at most one megabyte (1 MB) in size. The first block is called the Genesis block, and is special as it does not point to a previous block.

Several challenges though are already apparent: how do you ensure consensus is obtained on the validity of new transactions, especially when factoring in malicious actors who may attempt to distribute incorrect copies of the blockchain ledger to benefit themselves. In other words, how do you achieve fault tolerance in Bitcoin and in any other blockchain network?

2. Fault Tolerance

In a blockchain system, every participant maintains a copy of the entire blockchain ledger. For new transactions to be added to this ledger, participants must reach consensus on the validity of these proposed transactions. This is non-trivial for a great deal of reasons: sometimes vital information may not reach all participants and sometimes malicious actors may attempt to spread false information that benefits them. Fault tolerance then refers to how capable a system is at operating in the face of such challenges.

Consider the two general’s problem: General 1 is considered the leader while General 2 is the follower. They are both located on opposite sides of an enemy and must decide on a time to mount an attack. General 1 decides a time and sends a messenger to General 2 to convey his decision. General 2 must then send an acknowledgement (ACK) back to General 1. It’s easy to imagine how this system can be problematic: either message may not be delivered if the messenger is captured. Even if both messages are delivered, how can General 2 be certain his acknowledgement was received without devolving into infinite acknowledgements? And we haven’t even factored in a malicious messenger or general into the equation yet!

This problem is in fact, unsolvable. Why even bring it up then? Because an extension to this problem called the Byzantine General’s Problem plays a critical role in blockchain and other distributed systems.

2.1. The Byzantine General’s Problem

Byzantine refers to the old Byzantine Empire, also known as the Eastern Roman Empire and their army Generals apparently faced the same problem as above though with two major changes: there are now more than two Generals and any of the actors may be malicious which implies that some of the Generals and/or messengers may hold treacherous intentions, i.e., they may intentionally cast suboptimal decisions or deliver the wrong message respectively.

Consider nine Generals with four in favor of attacking and four in favor of retreating. To avoid a disastrous defeat, a final decision for either a collective attack or retreat must be reached. The ninth General though has other plans and with malicious intent casts two votes: one informing the attackers that he’s in favor of their stance and the other informing the retreat favoring Generals of the same. The resulting effect of this betrayal is not hard to imagine.

Here, while the malicious entity appeared to be functioning correctly to all parties, i.e. there wasn’t a “node failure”, the system was compromised. This makes Byzantine fault tolerance the most general and difficult class of failure modes and a system tolerant to this class of failures is unsurprisingly termed as ‘Byzantine fault tolerant’.

It is critical for distributed systems and blockchain networks to attain Byzantine fault tolerant operation and continue operating reliably and correctly even in the face of failed and malicious nodes. Without this, neither is blockchain viable for monetary systems such as Bitcoin and nor can any enterprise usecase ever adopt a blockchain centric solution for business processes.

3. Attaining Fault Tolerant Consensus

To attain fault tolerance necessitates honest majority participation. That means a blockchain network relies on a majority of its nodes being honest. How many, exactly though? While early Byzantine fault tolerant systems required near eighty percent honest participation, Bitcoin and most cryptocurrencies require just 51% honest participation, i.e. as long as honest nodes control a majority of the compute power on the network, the system remains fault tolerant.

To see how this works, let’s get an overview of the process of adding a block of new transactions to a blockchain ledger, and in the process explore consensus algorithms.

3.1. Transaction Overview

Proposals for new transactions are broadcast into the network and participants on the network gather these proposals into blocks. These blocks are then broadcast to other participants for verification, which then need to indulge in some book-checking to confirm that all transactions in that block are valid. For example, for transferring funds in a cryptocurrency based blockchain network this would involve confirming that a sender possesses the necessary balance to transfer the requested amount to a receiver’s wallet. Again, as long as a majority of the participants are honest the network stays fault-tolerant to malicious entities.

Blocks thus verified are then broadcast to all members for appending to their respective copies of the blockchain ledger. The set of transactions on that block are now part of the irrefutable record history that is the blockchain.

Obtaining the required consensus though is a challenging process and also led to the popularization of the whole “mining” term that is often synonymous with blockchain and cryptocurrencies in the minds of many. In reality, mining is just one of the ways for participants to reach consensus and is officially called the Proof of Work (PoW) consensus algorithm. Obtaining valid consensus in a fault-tolerant manner is absolutely critical for blockchain applications though. The following two sections delve deeper into mining and another popular consensus algorithm in use today.

3.2. Proof of Work Consensus, aka “mining”

Mining, officially called the Proof of Work (PoW) consensus algorithm, is one of the ways for participants to reach consensus. The idea is to have participants attempt to crack a hard, cryptographic puzzle to add new blocks to the chain. Here’s how this works:

1. Nodes on the network receive transaction proposals which they gather into blocks

2. All transactions in a block are individually hashed to obtain their respective hash values

3. These hash values are then summed to a single hash value representing all transactions in that block, this value serves as the root hash value for that block

4. The root hash value of the previous block is now obtained by referencing the ledger

5. Now, a special value called a nonce must be found which when summed with these two root hashes results in a number with a certain number of preceding zero bits

6. This nonce once found is stored in the block header along with the root hash value of the block and the root hash of the block preceding it

7. Once a node has found this nonce and thus generated a complete block it broadcasts the block to other participants for verification of the transactions within the block

8. The first transaction in every block is always a self-credit by the node that generated that block: when this block is accepted this transaction becomes true and new coins are minted and credited to the successful nodes wallet which can now claim to have successfully “mined” the block

Conflicting chains will arise during this process and are abandoned by nodes in favor of current longest PoW chain at that time, which is always regarded as correct as the most compute power or ‘work’ has been expended on that chain. If two nodes simultaneously generate a valid block and propagate it, receiving nodes will begin extending their chains based on the first block received. These conflicting chains dissolve gracefully when the next block is found thus extending one chain and making it the longest.

3.2.1. Incentivizing Honest Participation

Finding the nonce requires a surprisingly large amount of computational power, and the difficulty increases as the blockchain grows. Any attacker who attempts to propagate a false chain must not only compute the correct nonce and headers for many blocks in order to successfully create a fake chain but must also convince a majority of the network to ditch their ledgers in favor of the fake one so it becomes the most accepted chain. Such a 51% attack is simply not going to happen on a sufficiently large network: to do this on Bitcoin and Ethereum networks today would require more compute power than several supercomputers combined!

Further, if an attacker were to amass more compute power than the rest of the network they may simply be tempted to use it to find the next block, which they would have a high chance of mining and of claiming the newly mined coins. This would also alleviate risk for the attacker as their amassed wealth would greatly depreciate in worth if faith and trust in the currency is lost. Further, many networks restrict changes to blocks that are beyond a certain height, for example only the last ten blocks may be alterable with all prior staying completely immutable.

3.3. Proof of Stake

Proof of Work is by no means the only consensus mechanism. Proof of Stake (PoS) is an alternate method that barely uses any power to run thus alleviating the tremendous energy consumed (and wasted) by PoW networks which discard over 99% of calculations in their attempt to find that correct nonce. In a PoS network, ‘stakeholders’ verify new transactions and hold authority to append new blocks. A stakeholder’s chance to ‘forge’ this block is determined by the amount of assets held and the duration for which they have been held. As more assets are held for longer, the stakeholder has a greater “say” in the network. NXT was the first complete PoS based cryptocurrency, with Ethereum attempting a complete shift as well.

So, while PoW algorithms reward compute power, PoS networks reward the long-held assets of their stakeholders who incidentally have the most to lose as these assets greatly depreciate if trust in the network is lost.

4. Why Blockchain is Worth the Fuss

So having a truly decentralized ledger with fault-tolerant consensus mechanisms is great from a technological perspective, but it also sounds like a demanding system to set up from scratch, especially from a business perspective. What then makes blockchain worth the trouble?

1. Truly Decentralized

This is a bigger deal than it may sound like: a blockchain system along with a robust consensus mechanism requires no central servers for co-ordination between participants, for the minting of new coins or assets or for timestamping transactions and yet the end result is a technology fault-tolerant enough to operate monetary systems.

2. Participant Driven

Participants propose new transactions, verify pending transaction proposals, decline incorrect transaction requests and maintain the ledger collectively. No authority is required to overview any network operations.

3. Transparency

Since the ledger records all transactions on the network and every participant maintains a copy of the ledger in its entirety (or even just the block headers in the case of pure-verification nodes), the entire history of transactions can be viewed and verified for auditing and other reconciliation purposes as and when required.

4. Privacy

As only hashes of transactions are stored, the individual privacy of participants is still maintained. In a public currency system such as Bitcoin or Ethereum, anyone and everyone can view all transactions that have occurred, but these transactions take the form of wallet addresses which are themselves cryptographic signatures, thus maintaining privacy for the individuals behind the transactions. (See the image below).

5. Challenges to Adoption

Nothing is ever just a bed of roses, and blockchain is no different. Some major concerns when setting up a network are:

1. Ensuring Honest Participation

For a public system that anyone can join such as Bitcoin/Ethereum, it really depends on luck with the community at large as honest participation is crucial int the early stages to ensure longevity of the network. For closed networks setup within business domains, participation must be regulated tightly.

2. Infrastructure Investment

Infrastructure costs both in terms of hardware deployment and software development and subsequent maintenance can be significant, especially initially. But it can be argued that this goes for any new system a business house may venture towards and it really is the long term returns and value that should be deciding factors.

3. Fear of the Unknown

The biggest hurdle comes here. There’s no way to sugarcoat it: blockchain and processes around blockchain are radically different and adoption will take a while. With public networks, most people cannot comprehend let alone digest the concept of a decentralized currency which is truly self-regulating. Further, the enterprise space always tends to be a slow and cautious adopter and given the wide-reaching changes required from business processes at the fundamental level coupled with the mindset changes necessary for true blockchain adoption, you can bet that it will be a while before blockchain truly sets in.

6. Concluding Our Discussion: What We’ve Seen So Far

We’ve covered what blockchain is and have delved into the nitty-gritties of the fault-tolerance and consensus mechanisms it relies on. We’ve even taken an in-depth look at the most famous consensus mechanism, Proof of Work or mining as its better known and reviewed another great consensus mechanism, Proof of Stake. We’ve looked at why blockchain is worth the fuss and what the main challenges are to its adoption. We’ve taken the help of Bitcoin and cryptocurrencies along the way to better understand blockchain, and that leaves us on a good note to conclude this article:

· A blockchain is a ledger of transactions and is stored by every participant on the network

· It’s structured as a sequence of blocks, with each block holding a certain number of transactions and maintaining a link to its preceding block, thus forming a chain

· Transactions in a block are hashed until a root hash value is obtained, which is subsequently stored in the block header along with the root hash of the preceding block

· Participants on the blockchain network must reach consensus as to the validity of proposed transactions before they’re added to the immutable ledger

· Consensus must be approved in a fault-tolerant manner accounting for malicious and absent nodes

· Consensus algorithms are built for this purpose, and several exist

· The Proof-of-Work consensus algorithm emphasizes compute power, with participants dedicating resources towards the computation of a special value, dubbed a nonce

· Once the nonce is found, a new block is generated and the nonce is stored in the block header as well

· Multiple nodes may generate valid blocks simultaneously, leading to conflicting chains

· These conflicts are resolved as one PoW chain eventually grows larger and other nodes periodically switch over to the longest chain

· The longest chain is always considered the correct chain as the most compute power has been expended on it

· Proof of Stake is another consensus algorithm and instead of relying on large compute power being expended behind wasteful computations, it relies on stakeholders of its assets

· A stakeholder’s chance at forging the next block is proportional to the quantity of assets held by them and the duration for which those assets have been held

· Blockchain systems are highly decentralized and self-regulating, are entirely participant driven and facilitate high levels of transaction transparency while maintaining individual privacy

· The main challenges to blockchain adoption remain ensuring honest participation and infrastructure investments, though it can be argued that both these are critical and necessary for any system

· Widespread adoption is hindered by a fear-of-the-unknown, which necessitates greater awareness, hands-on experience and most of all: some more time!

This has been the first part in a multi-part series on blockchain technology. Future parts will focus on the enterprise, on the Hyperledger Fabric framework and on writing your own smart contracts and custom logic. Keep your eyes peeled for Part 2, which will focus on exploring blockchain in the enterprise world!

Demystifying Blockchain was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.

The article below is one I originally posted on Reddit in the September of 2017 on the r/buildapc subreddit, a community of over 1.4 million subscribers and nearly as many posts. It was very well received with over 5600 upvotes and a community awarded gold star, the highest award at the time. It remains in the top 30 r/buildapc posts of all time today, and may be found at the link here, check it out for the community discussions and comments that followed! https://www.reddit.com/r/buildapc/comments/70ih3a/a_guide_to_pc_building_some_advice_from_my/

Disclaimer: VERY long post ahead, one that almost saturates the 40,000-character Reddit limit. I hope to help the younger or more inexperienced system builders out there, and anyone confused or stuck at some point. I encourage you younger and inexperienced builders to read through the whole thing, and others to skim over at their pace. Also, since this is based on my experience over a decade of building AMD rigs and the Ryzen rig I built recently over the past couple of months, a lot of examples use AMD systems. Regardless, most if not all that advice and experience can be applied to Intel systems, and I’ve done my best to do so. Happy reading!

Greetings fellow Redditors,

I’m writing here today to share with those of you eyeing that shiny new build some advice on that topic. This will be from experience I have gained over the years and indeed, gained over the past few months that were spent obsessing over the Ryzen build I’ve just finished piecing together.

A little background on myself is in order: I’ve been a pc enthusiast for slightly over a decade now, and have recently earned a Master’s degree in the field of Computer Science. You most certainly do not need any qualification of that nature to be doing this simply because no engineering or science degree in the field will teach you this stuff. PC building ultimately relies on its community of enthusiasts and DIYers on continuing to enrich the knowledge pool for all of us out there. Indeed, I started building back in 9th grade and am happy to say I haven’t blown anything yet and the local power station still exists. So, without much ado, here’s the system I’ve just built:

Processor: AMD Ryzen 7 1700 OC’ed to 3.7GHz @ 1.275v

Motherboard: Gigabyte GA AB350 Gaming 3

Graphics Card: Gigabyte AORUS GTX 1080Ti 11GB

RAM: 16GB (8x2) Corsair Vengeance LPX 3000 @ 2933MHz CL16 (XMP)

PSU: Corsair RM750x

Samsung 850 EVO 250 GB (Primary OS & Applications)

Crucial MX300 275GB (Game Drive)

Western Digital 1TB mass storage hard drive

LG 34UM60 2560x1080 75Hz Freesync display (Soon to be replaced by an Acer Z35 G-Sync panel)

All this in a NZXT S340 Elite case

Yes, that’s a good high-end build beyond which point the effect of diminishing performance returns looms really large currently. Many of you reading this will be quick to point out the smaller game drive SSD (and you’d be right there) and will definitely point out that a 2560x1080 panel is not enough for this build and why pixel density for that resolution at 34” is poor. Allow me an opportunity to counter that in the section dedicated to monitors in this post. Maybe also the B350 chipset instead of an X370. Calm down, we’ll talk about it. Let’s break it all down, component by component:

Considerations in choosing a processor:

Risking backlash, I’m going to say this flat out: buy the fastest factory-clocked chip you can buy without telling yourself you’ll overclock it to match the faster chips. This is in stark contrast to the overclocked Ryzen 1700 I’ve put into my own build. Here’s why:

1. You’re going to tell yourself you’re saving money on the chip and being very smart, but will spend that money on a X370 board “for the beefier VRMs and overclocking features” and will then convince yourself you need a 280mm radiator because who buys a 140mm one, right?

2. Companies like AMD and their motherboard manufactures like to make overclocking sound very easy, they make many videos of it and when you see experts from these companies reiterate how easy and quick it is, you tend to believe it. It’s not that simple and requires more work.

The companies will make it look easy because the silicon manufacturer like AMD wants to sell you the X370 chipset for a multitude of reasons: one, they may have higher margins on those than on processors themselves and two, because once they’ve got you to invest in a top of the line board and told you chips up to the year 2020 will be supported by it, they’ve effectively sold you a couple more processors, or at least made you very unlikely to shift away from their ecosystem. Motherboard manufacturers like it because, come on, it’s obvious, the boards cost more.

While we’ll talk about the motherboards next, focusing on the processors for now, overclocking them is expensive. Let’s consider the cost: an Ryzen 1800x is $500 (can be found at $429 easily now days) and a Ryzen 1700 is $330 (can be found around $270 I believe). That’s less than a $200 price difference. An X370 board will cost at minimum a $100 over the B350 board I picked up, and the water coolers range from $110-$160. You’ve most probably spent more already, if not as much. Yes, you can overclock with a B350 + an air cooler, but most of you will not do that. You’ll see the videos, read the reviews and say “Oh I want to stick with Ryzen for long, I’ll just get the ‘better’ board with the overclocking features.” Factor in even a $50 air cooler, and the cost difference between a R1700 and a R1800x vanish. The difference is even smaller with the R1600 and R1600x. Yes, you’ll need to buy a cooler with the “x” marked chips, even then, you’re not saving a significant amount, if you’re saving any at all, that is.

As for overclocking, keep in mind you need to fidget a lot with voltages and every time you think you’ve found a stable voltage, you need to stress test it for many many hours before being sure. Some enthusiasts will say 24hrs, some will say 48, some will say “hey if doesn’t crash on a Prime95 SmallFFT or Blend test in an hour, you’re gold because gaming won’t stress it as much anyway” and they’re right with that. What do I say? Do it for as long as you’re patient. If you’re okay with keeping the rig on all night only to wake up and see it crashed and its experiment time again, go ahead. I don’t have that patience, I need something that just works, good and fast. Most of you do too.

Another point to this story: voltage requirements change over an extended period of time as the CPU mildly degrades from an overclock that’s not been thoroughly tested. I had my R1700 pass a couple hours of the Prime95 SmallFFT CPU torture test when clocked to 3.65GHz @1.18v. Fast forward a couple months when I started noticing mildly weird behavior from my system, and it bluescreened in 2 minutes of that test. Now it’s at 3.7GHz @ 1.275v, having passed several hours of SmallFFT again. Will it degrade again? Who knows. How can I ensure against it? Thorough stability testing. What comprises that? Who knows, you’ll get many different accounts. Get the fastest factory-clocked chip. Look at overclocking as something you’d do to extend the life of the CPU for a couple extra months or maybe a year before replacing it. The 1800x and 1600x overclock to 4.0GHz on 2 cores and 4.1GHz on one, that’s already great for most games out there that are single thread heavy (like GTA V and Far Cry 4). The couple extra frames you might get from future games is simply not worth it, especially if you manage a Freesync or G-sync display on your rig.

This advice applies just as well to Intel chips. Get the fastest factory clocked chip, but with Intel, try to get the ‘K’ or ‘X’ marked chips for the unlocked multiplier. Why? Because Intel upgrades are always costlier since you most likely need to change your motherboard as well. So a chip capable of some overclocking might really extend the life of your rig, just ask the i5 2500k owners!

Considerations in choosing a motherboard:

Now this is my favorite topic, and the component I spent the most time researching. Why? Because of how everyone seems to love justifying getting the most expensive board out there. There’s always a good reason: “beefier VRMs dude!!”, “more I/O dude!!”, “SLI dude!!”.

First up, I am not suggesting you get a 1800x and shove it onto a $60 A320/B350 board. Nah, that’s another extreme. Likely because those boards were not really manufactured with the thought that they’d be graced with a top-of-the-line chip, and may actually have weaker VRMs. But look at some of the top B350 boards which are known to officially support an 1800x very well. You’ll have no problems running it, maybe even overclocking it a bit. But really, consider the future proofing excuse, AMD recently stated these first-generation Ryzen chips were “the worst-case scenario” considering the new architecture on a new process node. Future chips will only have higher clocks at lower voltages, meaning an adequate VRM such as those on the top B350 boards would do just fine. Also, most VRMs on X370 boards are already known to be overkill for current Ryzen chips. Personally, I rank the beefier VRM reasoning as the weakest one for an expensive motherboard.

Continuing our look at some great B350 boards, they have the same awesome ALC1220 audio codec as their X370 counterparts, solid build quality, BIOS recovery facilities like Dual BIOS and some other goodies like the diagnostic LEDs on the Gigabyte boards, and they cost a $100 less than their equivalent X370 boards. Talking of I/O, since many fear the lack of adequate SATA and USB ports, speaking from my experience my Gaming 3 board comes with 7 USB ports, 4 of them are taken up by the keyboard, mouse, speaker and the USB WiFi dongle. That still leaves me with 3 USB3 ports. On top of this, I get an extra USB 3.0 port along with a couple USB2.0 ports for the front from the motherboard, and simply use them for any external device I connect. Point being, I still have adequate empty slots. The board I have comes with 6 SATA ports. Ryzen itself provides for 2 of these, and I believe you lose them should you attach an M.2 SATA drive. Considering the miniscule real-world difference of using a NVMe drive which we’ll also speak of soon, I’d say 6 ports is really overkill for 99% of the PC users out there. Granted my use case and yours would differ, but I doubt by very much even with high end gaming builds.

Let’s talk of multiple GPUs. The only cards you should consider a SLI setup with is the 1080Ti, that too if you have a lot of money AND are Enthusiast Number 1. Anything lesser, and you are universally advised to get a single, more powerful card. Not only is SLI support lacking, but with Nvidia having 70% of market share and game devs still not supporting it well, Crossfire really is worth even less of your time and money. For 1080Tis in SLI, Ryzen anyways provides for only a single x16 PCI-E 3.0 lane, splitting it into a x8/x8 configuration for multiple GPUs. This may hold back the 1080Tis, the only cards remotely capable of saturating this setup, if they are used in a game with great SLI support at very high resolutions. I doubt that’s your use case, if it is, get a Threadripper and take advantage of its 64 lanes. For the rest of us, save the money, get a better GPU, and listen to that advice that’s so universally accepted out there: stick with a single, powerful GPU. And so goes the SLI advantage of the X370.

You’ll likely have a similar situation with Intel boards, I’d just like to conclude by saying it’s hardly ever necessary to spend near $200 on a motherboard, when so many in the $90-$130 range are built so well and packed with essential features. Visit the manufacturers websites and do your own homework before you spend the extra amount just for the sake of it.

Considerations for CPU cooling:

Another topic that gets a lot of attention, only some of which is justified according to me, is CPU cooling. Yes, it is important to keep the chip cool, but not as cool as possible. Why? Simply because there’s no need to spend the money there. Consider this, my R7 1700 at 3.7GHz reached 85 degrees during a Prime95 SmallFFT torture test while using its default 95W Spire cooler. That is indeed high, yet 10 degrees lower than the 95 degrees limit AMD prescribes for it, but here’s the kicker: it gets that hot in a scenario very unrepresentative of a regular use case. While gaming, it doesn’t exceed 65 degrees, a full 30 degrees below its 95 degree limit, even with the AORUS 1080Ti in my system radiating it’s heat upwards and towards it over prolonged gaming sessions. Yes, a better air cooler would knock it down by maybe 10–12 degrees, a water cooler by 15–25. But, why? The chip is far below it’s 95 degrees limit, and I cannot hear the fan. A loud fan is indeed a good reason to get a better cooler, if the noise from your fan bothers you, by all means go ahead and get one. If however the sound from the fan is not at the point of bothering you yet, save that money. The sound from the Wraith cooler on my older Phenom II X6 1090T really bothered me, and I simply solved it by swapping out that fan for a better one; a much cheaper, faster and easier solution than changing the whole thing!

If you really like the idea of an AIO cooler, then go ahead, but I urge you to consider a 140mm radiator to save some dough. Yes 280mm is better, but a 140mm radiator is already as good or slightly better than most good air coolers, and that’s really perfect enough. But do consider that you’re adding another part to your rig that is candidate for an RMA: the pumps do fail, sometimes sooner than later, and worse: liquid leaks. Maybe it’s not as common, but it still happens. Also consider neither NZXT nor corsair are known for world class customer service when you think of that sexy Kraken or H100i. Just be sure of your decision and think of it from many angles, is all I’d like to reiterate. Personally, I wouldn’t be able to sleep at night thinking of any CPU water cooling equipment, the Kraken or the H100i or whatever lingering right over my 1080Ti, but that’s just me. You may differ.

One last thing: you may think you’re keeping your room cool in the summer by gaming with the CPU at just 40 degrees Celsius thanks to your 360mm radiator, but keep in mind that heat has got to go somewhere, and a better CPU or any component cooler is only considered better for its ability to dump more heat from the component that it is cooling into its surrounding environment. You may be running your chip cooler at the cost of a warmer zone around your PC, all for no added benefit to your CPU that’s already well below its thermal limit!

Considerations in choosing a power supply:

This is the one component I advise you to splurge on. Get a great PSU, after all, it’s going to be delivering power to your components, do you really want to skimp on that?! As for wattage, you must decide based on your use case. If this is a rig you may even have a remote probability of upgrading in the future, get at least a good 650W PSU. That should be enough for a high-end processor and a 1080Ti along with a good number of HDDs and SSDs. Were possible, getting a 700W or 750W PSU would not hurt so as to keep a little extra breathing room, though it may not be immediately essential. If you feel you may SLI or Crossfire in the future and have a rig with some watercooling and several drives, please do look at a 1200W PSU. If you’re beyond certain of a zero possibility of upgrading and are building a moderate system, a 550W PSU should suffice: but never skimp on quality. If there is any chance that you may upgrade, think how much it would suck to have to buy a new PSU again at that time. Invest in this one, it’s a long-term call.

There are also PSUs with digital interfaces that let you monitor things via software, I do not think that’s very useful, but if you like the idea, get one. One thing: the PSUs by Corsair that have the digital interface have one feature over the standard ones I like: a PSU fan test switch. With many PSUs having a silent operation mode and keeping their fans off when the power draw is below a certain amount (350W on my 750W RMx, I believe), the fan stays off and sometimes one gets concerned that the fan is at least operational. A fan test switch is indeed useful here. Corsair has the RMi and HXi series, with the ‘i’ demarcating the digital interface. Other PSU brands will have something similar. Check on it.

Considerations for storage:

This one is relatively simpler, yet I messed up a bit here. Definitely get a 250GB SSD for the OS. I find this capacity adequate. Remember, I also use my system for work, and have several IDEs and Dev tools such as Visual Studio with several SDKs and even Unreal Engine installed on my C drive, along with Ghost Recon Wildlands, Elite Dangerous and Far Cry 4, and I still have 40GB free. Yes, I keep all documents, music and videos on the mass storage hard drive and not on the C drive, and if you aren’t already, get with the program and move your files off the C drive ASAP.

Where I messed up is the slightly small 275GB game drive SSD. Large games with huge textures benefit immensely from an SSD, and having a SSD to shove those games that should be on an SSD on there is a very worthy investment. It would be ideal for me to have a 1TB SSD for them and have them all in one place, this makes Windows reinstalls a lot easier as all the games are on one drive and need minimal setup. Alas, that’s out of my budget for now and the 275GB drive is simply a stop gap arrangement. Based on the games that you play and their size, get a separate SSD for them as and when you can, it makes maintaining them far easier, especially when you are in that situation wherein a fresh Windows installation must be done. Use a large HDD for mass storage of documents and older games.

As for NVMe? I recently had a chat with a game developer who worked on Far Cry Primal and For Honor back when he was a Game Dev for Ubisoft and he’s currently running a Threadripper 1950X and two EVGA FTW3 1080Tis in SLI on his private development rig which has no NVMe drives. He said the 0.5 second savings in load times is not worth it to him. Here’s a dude with no shortage of reasons and money for a fast rig of that stature, and he didn’t need a NVMe drive. I doubt most of us do either. Save the dough, unless you need the space savings or are getting one for very near the price of a regular SSD, just get the regular SSD. Spend the money on a better GPU and PSU, or go buy a gym membership and a box of whey protein and stop obsessively spending all this time here!

Considerations in choosing RAM:

16GB is regarded as a good amount today, more than adequate for today’s stuff and enough legroom for tomorrows awesomeness. If you cannot currently afford that much, get 8GB. Most processors run dual channel memory, and so for budget builds many will advise a 4GB x 2 configuration. I advise against this. Why? Maybe you can’t get 16GB now, but it will be essential soon and maybe you can add a second 8GB stick 6 months later. Keep it as future proof as possible, why get around to buying another 2 4GB sticks later and struggle if you encounter issues with this setup? You’ll encounter no initial issues with the single channel setup, and in the future when you get a dual channel setup, you won’t encounter any issues then either.

Do check compatibility well. Ryzen is known to really favor Samsung B-die, this has been confirmed by AMD themselves. Other chips may have similar preferences. Google memory support on your platform of choice before pulling the trigger to be sure. I got lucky and my LPX runs on XMP with no issues and even passed a night of MemTest. But don’t rely on chance and do a little research on this, you’ll save yourself of a colossal headache with memory that creates all sorts of instabilities and prevents a boot in the worst case.

Considerations in choosing a case:

Go with what you need. Talk to yourself about what exactly are your requirements. I choose the S340 Elite because it’s the best looking compact ATX case, and there’s fair chance I travel around and I would hate to have to check-in this rig on flights. With a case as compact as this, I can just take it as carry-on baggage. I spent a ton of time sitting on manufactures websites with a scale in my hand pouring over the dimensions of these cases as mentioned on their specification sheets. See what your requirements are and invest accordingly. Again, it would suck to have to buy a new case at a later date and to reassemble.

Maybe you hope to move onto Threadripper or X299 in the future. Then you’ll need a case that can support the Extended ATX (E-ATX) boards those platforms require. Assess your requirements thoroughly and invest once and invest well.

Considerations in choosing a graphics card:

Yes, finally. I know many of you were waiting on this one. So there’s three ways I’d recommend to go about this: firstly, get the fastest card you can currently afford. But what if it’s not that great a card/you desire more? What if you can “only” get a GTX 1080 now but you know a couple months later can get a 1080Ti? That gets us to my second approach: if you need a working computer right away, get the cheapest card you can find. Maybe a reliable secondhand HD 7850 for $50. Maybe a GTX 1050 for $65 or a RX 550 or whatever. And hold it for a few months before getting a top of the line card.

But what if you don’t need a 1080Ti now, are sure you will not need it in the near future either and can get a 1060 or RX 570 or something and are completely okay with that? Maybe you are certain of sticking to 1080p@60Hz. Then just get the best you can get, and be happy. Just make sure you’re really certain. It’s easy to convince yourself that you are content with your parts at the time you’re paying for them, but it is far easier to start craving for more once the gaming begins. Don’t let that be you. Be sure of your requirements, if you’re keen on jumping onto 144Hz or 1440p or even 1440p 144Hz or 4k but can’t afford a 1080 or 1080Ti now, maybe you can plan to spend on that a couple months down the line. It’s better to sacrifice today and get a small, cute little 1050 only to make a really nice jump a short bit later.

This is also what I did. I bought a reference 4GB RX480 back in March, before the mining craze. Got it brand new from Newegg with Doom free and a $30 rebate for a final price of $160. Man, those were good times for GPUs…seems a long time back now. I sold that RX 480 slightly after the mining craze and moved on to this GTX 1080Ti. Why didn’t I sell earlier? I was convinced on RX Vega and so blinded by it, I didn’t even consider this move. So there is my final and most important point WRT GPU purchases: never, ever make plans on the basis of future releases. Even if the product is good, you may not get your hands on it at MSRP for months. If you can wait that out with your current build, then by all means do so. Otherwise, just get the best deal out there on the move you plan to make and get on with your life.

When choosing between various high-end parts, just get the best priced one. I picked the AORUS 1080Ti because it was the cheapest at the time, I also knew it’s copper baseplate cooling was superior to many others, but I didn’t really bother verifying this. Price ruled. What difference are you expecting between an AORUS 1080Ti, and a MSI Gaming X or an ASUS Strix? They’re aftermarket cards from good brands and are definitely better than the Founders Edition, that’s really all that matters. If the Strix was cheaper, I’d have got that even though I liked the AORUS card’s look more. Doesn’t matter. These cards are also incredibly fast and overclocking them really won’t be worth your time and thought. Just stick with a good brand with good after sales support and get the cheapest amongst them without worry. Really think 20MHz or even 50MHz on a 1080Ti will matter? Nvidia boost automatically takes my card up to 1984MHz, though Gigabyte claims just 1600MHz something on their website. All other cards will be the same, ditto for any other chip. As long as you’re getting one with a good AIB custom cooler, just get the best priced model and move on.

Considerations when choosing a monitor:

Now this is a touchy topic. You may be confused with all the terminology out there, I was too. First thing first, for a gaming setup, I absolutely recommend a Freesync or a G-Sync panel depending on your graphics card being an AMD or a Nvidia card. Either ways, I absolutely recommend active refresh, and not for the screen tearing which didn’t personally bother me as much, though again you may be different, but instead for the removal of stutter from those times your framerate falls below the refresh rate of the monitor and for extending the life of your rig by providing a smooth gaming experience for a longer time. Let me share my own experience:

I initially had a RX 480 in my system with a 1080p 60Hz monitor. With a good mix of high and very-high settings coupled with anti-aliasing, it rarely ever kept to a smooth 60 FPS. I hated the stutter, and soon switched to my current ultrawide 75Hz monitor. Aware of the increase in resolution, I was certain I’d have to slash settings down even further on my system, but thanks to Freesync, I never ended up having to care. I did not reduce my quality settings in a single game, GTA V or Witcher 3 or Elite Dangerous as framerates down to low 50s felt butter smooth. Following the Vega fiasco, I ended up with a 1080Ti. Now this card is amazing, no doubt at all. Where the RX 480 could not even run GTA V at a smooth 60 FPS with no MSAA and medium to high settings, I get a smooth 75FPS with 4xMSAA and every setting maxed other than grass and the advanced stuff. I get the 140FPS I should in the GTA V benchmark, and 95FPS on the Ghost Recon benchmark with Very High settings. But let me tell you something, there are still times when the framerate drops. Not very frequently, but not so infrequently that I can just dismiss it. Sometimes, you’ll end up in that fucked up scenario in GTA V where you’re downtown and it is nighttime and there’s tons of reflection and shadow and a lot of cars and people and it rains and there’s fog and my framerate actually tanks to 55–60 FPS. In Ghost Recon, keeping Turf Effects on along with ultra shadows and vegetation leads to a smooth 60 FPS, and yet a good number of times it will drop to 45–50. The stutter sucks. I hate it. Enough to have scouted out a G-Sync ultrawide somehow. Look, the benchmarks you see in card reviews are great, and I too get those figures and the 100 FPS on Witcher 3 at ultra with 4x Hairworks and the 28000 points on Fire Strike, but real-life gaming is not represented well by these benchmarks. There will be moments that bring even really powerful systems down, and the bigger problem: not all games are optimized well. Case in point, Far Cry 4 hates Ryzen in my experience, and Firewatch and Euro Truck Sim 2 aren’t the best optimized games either. No matter how fast your processor with how many cores and how expensive your GPU, not every game will run at those incredible framerates, at least not all the time, and not even the ones that benchmark really well. Active refresh technologies like G-Sync and Freesync are a very essential part of a gaming rig in my humble opinion.

Now you may have heard of LFC, or Low Framerate Compensation on Freesync monitors. Yes, this is another worthy feature. What does it do? Briefly put, if there are times your framerate falls really low, below the minimum boundary of the Freesync or G-Sync range, it’ll automatically display each frame twice so as effectively double the framerate and thus push it back into the Free/G sync range. So 20 FPS moments turn to 40 FPS. Again, a great feature to have. While only some Freesync panels have this, all G-Sync panels do. Keep in mind the new Freesync 2 certification from AMD requires all Freesync 2 panels to have this feature. G-Sync panels also have a feature called Ultra Low Motion Blur, or ULMB, which aims to combat the excessive blurring you may encounter when the framerates and refresh rates are so high that the monitors response time slows down to several milliseconds. Again, briefly put, it aims to strobe the backlight in sync with the refresh rate, but here’s the caveat: G-Sync must be off for it to take effect, and its generally effective at the 85+ FPS range. As G-Sync is often a better feature to have enabled, don’t overpay for a monitor that claims to have a superior implementation of it.

Now, for the very touchy topic of pixel density, or more so, the lack of it. Many will claim they absolutely need ‘x’ dpi, but really most of us will not notice. I’ve gamed on PS4’s hooked to 1080p 50” displays that looked fantastic, and using a 2560x1080 34” ultrawide has been a gorgeous experience. Yes, the extra screen space obtained for productivity from a 3440x1440 panel may be nicer, but you know what’s better? Not having extra space in my wallet due the $400-$600 extra I didn’t spend on the low refresh rate 1440p ultrawide panel. I was almost sold on them, and was reading reviews for the Acer X34, when on the conclusion page despite awarding it the “Editor Recommended” batch, Tom’s hardware felt compelled to mention how you shouldn’t dismiss the Z35 just for its lower resolution as its brilliant panel makes this a non-issue. Heading over to the conclusion page for the Z35 review, they call it “the most beautiful display that has graced our labs”. Now I doubt Acer would pay to show the Z35 over the X34, I believe they’re being honest, and if they can feel that after testing all those displays, you should reconsider your worry on “pixel density”. Nonetheless, I did manage to check them both out in person and was really glad Tom’s said that as they were absolutely right: I found no reason to pay that much extra from the X34 1440p display. I program, work and game on a 2560x1080 display, and am only thrilled each time. Don’t believe people on the internet who profess their necessity for high pixel densities and thank God for having spent on it: it’s the internet, there’s a very good chance they’re either just trying to get themselves to feel better for having spent that extra amount or are image/video professionals who for some reason feel the need to state why they need it when it’s obvious they do. Some will be nice and honest too. 34 inches is not “too much” for 2560x1080, but those many extra hundreds of dollars might be too much for you. Think twice and try to check them out in person always.

Also, an added bonus, many 2560x1080 panels including the Z35 have a very high refresh rate along the lines of 165–200 Hz. So, the 1080Ti might be rendering to a slightly lower resolution screen but really gets to stretch its legs with that refresh range on a G-Sync panel. Also, the 1080Ti just got more future proof thanks to this. See why I say don’t dismiss things easy?!

Lastly, would I recommend an ultrawide display? Absolutely. Again, a game changing and exhilarating experience not only the first time you play, but each time. And full screen modern movies. What’s not to love?

All this finally leads us to our conclusions:

Conclusions:

1. Buy the fasted factory-clocked chip you can, overclock only when necessary, or if you have the time and patience to learn to do it right. You can also spend on a CPU in stages, say if you’re building a system today on a limited budget and hope to upgrade in the future, get a good motherboard and PSU and Ryzen 1400 or 1500X. That’s adequate for now, and then around 18 months later when 2nd gen Ryzen is out, get the top notch 6 or 8 core model then. With intel, your choices in this regard may be limited as they always have so many motherboard sockets. Maybe you can get an i3/i5 now and get an i7 from the same generation later on from the secondhand market.

2. With motherboards, more expensive is hardly ever necessary. Look for a solid board in the $90-$130 range, it’ll probably have more than enough I/O, connectivity and PCI-E lanes for you. Then again, if going the X299 or X399 route, go all out and get the beefiest, fanciest board you can: you’re not upgrading for ages and no point getting onto the HEDT bandwagon and compromising even a single feature thanks to the board. And surely, going this route, you can afford it!

3. Check RAM compatibility on your platform. If not going with 16GB today, get a single 8GB stick and upgrade by adding another one later so as to get them on dual channel mode. Don’t think you’re going to get a world of difference by splurging on 3600MHz memory over 2600MHz one. The real-world performance difference is negligible, if you’re lucky, you may see a couple extra frames. Keep in mind with Ryzen, any RAM speed above 3200MHz depends on silicon lottery as RAM speeds directly correlates to the speed of the Infinity Fabric in the chip. You may splurge to get the highest speed RAM kit but have no luck running it.

4. Splurge on a solid PSU. If you’re definitely not upgrading, just get what you need for your current build and be done with it. Otherwise, get as much breathing space as possible. If you may go with SLI/Crossfire in the future and may have watercooling pumps and radiators and several drives, get at least a 1200W PSU.

5. A 250GB SSD for the OS and applications should be more than adequate. Try to get a separate SSD for the games, and be sure to add a mass storage hard drive.

6. Get a case that suits your needs. If you’ll be travelling, get a compact one. If you’ll be upgrading to an E-ATX based platform like Threadripper or X299, get a case that’ll accommodate them when it’s time.

7. Don’t get talked into the pixel density argument WRT monitors. Try them out yourself, but don’t over splurge beyond your means. Definitely try your best to score Free/G sync, and do consider ultrawide displays. A 29-inch ultrawide is as tall as a 24” 16:9 monitor and a 34” one is as tall as a 27” 16:9 display. Keep that in mind when choosing, and try to get a larger one, especially if you’re paying extra for G-Sync, as you may not upgrade soon.

8. Don’t obsess with running your CPU as cool as possible, it’s unnecessary as long as it’s running well within thermal range and keep in mind that all that excess CPU cooling will just dump more heat into the surrounding, thus creating a warmer atmosphere for you around your PC for no added benefit to the chip. Think also of the higher risk of pump failure and liquid leakage and the RMA hassles before you invest in watercooling. With custom cooling loops, think of the fuss each time you need to move anything inside, what with draining the reservoir and everything, do your research beforehand and be sure you’re ready for all this.

9. Lastly, along with Googling your doubts and asking them here and on other forums, spend time on the manufactures websites. They often have great comparison tools and detailed specs. Make use of that and your own reasoning, don’t blinded accept anything you’re told without researching this way yourself, and putting your own thought into it.

Some general things to keep in mind when building:

1. If your PSU has those daisy-chain 8 pin connectors for the GPU in a (6+2) + (6+2) config, for high powered cards that have two 8-pin power slots, use two different cables instead of daisy chaining. Remember the following:

6 pin = 75 watts

6+2 = 8 pin = 150 watts

So using:

8 pin is okay

6+2 is okay

6+6 to a single 8 pin is okay

But,

8+8 in one cable via daisy chaining is not

6+2 + 6+2 in one cable via daisy chaining is not and

6+2 + 6 via daisy chaining is not

2. Be sure to plug in RAM in the right slots. In my motherboard, the RAM acted very funny and never enabled XMP when I’d accidently left them in the black slots numbered 2 and 4. I moved them to the red slots numbered 1 and 3 and all has been good since. Your board may have such a need too. Check the manual.

3. Run MemTest once overnight on newly installed RAM with XMP enabled.

4. Don’t forget to enable XMP in the BIOS! This has to be done manually!

5. Enabling a global frame rate limit in Radeon Settings or the Nvidia Control Panel when you’re running Free/G sync may prevent some games from running in full screen mode. I had this problem when I had used Radeon settings and set the Global Frame Rate Target Control setting to 74 FPS, it made the display hang when trying full screen with Elite Dangerous and Far Cry 4. Disabling this setting fixed it, this was an accidental discovery I made that had me scratching my head for a long time wondering what’s wrong and I do hope this helps someone in a similar situation out.

6. Lastly, when enabling a frame rate limit such as above, set it to one below the max refresh rate of the monitor, so 143 FPS for a 144Hz panel.

And with that we’re done with this humongous post, congratulations and thank you for having gone through it! I hope this helped you in some way, and do feel free to reach out with any questions/suggestions you may have. Wish you the very best ahead!

A Guide to PC Building: Some Advice From My Experience Over The Years was originally published in High Tech Accessible on Medium, where people are continuing the conversation by highlighting and responding to this story.