This article was first published and is available free of charge and any form of tracking here: https://cloudhpc.news/dreams-and-wishes-for-hpc-in-2025/

I was going to write a piece about what to expect in 2025 in the world of HPC but then I realised my crystal ball is no better than yours. In fact, looking at my investment history, I’d say it’s a fair bet to say it’s worse 😆

I could regale you with fairy tales of what I think will happen in 2025 but all that does is give you ammunition to declare “You were wrong, I informed you thusly” later this year. Plus, as the finance types among you know all too well, “Markets can remain irrational longer than you can remain solvent” and that applies equally well to time boxed predictions. I’m looking at you AI hype.

So instead, I’m going to share some of my wishes for higher performance and not so high performance compute instead. Like most wishes they probably won’t come true, but if they did, I think we’d be in a better place. Maybe, just maybe, it might even inspire one or two of you to help make them reality.

Break out of the Niche

Several times in 2024 I asked the question, “What is HPC?”. I attempted to answer it a couple of times too. I wasn’t just being obstinate. Rather I’m trying to make a point.

At its inception, HPC was special. It was something needed by a very small number of people and institutions around the world. It made sense that it was something niche. After all, how many people really needed the ability to string together commoditised hardware to obtain teraflops of compute. A few scientists, a couple of engineers, some banks and that was your lot.

Let me ask another question though. What do HPC engineers do? At its most basic level they take compute that no longer fits onto a single machine, optimize the hell out of it and then run it across thousands of computers. In case you hadn’t noticed half the world seems to need to do that right now.

My first wish for 2025 is for HPC folk to realise this. To step up and out of their niche and use that ability to help solve the same problem in multiple domains that see HPC as a dinosaur and don’t have the first inkling of an idea that we have the solutions to their problems.

The whole world is slowly realising that even if they could buy all the compute they need, we simply don’t have the electricity to run all of it! We are going to have to do more with less. Guess who’s been doing that for decades already?

Power constrained architectures are nothing new in the mobile space but we’re now having to deal with that reality in the data centre. Arm and AMD’s relentless improvements in power efficiency in CPUs doesn’t mean diddly squat when NVIDIA’s GPUs are eating every last electron they can find.

HPC doesn’t have to be a niche offering in the clutches of select few. Let’s Think bigger.

Stop Reinventing the Wheel

Following on from the above, because other non HPC industries are either unaware, alienated or just repulsed by HPC there are a plethora of solutions in the same space being reinvented.

Don’t get me wrong. Choice is a good thing. I’d love to have multiple fantastic solutions to pick from. But I don’t.

Instead, I have a smorgasbord of poorly documented products that do half of what I need. In a particularly user unfriendly fashion. Honestly, I’d give that up for a handful of polished, products with great documentation and community support.

I know I’m being rather critical here but if you measure most HPC products against almost any recent popular software offering, whether that’s developer or user focused, the level of polish in anything HPC is far cry from what we generally expect from modern software.

Go From “It Kind of Works” to “It Just Works”

Something I’ve said to a few people in conversations during 2024 is that HPC needs its “Apple” moment. It needs to go from having functional products that require a PhD to operate to having products that are intuitive, with sensible defaults that are easy to adopt.

If the goal is to achieve wider adoption of HPC solutions to today’s problems those solutions are going to have to be a lot more usable.

Explode the Monolith

A good scheduler may well be the cornerstone of HPC but it’s about time we realised it’s not the only foundation we need. Heck let’s be honest shall we, most HPC isn’t even compute constrained anymore. That’s a dirty little secret we tend to keep to ourselves, and whisper is hushed voices when sipping coffee around the liquid cooled supercomputer.

Isn’t it time we acknowledged it. Isn’t it time we admitted that a data plane is just as important. A node, rack, zone and region aware data plane. And a hardware control plane. That works for cloud, on prem or colo.

But critically I don’t want my data plane to be part of my scheduler. Or my hardware control. No, I want an open standard to allow them to communicate.

As anyone that has attempted to swap schedulers in a HPC system will tell you, the API changes are trivial. The problems come because the choice of scheduler inevitably leaves its mark on the architecture of the rest of the system. The mark of the beast. Can we be done with that already?

I want my scheduler to schedule and be bloody good it at but do no more. I want it to talk to my data plane to figure out where data is to make those scheduling decisions. I want it to talk to my hardware control plane to spin up/down compute as needed.

If we ever have a hope of using ML/AI in HPC infrastructure we need to start by exposing the data needed to train it (more on this later).

Embrace Diversity

To those proclaiming the death of x86, I’d like to remind you that while we may already be calling the death of Intel, the x86_64 architecture was produced by more than just Intel. Not only that, but AMD’s latest (aptly named) EPYC CPUs are more power efficient (when performance adjusted) than their Arm based counterparts.

That’s not a reason to sit back that assume x86_64 is the only game in town. Sure, HPC clusters used to be comprised of dedicated homogenous hardware. Those days are gone. Get used to it. If you want cost-effective, large-scale compute it future you’re looking at a heterogenous compute environment that comprises multiple CPU architectures including Arm, x86_64, RISC_V as well as multiple accelerators including GPGPUs, FPGAs and ASICs.

That leaves us with another problem though. You can bet your compute allocation that scientists researching new cancer treatments (or the engineers trying to model new spacecraft or traders managing their risk) do not give a FLOP which flavour of silicon was used.

The HPC infrastructure needs to remove that complexity. We already have the technology needed to automatically select the most appropriate hardware based on any number of criteria including cost, performance and green credentials. It’s about time to spin that together into something more accessible by the masses.

HPC needs to embrace the DEI agenda and I don’t mean the carbon but rather the silicon based life form version.

HPC & AI

HPC and AI have this really odd incestuous love hate relationship. HPC was necessary for the initial training of modern LLMs. Speak to most AI startups though and they probably won’t know what HPC even stands for.

While LLMs might be attempting to replace traditional HPC calculations in some domains, what I’ve always been interested in using AI (well more strictly just ML) for is to improve the scheduler. I’ve wanted to do that since about the time Google released TensorFlow. Writing a scheduler to take advantage of ML isn’t that hard. What is hard is getting the HPC application that uses the scheduler to expose enough information with each task to be able to train anything.

This is slowly changing though and if some of the ideas above come to fruition then we stand half a chance of having AI optimized HP.

If Terminator or HAL 2000 style AI scenarios already scare you, wait till we can have AI train new models. At that point the current LLM doesn’t need to be able to take over the world, it just needs to be able to train up a new SkyNet.

Dreams and Wishes for HPC in 2025 was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article was first published and is available free of charge and any form of tracking here: https://cloudhpc.news/ai-are-we-confusing-intelligence-with-eloquence/

Is ChatGPT actually smart? Or is it just playing us with its good looks and way with words?

This last week I’ve been a little sick causing my ability to speak to range from completely mute to a speak-n-spell monotone coupled with short words and long pauses. It was genuinely scary how robotic I sounded at times. I cancelled most of my meetings naturally, but I do wonder how smart the odd few people I did have to meet (such as the doctor) thought I was. Scratch that, I was judging my own verbal output to be on par with a brick. There’s definitely no risk that anyone meeting me in that state was about to think I’m smart.

There’s a fairly well established halo effect that leads us to believe that people who are attractive or eloquent and articulate are also intelligent (one example). This led me to wonder, well what about AI?

Do we think AI is smart just because it is well spoken? Humans also have a tendency to project and anthropomorphise. For example, we will often refer to cars as having character or a soul (I mean dinosaur juice powered cars of course not those the electron powered kitchen appliances with wheels). While the default language output of ChatGPT might be fairly corporate vanilla (real Madagascan vanilla I grant you) it is still probably more articulate than the majority of the English speaking population.

I’m sure this is no accident and the smart folk at OpenAI, Anthropic and Google et al spent a long time tuning the output to get to this point. What if it wasn’t though? Google search has (or at least used to have) the same wealth of knowledge that ChatGPT does, we never attributed any kind of intelligence to that. How smart would we think ChatGPT is if its output sounded more like Joey from Friends instead?

LLMs are fantastic word calculators. They have no understanding of the output they generate. When you read reports like the recent spate about AI’s trying to escape, you’d do well to remember that. And maybe ask the question, who benefits if people believe the lie?

Not convinced? This is from part of the ChatGPT conversation to generate the header image

That was the fourth attempt to generate it too. I gave up and went back to Photoshop…

AI: Are we Confusing Intelligence with Eloquence? was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article was first published and is available free of charge and any form of tracking here: https://cloudhpc.news/how-good-is-gpt-o1-really/

Open AI ChatGTP o1: 0 — Hamza: 1

Relax your job is still safe.

I’ve been using a paid subscription to OpenAI’s ChatGPT for some time now, you may have noticed the AI generated images that often accompany my ramblings. That’s not all I use it for though and with the release of the o1 model I was curious to see how much better, if at all, its gotten.

You see dear readers; you have been an unwitting participant in a little experiment by myself this week. Human experimentation! How very Sheldonesque of me, making ol Sam Altman look good right here. I’m sorry. I promise no humans or AIs were hurt in the writing of this post.

Whilst I’ve been using ChatGPT to generate images, and sometimes code for quite a while, till this week I had never once posted an AI written article. Not for any moral or ethical reasons. No. It’s far simpler than that. For my domain at least (HPC) I always found them to be well… crap.

But perhaps that was just my own ego. So pitted Sam Altman’s latest and greatest model against my weakest skill, writing posts like this. (I’m pretty sure I write better code than I write blog posts). Earlier this week I picked a topic that has never gained much traction, and I wrote another post on it. I asked ChatGPT to write one too (same topic), asked it nicely to refine it, then to make it fit in LinkedIn’s word limit. Sheesh for a so called intelligence it doesn’t half need a lot of direction and coaxing.

I then generated two very similar images to go with both and instead of my usual single post I scheduled two to go live at the same time. For those of you that engaged with both (there were a small number) and were wondering if I’d lost the plot, well now you know.

The results after 2 days.

ChatGPT: 219 impressions, 5 likes and 2 reposts, 0 comments

Me: 885 impressions, 17 likes, 7 comments.

Suck it ChatGPT. The reposts on the ChatGPT version? From HMx Labs employees (they didn’t know about the experiment either)

Which one did you prefer and why?

EDIT: I went to generate the image to go with this post after having written it. This was the first version it gave me :lol

This is a great example of how ChatGPT has no understanding of the words that it is fed. All of the words are in the image but it swapped the victor. I come across this sort of thing surprisingly often.

Now I could start projecting and anthropomorphising ChatGPT and saying it doesn’t like to lose but that would ascribe consciousness and intent to a machine that has neither. It’s just dumb. The old adage of “do not ascribe to malice that which can be explained by incompetence” applies equally well to AI.

How Good is GPT o1 really? was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Let’s talk silicon. With everything that’s happened in this space just this week how can we not?

This article was first published and is available free of charge and any form of tracking here: https://cloudhpc.news/heterogenous-or-homogenous-hpc/

In just this week we’ve had news

• about Intel CEO Pat Gelsinger being forced out.
• during re:Invent AWS have announced that more than 50% of the capacity added in the last two years was using AWS Graviton.
• They have also backed Tenstorrent to the tune of $2.6Billion for its next generation silicon using RISC-V.
• Further, Apple has come out admitting it uses AWS including for its AI workloads
• Canada has become the latest country to announce plans for sovereign AI (and therefore compute).

And that’s just this week.

During the course of this year we’ve seen Google launch its own ARM powered silicon in the form of Axion and NVIDIA is bringing us the GB200 superchip.

In the consumer space we’ve obviously had Apple Silicon seriously disrupt x86’s dominance in end user computing in much the same way ARM chips previously have for mobile devices.

More and more we are looking at an increasingly heterogenous future for our compute needs. Not only does that mean a mixture of CPU and GPU but also a mixture of architectures in the CPU in the wild.

To my mind there are a few macro tecno-economic factors driving this change. Not least of which is the insane compute demands of the current AI hype cycle. Anyone following my writing will already know I’m not a believer when (and not if) we see that hype train fall into the trough of disillusionment much of that demand will vaporise. There will remain however two other important factors that will continue to drive diversity in the CPU space.

The first of these is the CHIPs Act and increasing global geopolitical tension. Since the USA relaxed export controls around encryption technology in 1999 we’ve essentially been in an era where the world’s technology has been provided mostly by one country, and in reality, the CPUs by one company, Intel. The CHIPs act has put an end to that. Not only are countries looking at investing in sovereign AI capabilities, but that also often comes with sovereign silicon capability to power that AI and by extension, other compute. If we continue down the path of increased export controls we are likely to see an increase in development of alternative silicon. Even if other countries and companies are not successful in developing competitive substitutes, the export controls themselves will transform the way we see compute. It will become even more heterogenous across regions and global corporations will have to distribute compute based on a complex algorithm with data residency legality and performance requirements as inputs (more than we already do today).

The second is the slow transition of compute to commodity status and the expected behaviour from the new cloud utility companies to counter this and retain their profit margins for as long as possible. As we’ve seen in other industries in the past, cloud utility providers will attempt increasingly to differentiate their offerings to entice customers and increase switching costs. Custom silicon is a natural extension to this and as such the large proportion of AWS’ new capacity being Graviton and their investment in Tenstorrent should come as no surprise.

So, what does all this mean for HPC? I think there are two possible paths, and we may follow them both.

Traditionally HPC workloads have been run on large homogenous (on prem) clusters. The typical approach has been to purchase a large quantity of silicon, all the same shape and flavour, and deploy this with a scheduler capable of distributing workload across it all. More recently though we’ve seen green shoots of heterogenous HPC (including the use of cloud) where multiple types of compute may be used.

We will either see this trend drastically accelerate and embrace highly heterogenous HPC or we will see a re-entrenchment back to owning and controlling a homogenous set of hardware.

The path we follow here will depend greatly on the how the scheduler industry steps up (or not). Quite frankly your average scientist, engineer or quant doesn’t care what kind of silicon their workload ran on. They just want it done as fast as possible. If we are able to abstract away that complexity, if we are able to produce portable workloads that can run across multiple architectures then we may well see the former use case.

On the other hand, if we remain stuck with primitive schedulers with limited visibility into their workload’s characteristics and old codebases capable of running only on a single architecture then you can guess where we will end up.

Heterogenous or Homogenous HPC? was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article was first published and is available free or charge and any form of tracking here: https://cloudhpc.news/cloud-is-a-utility/

[Extract from HPC Club 27Nov24]

I’ve been saying for some time, probably since 2016, that cloud providers will just become the next utility companies of our time. Well, I think it’s already happened.

Don’t believe me? Take a look at the state of GPU compute today. The massive demand from AI peddlers has accelerated not only the heat death of the universe but also the transition of cloud providers from bleeding edge innovators to utility companies. We still have the big three of course, but we have a rather large number of smaller providers snapping at their heels with much more competitive pricing too. AWS will not only let you reserve capacity in the future (GPUs and CPUs now in fact) but even give you a marketplace to resell capacity you no longer need!

It’s still early days though. A bit like when we couldn’t decide what an electrical plug should look like, if distribution should be AC or DC or at what voltage.

We still have multiple APIs to contend with. Moving workload from one cloud to another is hardly trivial. Moving it to your own hardware is usually even more fraught.

But this pattern isn’t anything new either. We’ve seen it play out in the past not only with electricity but also telephony, then mobile telephony and then mobile data. If you’re building enterprise systems with a shelf life of 30+ years rather than the latest SaaS product that will be canned or completely rebuilt within 2 years, then these macro technological trends are very important.

Think carefully about the portability of your workload, not only today but what that may look like in 3, 5 or 10 years. What will you do if you’re stuck on the equivalent of British Gas and Octopus would halve your bills? How hard is that switch today? In 5 years?

Cloud is a Utility was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article was first published and is available free or charge and any form of tracking here: https://cloudhpc.news/anatomy-of-a-risk-system/

Psst. Come closer. I’m going to tell you a secret from the world of financial services.

The first picture shows what anyone going into a cloud migration at an investment bank for the first time thinks what a risk system looks like. The second picture is what they’d draw after six months of missed spend targets, sleepless nights and face palms.

Ok it’s not really a secret. Anyone that’s worked in financial risk systems will tell you that, yet somehow it seems to get forgotten. Hope and cloud marketing always prove to be stronger than the reality of how long it can take to untangle complex enterprise systems.

The unfortunate trust however is that financial risk systems are horrifically complex aliens and migrating them anywhere least of all to cloud is a difficult endeavour even if you have a spaceship.

They’re not difficult to migrate because the math and code in calculating the risk is hard (though it is). They’re not difficult to migrate because they deal with enormous scale (though sometimes they do). They’re not even difficult to migrate because they’re written in highly complex tech stacks (quirky sometimes, complex not so much).

They/re complex due to handful of reasons that are often more to do with human behaviour and economics rather than anything technical. They’re difficult to migrate because like any large enterprise system they are a product not only of the technical choices of their creators but also of the org chart. They’re constrained by both technical and political decisions across the entire bank. And lately all of this gets done on a budget smaller than your average AI startup.

This is particularly problematic for a risk system due to the number of touch points or interfaces it has (both human and programmatic) with other parts of the bank. Slowly grown over a period of years these systems connect to multiple other applications in the bank but are usually the first to be targeted for cloud migration due their size and demand for compute. The result? A desperate call to Mario and Luigi to replumb everything to keep the risk flowing.

All this before anyone even says the word scheduler. Or orchestration.

Anatomy of a Risk System was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article was first published and is available free or charge and any form of tracking here: https://cloudhpc.news/intel-x-amd-and-hpc-in-financial-services/

If you work on financial risk systems, then last week’s announcement by Intel and AMD probably left you somehow both jumping for joy and scratching your head in some kind of HPC version of “pat your head and rub your belly”.

The sentiment in itself is great and sorely needed but the news left me with more questions than answers. I’m sure we will find out more in due course but in the meantime if anyone from Intel or AMD wants to share any insights (under NDA) I’m all ears.

Most of our banking clients are currently Intel only for two key reasons. Firstly, they use the Intel Math Kernel Library (MKL) which won’t necessarily play nicely with AMD processors. Secondly the results obtained between Intel and AMD processors are not the same and for various logistical reasons this makes it hard to impossible to use both at the same time.

I’d love to know if anything in the new x86 Ecosystem Advisory Group will help unify things to help resolve either or both of the above.

In any case I suspect it will give people enough pause for thought to consider what could happen with x86 that it stops them contemplating the herculean task of porting the code to run on ARM. At least till the next time AWS comes along, cheque in hand, proposing migrating to Graviton 🤣

Intel x AMD and HPC in Financial Services was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article was first published and is available free or charge and any form of tracking here: https://cloudhpc.news/the-copilot-pause-and-ai-generated-code/

How often do you suffer from Copilot Pause? Come on, be honest now. Share below for a bit of Friday fun and frolics.

I’ve been using GitHub Copilot since its release. Due to the nature of what I do, I don’t really get to specialize in a particular language or tech stack any more. I’m often helping someone else, switching from one project to another or one client to another.

Any polyglots on here? Do you find that you don’t really think in any one language anymore, but some weird combination or no language at all, just constructs and then ever so occasionally you need to turn that into a particular language, but that language doesn’t quite have the words to express what you want?

Well, I get that with computer programming languages too, except that while I don’t (generally) forget the syntax of real language, I will sometimes forget the syntax or function name in the programming language I happen to be working with at the time. Copilot is great in these scenarios and my main use for it. I don’t let it write any logic for me.

And I’ve found that’s all I really use it for. Most of the time when I’m writing code, I know exactly what I want it to do, on each line. I don’t need Copilot to do that for me. I do like that it reminds me how I express that in the particular programming language I’m working in today.

News this week then that Copilot doesn’t, overall, increase productivity and can actually increase the number of bugs in code wasn’t particularly surprising. (See comments for links).

I’ve seen a few posts from people choosing to wholesale abandon AI code tools too. This is a far cry from the general wisdom that was espoused a year ago that “AI will take your job” or at the very least another developer using AI will.

Overall, a bit more balance in how we use AI to create code is welcome!

The Copilot Pause and AI Generated Code was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article was first published and is available free or charge and any form of tracking here: https://cloudhpc.news/wordpress-feuds-cups-and-hpc/

The most important thing in HPC this week? WordPress drama and Linux CUPS CVEs of course.

No, seriously. Imagine this: you’re using CUPS print queues to schedule HPC jobs, because what better way to manage supercomputing resources than by tricking the system into thinking every simulation is a print job?

But wait — your cluster runs WordPress for the UI. Naturally, the hosting feud between WP Engine and Automattic means your queue breaks every time a cease-and-desist gets issued. Jobs are stuck in the print queue, HPC downtime is at an all-time high, and all because you couldn’t decide on the right WordPress plugin.

Now, go patch that CUPS CVE before your compute nodes accidentally try to print your entire data lake.

Happy Friday everyone.

CUPS CVE Details: https://www.thestack.technology/critical-9-9-linux-bug-cups-your-ears-the-details-are-now-here/

WordPress Feuds, CUPS and HPC was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article was first published and is available free or charge and any form of tracking here: https://cloudhpc.news/cloud-migrations-and-repatriations/

Customers are leaving the cloud in droves. Allegedly.

Or at least that’s the excuse AWS is using with the UK Competition and Markets Authority to avoid being pushed into changing its business practices according to a recent article in tech red top The Register.

A closer look by IDC however indicates the percentage of customers that are repatriating workloads to on prem are in the single figures. Who wants to bet these customers had a “lift n shift” approach to cloud adoption?

Now you probably think I’m knocking them. Using the term “lift n shift” disparagingly, as it so often is (I wonder who benefits from that…). But I’m not.

Cloud evangelists and your favourite Cloud Solution Architects will no doubt argue that your move to cloud should be accompanied by a re-engineering of your application to be more cloud native. This will make best use of the cloud resources, keep your costs down and give you the greatest benefits of cloud adoption. All of that is true. But its not the whole truth.

Cloud native is generally lauded as the goal. It shouldn’t be. Cloud native means your applications are locked into the cloud you’ve decided to adopt. It may also not even be the most coft effective option. That’s not where you want to be.

I prefer to aim for something I like to call Cloud Optimized for our clients. If the spectrum spans from Lift N Shift to Cloud Native then Cloud Optimizied sits somewhere between the two. Perhaps it looks a little like this

Lift N Shift ➡️ Cloud Aware ➡️ Cloud Optimized ➡️ Cloud Native

What you should be aiming for is taking advantage of the inherent elasticity of the cloud but not getting locked into a particular implementation of it. In the reams of high performance computing this is even comparatively easy. An HPC example of the above may look something like

24x7 always on servers ➡️ Capacity Reservations ➡️ Script based dynamic capacity scaling ➡️ AWS Parrallel Compute Service/ AWS Batch ➡️ AWS Lambda Functions

If you think AWS Lambda is a great idea for your HPC workloads please come and speak to me before signing those blank cheques!

Cloud Migrations and Repatriations was originally published in HMx Labs on Medium, where people are continuing the conversation by highlighting and responding to this story.