Introduction to High-Performance Computing

Overview

Teaching: 10 min
Exercises: 0 min

Questions

• What is a High-Performance Computing (HPC) system?

• Who uses HPC systems?

• Why HPC?

Objectives

• Understanding what a supercomputer is

• Get a basic understanding of what HPC is

What is a High-Performance Computer?

Figure: From top to bottom, left to right: The original 1976 Apple 1, Commodore Amiga 1000, Samsung Galaxy Tab 3; a Smart Watch, the Summit Supercomputer at the Oak Ridge National Laboratory (images courtesy of Wikimedia Commons).

A computer is an electronic device capable to carryout calculations of various sorts. Computers typically process data stored in binary form using a given program or application. A program or application is a set of instructions to execute on a given data. In order to fulfill its purpose, a computer has to be equipped with the right components. For example, personal computers such as laptops can be used for day to day tasks such as word processing, video gaming, and watching movies, for this, they feature a keyboard and a considerably wide screen, decent processor, and memory of the order of a few gigabytes. Smartphones are used for calling, taking pictures, messaging, online shopping, banking, navigation, connecting with friends, entertainment, and many more; for this reason they feature a touchscreen display, a rich set of communication capabilities (Internet, Bluetooth, near-field communication, etc.), good cameras, microphones, speakers, sensors, and many more.

While there are a large number of tasks that can be done using personal computers or mobile devices, there are many tasks that require much more powerful computers to accomplish them. For example, while a low-cost laptop can be sufficient to access emails, watch vidoes, and work on documents and spreadsheets, a powerful workstation is required for demanding tasks such as computer-aided design, video editing, three-dimensional modeling. Even more powerful computers take the form of servers that are stored in data centers. These servers typically have large amounts of memory, processors (cores), and storage, and they consume significantly more power than laptops or workstations. These servers power web applications that we access from web browsers or mobile devices, such as social media platforms (Facebook, Instagram), Google, Apple, Microsoft (mail, photos, cloud storage, productivity apps).

A high-performance computer (HPC) is a supercomputer consisting of a set of highly capable servers that are tightly connected using a high-speed network to enable theser servers to work together in an efficient manner. HPC servers are commonly equipped with high-performance processors, memory, and storage. What sets HPC apart from common servers are the high-speed network that interconnects the servers and allow them to tackle large problems that are otherwise impossible to do with just one server.

Figure: Pictures of HPC servers. Left: a server with four Nvidia V100 GPUs. Right: a server with three Nvidia A100 GPUs.

Why Do We Need an HPC?

The use of HPC is necessary when tackling extremely large and/or complex problems in various disciplines, such as weather forecasting, molecular modeling, material sciences, aerodynamics, cybersecurity… to name a few.

Figure: High performance computing applications (images courtesy of LLNL).

Some problems in science and business require days of calculations, but some are also time-sensitive. Take for example weather forecasting: A forecast would only be valuable if it can be obtaned before a weather event occurs. Weather forecast involves very large and complex computations of mathematical models which take into account a large number of inputs such as wind directions, air humidity, temperatures, pressures, cloud formation, and so on. All this is time consuming and every slight variation from what was expected may have dire consequences on projected results. Weather forecaster use several models in order a coverage of most alternatives. Doing all this in a timely way is primordial. Other problems are too big to be solved with a general-purpose computer. For this reason, some supercomputers feature very large memory where programs and data to operate on are stored for the time needed to solve the problem. The size of the memory of a computer is a big limiting factor on how much data can be used at a given time. For a program to be solvable, it has to fit in the computer memory. Supercomputers combine memories on compute nodes to seamlessly work as a single computer’s memory. This brings the ability to the computer to fit bigger problems. Advances in computer architecture also give HPC an advantage. Current computers feature components of different nature working together to deliver faster speeds. This is called heterogeneous computing. Components such as GPU (Graphics Processing Unit) can be used to accelerate computations by adding specific programming constructs to traditional programs. Concept such as vector processing make this possible. Tasks that would take a CPU much longer to complete could be done faster using co-processors or GPUs.

How Parallel Computing Works

Parallel computing entails the division of computational tasks across multiple processing units, enabling the simultaneous processing of different data segments. This methodology significantly reduces the overall execution time by leveraging the collective computational power of several workers. For instance, a task that would ordinarily take one worker approximately six hours to complete can be accomplished in just one and a half hours when divided among four workers. Extending this approach to a larger scale, distributing the workload among 400 processors could potentially decrease the total runtime to a mere 54 seconds. This illustrates the efficiency and scalability that parallel computing offers for complex computational tasks.

The Key to HPC Computing Power

The key to using HPC lies in parallel processing. In a nutshell, parallel computation entails breaking down a set of computations into many little sub-tasks so that they can be processed in parallel by the many processing units available in an HPC system. As we shall learn throughout this training program, parallel processing is not always straightforward to do. This is why training programs such as this one will help you to acquire basic understanding of an HPC and how to program it to solve your problem faster than your typical desktop or laptop computers.

What is Cyberstructure?

I still don’t find sidebat command.

Key Points

• Supercomputers are a collection of smaller computers into one big unit

• HPC is used in various domains, pretty much everywhere a computer could be used to solve a problem

• HPC helps get results faster than traditional computing.