Before diving into high-performance computing world and start writing C code, we need to first understand one of the important concept in programming general. These concepts can be implemented or used with different programming languages/technologies yet the understanding of these concepts do not change.

Early days of the computing did not require any of these concepts until we need to share the computing unit for different tasks, or we would have multiple nodes¹ to distribute the task, so we need some level of communication or agreement.

Let’s assume that you have only one computer. This computer’s operating system can run multiple processes in a single core CPU. Ideally, in your operating systems class, you would learned problematic states that can happen in any not well designed operating systems such as deadlock and race condition.

In this computer, we have to make sure that each processes will be executed within the time constraint that is either defined by the operating system or the program. What would you do if a program that needs to be executed immediately? Well, you would stop the current running process to run the job and the put the other one back in again. The problem starts when you need to run multiple jobs that has the same set of requirements. So, these processes will race for the computing resources.

We will solve this problem with consistent concurrency planning in the operating system level. Not many classic programming languages can provide this out of the box, but modern programming languages such as Golang, Rust or Julia provides that experience without adding any additional technological overhead to the developer or scientist. In the next chapter, we will look into libraries and operating system APIs.

Now, we have another problem to discover. Our computers and mobile devices are not single core CPUs. Many mainframes, database servers or load balancer services do not have only one CPUs but multiple of them. Now, we have to find a way to efficiently use all available cores.