Nowadays probably there isn’t a person out there who hasn’t heard the terms CPU and GPU, but only few know what these terms really stand for.

Today I’ll try to explain what the CPU and the GPU have in common, where they differ and by the end of this post you will have a better understanding of the weak and strong parts of each and why you should probably consider GPU acceleration if you already haven’t.

History

The Central Processing Unit (CPU) is the brain of your computer. It drives your Operating System, the applications running on top of it, the communication between different hardware components in the PC and with the outside world. With the popularization of the arcade gaming consoles in the 70s and the Graphical User Interface (GUI) in the 80s, we started using monitors not only to display text on the screen, but also arbitrary graphics. And ever since we’ve wanted bigger and better graphics. But graphics require a lot of pixels. For a display running at 320×240 resolution that is 76800 individual pixels for a single frame to display on the screen, and to make the illusion of animation we need to update this frame 30 times per second which yields just over 2.3 million pixels each second. For comparison, today we’re moving to 4K resolution on TVs and computer monitors and that means a resolution of 3840×2160. At 30 frames per second, this means 248,832,000 pixels per second. This was a task the CPU had a hard time keeping up with back then (remember the CPU has a whole lot of other things to do). And today it still is a task the CPU cannot possibly handle in a reasonable amount of time. So another piece of hardware is used to help the CPU in this highly demanding task – the Graphics Processing Unit.

How they differ

The GPU just like the CPU is a processing unit, a processor if you’d like, and just like the CPU it has a number of cores, internal cache memory, registers, etc, but it is designed with a completely different idea in mind. While the goal behind the CPU is to achieve low latency (to be able to do calculations quickly and return a response as fast as possible) and be as generic as possible, the task before the GPU is to achieve high throughput. What that means is to be able to parallelize a task and execute as many copies of it as possible and in parallel, so that we can get the maximum number of results all at once. Think about it, we usually don’t care how fast the first pixel was delivered, because without having all the pixels there isn’t really much we can do, so the GPU doesn’t really try to do the job as fast as the CPU, it tries to do the job in bulk. 

A neat metaphor for CPU and GPU are a sports car and a truck. If your goal is to get from point A to point B as fast as possible – by all means the sportscar is what you need, but if you have to transfer 10 tons of apples between A and B, while the sportscar will be faster to deliver some of the apples, it will need to make many courses back and forth until all the apples have been delivered, and while the truck will take 2 or 3 times as long to deliver the first batch, it will actually deliver everything at once and won’t have to do a second round trip. Well that’s in layman’s terms exactly what the GPU does.

In short:
> With Latency we measure how fast we can do a single piece of work, e.g. how fast can we deliver one apple from A to B.
> With Throughput we measure the amount of work done for a given time, e.g. how many tons of apples we can move from A to B in 24 hours.

Enough with the metaphors, let’s get back to the pixels. Back in the 90s and the early 2000s this was all the GPUs could do – calculate pixels and show them on the screen. The architecture that allowed the GPU to manage this was different from the one of the CPU. While as stated earlier the CPU and the GPU consist of more or less the same main components, these components are very different internally. For example, a modern CPU works on a frequency of about 4GHz, a modern GPU works only at about 1.5GHz. And while a modern CPU may have 12 or 16 cores, the NVidia 3060TI has a whopping 4864 cores (now you probably start to understand how the GPU is able to achieve such throughput). The GPU core however is very simple, not as highly sophisticated as the CPU core, but on the other hand, it doesn’t need to be any more complicated than that. 

And while in the 90s the GPUs were only used for graphics intensive tasks and showing images on the screen, with the rise of the internet and the enormous amount of data it generated, people started looking at the GPU as something more than just a pixel processing machine. The architecture of the GPU made it perfect for big data analysis. If only there was a way to program the GPU for things other than graphics. And while in the early 2000s people `cheated` by using graphics frameworks like OpenGL to trick the GPU into thinking it was calculating pixels while it was rather doing data analysis, nowadays there’s a number of frameworks (OpenCL, CUDA, Metal, Vulkan Compute) that allow for General Purpose GPU (GPGPU) programming. 

While the GPGPU frameworks allow you to write code directly for the GPU, it is not a good idea to take an existing program and simply run it on the GPU. Remember the architectural differences mentioned above? A program designed for CPU is not a good fit for the GPU architecture and will most probably run very poorly. To harness the power of the GPU you need to write your software specifically with the GPU architecture in mind, but once you do that, you can feast on the high throughput the GPU can give you. It’s also worth mentioning that modern computer motherboards can easily fit at least 2 or 3 GPUs inside, making upgrading much easier than a CPU upgrade.

In conclusion

I hope you now have a fairly thorough understanding of what the CPU and GPU are for and what problems they solve. Now let’s look at a few examples for tasks:
Here are some things that are not suited for a GPU:

  • Running your everyday OS or programs
  • Anything that is serial in nature and cannot easily be parallelized
  • Anything that is not computationally heavy and takes a short time to do.

And here’s a few examples of things that fit on the GPU nicely:

  • Image processing
  • Machine Learning and Deep Learning
  • Big Data analysis
  • Cryptocurrency mining

Finally check this cool Mythbusters video that demonstrates the concept of CPU vs GPU computing.

Enjoy!