What is the difference between concurrency and parallelism?

Definition

There are three types of code:

• Serial: runs some tasks in an order with one processor.
• Concurrent: runs many tasks simultaneously with a less number of processors.
• Parallel: runs n tasks simultaneously with n processors.

Example 1: cleaning bedroom

You want to clean four bedrooms in your house:

• Serial: You clean bedroom 1, when it is finished, you start cleaning bedroom 2, and so on.

• Concurrent: You clean a little of bedroom 1, then a little of bedroom 2, 3, and 4. You repeat this until all are cleaned.

• Parallel: you clean bedroom 1 and 2. Your friend, at the same time, cleans bedroom 3 and 4.

Example 2: Workshop

You own a workshop and you received two orders: a chair and a table.

• Serial: you work on the chair. When it’s done, you send it to its customer. Then, you start making the table.

• Concurrent: You work 1 hour on the chair and 1 hour on the table and repeat this until they are made. Then you send them to customers. Note that they may be finished at the same time, or one earlier than the other.

• Parallel: You make the chair, and your colleague makes the table at the same time.

In these examples, your friend, colleague, and you are independent processors.

What is a processor?

Nowadays, a CPU is made of multiple cores. A core is exposed as two virtual cores to operating systems via Hyper-threading (Intel term) or Multithreading (AMD term). Each virtual core is considered a processor. Therefore, a quad-core CPU with two virtual cores per core gives you 8 processors. Virtual cores are great for running light simultaneous tasks like listening to music on Spotify, writing a report, and googling some questions at the same time. However, for running a numerical code, using all virtual cores can degrade the speed of code. Therefore, if you have a parallel code for a number-crunching problem, consider each physical core as one processor.

Where to begin with concurrent code?

Most languages provide libraries to write concurrent codes:

• in Python the library is threading,
• in C++ it is the standard library of thread,
• in C# the library is System.Threading,
• and in Java class java.lang.Thread is extended.

A concurrent or multi-thread program is written similarly in different languages. From the main program, different threads or tasks are spawned which is run in the background. Via a handle, the status of a thread can be checked and the program can be halted for a thread to finish.

Threads can be assigned to send/receive network requests, read/write data, run background actions of a graphical user interface, and do many more tasks.

Where to begin with parallel code?

Parallel codes are used for heavy numerical programs that are run for hours to weeks. You can use threading libraries to create threads and have parallel code. Just ensure the number of threads (including the main program thread) is less than or equal to the number of cores and, hopefully, the OS scheduler assigns each thread to a different core.

To code professionally for parallel processing, OpenMP library is great for employing the CPU cores of a machine. MPI libraries like OpenMPI, MPICH, and Mpi4py are perfect for running a code on supercomputers (cluster of computers). OpenACC is suitable for codes to be run on GPUs.

Comments

2 comments