Multiple paradigm reigns on the concurrency/parallel space. They may or may not be exclusive to one another; they may be more or less scalable, more or less easy to implement, etc…
I will review them as fairly as I can.
The grand father of all paradigm. There each process is sequential but one can achieve concurrency using multiple processes. Communication is made through kernel managed resources, such as sockets/named pipes, (named) events, mutexes and semaphores.
- relatively easy to implement, including fault tolerance
- system status is completely transparent ( through OS mechanism).
- can leverage horizontal scalability
- compatible with monothreaded components
- hard to debug
- high overhead due to heavy reliance on kernel object
- no dynamic control of parallelism
- processes consume a significant share of system ressources
My opinion: this is an approach from the past, especially because cross process communications come with significant tax due to serialization needs. But it is the sole available solution when using legacy/sequential code.