Thread based software architecture vs Process based software architecture

Hi,
Have few queries on the best possible software architecture.

Processes are heavy weight and they appear to occupy more memory, more time to create/start, increased latency during context switches and separate memory space that necessitates heavy IPC mechanisms. Threads are light weight and share memory space. However, I realized that threads also enter into contention for resources/memory due to the shared resources among them that inturn becomes a kind of bottle neck for multi-threaded architecture but not for multiple process based architecture. Also the workaround for having thread local storage does not seem to be straight forward. This also makes me believe that maintaining multi-threaded application can be bit complex compared to that of multiple process architecture. Also that the performance of multi process architecture will be better due to separate memory space (This avoids locking or serialization of execution in case of multi process architecture) and this seems to take away the advantage of less context switch time in case of multi-threaded application !! Kindly let me know if this understanding is correct or correct with appropriate inputs.

I understand that the software architecture is mainly based on the type of application/requirement. Considering the development environment as Linux OS with C language on single core/multi-core processors, i would like to know for which type of applications should we need to go in for multi-threaded software architecture and for which type of applications should we need to go in for multiple process based software architecture ? Is there any matrix sheet  that maps/lists the type of requirements/applications and the possible software architecture for it ?

Thx in advans,
Karthik

On Sun, 21 Sep 2014 05:40:26 -0700, Karthik Balaguru wrote:

> Hi,
> Have few queries on the best possible software architecture.
> 
> Processes are heavy weight and they appear to occupy more memory, more
> time to create/start, increased latency during context switches and
> separate memory space that necessitates heavy IPC mechanisms. Threads
> are light weight and share memory space. However, I realized that
> threads also enter into contention for resources/memory due to the
> shared resources among them that inturn becomes a kind of bottle neck
> for multi-threaded architecture but not for multiple process based
> architecture. Also the workaround for having thread local storage does
> not seem to be straight forward. This also makes me believe that
> maintaining multi-threaded application can be bit complex compared to
> that of multiple process architecture. Also that the performance of
> multi process architecture will be better due to separate memory space
> (This avoids locking or serialization of execution in case of multi
> process architecture) and this seems to take away the advantage of less
> context switch time in case of multi-threaded application !! Kindly let
> me know if this understanding is correct or correct with appropriate
> inputs.
> 
> I understand that the software architecture is mainly based on the type
> of application/requirement. Considering the development environment as
> Linux OS with C language on single core/multi-core processors, i would
> like to know for which type of applications should we need to go in for
> multi-threaded software architecture and for which type of applications
> should we need to go in for multiple process based software architecture
> ? Is there any matrix sheet  that maps/lists the type of
> requirements/applications and the possible software architecture for it
> ?

You are mistaken in your notion that just because threads explicitly share 
a memory space and other resources that they have more contention.  
Threads share a memory _space_, but they don't have to use the same bits 
of memory within that space -- it is easy to set things up so that each 
thread has its own chunk of memory that it uses.

In a sense, once you get past the MMU, processes share the same memory 
space, too -- it's just that the MMU protects each process from having to 
know about the memory space occupied by other processes, or even, for that 
matter, from having to know what physical addresses it occupies.

The "processes have separate memory space" is an illusion, provided in 
hardware by the MMU.  At the point where activity is going on in physical 
memory, all the processes have to access the same memory space, and so 
they contend for that resource.  Ditto hard drive accesses, screen access, 
etc.

Really, the biggest thing that you give up with threads vs. processes is 
that -- assuming the OS is doing its job -- processes are safe from one 
another.  Threads, however, can easily stomp on one another, simply by 
writing into some part of memory that some other thread is using and 
thinks isn't going to be disturbed.

For me, the dividing line between threads and processes is one of work 
load, processor loading, and trust: do I trust whoever is developing that 
software entity over there not to stomp on my stuff, and is it less work 
for both of us, using threads, to not stomp on each other's stuff than it 
is to just use processes?  And can the job be done at all using 
processes?  If the answer to the first two questions is "yes", then 
threads are indicated.  If the answer to _either_ of the first two 
questions is "no", then processes are indicated -- and if the answer to 
the third question is then "no", the project is in jeopardy.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Hi Karthik,

On 9/21/2014 5:40 AM, Karthik Balaguru wrote:
> Hi, Have few queries on the best possible software architecture.

None -- without a clear definition of the application domain!  :>

> Processes are heavy weight and they appear to occupy more memory, more time
> to create/start, increased latency during context switches and separate
> memory space that necessitates heavy IPC mechanisms. Threads are light
> weight and share memory space.

The easiest way to think of the distinction is:  threads are active entities
(i.e., they are the "things" that "execute code").  Processes are containers
that hold resources -- which can include (one or more) *threads*!

I.e., a process is like its own little "machine" -- with its own memory,
access priviledges, priorities (in the context of the "machine" in which it
resides), etc.

So, if the "system"/machine has certain shared resources (I/O devices, etc),
it is the *process* that requests (by the actions of one of its threads)
those resources and, eventually, gains ownership/access to it.  (I.e., thread
#1 in process A can request a resource and, when made available, thread #5
in process A can *use* that resource -- but none of the threads in process B
can, at that time)

Given that processes contain threads (in this conceptualization), you can see
why it is "more expensive" to switch processes than it is to switch threads.

You can also see why two threads in a process can compete to access a resource
THAT THE PROCESS OWNS (either because it was explicitly requested from "the
system" by "some thread" in that process; OR it was implicitly granted to that
process when the process was instantiated:  e.g., "shared memory" IN the
process's address space).  You are glossing over the potential case where
two or more PROCESSES have to compete in "the system" for other "shared
resources".

> However, I realized that threads also enter
> into contention for resources/memory due to the shared resources among them
> that inturn becomes a kind of bottle neck for multi-threaded architecture
> but not for multiple process based architecture.

It's still a bottleneck.  If two or more processes want to share some data,
they either do so via "shared memory" (assuming the OS supports this between
processes) -- which requires SOME form of "access/contention resolution" -- or
by a more expensive solution (e.g., IPC/RPC).  In each case, SOMETHING is
handling the fact that contention can exist.

> Also the workaround for
> having thread local storage does not seem to be straight forward. This also
> makes me believe that maintaining multi-threaded application can be bit
> complex compared to that of multiple process architecture.

There is no concept of a thread's "(private) memory space" -- though you
can easily arrange for this (e.g., each thread has its own pushdown stack!
anything thread #1 does that is implemented via the stack is effectively
private -- though a rogue thread can still scribble on it!).

By contrast, each (single-threaded) process has a unique, disjoint memory
space "guaranteed" by the OS at the process's instantiation (I am assuming
you have a "real/nonTOY OS").

> Also that the
> performance of multi process architecture will be better due to separate
> memory space (This avoids locking or serialization of execution in case of
> multi process architecture) and this seems to take away the advantage of
> less context switch time in case of multi-threaded application !!

No.  If there is no contention, there is no locking required beyond what
is implicitly present when "thread #1" is scheduled to execute while the
other threads are (temporarily) blocked.

Contention has costs, period.  You can structure your code so that these
costs are minimized.  E.g., in a consumer/producer model of sharing,
the two threads never actually compete for the same "object" -- an
object that is being produced is invisible to a consumer waiting to
consume it!  Likewise, an object that has BEEN produced is no longer
of interest to its producer!

Process model gives an (incorrect) illusion of greater separation
only because it "makes sharing (between PROCESSES) harder".  If
you similarly impose the restrictions that different process spaces
impose on your code (i.e., never compete for data for which you have
no NEED to compete -- as if it was NOT POSSIBLE), then the costs
of sharing are the same -- none.

> Kindly let me know if this understanding is correct or correct with appropriate
> inputs.
>
> I understand that the software architecture is mainly based on the type of
> application/requirement. Considering the development environment as Linux OS
> with C language on single core/multi-core processors, i would like to know
> for which type of applications should we need to go in for multi-threaded
> software architecture and for which type of applications should we need to
> go in for multiple process based software architecture ? Is there any matrix
> sheet  that maps/lists the type of requirements/applications and the
> possible software architecture for it ?

(sigh)  *BIG* (complex) question.  Essentially, you have to look at the
benefits of "tightly coupled" execution (threads) vs. more "loosely
coupled" (processes).  And, the overhead involved in each sharing case.
Likewise, the potential for (the illusion of) concurrency and the
periods involved.

E.g., any time an "execution context" (threaded or single-thread) has to
block on <something> (resource, user, i/o, etc.), then there is an
opportunity for some other execution context to "do meaningful work"
(note that this is not the same thing as a GUARANTEE that they will
be able to do meaningful work!).

How often this occurs and the amount (percent?) of time that the
blocked process is suspended -- relative to the rate at which "new
work" arrives -- determines how much time you can afford to "waste"
in the overhead of your model (thread v. process).

E.g., if work is represented by cars arriving at a toll booth
(your job being to monitor the presence of individual cars, the
receipt of appropriate payment from each and the control of the
"gate" allowing paid vehicles to pass), you could (all else being
equal) create a single-threaded process that:
    wait for car;
    wait for payment;
    raise gate;
    lather, rinse, repeat
And, spawn N instances of this process -- one for each "lane"
at the toll booth (binding the appropriate instances of "car
sensor", "coin counter", "gate actuator" to each instance).
The "procedure" (I am trying to avoid using the word "process")
is inherently serial -- easily handled by a single thread.

[A car doesn't arrive at lane 4, pay at gate 7 and then exit
at gate 2!]

THE PROCESSES HAVE NOTHING TO SAY TO EACH OTHER!  So, there is
no contention *between* them.

Most of the time, a process is waiting for (the next thing)
to happen.  I.e., while waiting for payment, it doesn't have
to deal with "another car" -- even though another car *may*
be arriving in some other lane!  So, the cost of multiple
processes (time) is largely hidden in that "wait time".

You can, similarly, design this as a set of THREADS in the
exact same way!  Each thread has nothing to share with the
other threads!

[Keep this in mind as reading each of the following examples.
"Thread" can often be replaced by "process"; but, you will
have to think of everything else going on in the particular
example to evaluate how (in)effective that solution might be!]

Imagine, instead, writing this process as a set of threads:
one that waits for the car; another that waits for payment;
a third that raises the gate (and, presumably, ensures the
car has passed successfully).  These threads need to
share information -- you don't want the gate_raiser thread
to raise the gate before the payment_received thread has
vouched for the vehicle's compliance!

That shared information can be as simple as a shared "state"
variable:  {AWAITING_CAR, AWAITING_PAYMENT, RAISING_GATE}.
Each thread can be responsible for monitoring the variable
to determine when it is appropriate to "start" AND updating
the variable when it has finished its assigned chore.
I.e., only one thread is ever "holding" the variable
(able to write to it!).

[threads could also directly start/unblock each other in
succession... lots of ways to skin this cat]

You could likewise use a set of *processes* to do this:
each process (pedantically, the single thread *in* each
process) responsible for blocking on a particular condition,
etc.  But, processes cost more and are heavier-footed than
threads.

In the "process" implementation, the sharing has to happen
through some OS-supported mechanism -- *if* processes
are prohibited with accessing each other's (or *SYSTEM*!)
resources.  In the thread implementation, threads within a
shared "container" can freely exchange information
(relying on synchronization primitives provided by the OS
*or* by constraints inherent in the algorithm:  "YOU set
this, I will CLEAR it")

You could, also, have one giant "process" with lots of
threads -- that handles the entire toll-booth.  (again,
lots of ways to skin this cat... I'll let you sort out the
"more obvious" ones)

Threads could sit "awaiting events".  A set of "accepting payment"
threads (responsible for verifying proper payment) can sit
waiting for "CAR_ARRIVED" events (messages).  When such an event
is detected, the first WAITING/blocked thread consumes it and
begins execution (the event obviously has to specify the lane
on which the waiting car was detected!).

[The next "accepting payment" thread -- IF ANY (possibly a configurable
option... you might have fewer threads than lanes, etc.  depends on the
expected interarrival times of "cars") -- then steps up and awaits
the NEXT "CAR_ARRIVED" event.  This may be on the same lane as the
immediately preceding event -- or, another lane entirely!]

The "accepting payment" thread recently activated (above), now
sits waiting for "coin received" events (from the specific lane
that it is monitoring!).  When it has  processed enough of these
to indicate proper payment, it generates a PAYMENT_RECEIVED event
(tagged with the corresponding lane number) and then goes back to
waiting for another "CAR_ARRIVED" event.

[I.e., this flavor thread can only handle CAR_ARRIVED events!]

Similarly, another (set of one or more) "raising gate" threads
sit waiting for PAYMENT_RECEIVED events and act accordingly.

Here, you need as many "raising gate" threads as there are gates
that you want to be able to raise CONCURRENTLY!  (e.g., if you
don't mind letting other "paid customers" wait while you raise
the gate for customer X, then you only need enough of those
threads to raise *on* gate at a time!

Yet another way of doing this is to have N copies of generic threads
that are capable of processing *any* sort of event -- i.e., having
a dispatch table (switch statement) at the start to route the
event to the appropriate processing code fragment.  In this case,
you need only enough threads to handle the total number of "things"
that can be happening at one time (i.e., one thing on each lane).

Ah, but what is to prevent a cock-up in The System (or, an exploit
by a savvy user?) from preventing the vehicle's initial arrival
to be immediately followed by a PAYMENT_RECEIVED event?  I.e.,
BEFORE the "accepting payment" thread has even been activated!
(we have a technical term for this:  "bug")

In the initial "serial process", this wasn't possible:  the code
that was executed after payment was received COULDN'T run until
a car had been detected AND coins counted.  The design of the
code precluded that possibility.  To "exploit" the system,
a user would have to synthesize all of the preceding events
to "advance" the algorithm to the point where it was ready to lift
the gate.

OK, let's build a SHARED OBJECT that indicates the "state" of each
of the lanes!  That way, the "raising gate" thread won't invoke
the actuator unless it sees all of the required prerequisites in
place -- even if "signaled" by a PAYMENT_RECEIVED event!

Now, you have several entities trying to update that state AT THE
SAME TIME THAT OTHERS ARE TRYING TO EXAMINE IT.  "Contention"
that affects the entire application's performance -- ONE bottleneck
(instead of a "bottleneck per lane" -- or NO bottlenecks!)

Imagine if the cost of ATOMICLY accessing this object was a fat
system call -- because it resided somewhere that all PROCESSES
could access (contrast with THREADS)!

In each case, you decide how much information you are sharing and
who you are sharing it with.  A single thread that runs a single
lane from start to finish IMPLICITLY is sharing data with itself:
it saw the car arrive on its assigned lane, it watched as the
coins were deposited in the coin acceptor on that lane, then it
raised the gate for that lane -- before returning to await the
next arrival.

As you split the "chore" into finer pieces -- or, split the
handling of it into different/disjoint "execution contexts" -- you
need to pass more information between those objects.  E.g., passing
events of the form (<lane>, <event_type>) to a set of generic
"handlers" moves the sharing into the "event system".

[whether this is a fifo, shared memory, IPC, etc.]

OTOH, you increase the possibilities for concurrency and more
efficient use of resources (why have N "raise gate" processes
if drivers can afford to wait for THEIR gate to be lifted?
Perhaps the gate lift mechanism can ONLY lift a single gate at
a time (motor and gears/clutches).

Sorry for the long-winded explanation.  I will promptly be derided
for it.  But, hopefully it shows you different approaches (that
exploit "potential parallelism/decomposition" in different ways)
and the potential consequences of different approaches.

You have to look at your workload and see what approach makes the
most sense.  Interconnections are expensive in any algorithm!

On 9/21/2014 12:23 PM, Tim Wescott wrote:
> On Sun, 21 Sep 2014 05:40:26 -0700, Karthik Balaguru wrote:
>
> You are mistaken in your notion that just because threads explicitly share
> a memory space and other resources that they have more contention.
> Threads share a memory _space_, but they don't have to use the same bits
> of memory within that space -- it is easy to set things up so that each
> thread has its own chunk of memory that it uses.
>
> In a sense, once you get past the MMU, processes share the same memory
> space, too -- it's just that the MMU protects each process from having to
> know about the memory space occupied by other processes, or even, for that
> matter, from having to know what physical addresses it occupies.

MMU is not a requirement of a process model.  Nor excluded from
a thread model (even in a single process system).

There is no guarantee that processes are protected from each other.
That is an implementation detail.  I.e., you can adopt a "process
model" and have everyone living in a single, unified, FLAT memory
space.

The better way of thinking of processes is as resource containers
(threads being one sort of resource).  As such, they are bigger/heavier
than threads -- that only have to remember their current processor state.

E.g., a *process* can hold a resource.  A thread can not.  (the process
CONTAINING the thread holds it).  So, a process handling your "console"
can hold that console (hardware/software construct) and one thread
in it can paint the screen while another thread is responsible for
"ringing the bell" (which takes a sizable fraction of a second!)

> The "processes have separate memory space" is an illusion, provided in
> hardware by the MMU.  At the point where activity is going on in physical
> memory, all the processes have to access the same memory space, and so
> they contend for that resource.  Ditto hard drive accesses, screen access,
> etc.

In most modern OS's (I won't use the L-word!), an MMU enforces a separation
(partitioning) of this "hardware" address space.  *If* the processor
has such hardware available.

(how "violations" are handled is another subject)

> Really, the biggest thing that you give up with threads vs. processes is
> that -- assuming the OS is doing its job -- processes are safe from one
> another.  Threads, however, can easily stomp on one another, simply by
> writing into some part of memory that some other thread is using and
> thinks isn't going to be disturbed.

No, that isn't guaranteed -- unless you speak of a specific *port*
of a specific OS (think of OS's that claim to run on hardware WITHOUT
MMU's!  The process model still applies -- you just lose the protections!)

> For me, the dividing line between threads and processes is one of work
> load, processor loading, and trust: do I trust whoever is developing that
> software entity over there not to stomp on my stuff, and is it less work
> for both of us, using threads, to not stomp on each other's stuff than it
> is to just use processes?  And can the job be done at all using
> processes?  If the answer to the first two questions is "yes", then
> threads are indicated.  If the answer to _either_ of the first two
> questions is "no", then processes are indicated -- and if the answer to
> the third question is then "no", the project is in jeopardy.

For me, the criteria (if it has to be boiled down to a single one)
is "communications".  The more data that has to be "interactively"
shared (or, the higher the frequency of sharing), the more of an
annoyance process boundaries become.  Because there are BOUNDARIES
that must be crossed (at some cost) -- even if only in a conceptual
sense (i.e., just because you don't have hardware protections doesn't
mean scribbling in another process -- CONTAINER -- is "right")

On Sun, 21 Sep 2014 05:40:26 -0700 (PDT), Karthik Balaguru
<karthik.balaguru007@gmail.com> wrote:

>Processes are heavy weight and they appear to occupy more memory,
>more time to create/start, increased latency during context 
>switches and separate memory space that necessitates heavy IPC 
>mechanisms. Threads are light weight and share memory space.

Yes and no.  

The best way to think of a process is as a resource container - a
thread is a particular kind of resource (a computation resource) that
a process can contain.

Processes also typically are protection boundaries whereas threads
typically are not [though there are exceptions].


>Considering the development environment as Linux OS with C 
>language on single core/multi-core processors, i would like to 
>know for which type of applications should we need to go in 
>for multi-threaded software architecture and for which type 
>of applications should we need to go in for multiple process
>based software architecture ?

In Linux there is a system call "clone" (see clone(2)).  Clone
essentially creates new threads, but it permits detaching a new thread
into a separate process and specifying with relatively fine control
what parent resources should be copied to the child.

Clone wraps an even lower level call that provides even more control
over the environment of the new thread.  Using clone you can create
very lightweight processes, e.g., just a thread with MMU protection.

>Karthik
George

Hi Don,

Thanks for your quick reply !
That was indeed a pretty long & an interesting explanation !!

Karthik



On Monday, 22 September 2014 02:34:45 UTC+5:30, Don Y  wrote:
> Hi Karthik,
> 
> 
> 
> On 9/21/2014 5:40 AM, Karthik Balaguru wrote:
> 
> > Hi, Have few queries on the best possible software architecture.
> 
> 
> 
> None -- without a clear definition of the application domain!  :>
> 
> 
> 
> > Processes are heavy weight and they appear to occupy more memory, more time
> 
> > to create/start, increased latency during context switches and separate
> 
> > memory space that necessitates heavy IPC mechanisms. Threads are light
> 
> > weight and share memory space.
> 
> 
> 
> The easiest way to think of the distinction is:  threads are active entities
> 
> (i.e., they are the "things" that "execute code").  Processes are containers
> 
> that hold resources -- which can include (one or more) *threads*!
> 
> 
> 
> I.e., a process is like its own little "machine" -- with its own memory,
> 
> access priviledges, priorities (in the context of the "machine" in which it
> 
> resides), etc.
> 
> 
> 
> So, if the "system"/machine has certain shared resources (I/O devices, etc),
> 
> it is the *process* that requests (by the actions of one of its threads)
> 
> those resources and, eventually, gains ownership/access to it.  (I.e., thread
> 
> #1 in process A can request a resource and, when made available, thread #5
> 
> in process A can *use* that resource -- but none of the threads in process B
> 
> can, at that time)
> 
> 
> 
> Given that processes contain threads (in this conceptualization), you can see
> 
> why it is "more expensive" to switch processes than it is to switch threads.
> 
> 
> 
> You can also see why two threads in a process can compete to access a resource
> 
> THAT THE PROCESS OWNS (either because it was explicitly requested from "the
> 
> system" by "some thread" in that process; OR it was implicitly granted to that
> 
> process when the process was instantiated:  e.g., "shared memory" IN the
> 
> process's address space).  You are glossing over the potential case where
> 
> two or more PROCESSES have to compete in "the system" for other "shared
> 
> resources".
> 
> 
> 
> > However, I realized that threads also enter
> 
> > into contention for resources/memory due to the shared resources among them
> 
> > that inturn becomes a kind of bottle neck for multi-threaded architecture
> 
> > but not for multiple process based architecture.
> 
> 
> 
> It's still a bottleneck.  If two or more processes want to share some data,
> 
> they either do so via "shared memory" (assuming the OS supports this between
> 
> processes) -- which requires SOME form of "access/contention resolution" -- or
> 
> by a more expensive solution (e.g., IPC/RPC).  In each case, SOMETHING is
> 
> handling the fact that contention can exist.
> 
> 
> 
> > Also the workaround for
> 
> > having thread local storage does not seem to be straight forward. This also
> 
> > makes me believe that maintaining multi-threaded application can be bit
> 
> > complex compared to that of multiple process architecture.
> 
> 
> 
> There is no concept of a thread's "(private) memory space" -- though you
> 
> can easily arrange for this (e.g., each thread has its own pushdown stack!
> 
> anything thread #1 does that is implemented via the stack is effectively
> 
> private -- though a rogue thread can still scribble on it!).
> 
> 
> 
> By contrast, each (single-threaded) process has a unique, disjoint memory
> 
> space "guaranteed" by the OS at the process's instantiation (I am assuming
> 
> you have a "real/nonTOY OS").
> 
> 
> 
> > Also that the
> 
> > performance of multi process architecture will be better due to separate
> 
> > memory space (This avoids locking or serialization of execution in case of
> 
> > multi process architecture) and this seems to take away the advantage of
> 
> > less context switch time in case of multi-threaded application !!
> 
> 
> 
> No.  If there is no contention, there is no locking required beyond what
> 
> is implicitly present when "thread #1" is scheduled to execute while the
> 
> other threads are (temporarily) blocked.
> 
> 
> 
> Contention has costs, period.  You can structure your code so that these
> 
> costs are minimized.  E.g., in a consumer/producer model of sharing,
> 
> the two threads never actually compete for the same "object" -- an
> 
> object that is being produced is invisible to a consumer waiting to
> 
> consume it!  Likewise, an object that has BEEN produced is no longer
> 
> of interest to its producer!
> 
> 
> 
> Process model gives an (incorrect) illusion of greater separation
> 
> only because it "makes sharing (between PROCESSES) harder".  If
> 
> you similarly impose the restrictions that different process spaces
> 
> impose on your code (i.e., never compete for data for which you have
> 
> no NEED to compete -- as if it was NOT POSSIBLE), then the costs
> 
> of sharing are the same -- none.
> 
> 
> 
> > Kindly let me know if this understanding is correct or correct with appropriate
> 
> > inputs.
> 
> >
> 
> > I understand that the software architecture is mainly based on the type of
> 
> > application/requirement. Considering the development environment as Linux OS
> 
> > with C language on single core/multi-core processors, i would like to know
> 
> > for which type of applications should we need to go in for multi-threaded
> 
> > software architecture and for which type of applications should we need to
> 
> > go in for multiple process based software architecture ? Is there any matrix
> 
> > sheet  that maps/lists the type of requirements/applications and the
> 
> > possible software architecture for it ?
> 
> 
> 
> (sigh)  *BIG* (complex) question.  Essentially, you have to look at the
> 
> benefits of "tightly coupled" execution (threads) vs. more "loosely
> 
> coupled" (processes).  And, the overhead involved in each sharing case.
> 
> Likewise, the potential for (the illusion of) concurrency and the
> 
> periods involved.
> 
> 
> 
> E.g., any time an "execution context" (threaded or single-thread) has to
> 
> block on <something> (resource, user, i/o, etc.), then there is an
> 
> opportunity for some other execution context to "do meaningful work"
> 
> (note that this is not the same thing as a GUARANTEE that they will
> 
> be able to do meaningful work!).
> 
> 
> 
> How often this occurs and the amount (percent?) of time that the
> 
> blocked process is suspended -- relative to the rate at which "new
> 
> work" arrives -- determines how much time you can afford to "waste"
> 
> in the overhead of your model (thread v. process).
> 
> 
> 
> E.g., if work is represented by cars arriving at a toll booth
> 
> (your job being to monitor the presence of individual cars, the
> 
> receipt of appropriate payment from each and the control of the
> 
> "gate" allowing paid vehicles to pass), you could (all else being
> 
> equal) create a single-threaded process that:
> 
>     wait for car;
> 
>     wait for payment;
> 
>     raise gate;
> 
>     lather, rinse, repeat
> 
> And, spawn N instances of this process -- one for each "lane"
> 
> at the toll booth (binding the appropriate instances of "car
> 
> sensor", "coin counter", "gate actuator" to each instance).
> 
> The "procedure" (I am trying to avoid using the word "process")
> 
> is inherently serial -- easily handled by a single thread.
> 
> 
> 
> [A car doesn't arrive at lane 4, pay at gate 7 and then exit
> 
> at gate 2!]
> 
> 
> 
> THE PROCESSES HAVE NOTHING TO SAY TO EACH OTHER!  So, there is
> 
> no contention *between* them.
> 
> 
> 
> Most of the time, a process is waiting for (the next thing)
> 
> to happen.  I.e., while waiting for payment, it doesn't have
> 
> to deal with "another car" -- even though another car *may*
> 
> be arriving in some other lane!  So, the cost of multiple
> 
> processes (time) is largely hidden in that "wait time".
> 
> 
> 
> You can, similarly, design this as a set of THREADS in the
> 
> exact same way!  Each thread has nothing to share with the
> 
> other threads!
> 
> 
> 
> [Keep this in mind as reading each of the following examples.
> 
> "Thread" can often be replaced by "process"; but, you will
> 
> have to think of everything else going on in the particular
> 
> example to evaluate how (in)effective that solution might be!]
> 
> 
> 
> Imagine, instead, writing this process as a set of threads:
> 
> one that waits for the car; another that waits for payment;
> 
> a third that raises the gate (and, presumably, ensures the
> 
> car has passed successfully).  These threads need to
> 
> share information -- you don't want the gate_raiser thread
> 
> to raise the gate before the payment_received thread has
> 
> vouched for the vehicle's compliance!
> 
> 
> 
> That shared information can be as simple as a shared "state"
> 
> variable:  {AWAITING_CAR, AWAITING_PAYMENT, RAISING_GATE}.
> 
> Each thread can be responsible for monitoring the variable
> 
> to determine when it is appropriate to "start" AND updating
> 
> the variable when it has finished its assigned chore.
> 
> I.e., only one thread is ever "holding" the variable
> 
> (able to write to it!).
> 
> 
> 
> [threads could also directly start/unblock each other in
> 
> succession... lots of ways to skin this cat]
> 
> 
> 
> You could likewise use a set of *processes* to do this:
> 
> each process (pedantically, the single thread *in* each
> 
> process) responsible for blocking on a particular condition,
> 
> etc.  But, processes cost more and are heavier-footed than
> 
> threads.
> 
> 
> 
> In the "process" implementation, the sharing has to happen
> 
> through some OS-supported mechanism -- *if* processes
> 
> are prohibited with accessing each other's (or *SYSTEM*!)
> 
> resources.  In the thread implementation, threads within a
> 
> shared "container" can freely exchange information
> 
> (relying on synchronization primitives provided by the OS
> 
> *or* by constraints inherent in the algorithm:  "YOU set
> 
> this, I will CLEAR it")
> 
> 
> 
> You could, also, have one giant "process" with lots of
> 
> threads -- that handles the entire toll-booth.  (again,
> 
> lots of ways to skin this cat... I'll let you sort out the
> 
> "more obvious" ones)
> 
> 
> 
> Threads could sit "awaiting events".  A set of "accepting payment"
> 
> threads (responsible for verifying proper payment) can sit
> 
> waiting for "CAR_ARRIVED" events (messages).  When such an event
> 
> is detected, the first WAITING/blocked thread consumes it and
> 
> begins execution (the event obviously has to specify the lane
> 
> on which the waiting car was detected!).
> 
> 
> 
> [The next "accepting payment" thread -- IF ANY (possibly a configurable
> 
> option... you might have fewer threads than lanes, etc.  depends on the
> 
> expected interarrival times of "cars") -- then steps up and awaits
> 
> the NEXT "CAR_ARRIVED" event.  This may be on the same lane as the
> 
> immediately preceding event -- or, another lane entirely!]
> 
> 
> 
> The "accepting payment" thread recently activated (above), now
> 
> sits waiting for "coin received" events (from the specific lane
> 
> that it is monitoring!).  When it has  processed enough of these
> 
> to indicate proper payment, it generates a PAYMENT_RECEIVED event
> 
> (tagged with the corresponding lane number) and then goes back to
> 
> waiting for another "CAR_ARRIVED" event.
> 
> 
> 
> [I.e., this flavor thread can only handle CAR_ARRIVED events!]
> 
> 
> 
> Similarly, another (set of one or more) "raising gate" threads
> 
> sit waiting for PAYMENT_RECEIVED events and act accordingly.
> 
> 
> 
> Here, you need as many "raising gate" threads as there are gates
> 
> that you want to be able to raise CONCURRENTLY!  (e.g., if you
> 
> don't mind letting other "paid customers" wait while you raise
> 
> the gate for customer X, then you only need enough of those
> 
> threads to raise *on* gate at a time!
> 
> 
> 
> Yet another way of doing this is to have N copies of generic threads
> 
> that are capable of processing *any* sort of event -- i.e., having
> 
> a dispatch table (switch statement) at the start to route the
> 
> event to the appropriate processing code fragment.  In this case,
> 
> you need only enough threads to handle the total number of "things"
> 
> that can be happening at one time (i.e., one thing on each lane).
> 
> 
> 
> Ah, but what is to prevent a cock-up in The System (or, an exploit
> 
> by a savvy user?) from preventing the vehicle's initial arrival
> 
> to be immediately followed by a PAYMENT_RECEIVED event?  I.e.,
> 
> BEFORE the "accepting payment" thread has even been activated!
> 
> (we have a technical term for this:  "bug")
> 
> 
> 
> In the initial "serial process", this wasn't possible:  the code
> 
> that was executed after payment was received COULDN'T run until
> 
> a car had been detected AND coins counted.  The design of the
> 
> code precluded that possibility.  To "exploit" the system,
> 
> a user would have to synthesize all of the preceding events
> 
> to "advance" the algorithm to the point where it was ready to lift
> 
> the gate.
> 
> 
> 
> OK, let's build a SHARED OBJECT that indicates the "state" of each
> 
> of the lanes!  That way, the "raising gate" thread won't invoke
> 
> the actuator unless it sees all of the required prerequisites in
> 
> place -- even if "signaled" by a PAYMENT_RECEIVED event!
> 
> 
> 
> Now, you have several entities trying to update that state AT THE
> 
> SAME TIME THAT OTHERS ARE TRYING TO EXAMINE IT.  "Contention"
> 
> that affects the entire application's performance -- ONE bottleneck
> 
> (instead of a "bottleneck per lane" -- or NO bottlenecks!)
> 
> 
> 
> Imagine if the cost of ATOMICLY accessing this object was a fat
> 
> system call -- because it resided somewhere that all PROCESSES
> 
> could access (contrast with THREADS)!
> 
> 
> 
> In each case, you decide how much information you are sharing and
> 
> who you are sharing it with.  A single thread that runs a single
> 
> lane from start to finish IMPLICITLY is sharing data with itself:
> 
> it saw the car arrive on its assigned lane, it watched as the
> 
> coins were deposited in the coin acceptor on that lane, then it
> 
> raised the gate for that lane -- before returning to await the
> 
> next arrival.
> 
> 
> 
> As you split the "chore" into finer pieces -- or, split the
> 
> handling of it into different/disjoint "execution contexts" -- you
> 
> need to pass more information between those objects.  E.g., passing
> 
> events of the form (<lane>, <event_type>) to a set of generic
> 
> "handlers" moves the sharing into the "event system".
> 
> 
> 
> [whether this is a fifo, shared memory, IPC, etc.]
> 
> 
> 
> OTOH, you increase the possibilities for concurrency and more
> 
> efficient use of resources (why have N "raise gate" processes
> 
> if drivers can afford to wait for THEIR gate to be lifted?
> 
> Perhaps the gate lift mechanism can ONLY lift a single gate at
> 
> a time (motor and gears/clutches).
> 
> 
> 
> Sorry for the long-winded explanation.  I will promptly be derided
> 
> for it.  But, hopefully it shows you different approaches (that
> 
> exploit "potential parallelism/decomposition" in different ways)
> 
> and the potential consequences of different approaches.
> 
> 
> 
> You have to look at your workload and see what approach makes the
> 
> most sense.  Interconnections are expensive in any algorithm!

Hi George,

Thanks for pointing it out. I agree that clone can be really handy in system architectures based on multiple threads running concurrently in shared memory space by controlling different levels of sharing between the parent and child tasks uses flags.

Karthik

On Tuesday, 23 September 2014 00:45:58 UTC+5:30, George Neuner  wrote:
> On Sun, 21 Sep 2014 05:40:26 -0700 (PDT), Karthik Balaguru
> 
> <karthik.balaguru007@gmail.com> wrote:
> 
> 
> 
> >Processes are heavy weight and they appear to occupy more memory,
> 
> >more time to create/start, increased latency during context 
> 
> >switches and separate memory space that necessitates heavy IPC 
> 
> >mechanisms. Threads are light weight and share memory space.
> 
> 
> 
> Yes and no.  
> 
> 
> 
> The best way to think of a process is as a resource container - a
> 
> thread is a particular kind of resource (a computation resource) that
> 
> a process can contain.
> 
> 
> 
> Processes also typically are protection boundaries whereas threads
> 
> typically are not [though there are exceptions].
> 
> 
> 
> 
> 
> >Considering the development environment as Linux OS with C 
> 
> >language on single core/multi-core processors, i would like to 
> 
> >know for which type of applications should we need to go in 
> 
> >for multi-threaded software architecture and for which type 
> 
> >of applications should we need to go in for multiple process
> 
> >based software architecture ?
> 
> 
> 
> In Linux there is a system call "clone" (see clone(2)).  Clone
> 
> essentially creates new threads, but it permits detaching a new thread
> 
> into a separate process and specifying with relatively fine control
> 
> what parent resources should be copied to the child.
> 
> 
> 
> Clone wraps an even lower level call that provides even more control
> 
> over the environment of the new thread.  Using clone you can create
> 
> very lightweight processes, e.g., just a thread with MMU protection.
> 
> 
> 
> >Karthik
> 
> George

Hi Karthik,

On 9/26/2014 8:11 AM, Karthik Balaguru wrote:

> Thanks for your quick reply !
> That was indeed a pretty long & an interesting explanation !!

The point is to show how a single application can be approached in a
variety of different ways.  And, within those different approaches,
how the "sharing"/contention can manifest -- or not.

Finally, the relative differences in costs between process vs. threaded
implementations when it comes to that sharing.

Figuring out how to approach YOUR problem (how to decompose it) will
be your first step to determining the "most effective" implementation.

On Monday, 22 September 2014 00:53:32 UTC+5:30, Tim Wescott  wrote:
> On Sun, 21 Sep 2014 05:40:26 -0700, Karthik Balaguru wrote:
> 
> 
> 
> > Hi,
> 
> > Have few queries on the best possible software architecture.
> 
> > 
> 
> > Processes are heavy weight and they appear to occupy more memory, more
> 
> > time to create/start, increased latency during context switches and
> 
> > separate memory space that necessitates heavy IPC mechanisms. Threads
> 
> > are light weight and share memory space. However, I realized that
> 
> > threads also enter into contention for resources/memory due to the
> 
> > shared resources among them that inturn becomes a kind of bottle neck
> 
> > for multi-threaded architecture but not for multiple process based
> 
> > architecture. Also the workaround for having thread local storage does
> 
> > not seem to be straight forward. This also makes me believe that
> 
> > maintaining multi-threaded application can be bit complex compared to
> 
> > that of multiple process architecture. Also that the performance of
> 
> > multi process architecture will be better due to separate memory space
> 
> > (This avoids locking or serialization of execution in case of multi
> 
> > process architecture) and this seems to take away the advantage of less
> 
> > context switch time in case of multi-threaded application !! Kindly let
> 
> > me know if this understanding is correct or correct with appropriate
> 
> > inputs.
> 
> > 
> 
> > I understand that the software architecture is mainly based on the type
> 
> > of application/requirement. Considering the development environment as
> 
> > Linux OS with C language on single core/multi-core processors, i would
> 
> > like to know for which type of applications should we need to go in for
> 
> > multi-threaded software architecture and for which type of applications
> 
> > should we need to go in for multiple process based software architecture
> 
> > ? Is there any matrix sheet  that maps/lists the type of
> 
> > requirements/applications and the possible software architecture for it
> 
> > ?
> 
> 
> 
> You are mistaken in your notion that just because threads explicitly share 
> 
> a memory space and other resources that they have more contention.  
> 
> Threads share a memory _space_, but they don't have to use the same bits 
> 
> of memory within that space -- it is easy to set things up so that each 
> 
> thread has its own chunk of memory that it uses.
> 
> 
> 
> In a sense, once you get past the MMU, processes share the same memory 
> 
> space, too -- it's just that the MMU protects each process from having to 
> 
> know about the memory space occupied by other processes, or even, for that 
> 
> matter, from having to know what physical addresses it occupies.
> 
> 
> 
> The "processes have separate memory space" is an illusion, provided in 
> 
> hardware by the MMU.  At the point where activity is going on in physical 
> 
> memory, all the processes have to access the same memory space, and so 
> 
> they contend for that resource.  Ditto hard drive accesses, screen access, 
> 
> etc.
> 
> 
> 
> Really, the biggest thing that you give up with threads vs. processes is 
> 
> that -- assuming the OS is doing its job -- processes are safe from one 
> 
> another.  Threads, however, can easily stomp on one another, simply by 
> 
> writing into some part of memory that some other thread is using and 
> 
> thinks isn't going to be disturbed.
> 
> 
> 
> For me, the dividing line between threads and processes is one of work 
> 
> load, processor loading, and trust: do I trust whoever is developing that 
> 
> software entity over there not to stomp on my stuff, and is it less work 
> 
> for both of us, using threads, to not stomp on each other's stuff than it 
> 
> is to just use processes?  And can the job be done at all using 
> 
> processes?  If the answer to the first two questions is "yes", then 
> 
> threads are indicated.  If the answer to _either_ of the first two 
> 
> questions is "no", then processes are indicated -- and if the answer to 
> 
> the third question is then "no", the project is in jeopardy.
> 

Hi Tim,
It is a very practical input. Also, the point of view based on the skillset of person in not stomping of another person's memory area is a really a good one to consider in any kind of project management and appears to be a real practical implementation check-point.

Thanks,
Karthik

On Sun, 21 Sep 2014 05:40:26 -0700 (PDT), Karthik Balaguru
<karthik.balaguru007@gmail.com> wrote:

>Hi,
>Have few queries on the best possible software architecture.
>
>Processes are heavy weight and they appear to occupy more memory, more time to create/start, increased latency during context switches and separate memory space that necessitates heavy IPC mechanisms. Threads are light weight and share memory space. However, I realized that threads also enter into contention for resources/memory due to the shared resources among them that inturn becomes a kind of bottle neck for multi-threaded architecture but not for multiple process based architecture. Also the workaround for having thread local storage does not seem to be straight forward. This also makes me believe that maintaining multi-threaded application can be bit complex compared to that of multiple process architecture. Also that the performance of multi process architecture will be better due to separate memory space (This avoids locking or serialization of execution in case of multi process architecture) and this seems to take away the advantage of less context switch time in case of
>multi-threaded application !! Kindly let me know if this understanding is correct or correct with appropriate inputs.
>
>I understand that the software architecture is mainly based on the type of application/requirement. Considering the development environment as Linux OS with C language on single core/multi-core processors, i would like to know for which type of applications should we need to go in for multi-threaded software architecture and for which type of applications should we need to go in for multiple process based software architecture ? Is there any matrix sheet  that maps/lists the type of requirements/applications and the possible software architecture for it ?
>
>Thx in advans,
>Karthik

Why wondering about processes vs. threads, use both as I have done for
decades.

I prefer keeping individual programs relatively small to help
manageability, protection and updatable, with only a few threads
within each address space. For larger systems with multiple processes
and address spaces, just create some shared memory areas and map these
areas into multiple process address spaces. 

For simple items (byte/word/dword) that can be accessed atomically,
you don't need any synchronization, for complex items, process A moves
xx megabytes of data to a shared memory area and then sends using some
OS specific mechanism to process B "I just uploaded xx megabytes, go
ahead".

On modern virtual memory OSes (Linux/Windows) shared regions are
implemented as memory mapped files (with or without backup to a real
file). 

If the shared memory area is linked to a fixed virtual memory address,
it must also fit into the same virtual address in each process and you
can use pointers within that memory area directly. If the shared
memory can be loaded at any virtual address in each process, pointers
within each process in that shared area must be recalculated. 

For applications intended for a long (more than a decade) support, one
must be careful how that shared memory is structured. Put a version
number and pointers to key data structures into the absolute beginning
of that shared area and in that way, process from different software
versions can access the same data structure and the shared data area
composition can be changed at will.