On Thu, 16 Mar 2017 17:23:11 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>Hi George,
>
>On 3/15/2017 6:05 PM, George Neuner wrote:
>
>>> Assume applet, memory_manager_module, applet_parent.
>>>
>>> Applet_parent (for example) instantiates a memory object for applet.  Part
>>> of that requires binding parameters to that memory object (size, granularity,
>>> allocation_policy, release_policy, etc.).  Applet_parent need not persist
>>> beyond this point (assume its job is done).
>>>
>>> Where do the bound parameters get stored?  In the applet?  In the memory
>>> object?  Obviously, can't be part of applet_parent because that won't
>>> persist (unless you make other arrangements for it or portions of it).
>>>
>>> [I.e., applet_parent can't define an allocation handler in its body.]
>>>
>>> Now, migrate applet to another node.  Another (shared) instance of
>>> memory_manager_module likely already exists on that node.  If the
>>> allocation handler was a "stock" handler (contained in the
>>> memory_manager_module), can you simply copy the state over to
>>> the new node?
>>>
>>> What if the allocation handler was resident in some OTHER module
>>> on the original node?  Do you migrate that other module, as well?
>>> Or, instantiate another copy if the module has outstanding references
>>> on the first node??
>>
>> For one "applet" (process / security context) to allocate memory on
>> behalf of another, that memory has to be supplied by something that
>> exists outside of either of them.
>
>Actually, the memory can exist anywhere:  in some global space, in
>the "allocating entity" or in the "client entity".  As can the code
>that does the actual allocating (at the behest of the allocator).
>
>[Consider client environment not supporting arbitrary pointers.
>Unless something *gives* you a reference/handle to a piece of memory,
>there's no way you can access its contents (short of an exploit).
>E.g., no way you can go looking up/down the stack even though you
>know there *is* a stack supporting the implementation language]
>
>> Thus the system memory manager can't be a dynamic "module" loaded
>> locally into a process.  Its *interface* could be, but not the manager
>> itself.
>>
>> Leaving that aside.
>>
>> Ok, so you have a block of system memory, and you created one (or
>> more) heaps having certain properties within the block - heaps which
>> are "managed" locally in the process by code from a dynamic module.
>> And now you want to rehost the process.
>
>Objects (heaps in this example) are managed by <something>.  That
>something may be the entity "owning" the object, a proxy or the
>OS acting as the "proxy of last resort".
>
>The actions involved in management may reside in the OS, a service
>(created by <something> -- including the client in question!) or the
>client.  At the direct or *implied* request of the client, etc.
>
>> 1) The obvious: store the heap properties together with the heap so
>> that "copying" (serializing) the contents of the memory block to
>> another host replicates the heap state for the equivalent "manager"
>> module on that system.
>
>This potentially exposes those "trusted" parameters to abuse by
>the entiti(es) having access to the underlying object.  E.g., if client
>can tweek the "heapsize" parameter stored there, then it can potentially
>trick the "manager" on the remote system to instantiate a larger heap than
>it was originally allowed to create (unless you rerun the "create_heap"
>request as part of the migration effort)
>
>Likewise, if some of the "properties" reside *in* the client (e.g.,
>the algorithm by which allocations will be performed -- under the
>supervision of some remote/system service that actually *does* them),
>then you have to drag that/those things along with you.
>
>[Imagine these things reside in a *third* entity]
>
>> 2) Rehosting a running process necessarily requires checkpointing,
>> copying or reconstructing its entire dynamic state: heaps, stacks,
>> globals, loaded modules, kernel [meta]data, etc. - far more than one
>> memory block.
>>
>> Every piece of distributed state must be identifiable as belonging to
>> the process - regardless of what "module" may have created or is
>> currently managing it.
>>
>> You have to copy all data belonging to the process, so having heap
>> properties stored separately from the heap itself is not a problem.
>
>In my "portable" case, this is handled by moving the object handles.
>The system can then opt to "optimize" execution by (later?) moving the
>actual object instances (to minimize communication delays or take
>better advantage of processing power on some particular node -- which
>may differ from the source or destination nodes)
>
>But, you can (currently!) create "non-portable" objects.  And, objects
>that can't (easily) be shared.  (e.g., for a "conventional" heap, you
>can let the default policies available in a "heap manager" govern
>the way the heap operates.  *But*, you can't take advantage of any
>"enhanced capabilities" in much the same way that you can't in a
>more conventional process container, etc.)
>
>The problem with this (apparently arbitrary) distinction between
>portable/shareable objects and "legacy" variety implementations
>is that the developer has to explicitly decide what can be migrated,
>shared, and how (because the developer has to take extra steps at
>compile and link time to make those capabilities available).
>
>So, where these sorts of bindings get stored (and how they get
>tracked) varies based on this "other" information that the developer
>supplies.
>
>[You don't want to have to change the sources to support these
>abilities as you want to be able to experiment with letting them migrate,
>be shared, etc. without having to undertake a rewrite/refactoring.
>And, I can't see an easy way for the build tools to determine that
>without explicit directives from the developer (hence my related
>comments in this thread)]
>
>> [Unless the systems involved do not have MMU/VMM such that the process
>> address space can be faithfully reconstructed ... but if you really
>> need to do it, it can be done using a software based virtual machine
>> solution.]
>>
>>> Or, do you require declarations as to what *can* migrate and what
>>> is bound in place?  Then, contend with the consequences of
>>> cascaded dependencies?
>>>
>>> The "consistent" solution is to make all of these first class
>>> objects and let the OS manage them and their locations/instantiations.
>>> But, that adds considerably to the cost of *using* them.
>>
>> Yes, that certainly is the ridiculous, overcompensating solution.
>>
>> The "consistent" solution is the one done already by any decent OS:
>> keep a record of all the kernel structures and memory pages belonging
>> to the process.
>>
>> Program objects then are just ordinary data in memory blocks "owned"
>> by the process, or by the kernel on behalf of the process.
>
>But that ignores all of the "other" dependencies that can be in
>play at any given time (i.e., my "solution" being these portable
>handles that the OS tracks for the application/developer).  So,
><something> knows that there is an aspect of the current applet's
>instantiation that relies upon something else *or* that is relied
>upon BY something else.
>
>[My approach lets the run-time know of these possible external
>references by the presence of object handles held -- or exported -- by
>the applet in question.  *Absent* these, the task is just a block of
>memory and "processor state" that can be copied anywhere and "resumed".]
>
>> Ensuring that a rehosted program will work is another matter.  But as
>> I understand (???) your system, there is location transparent naming
>> and equivalence[*] hosts will expose the same set of service APIs.
>>
>> [*] capable of running the same applications.
>>
>>> Its a lot easier to deal with implementations where "everything" is
>>> a cohesive "blob" instead of being able to slice-and-dice it at
>>> run-time and redistribute it based on evolving workloads, resources,
>>> etc.
>>
>> You can somewhat mitigate the problem by making large allocations
>> piece wise: abusing VMM to make them appear address contiguous.
>>
>> E.g., if a process asks for 100MB, reserve the address space but
>> instantiate just a few megabytes at a time, on demand as the process
>> touches unbacked addresses.
>>
>> [Because (I assume) you don't want to overcommit memory, you need to
>> reserve requested address space both locally in the process, and
>> globally so that other processes can't accidentally grab it.]
>
>Resource constraints vary with the resource and the consumer/provider.
>You can overcommit many resources because many "jobs" don't exploit
>their worst-case resource needs.
>
>But, you do so at the risk of delaying the availability of those resources
>to the job in question.  Or, some other job interested in those resources.
>
>[E.g., if your job is diarising a previously recorded telephone conversation,
>it can *probably* afford to block waiting on memory that it needs to complete
>that task -- perhaps even indefinitely!  It shouldn't be prevented from
>starting just because the MAX memory that MIGHT be required isn't presently
>available.  Nor should all of that memory be wired down for its benefit
>without knowing that it *will* need it -- or, when!]
>
>> You need to reconstruct the address space on the new host, but you
>> need only copy as much physical memory as the process is actually
>> using.
>>
>> Aside: one of the nice features of "moving" GC is that it compacts
>> live data into a (usually) smaller address range.  You don't want to
>> have to copy a large block to get at a much smaller volume of data
>> contained within it.
>
>But you need a way of tying a "handle" to the data involved -- and,
>later, resolving that reference.
>
>> There are a number of tricks that involve using VMM to implement GC.
>> You can't completely avoid data copying if you want to compact the
>> live data, but using VMM techniques together with clustered BiBoP
>> allocation, you can reduce the amount of copying to bare minimum.
>>
>> It also is possible to use VMM and just the tracing step of GC to
>> identify the actual set of pages containing live data ... you don't
>> need to copy any more than that even if the process has much more
>> memory instantiated.
>>
>> And regardless of whether you (would want to) use GC in normal
>> operation, if it is supported by the language/runtime, a compacting
>> collection executed before rehosting a process would minimize both
>> what needs to be copied and the physical memory that must be
>> instantiated on the target.
>>
>>> Last 25# of oranges... yippee!
>>
>> 10 inches of snow, followed by sleet, followed by freezing temps.  The
>
>Yeah, caught a glimpse of that on the national news...
>
>> crust is ~3/4 of an inch thick - my 185 lbs can walk on top of it. Had
>> to use the heavy coal shovel to break it up so it could be removed.
>
>90's, here.  I think 95 expected this weekend.  Being outdoors is a
>chore as it's either too warm during the daylight hours *or* the air
>too "smelly" (everything is in bloom) in the cooler hours when the
>breeze has died down.
>
>I'd (personally) appreciate a cold spell (but not *now* as the new
>crop of fruit is setting in place) to slow the growth process.  I
>suspect it will be a bad year for insects!
>
>[Normally, we don't see 95 until May; and 90 until mid-April]

I am not sure where you did move the goal posts this time, but at
least the VAX/VMS VAXcluster addressed some of these issues in the
1980s. Switching processes between (cabinet size) CPUs was routine via
a shared (redundant) disk pool (checkpointing and virtual memory page
faulting).

These days the disk pool could be part of the L3 (or L4) cache
hierarchy.

On 3/18/2017 4:04 AM, Don Y wrote:
> On 3/18/2017 3:05 AM, Tauno Voipio wrote:
>> Just get the book
>>
>> Andrew S. Tanenbaum, Operating Systems: Design and Implementation,
>>
>> read and understand it, then come back.
>
> AST doesn't address distributed OS's in that tome concentrating only
> on a toy OS.  Get his _Distributed Operating Systems_.  Understand *it*.
> Make sure you can understand why a distributed OS is a very different
> animal from a monolithic one.  Pay particular attention to the case
> studies of Mach, Amoeba and Chorus and the tradeoffs made in their
> *different* implementations.
>
> Then, go chase down the *technical* information on each of those.
> Boykin et al. would be a good place for you to get some exposure to
> what you could do with the original Mach.  Amoeba will require a fair bit
> more digging (no source code available).  Then, you can look into the

Sorry, Amoeba *is* available; *Alpha* (Jensen et al.) isn't -- but has
similarly applicable concepts (particularly capability handling).  You'll
need to understand both approaches, and their consequences, before you
can appreciate how they can be leveraged for security and reliability
(esp redundancy).

> Inferno docs to see how they handle location transparency and followup
> issues addressed in their Grid implementation.  From there, imagine how
> you'd support resource migration in each of the above -- and how
> you'd expose it to the developer.
>
> Finally, sort out how to extend all of these concepts to real-time
> programming.
>
> [All this assumes you've already been through Organick, McKusick, etc.]
>
> Don't bother coming back -- been there, done that, T-shirt, etc.  :>
>

On 3/18/2017 3:05 AM, Tauno Voipio wrote:
> Just get the book
>
> Andrew S. Tanenbaum, Operating Systems: Design and Implementation,
>
> read and understand it, then come back.

AST doesn't address distributed OS's in that tome concentrating only
on a toy OS.  Get his _Distributed Operating Systems_.  Understand *it*.
Make sure you can understand why a distributed OS is a very different
animal from a monolithic one.  Pay particular attention to the case
studies of Mach, Amoeba and Chorus and the tradeoffs made in their
*different* implementations.

Then, go chase down the *technical* information on each of those.
Boykin et al. would be a good place for you to get some exposure to
what you could do with the original Mach.  Amoeba will require a fair bit
more digging (no source code available).  Then, you can look into the
Inferno docs to see how they handle location transparency and followup
issues addressed in their Grid implementation.  From there, imagine how
you'd support resource migration in each of the above -- and how
you'd expose it to the developer.

Finally, sort out how to extend all of these concepts to real-time
programming.

[All this assumes you've already been through Organick, McKusick, etc.]

Don't bother coming back -- been there, done that, T-shirt, etc.  :>

On 3/17/2017 11:25 PM, Don Y wrote:

>> When the time comes to replicate the process's memory on another host,
>> only mapped physical space needs to be copied - not the entire virtual
>> address space reservation.
>
> Again, that assumes the (active/mapped) memory represents the entire
> "state" of the process and all of its dependencies.  (what about any
> network connections -- managed by the OS -- that may be active at the
> time?  the contents of any NIC buffers AS the process is in transit?
> file handles, my "object handles", etc.)

ProcessA:
     ThreadA1();
     ThreadA2() {
        ...
        // some sort of synchronization primitive with ThreadA3
        send_resource(my_potato, ProcessC)
        ...
        exit
     }
     ThreadA3() {
        ...
        my_potato = create_potato(size, flavor, &my_masher)
        ...
        exit
     };
     my_masher();

ProcessB:
     ThreadB1();
     ThreadB2();
     create_potato();
     default_masher();

ProcessC:
     ThreadC1();
     ThreadC2() {
        ...
        delete(received_potato)
        ...
     };

Normally, when all ThreadAi have exited, ProcessA would die and
its resources be reclaimed.

If create_potato() was hosted by ProcessA *and* the send_resource()
targeted ProcessA, then everything is self-contained; when everything
dies/exits, the potato can be deleted (because nothing outside the
ProcessA references it).  There is no visibility outside of that
process container for any of the objects contained within.

If we assume ProcessB has the "potato server" (i.e., if create_potato()
is an aspect of that SERVICE), then ProcessB must remain active for
the potato to be created *and*, once created, must continue to remain
active to service requests made of the "my_potato" object (because
my_potato is just a handle to the potato abstraction reified by
the potato server in ProcessB).  Note that this dependency is extended
to ProcessC once the potato resource has been transfered to ProcessC
by ThreadA2!

In addition to ProcessB remaining active while the outstanding reference
to the potato exists, ProcessA must remain active even after the potato
has been passed to ProcessC -- because of the dependency on "my_masher"
that is bound to the instantiation of the potato as it occurred in
ThreadA3!

[Maybe "my_masher" leaves a certain amount of chunkiness in the potatoes
after the mash method is applied -- by the potato server -- while the
default_masher ensures a more uniform, pasty consistency]

<something> has to track these dependencies.  We (I) don't trust the
developer to write (bug-free) code to "manually" do that -- not for EVERY
such "distributed object".  The compiler can't do it because it
doesn't know where the objects will reside.

By assigning capabilities/handles to each of these objects (a masher
is an object just like a potato is an object and a potato server is an
object and a process is an object and a thread is an object and a...),
the OS can keep track of:
- where each object "resides"
- where each reference to a particular object resides
- how many outstanding references to an object exist

So, when ThreadC2 finally deletes the potato (which *it* knows by the
name "received_potato"), the resources bound up in that potato instance
can be freed (from the "potato server" in ProcessB as well as the
"masher service" in ProcessA -- in addition to the handle in ProcessC!

If ProcessA (and/or ProcessB and/or ProcessC) now has no remaining
active resources (e.g., all threads have exited), then *they* can
be reclaimed along with their resources.

If the system opts to migrate ProcessA (containing my_masher()) to
another node *while* that potato is still active, then the resources
previously used for that *instance* of ProcessA (on NodeX) can be
freed -- but, an outstanding claim levied against the resources that
ProcessA (now known as ProcessQ) *still* requires while being hosted
on NodeY.

So, if received_potato->mash() is invoked by a thread in ProcessC
*after* it has accepted the transfer of "received_potato" but before
it has been deleted, the potato server in ProcessB will receive that
RMI and, as part of its *mechanism*, will invoke the masher member
that will be implemented by my_masher() in ProcessA (if ProcessA
hasn't yet migrated) *or* by ProcessQ (if ProcessA *has* already
migrated).

There's no clear cut concept of the potato's "ownership".  The
process that has "rights" to it changes (from A to C in this example).
The process that *implements* it morphs (from B assisted by A to
B assisted by Q).  A single object has tied up resources in three
different process containers.

For a concrete EXAMPLE of similar objects, imagine the potato is a
video_clip and there are "compress", "store", "display", "motion_detect"
and "commercial_detect" methods available.  Further, that the user
can *bind* (returning to the concept of closures) particular functions
and criteria to each of these.

So, maybe an "MP4" functionality is bound to a Compressor object and
that compressor object bound to the clip.  I.e., my_clip->compress()
effectively causes an MP4 version of the clip to be created.  The
MP4 functionality is provided by an MP4 *module* in ProcessD running
on nodeI (because nodeI has the resources available to expedite that
processing).

Meanwhile, a Region object (polygon) may be created and bound to the
motion_detector object that is created to determine which portion(s)
of my_clip to examine for signs of motion in the motion_detect method.

And, criteria for a "commercial_detector" bound to a detector that
is eventually bound to that video_clip and subsequently invoked
by the my_clip->elide_commercials() method running in ProcessL on
nodeW.

Why force every compression algorithm to be implemented in a single
"Compressors" module?  Or, be supported by set of resources defined
by the UNION of their requirements?  Why force all commercial
detection algorithms to co-reside with these compressors?  And, all
to reside in a "video_clip" module?  Or, execute on the same hardware
node, concurrently?

Why force all of these bindings to persist if the client(s) that
*had* permission to invoke them have terminated?  (i.e., if there
is only one outstanding capability for the compress method and
the process holding that capability terminates, then NOTHING can
invoke that compressor!  So, the *binding* can be freed in my_clip.
And, in return, the service that implements it (in ProcessD) can be
allowed to die an orderly death (if no other outstanding references).

Put *all* of this in a single process and the OS has no way to
know when certain resources are no longer required -- because
the containing object (the process) can't inform the OS of "things"
of which the OS has no knowledge (this is A Good Thing because it
means the OS doesn't HAVE TO know about everything!)

On 18.3.17 08:25, Don Y wrote:
> On 3/17/2017 8:59 PM, George Neuner wrote:
>> On Thu, 16 Mar 2017 17:23:11 -0700, Don Y
>> <blockedofcourse@foo.invalid> wrote:
>>
>>> On 3/15/2017 6:05 PM, George Neuner wrote:
>>>
>>>> For one "applet" (process / security context) to allocate memory on
>>>> behalf of another, that memory has to be supplied by something that
>>>> exists outside of either of them.
>>>
>>> Actually, the memory can exist anywhere:  in some global space, in
>>> the "allocating entity" or in the "client entity".  As can the code
>>> that does the actual allocating (at the behest of the allocator).
>>>
>>> [Consider client environment not supporting arbitrary pointers.
>>> Unless something *gives* you a reference/handle to a piece of memory,
>>> there's no way you can access its contents (short of an exploit).
>>> E.g., no way you can go looking up/down the stack even though you
>>> know there *is* a stack supporting the implementation language]
>>
>> Moving the goal-post again.
>>
>> From previous discussions, I know your system works similarly to Mach
>> ... and while I don't know your system, I *do* know Mach.  You're
>> conflating things when you talk about "allocating entities" (aka Mach
>> servers), and introducing complexity where none should exist.
>>
>> E.g.,
>>
>> The Mach memory manager (MM) grabs most of the RAM for itself and
>> creates a heap from which it parcels blocks to other processes.
>>
>> An user allocated memory block physically exists IN THE MM's HEAP, but
>> semantically it *belongs* to the user process: it is mapped into the
>> address space of the user process, else that process could not use it.
>> When you go to rehost this process, you need to consult the MM only so
>> far as to know the attributes of the block.  The block itself is IN
>> THE USER PROCESS, and what it contains has to be replicated at the
>> same VMM "linear" address in the new process on the new host.
>>
>> The MM itself need not be involved in the replication.  In fact you
>> probably would not want it to be because you only want to copy live
>> data and not the entire address space of the rehosting process.
>>
>> The access/call and identity mechanisms: i.e. the "handles" by which
>> the MM is called, and by which the block is known to the MM - mostly
>> are beside the point.
>
> (sigh)  You're fixating on AN EXAMPLE of an OBJECT that can need things
> like "allocation_policy" bound to it.  (rewind the thread to see where I
> introduced a "heap").  Reread the thread with "potato" substituting
> for all instances of "heap".  And, "mashing_policy" for
> "allocation_policy",
> "weight" for "size", etc.
>
> THEN, lecture me as to how the "OS" owns this object and how *its*
> primitives (VMM in your example) apply to its management.
>
> [Should I, in the future, talk STRICTLY in terms of abstractions and avoid
> ANY pleas to reify those concepts?  For fear that a reader will seize on an
> example as representative of the entire universe of possibilities?  How
> much "imagination" should I expect of readers?]
>
>>>> Ok, so you have a block of system memory, and you created one (or
>>>> more) heaps having certain properties within the block - heaps which
>>>> are "managed" locally in the process by code from a dynamic module.
>>>> And now you want to rehost the process.
>>>
>>> Objects (heaps in this example) are managed by <something>.  That
>>> something may be the entity "owning" the object, a proxy or the
>>> OS acting as the "proxy of last resort".
>>
>> Now you are getting completely away from the "memory" question you
>> initially asked and into objects managed by servers on behalf of
>> clients.
>>
>> Again with the goal post.
>
> "Imagination".  Remember, we're talking about potatoes...
>
>> An IN-PROCESS server allocates from memory already owned by the client
>> process.  The client DOES NOT OWN an object which it can't touch
>> directly, and which it can only affect via RPC to another process.
>
> An in-process server allocates RESOURCES from resources that it
> can REFERENCE from ANY OTHER SERVICE (which may be the OS or any
> other object server to which the "in-process server" has rights.
>
> The potato server can call upon a "masher" supplied by a "mashing service"
> (and the poor OS is completely clueless as to the concept of "mashing").
> The handle for the object that it then instantiates is all the client
> ever needs to manipulate this particular potato.
>
> The potato server can migrate to another node.  The client referencing
> the potato can migrate.  The client can give the potato to someone else.
> The potato server can pass management of the potato to a different
> instance of a *compatible* potato server.
>
> The potato can be "serialized" to a persistent medium and later
> reconstituted ("instant potatoes"!  :> )
>
> The application need be aware of none of these things.
>
>> Replicating the memory state of the process on the new host
>> automagically handles the case of an in-process server.  It also
>> replicates client side state for anything controlled on its behalf by
>> an out-of-process server.
>>
>> The state of an out-of-process server does not factor directly into
>> rehosting a client.  All the server needs to know (if even this much)
>> is that the client's "session" is to be suspended pending reconnection
>> rather than terminated.
>
> Replicating "memory" doesn't replicate the state of the machine because
> other machines are involved.
>
> Copy the memory image of *my* PC's "shell", at this moment, onto your PC.
> Now, tell me which files I have open.  Which TCP connections are live.
> etc.
> The "state" is spread around in several places *besides* the process
> itself.  Including the mail server at the other end of this SMTP connection
> (when I click SEND).
>
> [I.e., unless the passphrase for my SMTP server persists somewhere in
> this process's memory, your copy of it's memory won't help you access
> that connection that exists *now*]
>
>>> The actions involved in management may reside in the OS, a service
>>> (created by <something> -- including the client in question!) or the
>>> client.  At the direct or *implied* request of the client, etc.
>>
>> You are failing to realize that it doesn't matter who manages the
>> data: it matters only who *owns* the data.
>>
>>> Likewise, if some of the "properties" reside *in* the client (e.g.,
>>> the algorithm by which allocations will be performed -- under the
>>> supervision of some remote/system service that actually *does* them),
>>> then you have to drag that/those things along with you.
>>
>> Or simply provide them in the new setting.  E.g., there is no reason
>> to copy in-memory code if the new host has access to it via a file
>> system.  If the new host has a different architecture, you can't copy
>> the code anyway and would have to provide checkpoint mechanisms
>> whereby the *program* (not "process") can be stopped and restarted
>> cleanly.
>
> The applets run in a virtual machine.  So, I can pick them up and
> move them to another heterogeneous host.  (The "servers" are currently
> designed for a homogeneous environment.  But, the same concepts apply)
>
>>>> 2) Rehosting a running process necessarily requires checkpointing,
>>>> copying or reconstructing its entire dynamic state: heaps, stacks,
>>>> globals, loaded modules, kernel [meta]data, etc. - far more than one
>>>> memory block.
>>>>
>>>> Every piece of distributed state must be identifiable as belonging to
>>>> the process - regardless of what "module" may have created or is
>>>> currently managing it.
>>>>
>>>> You have to copy all data belonging to the process, so having heap
>>>> properties stored separately from the heap itself is not a problem.
>>>
>>> In my "portable" case, this is handled by moving the object handles.
>>> The system can then opt to "optimize" execution by (later?) moving the
>>> actual object instances (to minimize communication delays or take
>>> better advantage of processing power on some particular node -- which
>>> may differ from the source or destination nodes)
>>>
>>> But, you can (currently!) create "non-portable" objects.
>>
>> So they are lost.  So what?  You knew they were non-portable when you
>> created them.
>
> But the apps that contain them can be portable!  You might have
> *chosen* to make a non-portable potato (by implementing a potato
> server as one of the threads in your process).  The potato is
> bound to the process.  But, the process isn't (necessarily) bound
> to the *node* (or process group, etc.).
>
> E.g., I can create a mutex to allow the local threads to cooperatively
> access <something> (maybe just a 4 byte counter).  As long as the threads
> remain bound in that process container AND the counter (or it's accessor)
> AND the mutex, the whole entity (process) can be moved at will.
>
> If some other process/agent needs to access that mutex, then it
> needs a handle that is visible from outside of the local process.
> If that other process can migrate to a different node, then it
> can retain the ability to interact with the mutex in this first
> process still residing on the original node.
>
> So, you end up with very different use/performance cases:
> - two or more colocated threads accessing the object
> - two or more LOCAL *processes* accessing the object
> - two or more REMOTE processes accessing the object
>
> [Note, of course, that any other *object* can also indirectly be accessing
> that object/potato]
>
> In the first case, you can conceivably optimize the access to that of
> a conventional mutex (in this example -- remember potatoes!)
> implementation.
>
> In the second case, one process might be able to access the object
> "from within" (a direct call to the functions that implement the
> object) while another process (same node) accesses it "from without".
>
> In the third case, one may be from within, or without, or "remote".
>
> I don't consider it a "good idea" to be forced to make these "mobility"
> decisions at compile time.  For the same reason that you don't decide
> where in
> memory a piece of code should execute.  Or, whether it WILL be optimized by
> the compiler, or not (compiler command line options aren't reflected in
> the sources)
>
> Just like I don't define which functions/methods should be accessible
> remotely (let the IDL compiler address that based on what I tell *it* to
> do)
>
>>> And, objects
>>> that can't (easily) be shared.  (e.g., for a "conventional" heap, you
>>> can let the default policies available in a "heap manager" govern
>>> the way the heap operates.  *But*, you can't take advantage of any
>>> "enhanced capabilities" in much the same way that you can't in a
>>> more conventional process container, etc.)
>>
>> There's no reason code can't use "enhanced capabilities" - as long as
>> the result is the same (for some definition).  E.g., if you can ignore
>> timing, then debug code is interchangeable with optimized code.
>
> That's been my goal!  But, it means the code has to reconfigure itself AT
> RUNTIME.  Because the compiler doesn't know where an instance of a potato
> will reside at *compile* time.
>
>>> The problem with this (apparently arbitrary) distinction between
>>> portable/shareable objects and "legacy" variety implementations
>>> is that the developer has to explicitly decide what can be migrated,
>>> shared, and how (because the developer has to take extra steps at
>>> compile and link time to make those capabilities available).
>>
>> It is completely arbitrary, and IMO unnecessary.
>>
>> If there is something - a device possibly - that only exists on one
>> host, then the module that interacts DIRECTLY with that <something> is
>> not portable.  Anything objects it creates as a server on behalf of a
>> client likewise are not portable.
>
> No.  A "something" that CAN only exist on one host (node) is not portable.
> A "something" that HAPPENS TO only exist on one node but CAN be migrated
> to another node doesn't suffer those same restrictions.
>
> I put a thin layer immediately above the hardware devices on each node
> (think of it as a sort of HAL).  At the top interface of this layer, I
> make a deliberate choice as to whether or not that interface shall be
> exportable across nodes.
>
> For example, there is little need to bind anything that uses the
> "irrigation valve" hardware (HAL layer) to the node that actually
> has the hammer drivers for those valve actuators.  They are inherently
> slow and largely sacrificial.  So, that interface is exported.  The
> "irrigation control task" can reside anywhere in the system -- including
> nowhere (zombie).
>
> OTOH, the interface to each of the video cameras is relatively high
> bandwidth.  The task(s) that directly interact with them *should*
> reside on those nodes (*if* you assume the camera output will see
> much consumption).  So, those interfaces are not exported.
>
> OToOH, the interface to the *compressed* video streams provided by those
> tasks *can* be exported.  Their bandwidth requirements might be
> considerably less than the "raw" video coming off each individual
> camera.  So, a "motion detection" task can run on the IRRIGATION
> NODE.  Or, mode to the LAUNDRY NODE if/when the irrigation node
> is taxed with some other responsibility (perhaps it is taken out of
> service as the valves -- and the proxies that run them -- are no longer
> needed)
>
> I.e., you gain the most flexibility by making everything exportable.
> But, that means you either incur the costs associated with that
> overhead -- even for the "local" case (e.g., motion detection task
> running on the same node as the camera!) -- *or* do run-time
> optimization of the communication path so that the remote costs
> are only incurred when necessary (and factored into the workload
> manager's reassignment of tasks to nodes)
>
>> So if this module is to be used by portable applications, it should
>> NOT be implemented as an in-process server.   Q.E.D.
>>
>>> So, where these sorts of bindings get stored (and how they get
>>> tracked) varies based on this "other" information that the developer
>>> supplies.
>>
>> This is where you and I have parted company in the past.  You like the
>> kernel capability approach whereas I prefer the user capability
>> approach ... precisely BECAUSE user capabilities avoid many of these
>> semantic problems by virtue of being ordinary data.
>
> The value of the in-kernel capability is that inherently tracks
> references.  If the capability is "just data", then you have to
> add a second mechanism to track *where* every copy of the data
> happens to reside!  There's no way for the system to know that
> you've just handed a COPY of a capability to something, deleted
> YOUR copy, then retrieved a copy of that copy!  (i.e., the
> "reference" is still "live", even though it looks like you
> deleted it).  The "system" has no idea who has outstanding
> references (handles/capabilities) nor how MANY.
>
> By contrast, if the kernel manages the capabilities, then they
> can't (by definition) be copied or transferred or deleted
> without the kernel being an active part of that operation.
> The kernel (and the manager for that particular capability)
> can mediate how each is used ("No, you can't give that
> capability away!  If you try to do so, I will abend your
> process and/or just silently DELETE the capability and let
> you and the other party wonder why that operation that
> you intended to have performed *isn't*!")
>
> "Is anyone using this video camera?  No, there are no active
> capabilities in existence ANYWHERE in the system so we can
> shut it down.  Is anyone (process) using the node that hosts
> that camera?  No, so we can shut down the node, as well!"
>
>> Everything other than kernel level process and address space control
>> can be done in user space, and rehosting then is simply a matter of
>> replicating the memory state [and reconstructing the kernel state].
>>
>>>>> The "consistent" solution is to make all of these first class
>>>>> objects and let the OS manage them and their locations/instantiations.
>>>>> But, that adds considerably to the cost of *using* them.
>>>>
>>>> Yes, that certainly is the ridiculous, overcompensating solution.
>>>>
>>>> The "consistent" solution is the one done already by any decent OS:
>>>> keep a record of all the kernel structures and memory pages belonging
>>>> to the process.
>>>>
>>>> Program objects then are just ordinary data in memory blocks "owned"
>>>> by the process, or by the kernel on behalf of the process.
>>>
>>> But that ignores all of the "other" dependencies that can be in
>>> play at any given time (i.e., my "solution" being these portable
>>> handles that the OS tracks for the application/developer).  So,
>>> <something> knows that there is an aspect of the current applet's
>>> instantiation that relies upon something else *or* that is relied
>>> upon BY something else.
>>
>> No it doesn't.
>>
>> You have created a whole set of artificial dependencies through your
>> (ab)use of kernel capabilities.  Nothing in your system is - or even
>> can be - self contained.
>
> A process need never export a handle to an object that it creates
> and manages -- for itself.  The OS needn't even be aware that the
> object exists.  Does the OS know that I have a 4 byte counter
> *here*?  Or here?
>
> OTOH, if something *else* DEPENDS on that counter (potato), then
> you want that dependency to be visible to the system.  You don't
> want the potato to silently disappear without the dependent entity
> knowing about it, *now* (not at some later date when it expects it
> to STILL exist).
>
> Likewise, if it "moves", you don't want the dependent entity to have to
> scramble to *find* it when it needs it -- for much the same reason as
> above.
>
>>> [My approach lets the run-time know of these possible external
>>> references by the presence of object handles held -- or exported -- by
>>> the applet in question.  *Absent* these, the task is just a block of
>>> memory and "processor state" that can be copied anywhere and "resumed".]
>>
>> But by doing that, you've inserted the operating system into all kinds
>> of things where IMO it has no business being.  You've created the MCC
>> from "Tron".
>
> An OS manages resources.  I consider dependencies to be resources.
> And, the permissions associated with them.
>
> I don't want a user-level task to have to keep track of where the
> things that it needs happen to be, *now* -- or if they even happen
> to EXIST!
>
> Nor do I want to have to require the developer to never try to do
> something with a particular thing/object (potato) that it *shouldn't*.
> Just like the OS shouldn't let process A peek into process B's memory.
> Or, access unmapped portions of memory, etc.
>
> You'd not advocate allowing a user to "add 3" to an arbitrary string.
> You'd expect the *compiler* to enforce the rules of the language on
> his *use* of the language.
>
> I expect the OS to enforce the rules of the *system* on the users
> of that system in much the same way!
>
>>>> You can somewhat mitigate [rehosting] by making large allocations
>>>> piece wise: abusing VMM to make them appear address contiguous.
>>>>
>>>> E.g., if a process asks for 100MB, reserve the address space but
>>>> instantiate just a few megabytes at a time, on demand as the process
>>>> touches unbacked addresses.
>>>>
>>>> [Because (I assume) you don't want to overcommit memory, you need to
>>>> reserve requested address space both locally in the process, and
>>>> globally so that other processes can't accidentally grab it.]
>>>
>>> Resource constraints vary with the resource and the consumer/provider.
>>> You can overcommit many resources because many "jobs" don't exploit
>>> their worst-case resource needs.
>>>
>>> But, you do so at the risk of delaying the availability of those
>>> resources
>>> to the job in question.  Or, some other job interested in those
>>> resources.
>>
>> No, you are misunderstanding.  The process owns the space ... but in
>> the scenario I presented, the physical memory backing the space is
>> mapped on demand - which gives the OS more visibility into the process
>> if/when it needs to rehost the process.
>
> The "handles" give the workload manager (via the OS) insight into how and
> where it can/should rehost the process (or any other object!).  It knows
> what the process "talks to" because it manages the capabilities FOR that
> process.  And, it knows the other (more tangible) resources that the
> process requires (RAM, MIPS) by examining its current ledger and historical
> performance.  So, it can balance the costs of "stretching" certain
> communication channels (RPC/RMI being more expensive than the local case)
> against the costs that some *other* process/object (a process is just a
> different sort of object, managed by similar handles) would incur if
> *this* process is not migrated.
>
> [You can't come up with an "optimal" solution.  But, you can remember the
> solutions you've tried and how they performed -- because the OS is
> involved in these communications and can measure them!  So, you can
> avoid solutions that performed poorly in favor of those that performed
> *well* or that "have yet to be tried.]
>
>> Remember that there are 3 forms of VMM addresses: "linear", "virtual"
>> and "physical".
>>  - "Linear" space is what is seen by an application.
>
> "Logical" -- it need not be linear/contiguous.
>
>>  - "Physical" space is the actual RAM as seen by the system.
>>  - "Virtual" space is what ties the other 2 together.
>
> And, how does this pertain to potatoes?  :>
>
>> In a system using MMU only for process isolation, the "virtual" and
>> "physical" spaces would be identical - mapped 1:1.   Overcommitting
>> memory is only possible when "virtual" space > "physical" space.
>
> Exactly.  Recall that there are multiple objects sharing that physical
> space on a particular node.  The physical space determines the maximum
> amount that can be mapped at a given time to the set of objects
> currently making demands on it.  Those demands can (do) change, over time.
>
> E.g., a process may load many modules and, by executing code *in*
> each of them, cause pages to be mapped to instantiate those "opcodes"
> JIT.  If a portion of a module is never actively referenced, then
> the memory required to back it never gets mapped -- not taxing the
> physical resources currently available.  If a module gets UNloaded,
> then all of the mapped pages that were referenced can be released for
> other uses (other potatoes).  Likewise, for the pages that were
> never actually mapped!
>
>      E := load Encyclopaedia
>      ...
>      article := E.lookup(Da Vinci)
>      ...
>      unload(E)
>
> You wouldn't foolishly load the entire encyclopaedia into physical
> (or virtual) memory to do this.  Instead, you'd load some accessor
> code that could, itself, load PORTIONS of the "index" *when* you
> initiated a search.  This would require some number of pages of
> physical memory to be mapped to hold the resulting "article".
> Anything beyond that would be waste!
>
>      E := load Encyclopaedia
>      ...
>      article := E.lookup(Da Vinci)
>      ...
>      article := E.lookup(R Crumb)
>      ...
>      unload(E)
>
> Would only require resources proportional to the LARGER of the Da Vinci
> and R Crumb entries (the resources held by the first "article" are
> freed when the second "article" is defined).
>
>> When an application requests 100MB [e.g., (s)brk] from the OS, then
>> [assuming the request can be granted] the OS adds 100MB to the linear
>> space of the process, and *reserves* 100MB in the virtual space of the
>> system.
>
> No.  I don't *impose* that requirement!  Because it makes the system
> brittle.  The application handles this at a different level dictated
> by the CM system.
>
> If, in your example, you ultimately *use* all 100MB of that memory
> (which may be mmap'd in page-at-a-time increments) and then need
> (or, someone else needs) 50MB *more*, the system decides how best to
> provide those resources based on parameters from the CM system.
>
> If "more resources" are available (though not necessarily "50" MB),
> the system can opt to satisfy the request and let you continue to
> mmap (by your usage patterns) additional pages.
>
> If there are NO more resources available *when* you mmap another
> page (indirectly by your actions), the system decides how best to
> handle that request.
>
> Can I unmap some portion of a loaded module (knowing that I can
> always reload that portion of the module AS IF it hadn't *yet*
> been referenced/mmap'd)?  Can I pageout some "really low" portion
> of the stack (on the assumption that it won't be accessed RSN)?
> Can I *migrate* some other task off of this node and reclaim the
> physical resources that it currently holds?  Can I migrate *this*
> task to some other node that has more resources available?  Can
> I bring another node on-line and migrate "something" to it?  Can
> I rely on some redundant (potato!) server to assume responsibility
> for managing some objects that happen to reside on the local system
> and, as such, free up the space they occupy *and* the space
> required for the code to manage them?  Can I *kill* some task
> (and associated objects) that are just running opportunistically,
> presently (let's do some commercial detection in that OTA TV
> broadcast we recorded two hours ago so its ready for viewing before
> the user comes home and wants to see it!)? etc.
>
> You don't want to make those decisions at compile time.  Because you
> don't know what will be active in the system at *run* time!  And,
> you want to build as much flexibility in how/where resources can
> reside so that you have more run-time options.  You don't want to
> overprovision just to handle a "worst case" that, in practice, will
> never occur!
>
> [Repeat this substituting "MIPS" for "memory", etc.]
>
>> Now the reserved space belongs to the requesting process - no other
>> process can take it.
>
> See above.  If a process (potato) *needs* to be sure that resource
> (memory, MIPS, etc.) *is* available, "sitting and waiting", then it
> has to place an active "reservation" through the CM system.  The
> CM system dictates limits for what individual tasks/"users" can
> consume.
>
> Reserves are A Bad Thing as they tie up those resources regardless
> of what other objects might happen to place similar demands on the
> system's resources.  What's to stop "everyone" from demanding EVERYTHING
> that they *might* need "just in case"?
>
> If, instead, you allow elasticity in how the resources are supplied,
> you can afford to loosen the constraints on individual "consumers"
> because they have less ACTUAL impact on other consumers.
>
> Joe User (not Bob Developer) could write a sloppy script that
> consumes "limitless" resources.  Should he be allowed to bring the
> system to its knees -- because he is the "owner" of the device?
>
> Similarly, should Bob Developer be allowed free reign at the risk
> of compromising core services (implemented by *me*)?
>
> It's a lot easier to ensure your system can meet all its goals if
> you provision *for* those goals.  But, with an open/expandable system
> that sees huge variations between actual and potential use, that
> translates into dollars that consumers aren't keen on spending.
>
> (why do you have more VM configured in your PC than physical memory?)
>
>> However, the OS can choose whether or not to immediately map the new
>> linear space to physical space.  By mapping physical space on demand
>> as it is touched by the process, the OS has a much better picture of
>> the *actual* memory use of the process.
>
> Yes.  Though that can vary with other "input data" that this instance
> of the process (potato) sees.  E.g., the resources required for that
> lookup(R Crumb) are probably far less than the earlier lookup(Da Vinci).
> Do you *know* what the user will opt to want to "lookup" when you
> instantiate that task?  :>
>
> You can either say "you can lookup ANYTHING" -- and provision for it.
> Or, "you can look up anything as long as the article doesn't exceed X"
> (and impose a disconcerting constraint on the user:  "How do I know
> how big the article is LIKELY to be?").  Or, you can make a best
> effort and trade your response based on your current needs and those
> just added by the user.
>
>> When the time comes to replicate the process's memory on another host,
>> only mapped physical space needs to be copied - not the entire virtual
>> address space reservation.
>
> Again, that assumes the (active/mapped) memory represents the entire
> "state" of the process and all of its dependencies.  (what about any
> network connections -- managed by the OS -- that may be active at the
> time?  the contents of any NIC buffers AS the process is in transit?
> file handles, my "object handles", etc.)
>
> [Think about it.  Send me an ISO with an image of your PC's memory
> at this point in time.  I'll let you include the OS's state in that
> image, as well!  Will I be able to read this USENET post at exactly
> THIS --> . <-- point when I load the image into my machine's memory?
> What if you mark the thread as read before the ISO gets loaded on
> my machine?  Or, if YOUR server expires the article?  Or, if
> your NNTP server happens to be offline?  That's all state that is part
> of your "reading this USENET post" activity -- yet, you've not captured
> it well enough for me to pick up where you left off!  :> ]


Just get the book

Andrew S. Tanenbaum, Operating Systems: Design and Implementation,

read and understand it, then come back.

-- 

-TV

On 18/03/17 00:19, Les Cargill wrote:
> Tom Gardner wrote:
>> On 17/03/17 12:16, Les Cargill wrote:
>>> Tom Gardner wrote:
>>>> On 17/03/17 11:01, Les Cargill wrote:
>>>>> Tom Gardner wrote:
>>>>>> On 14/03/17 02:23, Paul Rubin wrote:
>>>>>>> Tom Gardner <spamjunk@blueyonder.co.uk> writes:
>>>>>>>> "When I was 14 I thought my father was an idiot. When I became 21 I
>>>>>>>> was amazed at how much he had learned in the past 7 years"
>>>>>>>
>>>>>>> One of the profs at my old school said at his retirement dinner "I've
>>>>>>> been teaching these kids freshman calculus for FORTY YEARS and they
>>>>>>> still don't get it!".
>>>>>>
>>>>>> :)
>>>>>>
>>>>>> We learned integration and differentials of polynomials
>>>>>> except 1/x for exams at 15. I can still visualise the
>>>>>> teacher taking a double period (80 mins) for each,
>>>>>> deriving the concepts from first principles.
>>>>>>
>>>>>> OK, he was a good teacher (but not a good mathematician,
>>>>>> he knew his limits!), but even so I've never understood
>>>>>> why people think calculus is inherently difficult.
>>>>>>
>>>>>
>>>>>
>>>>> Limits are conceptually difficult. It takes work. You
>>>>> have to be able to suspend disbelief in things like infinity.
>>>>
>>>> Work? Yes, good :)
>>>>
>>>> Difficult? Yes, there is a graunch; It probably took
>>>> 40mins of teaching and discussion.
>>>
>>>
>>> I remain highly skeptical of that figure. That was
>>> for the initial gloss; it takes a considerably longer
>>> time to gain a working knowledge of limits.
>>
>> It was sufficient for us to believe in the concepts,
>> and to be able to understand the explanation of how
>> polynomials (except 1/x) could be differentiated.
>
> Sure.
>
>> We then went on to use differentiation for the
>> O-level exams at 16. Well, I say 16 because that
>> was the normal age, but everybody in my school
>> (a selective state "Grammar" school) took maths
>> and English language a year early, so that if we
>> failed we could have another go :)
>>
>
> :)
>
>> Curiously, 15/20 years after that I was talking to
>> a secondary school maths teacher who refused to
>> believe we did calculus at O-level. Now it was an
>> unusual syllabus (University of London Syllabus D),
>> but I have since seen a copy of the textbook we
>> used, and it did indeed have exactly what I remembered
>> in it.
>>
>
> I recall now a high school in a neighboring town that offered something
> similar. I went to a small school; we were luck to to have a trig class. They
> had to bring in a substitute to teach it.

Parents chose (and still choose) where they lived based
on the schools.

For O-levels, my school had ~90 people/year divided
into three sets loosely based on ability. Most people
went onto A-levels and university, since the intake was
the top 10% of pupils based on IQ tests at 10 (the
11+ tests).

I also sat some exams for a local public (i.e. fee-paying)
school (and got a scholarship), principally in case I
failed the 11+.


> And he had to walk uphill in the snow 30 miles both ways... :)

It *was* uphill both ways to my daughter's school,
but not 30 miles, not on foot, and she enjoyed the
few occasions we got snow :)

On 3/17/2017 8:59 PM, George Neuner wrote:
> On Thu, 16 Mar 2017 17:23:11 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>> On 3/15/2017 6:05 PM, George Neuner wrote:
>>
>>> For one "applet" (process / security context) to allocate memory on
>>> behalf of another, that memory has to be supplied by something that
>>> exists outside of either of them.
>>
>> Actually, the memory can exist anywhere:  in some global space, in
>> the "allocating entity" or in the "client entity".  As can the code
>> that does the actual allocating (at the behest of the allocator).
>>
>> [Consider client environment not supporting arbitrary pointers.
>> Unless something *gives* you a reference/handle to a piece of memory,
>> there's no way you can access its contents (short of an exploit).
>> E.g., no way you can go looking up/down the stack even though you
>> know there *is* a stack supporting the implementation language]
>
> Moving the goal-post again.
>
> From previous discussions, I know your system works similarly to Mach
> ... and while I don't know your system, I *do* know Mach.  You're
> conflating things when you talk about "allocating entities" (aka Mach
> servers), and introducing complexity where none should exist.
>
> E.g.,
>
> The Mach memory manager (MM) grabs most of the RAM for itself and
> creates a heap from which it parcels blocks to other processes.
>
> An user allocated memory block physically exists IN THE MM's HEAP, but
> semantically it *belongs* to the user process: it is mapped into the
> address space of the user process, else that process could not use it.
> When you go to rehost this process, you need to consult the MM only so
> far as to know the attributes of the block.  The block itself is IN
> THE USER PROCESS, and what it contains has to be replicated at the
> same VMM "linear" address in the new process on the new host.
>
> The MM itself need not be involved in the replication.  In fact you
> probably would not want it to be because you only want to copy live
> data and not the entire address space of the rehosting process.
>
> The access/call and identity mechanisms: i.e. the "handles" by which
> the MM is called, and by which the block is known to the MM - mostly
> are beside the point.

(sigh)  You're fixating on AN EXAMPLE of an OBJECT that can need things
like "allocation_policy" bound to it.  (rewind the thread to see where I
introduced a "heap").  Reread the thread with "potato" substituting
for all instances of "heap".  And, "mashing_policy" for "allocation_policy",
"weight" for "size", etc.

THEN, lecture me as to how the "OS" owns this object and how *its*
primitives (VMM in your example) apply to its management.

[Should I, in the future, talk STRICTLY in terms of abstractions and avoid
ANY pleas to reify those concepts?  For fear that a reader will seize on an
example as representative of the entire universe of possibilities?  How
much "imagination" should I expect of readers?]

>>> Ok, so you have a block of system memory, and you created one (or
>>> more) heaps having certain properties within the block - heaps which
>>> are "managed" locally in the process by code from a dynamic module.
>>> And now you want to rehost the process.
>>
>> Objects (heaps in this example) are managed by <something>.  That
>> something may be the entity "owning" the object, a proxy or the
>> OS acting as the "proxy of last resort".
>
> Now you are getting completely away from the "memory" question you
> initially asked and into objects managed by servers on behalf of
> clients.
>
> Again with the goal post.

"Imagination".  Remember, we're talking about potatoes...

> An IN-PROCESS server allocates from memory already owned by the client
> process.  The client DOES NOT OWN an object which it can't touch
> directly, and which it can only affect via RPC to another process.

An in-process server allocates RESOURCES from resources that it
can REFERENCE from ANY OTHER SERVICE (which may be the OS or any
other object server to which the "in-process server" has rights.

The potato server can call upon a "masher" supplied by a "mashing service"
(and the poor OS is completely clueless as to the concept of "mashing").
The handle for the object that it then instantiates is all the client
ever needs to manipulate this particular potato.

The potato server can migrate to another node.  The client referencing
the potato can migrate.  The client can give the potato to someone else.
The potato server can pass management of the potato to a different
instance of a *compatible* potato server.

The potato can be "serialized" to a persistent medium and later
reconstituted ("instant potatoes"!  :> )

The application need be aware of none of these things.

> Replicating the memory state of the process on the new host
> automagically handles the case of an in-process server.  It also
> replicates client side state for anything controlled on its behalf by
> an out-of-process server.
>
> The state of an out-of-process server does not factor directly into
> rehosting a client.  All the server needs to know (if even this much)
> is that the client's "session" is to be suspended pending reconnection
> rather than terminated.

Replicating "memory" doesn't replicate the state of the machine because
other machines are involved.

Copy the memory image of *my* PC's "shell", at this moment, onto your PC.
Now, tell me which files I have open.  Which TCP connections are live.  etc.
The "state" is spread around in several places *besides* the process
itself.  Including the mail server at the other end of this SMTP connection
(when I click SEND).

[I.e., unless the passphrase for my SMTP server persists somewhere in
this process's memory, your copy of it's memory won't help you access
that connection that exists *now*]

>> The actions involved in management may reside in the OS, a service
>> (created by <something> -- including the client in question!) or the
>> client.  At the direct or *implied* request of the client, etc.
>
> You are failing to realize that it doesn't matter who manages the
> data: it matters only who *owns* the data.
>
>> Likewise, if some of the "properties" reside *in* the client (e.g.,
>> the algorithm by which allocations will be performed -- under the
>> supervision of some remote/system service that actually *does* them),
>> then you have to drag that/those things along with you.
>
> Or simply provide them in the new setting.  E.g., there is no reason
> to copy in-memory code if the new host has access to it via a file
> system.  If the new host has a different architecture, you can't copy
> the code anyway and would have to provide checkpoint mechanisms
> whereby the *program* (not "process") can be stopped and restarted
> cleanly.

The applets run in a virtual machine.  So, I can pick them up and
move them to another heterogeneous host.  (The "servers" are currently
designed for a homogeneous environment.  But, the same concepts apply)

>>> 2) Rehosting a running process necessarily requires checkpointing,
>>> copying or reconstructing its entire dynamic state: heaps, stacks,
>>> globals, loaded modules, kernel [meta]data, etc. - far more than one
>>> memory block.
>>>
>>> Every piece of distributed state must be identifiable as belonging to
>>> the process - regardless of what "module" may have created or is
>>> currently managing it.
>>>
>>> You have to copy all data belonging to the process, so having heap
>>> properties stored separately from the heap itself is not a problem.
>>
>> In my "portable" case, this is handled by moving the object handles.
>> The system can then opt to "optimize" execution by (later?) moving the
>> actual object instances (to minimize communication delays or take
>> better advantage of processing power on some particular node -- which
>> may differ from the source or destination nodes)
>>
>> But, you can (currently!) create "non-portable" objects.
>
> So they are lost.  So what?  You knew they were non-portable when you
> created them.

But the apps that contain them can be portable!  You might have
*chosen* to make a non-portable potato (by implementing a potato
server as one of the threads in your process).  The potato is
bound to the process.  But, the process isn't (necessarily) bound
to the *node* (or process group, etc.).

E.g., I can create a mutex to allow the local threads to cooperatively
access <something> (maybe just a 4 byte counter).  As long as the threads
remain bound in that process container AND the counter (or it's accessor)
AND the mutex, the whole entity (process) can be moved at will.

If some other process/agent needs to access that mutex, then it
needs a handle that is visible from outside of the local process.
If that other process can migrate to a different node, then it
can retain the ability to interact with the mutex in this first
process still residing on the original node.

So, you end up with very different use/performance cases:
- two or more colocated threads accessing the object
- two or more LOCAL *processes* accessing the object
- two or more REMOTE processes accessing the object

[Note, of course, that any other *object* can also indirectly be accessing
that object/potato]

In the first case, you can conceivably optimize the access to that of
a conventional mutex (in this example -- remember potatoes!) implementation.

In the second case, one process might be able to access the object
"from within" (a direct call to the functions that implement the
object) while another process (same node) accesses it "from without".

In the third case, one may be from within, or without, or "remote".

I don't consider it a "good idea" to be forced to make these "mobility"
decisions at compile time.  For the same reason that you don't decide where in
memory a piece of code should execute.  Or, whether it WILL be optimized by
the compiler, or not (compiler command line options aren't reflected in
the sources)

Just like I don't define which functions/methods should be accessible
remotely (let the IDL compiler address that based on what I tell *it* to do)

>> And, objects
>> that can't (easily) be shared.  (e.g., for a "conventional" heap, you
>> can let the default policies available in a "heap manager" govern
>> the way the heap operates.  *But*, you can't take advantage of any
>> "enhanced capabilities" in much the same way that you can't in a
>> more conventional process container, etc.)
>
> There's no reason code can't use "enhanced capabilities" - as long as
> the result is the same (for some definition).  E.g., if you can ignore
> timing, then debug code is interchangeable with optimized code.

That's been my goal!  But, it means the code has to reconfigure itself AT
RUNTIME.  Because the compiler doesn't know where an instance of a potato
will reside at *compile* time.

>> The problem with this (apparently arbitrary) distinction between
>> portable/shareable objects and "legacy" variety implementations
>> is that the developer has to explicitly decide what can be migrated,
>> shared, and how (because the developer has to take extra steps at
>> compile and link time to make those capabilities available).
>
> It is completely arbitrary, and IMO unnecessary.
>
> If there is something - a device possibly - that only exists on one
> host, then the module that interacts DIRECTLY with that <something> is
> not portable.  Anything objects it creates as a server on behalf of a
> client likewise are not portable.

No.  A "something" that CAN only exist on one host (node) is not portable.
A "something" that HAPPENS TO only exist on one node but CAN be migrated
to another node doesn't suffer those same restrictions.

I put a thin layer immediately above the hardware devices on each node
(think of it as a sort of HAL).  At the top interface of this layer, I
make a deliberate choice as to whether or not that interface shall be
exportable across nodes.

For example, there is little need to bind anything that uses the
"irrigation valve" hardware (HAL layer) to the node that actually
has the hammer drivers for those valve actuators.  They are inherently
slow and largely sacrificial.  So, that interface is exported.  The
"irrigation control task" can reside anywhere in the system -- including
nowhere (zombie).

OTOH, the interface to each of the video cameras is relatively high
bandwidth.  The task(s) that directly interact with them *should*
reside on those nodes (*if* you assume the camera output will see
much consumption).  So, those interfaces are not exported.

OToOH, the interface to the *compressed* video streams provided by those
tasks *can* be exported.  Their bandwidth requirements might be
considerably less than the "raw" video coming off each individual
camera.  So, a "motion detection" task can run on the IRRIGATION
NODE.  Or, mode to the LAUNDRY NODE if/when the irrigation node
is taxed with some other responsibility (perhaps it is taken out of
service as the valves -- and the proxies that run them -- are no longer
needed)

I.e., you gain the most flexibility by making everything exportable.
But, that means you either incur the costs associated with that
overhead -- even for the "local" case (e.g., motion detection task
running on the same node as the camera!) -- *or* do run-time
optimization of the communication path so that the remote costs
are only incurred when necessary (and factored into the workload
manager's reassignment of tasks to nodes)

> So if this module is to be used by portable applications, it should
> NOT be implemented as an in-process server.   Q.E.D.
>
>> So, where these sorts of bindings get stored (and how they get
>> tracked) varies based on this "other" information that the developer
>> supplies.
>
> This is where you and I have parted company in the past.  You like the
> kernel capability approach whereas I prefer the user capability
> approach ... precisely BECAUSE user capabilities avoid many of these
> semantic problems by virtue of being ordinary data.

The value of the in-kernel capability is that inherently tracks
references.  If the capability is "just data", then you have to
add a second mechanism to track *where* every copy of the data
happens to reside!  There's no way for the system to know that
you've just handed a COPY of a capability to something, deleted
YOUR copy, then retrieved a copy of that copy!  (i.e., the
"reference" is still "live", even though it looks like you
deleted it).  The "system" has no idea who has outstanding
references (handles/capabilities) nor how MANY.

By contrast, if the kernel manages the capabilities, then they
can't (by definition) be copied or transferred or deleted
without the kernel being an active part of that operation.
The kernel (and the manager for that particular capability)
can mediate how each is used ("No, you can't give that
capability away!  If you try to do so, I will abend your
process and/or just silently DELETE the capability and let
you and the other party wonder why that operation that
you intended to have performed *isn't*!")

"Is anyone using this video camera?  No, there are no active
capabilities in existence ANYWHERE in the system so we can
shut it down.  Is anyone (process) using the node that hosts
that camera?  No, so we can shut down the node, as well!"

> Everything other than kernel level process and address space control
> can be done in user space, and rehosting then is simply a matter of
> replicating the memory state [and reconstructing the kernel state].
>
>>>> The "consistent" solution is to make all of these first class
>>>> objects and let the OS manage them and their locations/instantiations.
>>>> But, that adds considerably to the cost of *using* them.
>>>
>>> Yes, that certainly is the ridiculous, overcompensating solution.
>>>
>>> The "consistent" solution is the one done already by any decent OS:
>>> keep a record of all the kernel structures and memory pages belonging
>>> to the process.
>>>
>>> Program objects then are just ordinary data in memory blocks "owned"
>>> by the process, or by the kernel on behalf of the process.
>>
>> But that ignores all of the "other" dependencies that can be in
>> play at any given time (i.e., my "solution" being these portable
>> handles that the OS tracks for the application/developer).  So,
>> <something> knows that there is an aspect of the current applet's
>> instantiation that relies upon something else *or* that is relied
>> upon BY something else.
>
> No it doesn't.
>
> You have created a whole set of artificial dependencies through your
> (ab)use of kernel capabilities.  Nothing in your system is - or even
> can be - self contained.

A process need never export a handle to an object that it creates
and manages -- for itself.  The OS needn't even be aware that the
object exists.  Does the OS know that I have a 4 byte counter
*here*?  Or here?

OTOH, if something *else* DEPENDS on that counter (potato), then
you want that dependency to be visible to the system.  You don't
want the potato to silently disappear without the dependent entity
knowing about it, *now* (not at some later date when it expects it
to STILL exist).

Likewise, if it "moves", you don't want the dependent entity to have to
scramble to *find* it when it needs it -- for much the same reason as
above.

>> [My approach lets the run-time know of these possible external
>> references by the presence of object handles held -- or exported -- by
>> the applet in question.  *Absent* these, the task is just a block of
>> memory and "processor state" that can be copied anywhere and "resumed".]
>
> But by doing that, you've inserted the operating system into all kinds
> of things where IMO it has no business being.  You've created the MCC
> from "Tron".

An OS manages resources.  I consider dependencies to be resources.
And, the permissions associated with them.

I don't want a user-level task to have to keep track of where the
things that it needs happen to be, *now* -- or if they even happen
to EXIST!

Nor do I want to have to require the developer to never try to do
something with a particular thing/object (potato) that it *shouldn't*.
Just like the OS shouldn't let process A peek into process B's memory.
Or, access unmapped portions of memory, etc.

You'd not advocate allowing a user to "add 3" to an arbitrary string.
You'd expect the *compiler* to enforce the rules of the language on
his *use* of the language.

I expect the OS to enforce the rules of the *system* on the users
of that system in much the same way!

>>> You can somewhat mitigate [rehosting] by making large allocations
>>> piece wise: abusing VMM to make them appear address contiguous.
>>>
>>> E.g., if a process asks for 100MB, reserve the address space but
>>> instantiate just a few megabytes at a time, on demand as the process
>>> touches unbacked addresses.
>>>
>>> [Because (I assume) you don't want to overcommit memory, you need to
>>> reserve requested address space both locally in the process, and
>>> globally so that other processes can't accidentally grab it.]
>>
>> Resource constraints vary with the resource and the consumer/provider.
>> You can overcommit many resources because many "jobs" don't exploit
>> their worst-case resource needs.
>>
>> But, you do so at the risk of delaying the availability of those resources
>> to the job in question.  Or, some other job interested in those resources.
>
> No, you are misunderstanding.  The process owns the space ... but in
> the scenario I presented, the physical memory backing the space is
> mapped on demand - which gives the OS more visibility into the process
> if/when it needs to rehost the process.

The "handles" give the workload manager (via the OS) insight into how and
where it can/should rehost the process (or any other object!).  It knows
what the process "talks to" because it manages the capabilities FOR that
process.  And, it knows the other (more tangible) resources that the
process requires (RAM, MIPS) by examining its current ledger and historical
performance.  So, it can balance the costs of "stretching" certain
communication channels (RPC/RMI being more expensive than the local case)
against the costs that some *other* process/object (a process is just a
different sort of object, managed by similar handles) would incur if
*this* process is not migrated.

[You can't come up with an "optimal" solution.  But, you can remember the
solutions you've tried and how they performed -- because the OS is
involved in these communications and can measure them!  So, you can
avoid solutions that performed poorly in favor of those that performed
*well* or that "have yet to be tried.]

> Remember that there are 3 forms of VMM addresses: "linear", "virtual"
> and "physical".
>  - "Linear" space is what is seen by an application.

"Logical" -- it need not be linear/contiguous.

>  - "Physical" space is the actual RAM as seen by the system.
>  - "Virtual" space is what ties the other 2 together.

And, how does this pertain to potatoes?  :>

> In a system using MMU only for process isolation, the "virtual" and
> "physical" spaces would be identical - mapped 1:1.   Overcommitting
> memory is only possible when "virtual" space > "physical" space.

Exactly.  Recall that there are multiple objects sharing that physical
space on a particular node.  The physical space determines the maximum
amount that can be mapped at a given time to the set of objects
currently making demands on it.  Those demands can (do) change, over time.

E.g., a process may load many modules and, by executing code *in*
each of them, cause pages to be mapped to instantiate those "opcodes"
JIT.  If a portion of a module is never actively referenced, then
the memory required to back it never gets mapped -- not taxing the
physical resources currently available.  If a module gets UNloaded,
then all of the mapped pages that were referenced can be released for
other uses (other potatoes).  Likewise, for the pages that were
never actually mapped!

      E := load Encyclopaedia
      ...
      article := E.lookup(Da Vinci)
      ...
      unload(E)

You wouldn't foolishly load the entire encyclopaedia into physical
(or virtual) memory to do this.  Instead, you'd load some accessor
code that could, itself, load PORTIONS of the "index" *when* you
initiated a search.  This would require some number of pages of
physical memory to be mapped to hold the resulting "article".
Anything beyond that would be waste!

      E := load Encyclopaedia
      ...
      article := E.lookup(Da Vinci)
      ...
      article := E.lookup(R Crumb)
      ...
      unload(E)

Would only require resources proportional to the LARGER of the Da Vinci
and R Crumb entries (the resources held by the first "article" are
freed when the second "article" is defined).

> When an application requests 100MB [e.g., (s)brk] from the OS, then
> [assuming the request can be granted] the OS adds 100MB to the linear
> space of the process, and *reserves* 100MB in the virtual space of the
> system.

No.  I don't *impose* that requirement!  Because it makes the system
brittle.  The application handles this at a different level dictated
by the CM system.

If, in your example, you ultimately *use* all 100MB of that memory
(which may be mmap'd in page-at-a-time increments) and then need
(or, someone else needs) 50MB *more*, the system decides how best to
provide those resources based on parameters from the CM system.

If "more resources" are available (though not necessarily "50" MB),
the system can opt to satisfy the request and let you continue to
mmap (by your usage patterns) additional pages.

If there are NO more resources available *when* you mmap another
page (indirectly by your actions), the system decides how best to
handle that request.

Can I unmap some portion of a loaded module (knowing that I can
always reload that portion of the module AS IF it hadn't *yet*
been referenced/mmap'd)?  Can I pageout some "really low" portion
of the stack (on the assumption that it won't be accessed RSN)?
Can I *migrate* some other task off of this node and reclaim the
physical resources that it currently holds?  Can I migrate *this*
task to some other node that has more resources available?  Can
I bring another node on-line and migrate "something" to it?  Can
I rely on some redundant (potato!) server to assume responsibility
for managing some objects that happen to reside on the local system
and, as such, free up the space they occupy *and* the space
required for the code to manage them?  Can I *kill* some task
(and associated objects) that are just running opportunistically,
presently (let's do some commercial detection in that OTA TV
broadcast we recorded two hours ago so its ready for viewing before
the user comes home and wants to see it!)? etc.

You don't want to make those decisions at compile time.  Because you
don't know what will be active in the system at *run* time!  And,
you want to build as much flexibility in how/where resources can
reside so that you have more run-time options.  You don't want to
overprovision just to handle a "worst case" that, in practice, will
never occur!

[Repeat this substituting "MIPS" for "memory", etc.]

> Now the reserved space belongs to the requesting process - no other
> process can take it.

See above.  If a process (potato) *needs* to be sure that resource
(memory, MIPS, etc.) *is* available, "sitting and waiting", then it
has to place an active "reservation" through the CM system.  The
CM system dictates limits for what individual tasks/"users" can
consume.

Reserves are A Bad Thing as they tie up those resources regardless
of what other objects might happen to place similar demands on the
system's resources.  What's to stop "everyone" from demanding EVERYTHING
that they *might* need "just in case"?

If, instead, you allow elasticity in how the resources are supplied,
you can afford to loosen the constraints on individual "consumers"
because they have less ACTUAL impact on other consumers.

Joe User (not Bob Developer) could write a sloppy script that
consumes "limitless" resources.  Should he be allowed to bring the
system to its knees -- because he is the "owner" of the device?

Similarly, should Bob Developer be allowed free reign at the risk
of compromising core services (implemented by *me*)?

It's a lot easier to ensure your system can meet all its goals if
you provision *for* those goals.  But, with an open/expandable system
that sees huge variations between actual and potential use, that
translates into dollars that consumers aren't keen on spending.

(why do you have more VM configured in your PC than physical memory?)

> However, the OS can choose whether or not to immediately map the new
> linear space to physical space.  By mapping physical space on demand
> as it is touched by the process, the OS has a much better picture of
> the *actual* memory use of the process.

Yes.  Though that can vary with other "input data" that this instance
of the process (potato) sees.  E.g., the resources required for that
lookup(R Crumb) are probably far less than the earlier lookup(Da Vinci).
Do you *know* what the user will opt to want to "lookup" when you
instantiate that task?  :>

You can either say "you can lookup ANYTHING" -- and provision for it.
Or, "you can look up anything as long as the article doesn't exceed X"
(and impose a disconcerting constraint on the user:  "How do I know
how big the article is LIKELY to be?").  Or, you can make a best
effort and trade your response based on your current needs and those
just added by the user.

> When the time comes to replicate the process's memory on another host,
> only mapped physical space needs to be copied - not the entire virtual
> address space reservation.

Again, that assumes the (active/mapped) memory represents the entire
"state" of the process and all of its dependencies.  (what about any
network connections -- managed by the OS -- that may be active at the
time?  the contents of any NIC buffers AS the process is in transit?
file handles, my "object handles", etc.)

[Think about it.  Send me an ISO with an image of your PC's memory
at this point in time.  I'll let you include the OS's state in that
image, as well!  Will I be able to read this USENET post at exactly
THIS --> . <-- point when I load the image into my machine's memory?
What if you mark the thread as read before the ISO gets loaded on
my machine?  Or, if YOUR server expires the article?  Or, if
your NNTP server happens to be offline?  That's all state that is part
of your "reading this USENET post" activity -- yet, you've not captured
it well enough for me to pick up where you left off!  :> ]

On Thu, 16 Mar 2017 17:23:11 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 3/15/2017 6:05 PM, George Neuner wrote:
>
>> For one "applet" (process / security context) to allocate memory on
>> behalf of another, that memory has to be supplied by something that
>> exists outside of either of them.
>
>Actually, the memory can exist anywhere:  in some global space, in
>the "allocating entity" or in the "client entity".  As can the code
>that does the actual allocating (at the behest of the allocator).
>
>[Consider client environment not supporting arbitrary pointers.
>Unless something *gives* you a reference/handle to a piece of memory,
>there's no way you can access its contents (short of an exploit).
>E.g., no way you can go looking up/down the stack even though you
>know there *is* a stack supporting the implementation language]

Moving the goal-post again.  

From previous discussions, I know your system works similarly to Mach
... and while I don't know your system, I *do* know Mach.  You're
conflating things when you talk about "allocating entities" (aka Mach
servers), and introducing complexity where none should exist.

E.g.,

The Mach memory manager (MM) grabs most of the RAM for itself and
creates a heap from which it parcels blocks to other processes.

An user allocated memory block physically exists IN THE MM's HEAP, but
semantically it *belongs* to the user process: it is mapped into the
address space of the user process, else that process could not use it.
When you go to rehost this process, you need to consult the MM only so
far as to know the attributes of the block.  The block itself is IN
THE USER PROCESS, and what it contains has to be replicated at the
same VMM "linear" address in the new process on the new host.  

The MM itself need not be involved in the replication.  In fact you
probably would not want it to be because you only want to copy live
data and not the entire address space of the rehosting process.

The access/call and identity mechanisms: i.e. the "handles" by which
the MM is called, and by which the block is known to the MM - mostly
are beside the point.



>> Ok, so you have a block of system memory, and you created one (or
>> more) heaps having certain properties within the block - heaps which
>> are "managed" locally in the process by code from a dynamic module.
>> And now you want to rehost the process.
>
>Objects (heaps in this example) are managed by <something>.  That
>something may be the entity "owning" the object, a proxy or the
>OS acting as the "proxy of last resort".

Now you are getting completely away from the "memory" question you
initially asked and into objects managed by servers on behalf of
clients.

Again with the goal post.


An IN-PROCESS server allocates from memory already owned by the client
process.  The client DOES NOT OWN an object which it can't touch
directly, and which it can only affect via RPC to another process.

Replicating the memory state of the process on the new host
automagically handles the case of an in-process server.  It also
replicates client side state for anything controlled on its behalf by
an out-of-process server.

The state of an out-of-process server does not factor directly into
rehosting a client.  All the server needs to know (if even this much)
is that the client's "session" is to be suspended pending reconnection
rather than terminated.


>The actions involved in management may reside in the OS, a service
>(created by <something> -- including the client in question!) or the
>client.  At the direct or *implied* request of the client, etc.

You are failing to realize that it doesn't matter who manages the
data: it matters only who *owns* the data.



>Likewise, if some of the "properties" reside *in* the client (e.g.,
>the algorithm by which allocations will be performed -- under the
>supervision of some remote/system service that actually *does* them),
>then you have to drag that/those things along with you.

Or simply provide them in the new setting.  E.g., there is no reason
to copy in-memory code if the new host has access to it via a file
system.  If the new host has a different architecture, you can't copy
the code anyway and would have to provide checkpoint mechanisms
whereby the *program* (not "process") can be stopped and restarted
cleanly.


>> 2) Rehosting a running process necessarily requires checkpointing,
>> copying or reconstructing its entire dynamic state: heaps, stacks,
>> globals, loaded modules, kernel [meta]data, etc. - far more than one
>> memory block.
>>
>> Every piece of distributed state must be identifiable as belonging to
>> the process - regardless of what "module" may have created or is
>> currently managing it.
>>
>> You have to copy all data belonging to the process, so having heap
>> properties stored separately from the heap itself is not a problem.
>
>In my "portable" case, this is handled by moving the object handles.
>The system can then opt to "optimize" execution by (later?) moving the
>actual object instances (to minimize communication delays or take
>better advantage of processing power on some particular node -- which
>may differ from the source or destination nodes)
>
>But, you can (currently!) create "non-portable" objects.

So they are lost.  So what?  You knew they were non-portable when you
created them.


>And, objects
>that can't (easily) be shared.  (e.g., for a "conventional" heap, you
>can let the default policies available in a "heap manager" govern
>the way the heap operates.  *But*, you can't take advantage of any
>"enhanced capabilities" in much the same way that you can't in a
>more conventional process container, etc.)

There's no reason code can't use "enhanced capabilities" - as long as
the result is the same (for some definition).  E.g., if you can ignore
timing, then debug code is interchangeable with optimized code.



>The problem with this (apparently arbitrary) distinction between
>portable/shareable objects and "legacy" variety implementations
>is that the developer has to explicitly decide what can be migrated,
>shared, and how (because the developer has to take extra steps at
>compile and link time to make those capabilities available).

It is completely arbitrary, and IMO unnecessary.  

If there is something - a device possibly - that only exists on one
host, then the module that interacts DIRECTLY with that <something> is
not portable.  Anything objects it creates as a server on behalf of a
client likewise are not portable.

So if this module is to be used by portable applications, it should
NOT be implemented as an in-process server.   Q.E.D.


>So, where these sorts of bindings get stored (and how they get
>tracked) varies based on this "other" information that the developer
>supplies.

This is where you and I have parted company in the past.  You like the
kernel capability approach whereas I prefer the user capability
approach ... precisely BECAUSE user capabilities avoid many of these
semantic problems by virtue of being ordinary data.

Everything other than kernel level process and address space control
can be done in user space, and rehosting then is simply a matter of
replicating the memory state [and reconstructing the kernel state].



>>> The "consistent" solution is to make all of these first class
>>> objects and let the OS manage them and their locations/instantiations.
>>> But, that adds considerably to the cost of *using* them.
>>
>> Yes, that certainly is the ridiculous, overcompensating solution.
>>
>> The "consistent" solution is the one done already by any decent OS:
>> keep a record of all the kernel structures and memory pages belonging
>> to the process.
>>
>> Program objects then are just ordinary data in memory blocks "owned"
>> by the process, or by the kernel on behalf of the process.
>
>But that ignores all of the "other" dependencies that can be in
>play at any given time (i.e., my "solution" being these portable
>handles that the OS tracks for the application/developer).  So,
><something> knows that there is an aspect of the current applet's
>instantiation that relies upon something else *or* that is relied
>upon BY something else.

No it doesn't.  

You have created a whole set of artificial dependencies through your
(ab)use of kernel capabilities.  Nothing in your system is - or even
can be - self contained.


>[My approach lets the run-time know of these possible external
>references by the presence of object handles held -- or exported -- by
>the applet in question.  *Absent* these, the task is just a block of
>memory and "processor state" that can be copied anywhere and "resumed".]

But by doing that, you've inserted the operating system into all kinds
of things where IMO it has no business being.  You've created the MCC
from "Tron".



>> You can somewhat mitigate [rehosting] by making large allocations
>> piece wise: abusing VMM to make them appear address contiguous.
>>
>> E.g., if a process asks for 100MB, reserve the address space but
>> instantiate just a few megabytes at a time, on demand as the process
>> touches unbacked addresses.
>>
>> [Because (I assume) you don't want to overcommit memory, you need to
>> reserve requested address space both locally in the process, and
>> globally so that other processes can't accidentally grab it.]
>
>Resource constraints vary with the resource and the consumer/provider.
>You can overcommit many resources because many "jobs" don't exploit
>their worst-case resource needs.
>
>But, you do so at the risk of delaying the availability of those resources
>to the job in question.  Or, some other job interested in those resources.

No, you are misunderstanding.  The process owns the space ... but in
the scenario I presented, the physical memory backing the space is
mapped on demand - which gives the OS more visibility into the process
if/when it needs to rehost the process.


Remember that there are 3 forms of VMM addresses: "linear", "virtual"
and "physical".  
 - "Linear" space is what is seen by an application.
 - "Physical" space is the actual RAM as seen by the system.  
 - "Virtual" space is what ties the other 2 together.  

In a system using MMU only for process isolation, the "virtual" and
"physical" spaces would be identical - mapped 1:1.   Overcommitting
memory is only possible when "virtual" space > "physical" space.

When an application requests 100MB [e.g., (s)brk] from the OS, then
[assuming the request can be granted] the OS adds 100MB to the linear
space of the process, and *reserves* 100MB in the virtual space of the
system.  

Now the reserved space belongs to the requesting process - no other
process can take it.

However, the OS can choose whether or not to immediately map the new
linear space to physical space.  By mapping physical space on demand
as it is touched by the process, the OS has a much better picture of
the *actual* memory use of the process.

When the time comes to replicate the process's memory on another host,
only mapped physical space needs to be copied - not the entire virtual
address space reservation.


George

Tom Gardner wrote:
> On 17/03/17 12:16, Les Cargill wrote:
>> Tom Gardner wrote:
>>> On 17/03/17 11:01, Les Cargill wrote:
>>>> Tom Gardner wrote:
>>>>> On 14/03/17 02:23, Paul Rubin wrote:
>>>>>> Tom Gardner <spamjunk@blueyonder.co.uk> writes:
>>>>>>> "When I was 14 I thought my father was an idiot. When I became 21 I
>>>>>>> was amazed at how much he had learned in the past 7 years"
>>>>>>
>>>>>> One of the profs at my old school said at his retirement dinner "I've
>>>>>> been teaching these kids freshman calculus for FORTY YEARS and they
>>>>>> still don't get it!".
>>>>>
>>>>> :)
>>>>>
>>>>> We learned integration and differentials of polynomials
>>>>> except 1/x for exams at 15. I can still visualise the
>>>>> teacher taking a double period (80 mins) for each,
>>>>> deriving the concepts from first principles.
>>>>>
>>>>> OK, he was a good teacher (but not a good mathematician,
>>>>> he knew his limits!), but even so I've never understood
>>>>> why people think calculus is inherently difficult.
>>>>>
>>>>
>>>>
>>>> Limits are conceptually difficult. It takes work. You
>>>> have to be able to suspend disbelief in things like infinity.
>>>
>>> Work? Yes, good :)
>>>
>>> Difficult? Yes, there is a graunch; It probably took
>>> 40mins of teaching and discussion.
>>
>>
>> I remain highly skeptical of that figure. That was
>> for the initial gloss; it takes a considerably longer
>> time to gain a working knowledge of limits.
>
> It was sufficient for us to believe in the concepts,
> and to be able to understand the explanation of how
> polynomials (except 1/x) could be differentiated.

Sure.

> We then went on to use differentiation for the
> O-level exams at 16. Well, I say 16 because that
> was the normal age, but everybody in my school
> (a selective state "Grammar" school) took maths
> and English language a year early, so that if we
> failed we could have another go :)
>

:)

> Curiously, 15/20 years after that I was talking to
> a secondary school maths teacher who refused to
> believe we did calculus at O-level. Now it was an
> unusual syllabus (University of London Syllabus D),
> but I have since seen a copy of the textbook we
> used, and it did indeed have exactly what I remembered
> in it.
>

I recall now a high school in a neighboring town that offered something
similar. I went to a small school; we were luck to to have a trig class. 
They had to bring in a substitute to teach it.

And he had to walk uphill in the snow 30 miles both ways... :)

-- 
Les Cargill

On 17/03/17 20:07, Tom Gardner wrote:
> It was 34 years ago, and I've neither been nor wanted
> to be a pure mathematician :)

Ahem. 45 years ago :(