Hi George, [watch mail; I'm pondering yet another DB abuse but have to sort out what I want, first :-/ ] On 11/17/2013 5:05 AM, George Neuner wrote:>> E.g., "23" interpreted as a file descriptor in task (process) A can >> refer to a particular pty. Passing "23" to some other task breaks the >> association with that particular pty. "23" is *just* "23" -- nothing >> more. > > On many systems, that actually is not true - the descriptor is a > global identifier for the referenced object. Opening the same object > in separate processes you may discover that the descriptors all have > the same value.But there is no *requirement* that this be the case! I.e., if you printed the value of a file descriptor in three instances of an app running on the same machine -- along with three instances under some other OS -- you could expect up to 6 different "local names" for the same physical object. By contrast, under Amoeba, 0xDEADBEEF12345678 pertains to exactly *one* object regardless of *who* has that value present in their local memory (as a capability_t). The identity of theobject (along with the allowed permissions) is encoded *in* that value. "23" has no such encoding (as a file descriptor *or*, in my case, a Handle) I.e., Amoeba tasks can pass capabilities to each other via any number of communication methods WITHOUT THE CONSENT OR INVOLVEMENT OF THE KERNEL! So, for example, a task can display a hex representation of a capability ON A CRT. A human can read that value and type it into another terminal. The task attached to *that* terminal can then store this in memory and present it AS A VALID CREDENTIAL with exactly the same permissions as the original! No record of this transfer exists. In Mach, all that first task could do is display the equivalent of a file descriptor to the user. If the user typed that into another process, it would be meaningless, there. You have to tell the kernel to transfer the capability (or a copy of it). It's not just "data".>> OTOH, 0xDEADBEEF010204302893740 passed from Amoeba task A to Amoeba >> task B carries "rights" with it. Encoded within the cryptographic >> envelope! > > So what? The envelope validates the contents. Encoded rights are > useless if the envelope is broken or even just sent to the wrong > server.But not invalidated if possessed by another. I.e., I can send copies of a capability to 100 other tasks. They have exactly the permissions that I had. The kernel is unaware that I have made these transfers -- it's just data! (that was the goal of Amoeba -- to get the kernel out of that business, contrast Mach). Because the kernel is NOT involved in the duplication of existing capabilities (same permissions), it never knows how many such outstanding references exist.>>> - "rights" are N-bit wide fields. The meanings of the bits are defined >>> by the issuing server. >> >> What I've called "authorizations". Except there is no visible >> "bit field" in my implementation. Each Handler decides how it wants >> to implement a set of "authorizations". > > Amoeba chose a public ticket representation specifically to facilitate > distributed agents. The entire point of it was that agents be able to > delegate modified rights on their own.Yes, as can mine. Same limitations -- the kernel is involved in creating *new* capabilities (different permissions on same object or different objects entirely). But, Amoeba allows exact duplicates to be made "for free". Mach doesn't. The kernel is involved in all Handle (port) operations. Even literal copies. So, it knows how many copies of each Handle exist at any given time (fudging the truth here in the distributed case). For example, Mach had a "send-once" right. It allowed the holder to send *one* message to a port. It could not be copied by the holder (had to be created by the holder of the corresponding *receive* right) even though holders of regular send rights could ask the kernel to copy (or move/transfer) them. So, a task could be given 38 send rights for a particular port. 38 messages later, the port stops working. In Amoeba, once you know the ticket's "binary value", you can use it as many times as you want (unless the server backing it has imposed some limitation on its use -- which a mach server could also layer atop any send right!). I can *move* that send-once right to another task of my chosing thereby giving it the "permissions" that were tied to it (by its server). But, in doing so, *I* lose the ability to use it! In Amoeba, we would *both* retain the capability to use it!>>> It's important to understand that Amoeba capabilities are, in fact, >>> "versioned", though versioning is neither sequential nor readily >>> predictable. >> >> This can just be seen as extending the namespace of the capability >> in a manner that makes for easier management. I.e., so a Handler >> (server) can opt to ignore "older" rights (because it can't vacuum >> memory to find and remove all instances of a particular "ticket") > > I think "extending" is the wrong term. A particular version of a > capability may be seen as defining a name space in which its > referencing tickets exist, but the effect of versioning the capability > is to destroy one space (and everything in it) and to open another.Yes. My point was it allows a name to be revised without being "changed". The title of the book remains the same though its content/role has been altered. Last year's pass for the park doesn't work this year.> Moreover, the capability itself exists within a name space which is > not changed by versioning the capability. > >> But, there is no direct *tie* to the original ticket from which >> this one (subset) was created! (or the one before *that*; or the one >> before *that*; etc.) > > The *tie* is through the capability. The ticket is only a reference > to it.But the reference is the same regardless of who "uses" it! If you give me a binary copy of a ticket, the kernel doesn't know that I now possess that capability (object, permission). I can freely create a "constrained" capability from this and hand it to someone else. Who, in turn, can *duplicate* that and hand it to yet another. The only way for any of us to know a ticket (capability) has been revoked is to try to *use* it. If I use mine or you use yours, we get told THAT capability has been revoked. But these other (nameless) two guys in my example have no idea that this has happened (unless one of us knows to tell BOTH of them).>> So, when a ticket is presented, you can't look at the ticket and >> decide that the ticket from which it was created was revoked and, >> therefore, so should this one! > > You misunderstanding something. > > Tickets don't create tickets, servers do. You can't modify the rights > of an existing ticket and copy it to someone else - it simply won't > work. Creating a valid ticket requires knowledge of the capability's > secret check (version) number, which only the server has.Yes. But literal duplicates of a ticket can be made "for free".> You have to present your ticket to the server holding the capability > and request issue of a new ticket having the desired rights. The > server will tell you to go to hell if you don't have the privilege to > create new tickets.Correct. Ditto Mach.> If you have the privilege, you may ask the server to create an > *additional* capability for the original object that you can > administer independently. This does not alter the original capability > that created your ticket.Correct, also ditto Mach.> Only the holder of an "owner" ticket can request the server to version > the capability. That invalidates *every* ticket referencing the old > version ... including the owner's own ticket!. When versioning a > capability, a server must immediately issue a new "owner" ticket in > response.In mach, deleting the port (Handle) deletes all instances of it. And, since the kernel knows where those instances are, those Holders can be notified when that happens.>> (hence version. But, now the handler/server needs to keep track >> of which version is current for each outstanding!) > > No. A server needs to track only those (versions of) capabilities for > which it intends to honor tickets. It does *not* need to retain any > information regarding revoked capabilities.It needs to know the current version of each capability that it will honor. Because actors can present stale tickets. In Mach, the Handle is *gone*. The Handler never sees it again. Nothing to "remember".>>> At this point, it should be clear that every issued ticket is tied to >>> a specific "version" of a capability by the capability's secret check >>> number. If the capability is versioned - i.e. the check number >>> modified- or if the capability record is deleted, then every ticket >>> issued referencing that (no longer existing) capability is immediately >>> rendered invalid. >> >> So, the Handler/server needs to effectively treat the version >> as the ID of the actor to which a ticket is granted IF IT WANTS TO >> REVOKE THOSE CAPABILITIES and only those WITHOUT AFFECTING OTHER >> TICKET-HOLDERS. > > To a 1st approximation, yes. > > A particular version of a capability is known by the pairing of its > public identifier and private check number. A different version is, > in a real sense, a different capability. > > However, it makes little sense to have multiple versions of a > capability all which have the same public identifier. If you want > simultaneously *valid* tickets to reference unique capabilities, you > create simultaneous capabilities with unique identifiers. > > The multiple version issue is avoided and irrelevant.This is the Mach approach -- a Handle is a Handle (a port is a port). The only version that exists is the current version. If I don't want you to have a Handle, I delete that Handle -- there's no way you can reconstruct it "from memory". The way the "permissions/authorizations" associated with a particular handle are interpreted can change -- without the consent of the Holders (also true in Amoeba). E.g., if he File Server is flushing all buffers to disk because of an impending shutdown, it can choose to ignore the "write" permissions that had previously been granted in EXISTING Handles. If it *wants* to notify each Holder of this, it can (because it knows which tasks are holding those Handles *now* -- no duplicates without the kernel's consent).>>> Because capabilities are kept private by the issuing server, >>> surrogates capabilities can be linked to the owner's capability, >>> allowing the owner to void delegate and/or surrogate tickets by >>> versioning/deleting the appropriate capability. The surrogate, of >>> course, can void delegate tickets by versioning/deleting its own >>> capability. >> >> But that means the handler/server has to do all this work! "Remembering". > > In your case the kernel does all this work for _all_ of the servers.Yes! This is the single biggest drawback! But, it is the only way to enforce communciation restrictions, etc. E.g., in Amoeba, even bad tickets have to be processed by the backing server. On *that* server's dime (timeslice/resources). In Mach, bad Handles (no send rights) are caught at the local server on the offending actor's dime! If a Handler sees an actor abusing his privilege (whatever that means), he can revoke the Handle (because the Handler has the receive right for the port!) and shut the actor up *at* the actor's kernel. In Amoeba, the server is forced to validate (invalidate) each request as it comes in -- regardless of how often, etc. And, tie up RPC/IPC resources as well! In Amoeba's case, RPC could be reasonably cheap because Amoeba expected processor *pools* -- lots of SBC's in a big box. You could afford fat pipes between boards, shared memory cards, etc. regardless of whether you created them initially. Mach could also support SMP & NUMA multiprocessors but handles NORMA by proxy. Invoking the communications for RPC is expensive. If you can determine that the connection shouldn't be attempted and inhibit it locally, you gain a lot. (Imagine all the network traffic if a roue/corrupt/adversarial actor kept issuing RPC's to an Amoeba server that was just going to repeatedly say NO_PERMISSION!) [I am obsessed with DoS attacks as I see this as the achilles heel that will always remain vulnerable in interconnected systems. Witness the extremes I taken in the design of my router/switch! :< Don't even *make* a coonection if its not supposed to exist!]>>> All versions of Amoeba's tickets specified a server (or service) ID - >>> the field wasn't sign protected because the ID might be a task >>> instance, but it allowed servers to immediately reject tickets they >>> couldn't possibly have issued. >> >> In my case, the Handler/server never is *presented* the communication >> in the first place -- unless the path has been previously created by >> possession of the Handle. > > In Amoeba's case, the sender needs a ticket that says it can ride the > communication channel. 6 of one ...It's just an IPC! Ameoba won't let tasks issue IPC's without a capability?? Who validates *those* (i.e., acts as handler/server for IPC)?> It's easy enough to create separate channels for each service with > unique tickets needed to access them.It's still an IPC. Something on the other end has to validate on *its* dime. In my case, that's the kernel EXECUTING IN YOUR TIMESLOT. In Amoeba's case, the kernel has no role in IPC other than to pass the data along to the service encoded in the ticket -- wherever it may currently reside. That service has the only and final word on whether it wants to honor the ticket (at this time). Am I missing some *new* addition to Amoeba that allows the local kernel to act as selective gatekeeper?>> The "problem" with my approach is that all of these things -- for >> the complete set of tasks executing on a host -- are contained in the >> kernel. Amoeba (et al.) allows the references to be moved *out* >> of the kernel into task-space (whether that's user-land or not). >> REGARDLESS OF WHETHER AN OBJECT IS LIVE OR NOT. > > But under your system, the kernel has to track capabilities for every > live object even though it isn't *responsible* for them.Correct. It also has to track the current state of each executing task, current memory allocations, runtime priorities, etc. I consider communication "rights" to be a key aspect of providing integrity to applications. Think about it. How can an actor take down my system? It has the final say in only those objects that it backs. In all other cases, it is subservient to other on whom it depends. You can't abuse a service unless the service chooses to let you. If you become abusive, the service can revoke your ability to even *talk* to it!>> One of the Mach problems, IMnsHO, was their desire/goal of trying to >> reimplement UN*X. So, any "impedance mismatches" between their >> model and the one used by the UN*X implementors was a performance >> or conceptualization "hit" (hence my deliberate choice of "impedance >> mismatch"). None of these things were "deal breakers" but they >> conspired to make it a bad fit. overall. > > That's a misconception ... I know someone who worked on the original > implementation. > > The designers did not *want* to reimplement Unix - rather they > believed that [recompile] compatibility with Unix was necessary for > Mach to gain acceptance while a critical mass of native programs was > being written.It amounts to the same thing. They chased after UN*X becaue it would give them a "real application" to demonstrate. Hardly anyone would be interested in watching an "inverted pendulum" controller implemented on yet-another-OS. So, much of the work went into supporting mechanisms that had no use outside of the UN*X context. E.g., the "environment server". And, many hooks were created just for these things! (e.g., the list of special ports provided to newly created tasks should only have been a *single* port from which all others could be derived).> Mostly due to time pressure [Mach was developed on a grant] they took > a disastrous shortcut. Realizing that they couldn't *quickly* > implement a *full* Unix API translation on top of native services, > they grabbed an existing BSD Unix kernel, modified it into a Mach > server task and hacked the system libraries to RPC the "compatibility > server".The "BSDSS" (Single Server). Ditto POE (DOS single server). Then MachUX and eventually MachUS (or perhaps I've got the multiserver mnemonic backwards). Along with the RTMach and Mach4 branches. (I believe I probably have the largest "private" Mach archive in existence. I'm sure many of the older releases are no longer available on *any* public servers!)> Their problem was that too much of the BSD kernel they used was serial > non-reentrant code. Fixing it would have taken as much effort as > doing a real API translation - effort they weren't prepared to give at > the beginning. Running as one task among many, the kernel inside the > compatibility server was painfully slow.NeXT followed this route with 2.5> Mach 3.0 brought out an almost complete, lightweight API translation > library, and saw a complete rewrite of the compatibility service to > offer only those few Unix things that had no analogue in Mach. But by > then it was too late.But even the first "UX" server had too much of a performance hit precisely because of the "kernel congestion" you mentioned above. That's the mistake with pursuing a UN*X goal -- even temporarily. The mechanisms don't fit the application as well as existing UN*X "monolithic kernels" did. You're gaining something (additional isolation, multiprocessing support, distributed processing, user-level services, etc.) and there's obviously going to be a cost. UN*X itself didn't have (nor apparently *want* those features so it could be more efficient. The RTMach crew just built on these same assumptions -- instead of saying, "UNIX is great -- for the problems that UNIX solves THE WAY UNIX SOLVES THEM. We're going after a different class of problems and a different methodology for solving them.> Creating the compatibility service in the first place turned out to be > Mach's fatal mistake. Because Mach could run Unix programs, few > people bothered to port anything to the native API. And because Mach > ran their Unix programs slowly, people judged the OS itself to be > unworthy. [Which was unfair: native Mach programs were faster than > equivalent Unix programs on the same hardware.]Exactly. But, the Mach API *under* another layer of abstraction just makes things worse, not better. Also, there were some problems with the possibility of cross contamination *within* the library -- no doubt another sacrifice to efficiency. I think the Mach (and Amoeba) focus on application to existing usage paradigms is the fatal flaw. E.g., applications there are transient and invoked by a multitude of users on a multitude of objects. BIG objects. OTOH, appying those mechanisms to a *control* system where: - connections tend to persist for a LONG period of time (24/7/365) - isolation means real dollars (task A doesn't end up scrapping all the product produced by task B) - processors can have multiple cores - as well as being physically distributed - and compute costs decrease considerably faster than memory or communication costs THEN, burning cycles in a kernel (cached) to save references to memory (local or remote) AND unwanted communication costs can have a big payoff. But, I think you have to approach the OS with these things in mind and not "files" and TTYs, etc. Do you *really* need all these traps? All these function arguments? etc. At the opposite extreme, see how things like L4 struggle to "run with the big boys". Yah, you can be lean and efficient and fast -- but, what does it cost to implement expensive feature with a few primitives? And, what level of skill to do so with the same sort of efficiency? ("Wow! Kernel traps are twice as fast!" "Yeah, but you need twice as many...")
Managing "capabilities" for security
Started by ●November 1, 2013
Reply by ●November 17, 20132013-11-17
Reply by ●November 18, 20132013-11-18
Hi George, Brief (ha!) description of how objects come into being and how actors get access to them. And how their interactions are controlled and revoked. [mixture of Mach-speak and each of the other lexicons we've loosely employed] Sketching as you read may help! When a task is created (by <someone>/<something>), in addition to the typical resources packed in that container (memory, executable, etc) there is a *Handle* to a Namespace_t. The Handle is effectively a send right to a particular port_t. That Handle is *handled* by the Namespace Server (Handler). I.e., the Namespace Handler holds the receive right for that port_t. (remember the task is an object and *also* is represented by a Handle!) Any communications directed to that port_t (Handle) end up *at* the Namespace Handler -- arriving on that specific port ("assigned" to that particular task). I.e., the Handle represents the task's Namespace. All Namespace operations (methods) expect a Handle BACKED BY SOMETHING THAT UNDERSTANDS NAMESPACE IDL as an argument. Other arguments vary with the operation (method) being invoked. For example, lookup(name), create(name), destroy(name), etc. The <someone>/<something> that caused this Namespace object to be created (evident below), may have prepopulated the namespace with certain names that the task will recognize, by convention (can vary with task; e.g., may also include a name like "[parent]"). The <someone> may elect to allow the task to modify this namespace (add, delete, rename, etc). Or, it may *lock* the namespace from the task's perspective. Once the task is executing, it can lookup() particular names in its namespace that have been previously agreed upon, by convention. [note that the conventions need only be consistent between a task and he creator of its namespace! one task can know an object by the name "trunk release" and another could know an object by the name "boot opener". Names are just convenient mnemonics.] Invoking the lookup() stub with the args required causes an IPC/RPC message to be created, args marshalled, etc. and then delivered to the port *in* the Namespace Handler that is associated with that task's Handle (Namespace). *Because* the task holds a valid send right for the port, the kernel allows the message to be delivered. The Namespace Handler receives the message, examines the method being sought and decides if messages of this type (method) are permitted ON THIS PORT! I.e., if the Handle has the appropriate "authorizations" defined when <someone> created this binding. If not, it returns FAIL (NO PERMISSION). Assuming the Handle for the task's namespace ALLOWS the lookup() method to be invoked by that task (WTF?), the args are unmarshalled and the operation (method) attempted. A result (if required) is then propagated back to the calling task (using a *reply* port that the task has included in the message -- i.e., you can support async methods if you plan on watching that reply port *later* for the reply to this operation!) [the IDL allows various types of "methods" to be designed -- including "routines" that return no value.] In the event of a lookup(), the task is often trying to establish initial contact with some other Handler -- to create an object of the type Handled by the Handler. Pretend we're interested in Files (bad choice but easy to spell! :> ) [Of course, objects of that type may already have been created by <someone> and placed into task's Namespace before the task began executing! The Handles for any of these will already have been bound to appropriate names in the task's Namespace. E.g., "log".] Now it gets clever... For initial contact with a named Handler, you need a "public" access method that is enabled "for all". The Handles to this perhaps just implement create() -- for the type of object managed by that Handler (server). So, a task can contact the File Server to create a new file! For untrusted tasks, when <someone> builds the initial Namespace for the task, he doesn't bind the "trusted" public Handle into the server. But, instead, has the Handler creates *another* such Handle. Same permissions (typically). Then, <someone> binds the name for this Handler into the Namespace that it has been building for the task. So, "__File_Server__" is bound to a particular port *in* the File Server. And, this port may be different for each such task! All such "public" Handles into the File Server are serviced by one or more threads. I can assign a thread per port (Handle), a thread for the entire service INTERFACE (not counting threads that actually BACK specific Files), or lump groups of ports into port sets so a single thread can handle requests coming in on ANY of these ports. The ask creates a File, for example. This causes the File Server to create a new port for that File and pass a send right (Handle) to that "object" back to the task in the reply to the create() method. Note that this object now exists yet has no *name*. No problem! The task knows it by its handle. It can read and write it and even destroy it -- without ever giving it a formal *name*! Furthermore, it can pass the Handle on to someone else and *they* can read and write it as well (assuming the task doesn't tell the File Server to reduce the authorizations on a *new* Handle passed instead!). As long as there are Handles referencing the File, it exists. If it wants to *name* the File, it can (assuming <someone> gave him that authorization for HIS Namespace) bind an Name to the object. THIS NAME ONLY MEANS SOMETHING IN THE TASK'S NAMESPACE! Meanwhile, some other task holding a copy of that Handle can create a different name for this same object (File) in *its* namespace! Or, remove the name, etc. [Namespaces are just convenience objects. BUT, have the effect of limiting what you can talk to!! If you can't reference it, you can't talk to it!] Now, assume someone holding a Handle to this File starts misbehaving. Keeps trying to write to the file. Doing this repeatedly when you have already been told (error return) that you don't have permission is abusive -- wastes the File Server's resources! But, because that offender holds a valid send right to this object, the kernel will keep passing along those RPC's! The File Server can take matters into its own hands and destroy the port on which the RPCs are arriving. The kernel will then know not to pass the RPC along -- the task has effectively lost the right to send to that port! Ah, but a determined adversary just reconnects to the File Server via the public port and starts the process all over again! *BUT*, the only way the adversary can talk to the File Server is via the public port that was created JUST FOR HIM! I.e., __File_Server__ maps to a specific port that, effectively, identifies *him* (his "public" connection). When the File Server gave permission to access that particular File that was later abused, it knew which pubic port was associated with it! So, the File Server can delete *that* port thereby severing the task's only means of talking to the File Server publicly. [You can also create initial "credentialed" connections whereby each actor identifies itself with a unique token -- a *port* that it creates exclusively for that particular connection creation operation. As ports are unforgeable, only a task holding this port can act *as* this actor!] He can still exercise any Handles that he has outstanding for other File objects -- until he gets abusive with those, etc. So, a server can isolate itself from its clients, selectively. (remember, everything is essentially a server -- for *something*!) From the other side, when everything drops (destroys, discards) every Handle to an object, that object can no longer be accessed! So, it can be deinstantiated and its resources freed. (this of course is done by the Handler for that type of object) E.g., when the last Handle into a particular Namespace is dropped, the Namespace itself can disappear (which, in turn, causes handles to all the objects referenced in the namespace to go away... etc.) Simiarly, when the last Handle into the Namespace Handler (!!) goes away, the Namespace Handler itself can go away!! [an unreferenceable object is just a waste of resources!] This is where the notification of Handler death comes into play. Automatic self-GC without a sweeper! So, "init()" can do its job and then release the Handles that it held to all these "top level servers" *knowing* that the last task to reference any of them will effectively determine their fate -- without init() acting like a zombie clinging to them. <frown> As I said, a picture makes things more intuitive.
Reply by ●November 18, 20132013-11-18
On Sun, 17 Nov 2013 11:31:44 -0700, Don Y <This.is@not.Me> wrote:> you missed the point of my question.No I didn't. Amoeba doesn't have any notion of reference counting BECAUSE reference counting doesn't scale [nor does garbage collection, but that's a different discussion]. Amoeba was designed for campus area networks with an eye toward wide area networks. If every owner ticket is lost, a persistent capability exists until an administrator manually deletes it (or the object it is associated with). That's why administrators get the big bucks. George
Reply by ●November 18, 20132013-11-18
Hi Don, On Sun, 17 Nov 2013 16:55:26 -0700, Don Y <This.is@not.Me> wrote:>>> OTOH, 0xDEADBEEF010204302893740 passed from Amoeba task A to Amoeba >>> task B carries "rights" with it. Encoded within the cryptographic >>> envelope! >> >> So what? The envelope validates the contents. Encoded rights are >> useless if the envelope is broken or even just sent to the wrong >> server. > >But not invalidated if possessed by another. I.e., I can send copies of >a capability to 100 other tasks. They have exactly the permissions that >I had. The kernel is unaware that I have made these transfers -- it's >just data!That's only true of the original implementation. Later versions expanded the ticket to include an "authorized user" field which variously could be a process, host or host:process identifier, or a admin level user/group id. The user field was protected by a 2nd crypt signature. This enabled servers to restrict who could use a ticket.>For example, Mach had a "send-once" right. It allowed the holder to >send *one* message to a port. It could not be copied by the holder >(had to be created by the holder of the corresponding *receive* >right) even though holders of regular send rights could ask the >kernel to copy (or move/transfer) them. > >So, a task could be given 38 send rights for a particular port. >38 messages later, the port stops working. > >In Amoeba, once you know the ticket's "binary value", you can use >it as many times as you want (unless the server backing it has >imposed some limitation on its use -- which a mach server could >also layer atop any send right!).In Amoeba, such things are entirely up to the server. It can choose to issue single use tickets, or "numbered" tickets if it desires. The rights field is completely server defined - it can mean anything. And nothing prevented a server from associating additional state with a capability. [That, in fact, was facilitated by the capability handling library.] There's no inherent limitation - just a different implementation. However, AFAIK, Mach did not allow copying/marshaling port handles to another host: i.e. to enable a remote task, a server had to accept connections from anyone on the remote host. [Same model as Amoeba: first connect, then accept or reject.]>I can *move* that send-once right to another task of my chosing >thereby giving it the "permissions" that were tied to it (by its >server). But, in doing so, *I* lose the ability to use it! >In Amoeba, we would *both* retain the capability to use it!Only if the server allows it.>>> But, there is no direct *tie* to the original ticket from which >>> this one (subset) was created! (or the one before *that*; or the one >>> before *that*; etc.) >> >> The *tie* is through the capability. The ticket is only a reference >> to it. > >But the reference is the same regardless of who "uses" it! >If you give me a binary copy of a ticket, the kernel doesn't >know that I now possess that capability (object, permission).About 4 times now I have told you that Amoeba could restrict the user of a ticket. No, the kernel doesn't know who has tickets or how many copies exist. Yes, a server can tell who is presenting it a ticket and decide whether the user is authorized.>I can freely create a "constrained" capability from this and >hand it to someone else. Who, in turn, can *duplicate* that and >hand it to yet another. > >The only way for any of us to know a ticket (capability) has >been revoked is to try to *use* it. If I use mine or you >use yours, we get told THAT capability has been revoked. >But these other (nameless) two guys in my example have no idea >that this has happened (unless one of us knows to tell BOTH >of them).They get told when they try to use it. That's an administrative issue which has little to do with mechanism. In your system, revoked ports just evaporate leaving clients equally clueless unless *somebody* deigns to tell them why. Your clients don't know their port privileges have been revoked until they try to use their handles. Someone could be sitting on a useless handle for hours, days or weeks (how long since last reboot?). How is that *substantively* different from a cached ticket in Amoeba? Coding to force periodic communication isn't an answer.>> Only the holder of an "owner" ticket can request the server to version >> the capability. That invalidates *every* ticket referencing the old >> version ... including the owner's own ticket!. When versioning a >> capability, a server must immediately issue a new "owner" ticket in >> response. > >In mach, deleting the port (Handle) deletes all instances of it. >And, since the kernel knows where those instances are, those >Holders can be notified when that happens.Only if they were coded respond to the appropriate signal. And Mach didn't allow signaling a task on a remote host.>> No. A server needs to track only those (versions of) capabilities for >> which it intends to honor tickets. It does *not* need to retain any >> information regarding revoked capabilities. > >It needs to know the current version of each capability that it >will honor. Because actors can present stale tickets. > >In Mach, the Handle is *gone*. The Handler never sees it again. >Nothing to "remember".That isn't substantively different. Amoeba servers don't have to remember capabilities for stale tickets. And again, a Mach host serving tasks on a remote machine couldn't simply pull the plug on the remote machine. Mach was a bit more efficient at preventing unwelcome *local* communication, but the remote situation (sans proxies) was the same as with Amoeba. (And proxies could be done under any system).>>>> Because capabilities are kept private by the issuing server, >>>> surrogates capabilities can be linked to the owner's capability, >>>> allowing the owner to void delegate and/or surrogate tickets by >>>> versioning/deleting the appropriate capability. The surrogate, of >>>> course, can void delegate tickets by versioning/deleting its own >>>> capability. >>> >>> But that means the handler/server has to do all this work! "Remembering". >> >> In your case the kernel does all this work for _all_ of the servers. > >Yes! This is the single biggest drawback! But, it is the only way >to enforce communciation restrictions, etc. > >E.g., in Amoeba, even bad tickets have to be processed by the backing >server. On *that* server's dime (timeslice/resources). In Mach, bad >Handles (no send rights) are caught at the local server on the >offending actor's dime!Not exactly - that's a system design issue. Remember that an Amoeba task needs a ticket to access the network service. It's perfectly reasonable to assign virtual channels to remote services each having their own access tickets. The result is more or less the same: "this party does not accept collect calls, enter your phone card digits now". No card, no call.>If a Handler sees an actor abusing his privilege (whatever that means), >he can revoke the Handle (because the Handler has the receive right >for the port!) and shut the actor up *at* the actor's kernel. > >In Amoeba, the server is forced to validate (invalidate) each request >as it comes in -- regardless of how often, etc. And, tie up RPC/IPC >resources as well!That's just shifting the load - in either case *somebody* has to do the work to reject the invalid attempt. As I said above, Amoeba was perfectly able to isolate a process on its own host - it's a matter of system design unrelated to the capability mechanism.>>>> All versions of Amoeba's tickets specified a server (or service) ID - >>>> the field wasn't sign protected because the ID might be a task >>>> instance, but it allowed servers to immediately reject tickets they >>>> couldn't possibly have issued. >>> >>> In my case, the Handler/server never is *presented* the communication >>> in the first place -- unless the path has been previously created by >>> possession of the Handle. >> >> In Amoeba's case, the sender needs a ticket that says it can ride the >> communication channel. 6 of one ... > >It's just an IPC! Ameoba won't let tasks issue IPC's without >a capability?? Who validates *those* (i.e., acts as handler/server >for IPC)?Bootstrap tickets were provided by IDL, compiled into the code. Dynamic tickets accumulated as the process executed had to be managed by the program. All the system calls had 2 versions: one that used the default IDL ticket and another that required an explicit ticket parameter [or sometimes multiple ticket parameters, depending].>> It's easy enough to create separate channels for each service with >> unique tickets needed to access them. > >It's still an IPC. Something on the other end has to validate on *its* >dime.There's a *BIG* difference between local IPC and remote IPC. By giving each "channel" separate access, the _local_ communication server gates usage.>In my case, that's the kernel EXECUTING IN YOUR TIMESLOT.So you save 1/2 a local IPC call. I don't see that as a tremendous benefit. Local IPC can be made extremely lightweight: e.g., you can do it without message copying.>> Mostly due to time pressure [Mach was developed on a grant] they took >> a disastrous shortcut. Realizing that they couldn't *quickly* >> implement a *full* Unix API translation on top of native services, >> they grabbed an existing BSD Unix kernel, modified it into a Mach >> server task and hacked the system libraries to RPC the "compatibility >> server". > >The "BSDSS" (Single Server).Was it really the SS version? I may be misremembering because I played with different Mach versions, but ISTM that a server instance could run processes under multiple user ids ... you just couldn't run very many processes over any particular server instance.>(I believe I probably have the largest "private" Mach archive in >existence. I'm sure many of the older releases are no longer available >on *any* public servers!)Cool.>... But, the Mach API *under* another layer of abstraction >just makes things worse, not better. Also, there were some >problems with the possibility of cross contamination *within* >the library -- no doubt another sacrifice to efficiency.My recollection of v3 was that, so long as you stayed away from features that relied on Un*x signals, performance was quite good. ISTM that signaling was the big stake they couldn't drive. YMMV.>I think the Mach (and Amoeba) focus on application to existing >usage paradigms is the fatal flaw. E.g., applications there are >transient and invoked by a multitude of users on a multitude of >objects. BIG objects. > >OTOH, appying those mechanisms to a *control* system where: >- connections tend to persist for a LONG period of time (24/7/365) >- isolation means real dollars (task A doesn't end up scrapping > all the product produced by task B) >- processors can have multiple cores >- as well as being physically distributed >- and compute costs decrease considerably faster than memory or > communication costs >THEN, burning cycles in a kernel (cached) to save references to >memory (local or remote) AND unwanted communication costs can have >a big payoff. > >But, I think you have to approach the OS with these things in >mind and not "files" and TTYs, etc. Do you *really* need all >these traps? All these function arguments? etc.I agree. There is no mechanism that is universally applicable. George
Reply by ●November 19, 20132013-11-19
Hi George, On 11/18/2013 10:43 AM, George Neuner wrote:> On Sun, 17 Nov 2013 11:31:44 -0700, Don Y <This.is@not.Me> wrote: > >> you missed the point of my question. > > No I didn't. > > Amoeba doesn't have any notion of reference counting BECAUSE reference > counting doesn't scale [nor does garbage collection, but that's a > different discussion]. Amoeba was designed for campus area networks > with an eye toward wide area networks.> If every owner ticket is lost, a persistent capability exists until an > administrator manually deletes it (or the object it is associated > with). That's why administrators get the big bucks.Actually, Amoeba *does* GC! Each server is responsible for discarding "unreferenced" objects whenever a GC cycle is invoked for/on that server. The difference is, Amoeba doesn't know where all the "references" (handles/tickets) are. So, it has to, instead, implement GC by noticing which objects haven't been *referenced* (actively operated upon). So, a ticket holder has to periodically tickle the object for which he's holding the ticket -- lest the server decide the object is no longer being referenced ("of interest") to anyone. Amoeba had a "notice" (can't recall the name of the actual method) operation for each object for this express purpose. By moving the capabilities out of the kernel, *everything* associated with them was moved out of the kernel as well. Including the GC! The "campus area network" goes to my other point re: the "mistake" Mach, Amoeba made in targeting "UNIX" (or, generic time-sharing service). It means they could assume things about the deployment environment (a sysadm) which, in turn, ensured they could only be applied to such environments. Imagine (c.a.e) having to clear some accumulated references *manually* in an embedded device that runs 24/7 ("Ah, there's a resource leak in my HVAC controller. Guess I'll have to CYCLE POWER to clear those stale handles!" :> ) Contrast that with an approach where <something> (kernel) is doing it continuously as a matter of its natural function. [Note *my* assumptions about how I intend to deploy my system also place limits on how/where it can be applied. It would be an inefficient solution to a distributed time-sharing system implementation (though, given the resources available in "real computers" nowadays, that might not be a deal-breaker). All my design choices are oriented towards systems that have to stay up "forever" AND be unattended.]
Reply by ●November 19, 20132013-11-19
Hi George, On 11/18/2013 2:07 PM, George Neuner wrote:> On Sun, 17 Nov 2013 16:55:26 -0700, Don Y <This.is@not.Me> wrote: > >>>> OTOH, 0xDEADBEEF010204302893740 passed from Amoeba task A to Amoeba >>>> task B carries "rights" with it. Encoded within the cryptographic >>>> envelope! >>> >>> So what? The envelope validates the contents. Encoded rights are >>> useless if the envelope is broken or even just sent to the wrong >>> server. >> >> But not invalidated if possessed by another. I.e., I can send copies of >> a capability to 100 other tasks. They have exactly the permissions that >> I had. The kernel is unaware that I have made these transfers -- it's >> just data! > > That's only true of the original implementation. Later versions > expanded the ticket to include an "authorized user" field which > variously could be a process, host or host:process identifier, or a > admin level user/group id. The user field was protected by a 2nd > crypt signature. This enabled servers to restrict who could use a > ticket.OK. But, did the *kernel* make the decision as to whether to pass the message (IDL) on to the service (encoded in the ticket) based on an examination of that field? Or, did the kernel pass it on to the service and rely on the *server* to decide "You aren't the right person"? (in addition to "You don't have the right permission(s)"?) I.e., as the kernel coesn't treat tickets as anything other than "data", it can't prevent me from sending "copies" of a ticket to lots of tasks, neither of whom are "authorized user" of that ticket. Who prevents them from pestering the server with this "inapplicable" ticket?>> For example, Mach had a "send-once" right. It allowed the holder to >> send *one* message to a port. It could not be copied by the holder >> (had to be created by the holder of the corresponding *receive* >> right) even though holders of regular send rights could ask the >> kernel to copy (or move/transfer) them. >> >> So, a task could be given 38 send rights for a particular port. >> 38 messages later, the port stops working. >> >> In Amoeba, once you know the ticket's "binary value", you can use >> it as many times as you want (unless the server backing it has >> imposed some limitation on its use -- which a mach server could >> also layer atop any send right!). > > In Amoeba, such things are entirely up to the server. It can choose > to issue single use tickets, or "numbered" tickets if it desires. The > rights field is completely server defined - it can mean anything. AndThere are two different levels of "rights" (permissions, authorizations, etc.) in this discussion. In Mach, there are "rights" (their term) that apply to ports. I.e., to communication. A task can own the "receive right" for a port (only one such right exists for each port). A task can be given *send* rights for a port -- this essentially allows it to pass messages to whomever is in possession of the *receive* right for that particular port, at the present time (ports can migrate). I.e., for a client to talk to a service, it needs a SEND right to a port on which the service is actively listening -- holding a RECEIVE right. Without the send right, the kernel won't even implement the IPC. A "send once" right is a send right that is automatically destroyed by the kernel when used. I.e., send *one* message, no more. For example, if I contact a service and want it to return an object to me, I can give it a port ON WHICH I AM LISTENING (holding the receive right). The service can (whenever -- incl async I/O) deliver that object to me (or, whomever I may have passed that port's receive right along to!) via that send right. But, I am expecting ONE message from that service. I shouldn't have to delete that "send right" once the reply has been received. Nor should I have to leave that communication path open! If the right persisted, the service could keep pushing messages down the pipe that I would have to handle! *Above* this set of "communication rights", there are "permissions", "authorizations", <whatever> that the object's server implements (and enforces) for each object type and operation. The kernel has no involvement in these. If task A has the RIGHT to send to a port being received by task B, then the kernel arranges for whatever messages A opts to send to be passed along. So, task B (server) could implement a "allow 38 creat()'s" capability -- and enforce it, itself (counting the number of create() IPC's that it handles for that particular sender). Or, it could pass out 38 instances of a "send once" right knowing that the task on the other end can only invoke the service 38 times before the *kernel* blocks further communications. I.e., where the counting is done is up to the service implementor. I can let you read my password *exactly* once.> nothing prevented a server from associating additional state with a > capability. [That, in fact, was facilitated by the capability handling > library.]Yes. The same is true for Mach servers. I.e., the thread that handles a particular port (which is being fed by some other client) can implement some particular set of methods and some particular set of limits on the arguments for each of those individual methods. Permissions aren't a "bit set" that you can add to or subtract from. I.e., there's no limit to the number of individual permissions implemented by a single "Handle". Nor do the permissions associated with one Handle have to be the same as the set supported for some other Handle (of the same object type). Policy resides in the servers.> There's no inherent limitation - just a different implementation. > > However, AFAIK, Mach did not allow copying/marshaling port handles to > another host: i.e. to enable a remote task, a server had to accept > connections from anyone on the remote host. > [Same model as Amoeba: first connect, then accept or reject.]The NORMA server made ports transparent. You didn't know whether an IDL was going to result in an "IPC" or an "RPC". You just "talked to a port for which you had send right(s)" and the kernel did the rest. No separate permission required to use the network.>> I can *move* that send-once right to another task of my chosing >> thereby giving it the "permissions" that were tied to it (by its >> server). But, in doing so, *I* lose the ability to use it! >> In Amoeba, we would *both* retain the capability to use it! > > Only if the server allows it.No. The send right is managed by the kernel. If I *have* it, I can "give" it to someone else. But, since it is a one-time use, doing so means *I* can't use it! (contrast this with giving a *copy*). I've made changes to my implementation that allow me to condition the operations ("methods"!) that can be applied to ports. I.e., I treat the ports (Handles) as objects themselves! Not just references to OTHER objects!>>>> But, there is no direct *tie* to the original ticket from which >>>> this one (subset) was created! (or the one before *that*; or the one >>>> before *that*; etc.) >>> >>> The *tie* is through the capability. The ticket is only a reference >>> to it. >> >> But the reference is the same regardless of who "uses" it! >> If you give me a binary copy of a ticket, the kernel doesn't >> know that I now possess that capability (object, permission). > > About 4 times now I have told you that Amoeba could restrict the user > of a ticket.But you haven't said that the kernel ignores any tickets presented to it by anyone other than the "intended owner"! Sure, the *server* can decide to ignore the ticket -- AFTER it has passed through the kernel *to* the server! Does the "enhanced" Amoeba implementation make this restriction?> No, the kernel doesn't know who has tickets or how many copies exist. > Yes, a server can tell who is presenting it a ticket and decide-------^^^^^^^^ See my point?> whether the user is authorized. > >> I can freely create a "constrained" capability from this and >> hand it to someone else. Who, in turn, can *duplicate* that and >> hand it to yet another. >> >> The only way for any of us to know a ticket (capability) has >> been revoked is to try to *use* it. If I use mine or you >> use yours, we get told THAT capability has been revoked. >> But these other (nameless) two guys in my example have no idea >> that this has happened (unless one of us knows to tell BOTH >> of them). > > They get told when they try to use it. > > That's an administrative issue which has little to do with mechanism. > In your system, revoked ports just evaporate leaving clients equally > clueless unless *somebody* deigns to tell them why.No. There are two port-related "notification mechanisms" in Mach (i.e., concerned with things that happen in the normal course of a port's life). When the last "send right" *to* a particular port dies (is destroyed, etc.), a "No More Senders" notification is delivered to the task holding the receive right. I.e., no one is interested in the object that you represent any longer. THERE IS NO WAY FOR ANYONE TO REFERENCE IT (other than you!). Please feel free to deinstantiate it and GC it's resources. When the receive right to a port dies (i.e., when the server that is handling the incoming messages SENDed to that port dies or destroys the port), a "Dead Name" notification is sent to holders of send rights. I.e., the object that you were referencing no longer exists. Don't bother trying to access it! In practice, I don't use this but, instead, rely on the task who deleted the port to explicitly notify the necessary clients. Or, let them figure it out at their leisure (when they try to invoke their "send rights", they will see the port is now a "dead name". This Handle is no longer attached to anything!> Your clients don't know their port privileges have been revoked until > they try to use their handles. Someone could be sitting on a useless > handle for hours, days or weeks (how long since last reboot?). How is > that *substantively* different from a cached ticket in Amoeba?Above.> Coding to force periodic communication isn't an answer.[though that's how Amoeba handles GC! :> ]>>> Only the holder of an "owner" ticket can request the server to version >>> the capability. That invalidates *every* ticket referencing the old >>> version ... including the owner's own ticket!. When versioning a >>> capability, a server must immediately issue a new "owner" ticket in >>> response. >> >> In mach, deleting the port (Handle) deletes all instances of it. >> And, since the kernel knows where those instances are, those >> Holders can be notified when that happens. > > Only if they were coded respond to the appropriate signal. And Mach > didn't allow signaling a task on a remote host.If a task doesn't care to deal with "port death", it doesn't have to. If it is important to the task's proper operation, it will! Mach allows any object (port) in the "system" to be accessed from anywhere else in the "system". E.g., if a debugger task running on host A wants to stop execution of some task running on host B, it can do so -- assuming it has been granted access to the port that *represents* that task (tasks are objects, too!). If a task on host B wants to access a file BACKED by a server on host C, it can do so. AND NOT BE AWARE THAT AN RPC IS BEING INVOKED instead of a simple, local IPC. This is important to me. I want to be able to move the endpoints of communication links freely within the system. The task at one end of a link shouldn't need to know that I've moved the other end to a different task -- OR to a different task on a different HOST! Likewise, I want to be able to migrate the actual *servers* (executable) while the system is "hot". I can't afford to signal all clients and say, "I'm going to change the network address of the server that will be handling your future requests". Instead, I bring up a new copy of the server on the intended host, then *move* the ports that the old one was listening to (for incoming requests) over to that new task. Just as if it was a second copy of the task executing on the same host as the first!>>> No. A server needs to track only those (versions of) capabilities for >>> which it intends to honor tickets. It does *not* need to retain any >>> information regarding revoked capabilities. >> >> It needs to know the current version of each capability that it >> will honor. Because actors can present stale tickets. >> >> In Mach, the Handle is *gone*. The Handler never sees it again. >> Nothing to "remember". > > That isn't substantively different. Amoeba servers don't have to > remember capabilities for stale tickets. > > And again, a Mach host serving tasks on a remote machine couldn't > simply pull the plug on the remote machine. Mach was a bit more > efficient at preventing unwelcome *local* communication, but the > remote situation (sans proxies) was the same as with Amoeba. > (And proxies could be done under any system).See above.>>>>> Because capabilities are kept private by the issuing server, >>>>> surrogates capabilities can be linked to the owner's capability, >>>>> allowing the owner to void delegate and/or surrogate tickets by >>>>> versioning/deleting the appropriate capability. The surrogate, of >>>>> course, can void delegate tickets by versioning/deleting its own >>>>> capability. >>>> >>>> But that means the handler/server has to do all this work! "Remembering". >>> >>> In your case the kernel does all this work for _all_ of the servers. >> >> Yes! This is the single biggest drawback! But, it is the only way >> to enforce communciation restrictions, etc. >> >> E.g., in Amoeba, even bad tickets have to be processed by the backing >> server. On *that* server's dime (timeslice/resources). In Mach, bad >> Handles (no send rights) are caught at the local server on the >> offending actor's dime! > > Not exactly - that's a system design issue. > > Remember that an Amoeba task needs a ticket to access the network > service. It's perfectly reasonable to assign virtual channels to > remote services each having their own access tickets. > > The result is more or less the same: "this party does not accept > collect calls, enter your phone card digits now". No card, no call.In Mach, the network isn;t visible to the tasks. Everything just looks like an IPC -- even if, in fact, it is implemented *now* as an RPC! (hence my original RPC/IPC references -- tired of maintaining that level of detail) I actually have no way for a task to determine where the "other end" of a communication (IPC) is handled -- locally or remotely.>> If a Handler sees an actor abusing his privilege (whatever that means), >> he can revoke the Handle (because the Handler has the receive right >> for the port!) and shut the actor up *at* the actor's kernel. >> >> In Amoeba, the server is forced to validate (invalidate) each request >> as it comes in -- regardless of how often, etc. And, tie up RPC/IPC >> resources as well! > > That's just shifting the load - in either case *somebody* has to do > the work to reject the invalid attempt.Doing the work locally means you don't incur the cost of the RPC (cuz any "send" could potentially end up being an RPC instance). And, the only way to "punish" rogues and adversaries (if they don't feel the cost of their actions, then the *system* must be feeling it!) is to make the cost of those actions be accountable *in* the actor's resource budget. E.g., if an actor can cause *you* to consume gigabytes of memory, then *you* bear the cost and it can keep pounding on you. OTOH, if *it* tries to do so directly, then it bears the cost ("Sorry, your memory constraints does not allow that amount")> As I said above, Amoeba was perfectly able to isolate a process on its > own host - it's a matter of system design unrelated to the capability > mechanism. > >>>>> All versions of Amoeba's tickets specified a server (or service) ID - >>>>> the field wasn't sign protected because the ID might be a task >>>>> instance, but it allowed servers to immediately reject tickets they >>>>> couldn't possibly have issued. >>>> >>>> In my case, the Handler/server never is *presented* the communication >>>> in the first place -- unless the path has been previously created by >>>> possession of the Handle. >>> >>> In Amoeba's case, the sender needs a ticket that says it can ride the >>> communication channel. 6 of one ... >> >> It's just an IPC! Ameoba won't let tasks issue IPC's without >> a capability?? Who validates *those* (i.e., acts as handler/server >> for IPC)? > > Bootstrap tickets were provided by IDL, compiled into the code. > Dynamic tickets accumulated as the process executed had to be managed > by the program. > > All the system calls had 2 versions: one that used the default IDL > ticket and another that required an explicit ticket parameter [or > sometimes multiple ticket parameters, depending].The difference is Mach doesn't treat the network as "special". It has no (convenient) way of telling you when you are using network resources. So, *I* don't want you to be able to use them (as they are scarce -- in addition to bandwidth, network services are typically their own "tasks" and your use of its resources interacts with other users of that same resource) unless you *should* be able to use them. I.e., nothing prevents you from picking up the phone and calling every number possible. The folks on the other end can choose to hang up on you -- AFTER they have been bothered to answer the phone, etc. CID prevents you (in theory) from being a victim of this sort of "abuse" (DoS)>>> It's easy enough to create separate channels for each service with >>> unique tickets needed to access them. >> >> It's still an IPC. Something on the other end has to validate on *its* >> dime. > > There's a *BIG* difference between local IPC and remote IPC. By giving > each "channel" separate access, the _local_ communication server gates > usage.Again, if a service moves (which was facilitated in Amoeba by things like FLIP), then how does the task wanting to talk to it NOW KNOW that it's a "long distance call"? That he NOW needs permission to use the network to contact that service? ("WTF? I was just talking to it 12 microseconds ago...")>> In my case, that's the kernel EXECUTING IN YOUR TIMESLOT. > > So you save 1/2 a local IPC call. I don't see that as a tremendous > benefit. Local IPC can be made extremely lightweight: e.g., you can > do it without message copying.No! I save the server having to process that incoming message (remember, messages take up spade in queues; LEGITIMATE tasks opting to use those queues end up waiting; etc.) AND the possibility of network services coming into play if the service is not local. I.e., a round trip to the remote host (through two network stacks, TWICE) just to say, "NO_PERMISSION" -- when the kernel could have done so locally ON YOUR DIME. You have to look graphically at where each test is being imposed and watch to see how the data flows through the system wrt locations of different services, etc. Remember, I'm *deliberately* using lots of physically distributed processors instead of *any* "big processors". So, I *expect* lots of chatter on the wire.>>> Mostly due to time pressure [Mach was developed on a grant] they took >>> a disastrous shortcut. Realizing that they couldn't *quickly* >>> implement a *full* Unix API translation on top of native services, >>> they grabbed an existing BSD Unix kernel, modified it into a Mach >>> server task and hacked the system libraries to RPC the "compatibility >>> server". >> >> The "BSDSS" (Single Server). > > Was it really the SS version? I may be misremembering because I > played with different Mach versions, but ISTM that a server instance > could run processes under multiple user ids ... you just couldn't run > very many processes over any particular server instance.Bad choice of names. Think of single server as "monolithic". Sort of like "real-time Linux" (no, the kernel hasn't been rewritten in a more deterministic fashion! Rather, it runs as a task UNDER an RTOS...). The final Mach "Unix" implementation was the "multiserver". It was what Mach was initially intended to facilitate. I.e., a "tty" was an object. Anything related to tty's was handled by the tty server. A directory was an object (list of "files"). Directories were handled by the directory service. Files (within directories) were objects -- handled by File Server. Processes were objects handled by a process server. etc. And, all of these operating in user-space. So, a much bigger performance hit as accessing each "service" meant kernel traps, IPC's, etc. By contrast, the BSDSS was "just a big app" -- far more efficient.>> (I believe I probably have the largest "private" Mach archive in >> existence. I'm sure many of the older releases are no longer available >> on *any* public servers!) > > Cool.(sigh) A lot of digging to gather up all the pieces over the years! Unfortunately, a paper that I have been looking for recently is not present in the archive. This leaves me uneasy wondering what else may have leaked away (it is entirely possible that it was a print copy and I never had an electronic version). The CMU "reports" archive no longer seems to be accessible.>> ... But, the Mach API *under* another layer of abstraction >> just makes things worse, not better. Also, there were some >> problems with the possibility of cross contamination *within* >> the library -- no doubt another sacrifice to efficiency. > > My recollection of v3 was that, so long as you stayed away from > features that relied on Un*x signals, performance was quite good. ISTM > that signaling was the big stake they couldn't drive.Mach tried to address "signals"/exceptions from the very beginning. (UNIX signals seem to be an afterthought) Lots of things could raise exceptions -- without really being "bugs" (e.g., the "no more senders" notification). As Mach was multithreaded from the outset, the whole notion of how to *handle* exceptions had several possible solutions. (In a single thread UNIX *process*, the choice is pretty obvious!) E.g., a signal is delivered to a task (contains threads and resources). Who should "handle" it? Just interrupt one thread at random (sort of like a UNIX process handles signals)? The thread that caused the exception? (what happens if it wasn't really associated with any ONE thread -- e.g., the no more senders). So, you can nominate a thread to handle your exceptions for the entire task. This is GREAT! The thread need not concern itself with anything other than exceptions! And, its resources can be apportioned such that it *can* handle those exceptions WITHOUT having to also worry about the "normal work" (e.g., what happens when a UNIX process is at its peak stack penetration and a signal comes along?) But, it means that the exception handler has to be coded to handle *all* these exceptions! You can't just take small mouthfuls like in UNIX (register different handlers for different signals, etc.)>> I think the Mach (and Amoeba) focus on application to existing >> usage paradigms is the fatal flaw. E.g., applications there are >> transient and invoked by a multitude of users on a multitude of >> objects. BIG objects. >> >> OTOH, appying those mechanisms to a *control* system where: >> - connections tend to persist for a LONG period of time (24/7/365) >> - isolation means real dollars (task A doesn't end up scrapping >> all the product produced by task B) >> - processors can have multiple cores >> - as well as being physically distributed >> - and compute costs decrease considerably faster than memory or >> communication costs >> THEN, burning cycles in a kernel (cached) to save references to >> memory (local or remote) AND unwanted communication costs can have >> a big payoff. >> >> But, I think you have to approach the OS with these things in >> mind and not "files" and TTYs, etc. Do you *really* need all >> these traps? All these function arguments? etc. > > I agree. There is no mechanism that is universally applicable.Exactly! I latched onto Mach in the early/mid 90's NOT because it would be a novel (better?) way of building big time-sharing systems (which is what all their literature described, at the time). But, rather, for how it could be applied FROM THE GROUND UP to the sorts of embedded systems I foresaw -- cheaper memory, faster MPU's, then MCU's and SoC's, etc. I.e., I don't have to have a physical disk drive to implement a file system. It can be a FLASH card. Or BBRAM. Or ROM! I don't have to set aside big buffers to move data out of and into those I/O devices -- I can just define a 512 byte "page" that a SERVICE uses to mmap() 512 bytes of a FLASH chip into userland. (yeah, its slow, but it doesn't consume any resources!) I don't have to worry about task A stomping on task B's resources -- MMUs *will* be available and affordable. And, I can make the system dynamically reconfigurable so load balancing doesn't become a nightmare ("Crap! I exec()'d the binary on the wrong host! In hindsight, that other host over there has nothing to do, now!") Of course, there is a steep cost for all this (complexity, etc.). But, CPUs are getting faster more quickly than memory, comms, etc. So, burning "100" instructions (assuming some sort of cache) isn't a big deal compared to "50" -- esp if it saves a trip down the network, etc. I am *sure* this isn't exactly how things will be done in the future. There need to be more economical ways of getting these results. But, I'm *reasonably* sure "systems" will be more physically distributed. More processors. etc. And, forcing people to use things like Berkeley sockets for all those comms is like forcing a sprinter to wear scuba flippers! (Limbo/Inferno was the final piece of the puzzle to fall into place. Many of its ideas map naturally to this framework -- though with an entirely different implementation! :< Now, I just have to beat it all into submission! :> )
Reply by ●November 21, 20132013-11-21
Hi Don, On Tue, 19 Nov 2013 15:13:07 -0700, Don Y <this@isnotme.com> wrote:>Actually, Amoeba *does* GC! Each server is responsible for >discarding "unreferenced" objects whenever a GC cycle is >invoked for/on that server. > >The difference is, Amoeba doesn't know where all the "references" >(handles/tickets) are. So, it has to, instead, implement GC >by noticing which objects haven't been *referenced* (actively >operated upon). > >So, a ticket holder has to periodically tickle the object for >which he's holding the ticket -- lest the server decide the >object is no longer being referenced ("of interest") to >anyone.Not exactly.>Amoeba had a "notice" (can't recall the name of the actual >method) operation for each object for this express purpose.Amoeba's distributed filesystem is comprised of peer/proxy servers. When a non-local file is opened, the local server obtains a copy of the file's capability to accelerate future accesses. Then it acts as a proxy to the remote server, performing access validations locally and relaying requests/data over a trusted peer channel. The file servers time out and remove cached non-local capabilities if they aren't used. They do NOT time out capabilities for their own locally hosted objects. AFAIK, the filesystem is the only included peer/proxy service - all the others either are host local or are simply peer. I'm not aware that any of the other services time out capabilities. George
Reply by ●November 21, 20132013-11-21
Hi Don, On Tue, 19 Nov 2013 16:49:10 -0700, Don Y <this@isnotme.com> wrote:>OK. But, did the *kernel* make the decision as to whether to >pass the message (IDL) on to the service (encoded in the >ticket) based on an examination of that field? Or, did the >kernel pass it on to the service and rely on the *server* to >decide "You aren't the right person"? (in addition to >"You don't have the right permission(s)"?)Yes and no. Remember, to make ANY system call requires a ticket. Normally, a process is created with a set of default tickets that enable it to access basic services (filesystem, network, etc.). The default tickets are held by the kernel and can be chosen by the parent when the process is created. The default ticket for any particular call is assumed unless the process explicitly presents a different one when the call is made. However, the kernel IPC mechanism doesn't validate the message other than to check that the server address on the ticket is good. E.g., if the process has no ticket for the socket service,, it will be prevented from even trying to contact the socket service.>A [Mach] "send once" right is a send right that is automatically >destroyed by the kernel when used. I.e., send *one* message, >no more.You can do similar with numbered use tickets/capabilities in Amoeba. Everything you can think of has an implementation analogue in either system. You're just hung up on kernel enforcement. E.g., you could design an Amoeba based system where all the system capabilities were held by [or accessible to] a "validation server" to which all client calls are directed. The validation server would authenticate the client's ticket and then hand off the "connection" to the appropriate server.>Permissions aren't a "bit set" that >you can add to or subtract from. I.e., there's no limit to >the number of individual permissions implemented by a single >"Handle". Nor do the permissions associated with one Handle >have to be the same as the set supported for some other Handle >(of the same object type).Permissions in Amoeba aren't a "bit set" either ... unless you want them to be. You have a 32 bit field in the ticket to use however you want. The field can represent a bit map, or an enumerated value, or be divided into multiple fields. If you don't need to validate ticket holders, you've got another 96 bits in the v2 user id and 2nd signature field to play with. There is no *practical* limit to the set of privileges an Amoeba capability can administer or that a ticket can claim.>> [Same model as Amoeba: first connect, then accept or reject.] > >The NORMA server made ports transparent. You didn't know >whether an IDL was going to result in an "IPC" or an "RPC". >You just "talked to a port for which you had send right(s)" >and the kernel did the rest. > >No separate permission required to use the network.That's not my point. To do much of anything, a Mach server has to register a public service port with send rights - which any task in the network can scan for and try to connect to. Aside from limiting the count of available send rights to the port, there is no way to prevent anyone from connecting to it. Only *after* determining that the connection is unwanted can the server decide what to do about it. Using MIG didn't affect this: you couldn't encode port ids into the client using IDL - the only way for a client to find the server was to look up its registered public port.>>> The only way for any of us to know a ticket (capability) has >>> been revoked is to try to *use* it. If I use mine or you >>> use yours, we get told THAT capability has been revoked. >>> But these other (nameless) two guys in my example have no idea >>> that this has happened (unless one of us knows to tell BOTH >>> of them). >> >> They get told when they try to use it. >> >> That's an administrative issue which has little to do with mechanism. >> In your system, revoked ports just evaporate leaving clients equally >> clueless unless *somebody* deigns to tell them why. > >No. There are two port-related "notification mechanisms" in Mach >(i.e., concerned with things that happen in the normal course of >a port's life). > >When the last "send right" *to* a particular port dies (is destroyed, >etc.), a "No More Senders" notification is delivered to the task >holding the receive right. I.e., no one is interested in the >object that you represent any longer. THERE IS NO WAY FOR ANYONE >TO REFERENCE IT (other than you!). Please feel free to deinstantiate >it and GC it's resources.That tells the server all existing clients have disconnected ... it does NOT tell new clients they can't connect.>When the receive right to a port dies (i.e., when the server that >is handling the incoming messages SENDed to that port dies or >destroys the port), a "Dead Name" notification is sent to holders >of send rights. I.e., the object that you were referencing >no longer exists. Don't bother trying to access it!This tells the server or client the other side has disconnected. It does not tell them why.>> Your clients don't know their port privileges have been revoked until >> they try to use their handles. Someone could be sitting on a useless >> handle for hours, days or weeks (how long since last reboot?). How is >> that *substantively* different from a cached ticket in Amoeba? > >Above.You assume that both tasks [server and client] have implemented notify_server and do-mach-notify-<whatever>. Unless both sides are using MIG, you can't assume any notifications will be sent, received or handled - it isn't *required*. If a task doesn't implement the relevant notification handler, it won't know anything has happened until it gets an error trying to use the port.>>> In mach, deleting the port (Handle) deletes all instances of it. >>> And, since the kernel knows where those instances are, those >>> Holders can be notified when that happens. >> >> Only if they were coded respond to the appropriate signal. And Mach >> didn't allow signaling a task on a remote host. > >If a task doesn't care to deal with "port death", it doesn't have >to. If it is important to the task's proper operation, it will!You're assuming again.>>> The "BSDSS" (Single Server). >> >> Was it really the SS version? > >Bad choice of names.Ok. I misunderstood you to mean it was BSD's single user kernel - not simply Mach's "single server" model.>But, I'm *reasonably* sure "systems" will be more physically >distributed. More processors. etc. And, forcing people to >use things like Berkeley sockets for all those comms is like >forcing a sprinter to wear scuba flippers! (Limbo/Inferno >was the final piece of the puzzle to fall into place. Many >of its ideas map naturally to this framework -- though with >an entirely different implementation! :< Now, I just have >to beat it all into submission! :> )I really don't like network agnosticism. It simplifies coding for simple minded tasks, but it makes doing real time applications like audio/video streaming, and remote monitoring/control a b*tch because you don't have visibility to adjust to changing network conditions. I hate RPC for the same reason. I often use loopback connections to communicate between co-hosted tasks. At the very least, using a network API reminds me that the tasks may, in fact, be separated by a network. Sometimes it doesn't matter, but sometimes it does. I had to deal with an extreme example of this in a component SRT system I worked on back before high speed Internet was common. The system was designed for use on a LAN, but the components were separable and sometimes were used that way. The system could be run successfully over dial-up lines [but it had to *know* which of its components were on dial-up]. The problem was, at that time, businesses often interconnected LANs using dial-up bridge routers: an apparently local LAN address could, in fact, be in another city! [Or even another state!!] Many of these bridges could be configured to ignore keep-alive packets and hang up if you weren't sending real data. The next real message then would be delayed - sometimes for many seconds - while the dial-up link was reestablished. To save on phone bills, IT managers tended to configure their bridges to shut down phone links quickly. The story is long and I won't bore you with it ... suffice to say that things became a lot more complicated having to deal with the potential for a customer to configure the system for a LAN and then try to run it distributed over dial-up bridges. That caused me [and others] an awful lot of headaches. George
Reply by ●November 21, 20132013-11-21
Hi George, ["pro bono" day so I've got to get my *ss out the door, RSN] On 11/21/2013 5:16 AM, George Neuner wrote:> On Tue, 19 Nov 2013 16:49:10 -0700, Don Y<this@isnotme.com> wrote: > >> OK. But, did the *kernel* make the decision as to whether to >> pass the message (IDL) on to the service (encoded in the >> ticket) based on an examination of that field? Or, did the >> kernel pass it on to the service and rely on the *server* to >> decide "You aren't the right person"? (in addition to >> "You don't have the right permission(s)"?) > > Yes and no. Remember, to make ANY system call requires a ticket. > > Normally, a process is created with a set of default tickets that > enable it to access basic services (filesystem, network, etc.). The > default tickets are held by the kernel and can be chosen by the parent > when the process is created. > > The default ticket for any particular call is assumed unless the > process explicitly presents a different one when the call is made. > > However, the kernel IPC mechanism doesn't validate the message other > than to check that the server address on the ticket is good. E.g., if > the process has no ticket for the socket service,, it will be > prevented from even trying to contact the socket service.Ticket encodes "server port" in a virtual sense. I.e., for a "name service" analogy, the "server" portion would be "DNS" (not a specific IP address). The system was responsible for locating and tracking which "processor" was currently "hosting" that service. (i.e., DNS might be "29" -- today! And, running on 10.10.2.27 -- NOW!) If you needed a special ticket to use the socket service, how did you talk to *any* service as you had no knowledge of what was "local" vs. "remote"? I.e., that was the appeal of FLIP! I.e., the kernel had to invoke the RPC if the service port IN YOUR TICKET was located on a remote host. (recall Amoeba used smallish processors originally. 16MB nominal size physical memory and NO paging!)>> A [Mach] "send once" right is a send right that is automatically >> destroyed by the kernel when used. I.e., send *one* message, >> no more. > > You can do similar with numbered use tickets/capabilities in Amoeba. > > Everything you can think of has an implementation analogue in either > system. You're just hung up on kernel enforcement.Exactly! -------------------------^^^^^^^^^^^^^^^^^^ You could implement some silly VM scheme whereby any task (thread) could write anywhere in physical memory -- and, give the owner of that piece of memory the power to allow or inhibit the write. This would allow you to implement protected memory domains whereby task A can't stomp on task B's memory. It would give you an even finer grained (i.e., BETTER!) scheme than current systems use! A task could opt to let another task write on one *byte* and not the byte next to it! You could share objects at whatever granularity you choose! BUT, IT WOULD REQUIRE THE "owning" TASK TO PROCESS EACH INCOMING WRITE ATTEMPT! If tasks all were well-behaved, not a problem -- each write attempt would be a DESIRED one! OTOH, a rogue/adversary could intentionally sit there doing "bad" things. Your memory integrity won't br compromised because you were implemented correctly. But, he's SPENDING YOUR DIME! His actions are causing you to have to burn cycles invalidating his accesses. You can't "block" his actions (in this hypothetical model). So, while it is fine-grained and *could* give you great performance advantages in a BENEVOLENT environment, it proves to be a vulnerability in an environment where adversaries (or, just "buggy" programs) reside. I can put a "perfect" firewall on my home internet connection. But, that doesn't stop adversaries from consuming my incoming (and, as such, outgoing) bandwidth! If that firewall executes *in* my host, then I am burning CPU cycles AT SOMEONE ELSE'S BIDDING.> E.g., you could design an Amoeba based system where all the system > capabilities were held by [or accessible to] a "validation server" to > which all client calls are directed. The validation server would > authenticate the client's ticket and then hand off the "connection" to > the appropriate server.Perfect! *If* that validation server executes using the *client's* resources! I.e., you present a (bogus) credential and *it* executes in your time slot (e.g., as a library instead of a server). So, if you want to waste YOUR time presenting bogus credentials, fine. You don't tie up network resources or resources in the object's server just to invalidate your request. *That* is my goal. You and only you pay for your actions -- whether they are careless mistakes or deliberate attacks! I.e., you can't "steal" from the system. Note any *server* that is used to do this can potentially be overwhelmed by "non-benevolent" actors presenting it with bogus credentials KNOWING that the server will have to spend CPU cycles (that it *could* have spent on other, LEGITIMATE services for "benevolent" actors!) "As is", I don't see any way of doing this in Amoeba. It was *intended* to have the servers validate tickets/capabilities -- so the kernel didn't have to understand "what they meant", just "where to send them".>> Permissions aren't a "bit set" that >> you can add to or subtract from. I.e., there's no limit to >> the number of individual permissions implemented by a single >> "Handle". Nor do the permissions associated with one Handle >> have to be the same as the set supported for some other Handle >> (of the same object type). > > Permissions in Amoeba aren't a "bit set" either ... unless you want > them to be. You have a 32 bit field in the ticket to use however you > want. The field can represent a bit map, or an enumerated value, or be > divided into multiple fields. > > If you don't need to validate ticket holders, you've got another 96 > bits in the v2 user id and 2nd signature field to play with. > > There is no *practical* limit to the set of privileges an Amoeba > capability can administer or that a ticket can claim.In my scheme, the permissions are implied *in* the Handle. If the server backing a particular Handle is willing to implement a set of N methods for *that* Handle, then the Holder has those permissions. If it chooses to only implement M methods -- possibly even *different* methods -- for another Handle, there is no inconsistency. Everything has a context and only makes sense in that context.>>> [Same model as Amoeba: first connect, then accept or reject.] >> >> The NORMA server made ports transparent. You didn't know >> whether an IDL was going to result in an "IPC" or an "RPC". >> You just "talked to a port for which you had send right(s)" >> and the kernel did the rest. >> >> No separate permission required to use the network. > > That's not my point. > > To do much of anything, a Mach server has to register a public service > port with send rights - which any task in the network can scan for and > try to connect to. Aside from limiting the count of available send > rights to the port, there is no way to prevent anyone from connecting > to it.No. That's the way Mach was *applied* -- because they were looking towards "UNIX as an application". *Obviously* (?) you need to be able to find services! Or, *do* you? What's to stop me from creating a task with the following namespace: CurrentTime Display AND NOTHING MORE! Further, the Handles that I endow the task with are: (CurrentTime, read) (Display, write) I.e., even though the IDL for the "Time" object supports a "set_time()" method, the task can't use it (it can *try* to invoke it but the server that backs the Time object will return NO_PERMISSION if it does! This task just spends it's life running: while(1) { now = get_time(); // uses CurrentTime segments = decode_7segments(now); display(segments); // uses Display }> Only *after* determining that the connection is unwanted can the > server decide what to do about it. Using MIG didn't affect this: you > couldn't encode port ids into the client using IDL - the only way for > a client to find the server was to look up its registered public port.But the client doesn't have to even have access to a particular server's public port! E.g., there is no reason you ("application") should have access to the device driver for the disk subsystem. Why should I even let you try to *find* it? In *your* namespace (that YOUR creator sought fit to establish KNOWING WHAT YOU SHOULD BE RESPONSIBLE FOR DOING), there is no entry called "DiskSubsystem"! Because there is no entry, you can scream all you want but never get a message to that server! No connection possible! If you *should* have the ability to connect to a server (e.g., the server backing the Time object, above), *it* decides whether to allow particular requests (method invocations) based on "permissions" that have been associated with the "Handle" (incoming port) on which your messages arrive. YOU CAN BE SEEN TO ABUSE THIS. Abuse is something that a server determines -- not the client *nor* the system! 500 attempts to write() a file that has been closed may not be abuse. An attempt to turn off the system's power, OTOH, *may* be! (i.e., you have permission to EXAMINE the current power status: on, battery, off -- not change it! Your attempt to do so indicates a significant bug or a deliberate attack!) In that event, the server can pull the plug on your Handle -- i.e., kill the port on which YOUR requests are received. If you manage to acquire another Handle, it can kill the port FROM WHICH your Handles were issued (even if some other task actually was responsible for creating them and passing them on to you -- an accomplice? Or, just someone who "trusted you more than they should have!"). The point is, the system can take steps to isolate your potential for damage. Can prevent you (eventually) from abusing resources.>>>> The only way for any of us to know a ticket (capability) has >>>> been revoked is to try to *use* it. If I use mine or you >>>> use yours, we get told THAT capability has been revoked. >>>> But these other (nameless) two guys in my example have no idea >>>> that this has happened (unless one of us knows to tell BOTH >>>> of them). >>> >>> They get told when they try to use it. >>> >>> That's an administrative issue which has little to do with mechanism. >>> In your system, revoked ports just evaporate leaving clients equally >>> clueless unless *somebody* deigns to tell them why. >> >> No. There are two port-related "notification mechanisms" in Mach >> (i.e., concerned with things that happen in the normal course of >> a port's life). >> >> When the last "send right" *to* a particular port dies (is destroyed, >> etc.), a "No More Senders" notification is delivered to the task >> holding the receive right. I.e., no one is interested in the >> object that you represent any longer. THERE IS NO WAY FOR ANYONE >> TO REFERENCE IT (other than you!). Please feel free to deinstantiate >> it and GC it's resources. > > That tells the server all existing clients have disconnected ... it > does NOT tell new clients they can't connect.Ah, but it does! ALL THE REFERENCES TO THE OBJECT ARE GONE! The object still exists IN THE SERVER, but no one can *talk* to "it" (the representation of the object) any more! I.e., it's like: object = &foo; ... object = NULL; (or, some random number) *object = .... won't work (at least, not on *foo*!) So, why *keep* foo around if no one can access it?>> When the receive right to a port dies (i.e., when the server that >> is handling the incoming messages SENDed to that port dies or >> destroys the port), a "Dead Name" notification is sent to holders >> of send rights. I.e., the object that you were referencing >> no longer exists. Don't bother trying to access it! > > This tells the server or client the other side has disconnected. It > does not tell them why.It *can* tell the clients that the *server* is no longer backing the object (i.e., the object no longer exists). Mach used to have a "dead name notification" whereby all actors holding ports (Handles) that referenced that port (object) were notified (sent an exception) of the death. But, this uses a lot of resources and is seldom necessary. I.e., why not just wait for the actors to "discover" this fact when they try to operate on that corresponding "port"? (the port is *marked* as "dead" but the task isn't "informed" of the death). Imagine some well known port dying and having to notify hundreds of actors of this fact. Chances are, most will say, "Gee, that's too bad! Where should I send the FLOWERS??" I.e., there's usually nothing they can DO about it. All you've told them is that, IN THE FUTURE, when/if they try to contact that port/Handle/object, they will receive a DEAD_NAME result in whichever method they invoke. OTOH, for cases where it is important to notify a "Holder" of this sort of "revoked capabilty", the server that was holding that "receive right" (i.e., the port that just died) can *choose* to asynchronously notify the actor on the other end. The actor on the other end can also choose to ignore this notification. AND, things still work "predictably" in each case! If/wehn you try to access the "dead name", the method fails. Too bad, so sad.>>> Your clients don't know their port privileges have been revoked until >>> they try to use their handles. Someone could be sitting on a useless >>> handle for hours, days or weeks (how long since last reboot?). How is >>> that *substantively* different from a cached ticket in Amoeba? >> >> Above. > > You assume that both tasks [server and client] have implemented > notify_server and do-mach-notify-<whatever>. Unless both sides are > using MIG, you can't assume any notifications will be sent, received > or handled - it isn't *required*.The kernel does the no-more-senders notification. Of course, the server doesn't have to *use* this information. If it has plenty of resources, it might just choose to mark the port and GC it later. Or, it might decide to free all the resources associated with backing that object. E.g., if the object is a file and it has writes pending, it might mark the buffers to be flushed, then return them to the heap. If it has read-ahead in anticipation of future read() methods, it can just drop that memory (because no one can send read() methods to that object any more!) Dead names have more flexibility. In either case, kernel just provides means to tell folks what is happening. Amoeba can't do this because it doesn't know where tickets are "hiding".> If a task doesn't implement the relevant notification handler, it > won't know anything has happened until it gets an error trying to use > the port.For a "dead name", yes, exactly. If a task can *tolerate* not knowing until that point, then why bother *telling* it? You could also have tasks that *never* try to talk to an object even if they have been GIVEN a Handle to it! E.g., if you never write to (your) stderr, what do you care if the file (or process!) it represents has gone away?>>>> In mach, deleting the port (Handle) deletes all instances of it. >>>> And, since the kernel knows where those instances are, those >>>> Holders can be notified when that happens. >>> >>> Only if they were coded respond to the appropriate signal. And Mach >>> didn't allow signaling a task on a remote host. >> >> If a task doesn't care to deal with "port death", it doesn't have >> to. If it is important to the task's proper operation, it will! > > You're assuming again.I *said* so! :> "If a task doesn't care to deal with 'port death'..." OTOH, if it *does*, then there needs to be a mechanism whereby the task can be *told* about this -- because it can't "see" what is happening to a port unless it deliberately tries to "use" it.>>>> The "BSDSS" (Single Server). >>> >>> Was it really the SS version? >> >> Bad choice of names. > > Ok. I misunderstood you to mean it was BSD's single user kernel - not > simply Mach's "single server" model.BSDSS was an old BSD (license required) kernel ported to run basically as a single Mach "task". I.e., slide Mach *under* an existing BSD UNIX. You gain "nothing" -- except the ability to run other things (even other OS's!) ALONGSIDE that "UNIX"! E.g., you could run BSDSS and POE (DOS server) on the same hardware at the same time -- not "DOS under BSD". Mach-UX was a more modern, less encumbered, kernel running in much the same way. Recall, about this time, the USL lawsuits were making life difficult for the UN*X clones. Lites was a similar project. Mach-US was the "application" that set out to illustrate why the mk approach would be "better" for systems than the monolithic kernel approaches had been, historically. It decomposed "UNIX" into a bunch of different "services" -- typically executing in UserLand. Then, tied them together *via* the microkernel -- instead of letting them rid in a single container atop it (as the single servers had). Unfortunately, Mach-US was too late and folks had already moved on. Trying to retrofit a reasonably mature system (UN*X) into a totally different framework (mk) was a losing proposition from the start. *Obviously*, the mechanisms developed *in* UNIX to handle <whatever> had been tweeked to fit it's idea of how a kernel should be built. To realize any significant advantages, you'd have to rethink what a "UNIX" should *be* -- not what it *is*!>> But, I'm *reasonably* sure "systems" will be more physically >> distributed. More processors. etc. And, forcing people to >> use things like Berkeley sockets for all those comms is like >> forcing a sprinter to wear scuba flippers! (Limbo/Inferno >> was the final piece of the puzzle to fall into place. Many >> of its ideas map naturally to this framework -- though with >> an entirely different implementation! :< Now, I just have >> to beat it all into submission! :> ) > > I really don't like network agnosticism. It simplifies coding for > simple minded tasks, but it makes doing real time applications like > audio/video streaming, and remote monitoring/control a b*tch because > you don't have visibility to adjust to changing network conditions. > > I hate RPC for the same reason. I often use loopback connections to > communicate between co-hosted tasks. At the very least, using a > network API reminds me that the tasks may, in fact, be separated by a > network. Sometimes it doesn't matter, but sometimes it does.I'm treating the network in much the same way that PC designers treat the PCI/memory busses. Could you design a "centralized" (non-networked) system with predictable responses if anyone could open the machine and add any BUS MASTERING hardware they wanted? :> How could you reliably know when/if you could access particular I/O's, etc. I.e., don't design for an "open" network onto which random, uncontrolled entities can be added at will. Instead, use it as a "really long bus" with devices having known characteristics along it's length. Just like when you design a system KNOWING which PCI cards will be inside! OTOH, you don't want to have to have a bunch of different mechanisms to access different devices *on* that bus! I.e., memory is memory regardless of whether its a buffer being used on a SCSI controller or an mbuf in the kernel!> I had to deal with an extreme example of this in a component SRT > system I worked on back before high speed Internet was common. The > system was designed for use on a LAN, but the components were > separable and sometimes were used that way. The system could be run > successfully over dial-up lines [but it had to *know* which of its > components were on dial-up]. > > The problem was, at that time, businesses often interconnected LANs > using dial-up bridge routers: an apparently local LAN address could, > in fact, be in another city! [Or even another state!!] Many of these > bridges could be configured to ignore keep-alive packets and hang up > if you weren't sending real data. The next real message then would be > delayed - sometimes for many seconds - while the dial-up link was > reestablished. To save on phone bills, IT managers tended to > configure their bridges to shut down phone links quickly. > > The story is long and I won't bore you with it ... suffice to say that > things became a lot more complicated having to deal with the potential > for a customer to configure the system for a LAN and then try to run > it distributed over dial-up bridges. That caused me [and others] an > awful lot of headaches.If your system had a *requirement* for a particular transit time, then its up to the implementors to guarantee that. You can't expect me to spool real time HD video to a 360K floppy disk! :> OTOH, you don't have to record video on *tape* (historically)! Avail yourself of whatever new technologies offer -- knowing what *new* capabilities this affords AND new limitations! I.e., splicing acetate film was a lot easier than digital video! All you needed was a sharp knife and some tape -- no high speed computers and fancy software... [C is ready. Off we go! :> ]
Reply by ●November 22, 20132013-11-22
Hi George, On 11/21/2013 5:16 AM, George Neuner wrote:> On Tue, 19 Nov 2013 15:13:07 -0700, Don Y<this@isnotme.com> wrote: > >> Actually, Amoeba *does* GC! Each server is responsible for >> discarding "unreferenced" objects whenever a GC cycle is >> invoked for/on that server. >> >> The difference is, Amoeba doesn't know where all the "references" >> (handles/tickets) are. So, it has to, instead, implement GC >> by noticing which objects haven't been *referenced* (actively >> operated upon). >> >> So, a ticket holder has to periodically tickle the object for >> which he's holding the ticket -- lest the server decide the >> object is no longer being referenced ("of interest") to >> anyone. > > Not exactly.Argh! Had to go wade through notes -- which then coerced me to dig through boxes of books that *HAD* been on their way to be donated. Now, of course, having "re-noticed" their actual titles, I will be strongly tempted to start cherry picking through the boxes... and the cycle will repeat. :< Anyway, _Distributed Operating Systems_ (Tannenbaum -- of Amoeba fame) describing "Objects and Capabilities in Amoeba" (section 7.2). "7.2.1 Capabilities" covers the idea of capabilities (128b) and the fields therein (server port, object, rights, check). "7.2.2 Object Protection" describes cryptographic seal and how impossible to forge added rights. "7.2.3 Standard Operations" deals with operations (methods) that can be invoked on objects. Fig 7-5 tabulates "The standard operations valid on most objects". These are: Age perform a single GC cycle (!) Copy duplicate the object Destroy destroy object and reclaim storage Getparama get params associated with server (backing the object) Info ASCII string describing the object Restrict Produce a new *restricted* capability for the object Setparams set params associated with server Status current status information from the server Touch pretend the object was just used Second paragraph begins: It is possible to create an object in Amoeba and then lose the capability, so some mechanism is needed to get rid of old objects that are no longer accessible. The way that has been chosen is to have servers run a garbage collector periodically, removing all objects that have not been used in /n/ garbage collection cycles. The AGE call starts a new garbage collection cycle. The TOUCH call tells the server that the object touched is still in use. I.e., the system has no way of knowing where all outstanding tickets for an object may be. So, it can't try to *resolve* references and use that to identify EVERY object that remains of interest. Even if the kernel were to grep() each process's memory, it would never be able to say, "Aha! Here's another ticket! Let's see what it REFERENCES and MARK that object as STILL OF INTEREST." There's no way to *recognize* a ticket in memory -- it's just bits! So, instead, Amoeba has servers maintain a time to live field for each object. Whenever an object is acted upon, the TTL is reset. An explicit TOUCH operation also resets it -- though without doing anything "actively" to the object (sort of a NOP -- this is what I previously erroneously called "notice") Whenever an AGE operation is invoked on an object, the TTL decreases by one unit. Once it reaches "0", the server managing the object (i.e., the only entity that is aware of things like TTL) can delete the object. No one has "noticed" it during the course of it's TTL (i.e., those "/n/" GC cycles (invocations of the AGE operation). So, if an actor holds a ticket that references an object (regardless of what type of object it may be) and just *sits* on it, when some *other* agency has invoked the GC on the server that manages said object those /n/ times, the object evaporates! "Use it or lose it". Note that the actor has no idea that /n/-1 GC cycles may have been performed in the time since he acquired the ticket. (remember, each object's TTL can be different at any instant in time -- it's not like *all* "foo" objects will expire at the same instant or GC cycle so no way for the system to alert everyone of an upcoming sweep: "TOUCH anything you want to keep cuz there's a GC cycle up ahead!".) In practical terms, this means that you have to artificially use every object (ticket) that you hold "often enough" to prevent the system (actually, the individual servers backing each of those objects) from deciding that no ne is interested in the object any longer and GC'ing it. So, for example, if you have stderr wired to "something" and haven't had any "errors", so far, in your execution (i.e., no need to write() to stderr), you risk having the object that your stderr is attached to (file, pty, etc) being GC'ed *before* you eventually have a need to use it! UNLESS you deliberately tickle that object periodically. Remember, if the object you are tickling is backed by a server on another node/host, that "tickle"/TOUCH means an RPC across the network! For *every* object that you want to "keep alive". In addition to this waste, how can an application designer (resource consumer) RELIABLY design to protect against such ASYNCHRONOUS, UNNOTIFIED revocations when designing his app? What is /n/ for EACH particular type of object? How often will the server for each type of object be called upon to perform a GC cycle (AGE)? How much time will my application spend NOT BLOCKED (i.e., able to execute code) in that interval? To be "safe", do I need to write my code like: payroll() { ... wages = hours * rate; tickle_objects(); benefits = lookup(my_options); tickle_objects(); taxes = (wages - benefits) * tax_rate; tickle_objects(); net = wages - (benefits + taxes); output(net); ... } Tanenbaum continues: When objects are entered into the directory server, they are touched periodically, to keep the garbage collector at bay. I.e., for *named* objects and a server whose role it is to maintain that (name, object) tuple, a process periodically runs through that service TOUCHing every object therein. I.e., every named object is accessed continuously just to make sure it doesn't evaporate. Who handles anonymous objects? Or, objects that may not need to be accessible beyond some group of cooperating actors?>> Amoeba had a "notice" (can't recall the name of the actual >> method) operation for each object for this express purpose."TOUCH"> Amoeba's distributed filesystem is comprised of peer/proxy servers. > When a non-local file is opened, the local server obtains a copy of > the file's capability to accelerate future accesses. Then it acts as > a proxy to the remote server, performing access validations locally > and relaying requests/data over a trusted peer channel. > > The file servers time out and remove cached non-local capabilities if > they aren't used. They do NOT time out capabilities for their own > locally hosted objects.It's got nothing to do with local v remote. In Amoeba, you simply have no way of knowing if there are outstanding references to an object. You (Amoeba implementor) have two options on how you handle this: - *expect* any object that has not explicitly destroyed to be of some interest to someone, somewhere. - expect people interested in objects to periodically RE-express that interest and, after a time, decide that anything for which an interest ha snot been expressed (TOUCHed or otherwise acted upon) is of no interest to anyone! The latter is Amoeba's approach as the former would result in "capability leakage" -- objects persisting long after the capabilities involving them have passed -- with no way of harvesting these. But, as you can see, it means extra (network!) traffic as a certain level of "interest" must be shown in EVERY object in the distributed system lest some objects be PREMATURELY harvested! And, still provides no guarantee that this premature harvest won't occur! No way for an application designer to guarantee that he has been "interested enough" in the objects for which he may be the sole ticket-holder! Nor any way for him to be sure he has had enough "CPU time" to *express* that interest! I can see apps being "tuned" (effectively) as system load changes. E.g., "Hmmm... we'll have to increase /n/ as some tickets are timing out too soon and we can't afford to rewrite all those apps that *use* those objects!" Sort of like arbitrarily increasing a thread's stack each time you get a SIGSEGV... always *hoping* this latest increase is "big enough". In Mach's case, the kernel (kernelS) know where all outstanding port rights (send/send-once) are. So, they can tell a server when no one is interested in a particular object (receive right) any longer. As soon as that interest goes away (as soon as the last send right is destroyed). Mach burdens the kernel with this sort of thing so the actors don't have to worry about it. So they don't have to be "tuned" to a particular server implementation (there's no /n/!)> AFAIK, the filesystem is the only included peer/proxy service - all > the others either are host local or are simply peer. I'm not aware > that any of the other services time out capabilities.And, of course, time for my morning tea! :>