Locking semantics

Hi Dimiter,

On 5/20/2015 5:54 AM, Dimiter_Popoff wrote:
> On 20.5.2015 г. 07:49, Don Y wrote:
>> On 5/19/2015 4:24 AM, Dimiter_Popoff wrote:
>>> On 19.5.2015 г. 10:10, Don Y wrote:
>>
>>>> I'm trying to sort out the best semantics for (file) locking
>>>> operations.
>>>
>>> I suppose I have been through all of this last 20+ years writing
>>> and "living" inside DPS and I think things are pretty straight
>>> forward.
>>>
>>> If it is a file, applications open it in a "registered" manner,
>>> i.e. each open file has only one IOCB (input/output control block).
>>> Tasks can "check in" to access it (there is a bitmap attached
>>> to the IOCB for that purpose).
>>> The IOCB can be "locked" by a task in which case the rest of the
>>> tasks just cannot access the file. Renaming a file will take
>>> locking the IOCB - having write access to this file - and doing
>>
>> So, how do you support two (or more) names pointing to the same file?
>
> I could easily do but I do not on purpose. Nothing is stopping me
> from having multiple directory entries pointing to the same piece
> of data or to some other directory entry which points to the data
> etc.

Yes.  But, now when you take a lock on the file's *contents*,
which file NAME is associated with that lock?  I.e., traditionally,
a name is treated as a separate entity from contents.

Most existing systems don't care *how* you got to the "contents"
because the lock applies to the *content*, not the "name" that
you resolved to access it.

When locking content, I work similarly:  an object O "physically"
residing on processor P can have the name N1 in task T1's address
space (on processor P1) and name N2 in task T2 on processor P2, etc.
When T1 takes the lock on "N1", it is dealing with the same
"physical" object that T2 may try to access as "N2".  And, none
of the tasks are aware that they are executing on different
physical processors (i.e., true parallelism) or that the object
they are interested in actually resides on yet another processor.

The problem I'm addressing is conflict resolution in the *namespaces*.
Note that T1 and T2 -- residing on different processors! -- may share
all, part or *none* of their respective namespaces.  Their namespaces
are just like any other object -- they can "physically" reside
anywhere in the system (though "locally" -- i.e., on P1 for T1 and
P2 for T2) obviously gives better performance if they are doing lots
of namespace operations!).

> However, this has implications - new directory entry type (so the
> data gets deallocated only for one of the directory entries) or just
> making a "link" type directory entry which may easily turn out
> to point to nothing (user deletes the file it points to). Solvable
> issues but adding complexity and no serious benefit.

For me, a namespace is just a set of ("name", handle) pairings.
E.g., "subdirectory" is a pairing of the text "subdirectory"
with a handle to the "directory object" that will contain more
such ("name", handle) pairings.  The "leaf's" are pairings in
which the handles point to non-directory objects.

The namespace code doesn't care about the types of these handles;
it just walks the chain stripping off each successive "subdirectory"
as it decends into the hierarchy and passes the remaining portion
of the "filename/pathname" to the object referenced by the portion
of the "name" that remains.  In this way, things *inside* an object
can be accessed.

For example:  when resolving ">don_private&devices+UART:baudrate"
(note the "separators aren't "reserved characters" -- they can be
anything that makes sense to the object in which they occur!),
the top level of the namespace is presented with this "string".
As it is typically a directory object, it looks through the list
of (name,handle) pairs that it currently contains and eventually
finds ">don" (but not ">do" or ">don_").  It takes the handle
associated with ">don" and passes "_private&devices+UART:baudrate"
to the resolver for that handle.

As this is probably also a directory object, it scans *its* list
of bindings and finds "_private".  Or, perhaps "_private&de" (!!!).
After stripping off the matched prefix, it again passes the remaining
string to the object referenced by *that* handle and the process
repeats.

In this example, lets assume we eventually get to a point where
we have resolved "...e" (remember, the name of the uart device can
be any contiguous substring up to and excluding the final character
in the "pathname", for this example:  "UART", "UART:", "+UART",
"ces+UART:", "evices+UART:baudrat", etc.  Of course, teh *logical*
name of the UART would be something like "UART" -- with everything
preceding it being part of the names of the objects in which it
nested!) to reference a handle that happens to represent a *uart*
object!

When we pass the remaining portion of the string (just like before)
to that (uart) object and ask for *it* to resolve "baudrate" (or
whatever portion of the pathname remains unresolved -- at least the
final "e"), then the uart object decides we are referencing the
component that (for example) governs baudrate.

The handle for *that* "baudrate object" then allows "baudrate
operations" to be performed on it.  E.g., things like:
     result_t set_baudrate(baudrate_handle h, baud_rate br);
     result_t get_baudrate(baudrate_handle h, baud_rate &br);
etc.

It wouldn't make sense for you to try to "fseek()" on a baudrate
object, for example!

>> E.g., I might have, on one of my UN*X boxes, 20 or 30 file *names*
>> (entries in a directory) that all reference the same "set of bytes"
>> (i.e., "program") on the disk.  This is a technique used in creating
>> a "crunch'ed" file -- a single executable that actually represents
>> many different programs mashed together.
>
> Under DPS this is solved in a much more general - and clean - way.
> Each piece of executable code is a "program module" which is accounted
> for by the OS (keeping track on where it came from into RAM) and,
> if reentrant/position independent (all I do are that) it is used by
> any task which wants to use it.
> Then each module can be of a "program library module" type containing
> multiple, 64 bit identifiable, entry points. DPS keeps track by
> whom a module is used so it can be discarded/deallocated when no
> longer needed etc. This mechanism is widely used by various layers
> (e.g. some "programs" call such module functions directly, whereas
> a DPS object can have some of its actions located in a .plm file
> (.plm being obviously "program libray module" :D ).

A library is a different entity.  It is designed to have multiple
"access points" but no real "entry point" ("shared libraries" are
a special case).  An executable file has *one* entry point.  The loader
copies it from disk into memory and then transfers control to that entry
point.

So, if you want a "program" to behave differently based on the name
by which it is invoked, that single entry point (that eventually brings
you to "main()") must examine the command line that caused the
loader to place it into memory:  "What was this called BY THE USER
to get it here?"

[These aren't really material to what follows; just trying to put UN*X in
its perspective]

>>>> How does process3 "lock" the name binding to that object -- at
>>>> least for the duration of this atomic operation it is attempting?
>>>
>>> The more generic naming - or whatever parameter of whatever
>>> object - I have been through (and still am when programming)
>>> is with the DPS runtime objects. You could do all you want to
>>> do with one of them, say "lock" for modification - save former
>>> lock state (in case your task is locking an object it has
>>> previously locked already) - modify - restore lock state.
>>> The number of "lock" bits you can implement is obviously
>>> object specific, you can grow them with new types as needed
>>> etc.
>>>
>>> I suppose what you are describing is still a bit too general.
>>> While it is crucial to get things right at the foundations
>>> I suppose your questions will get their answers automatically
>>> as you go :-). So I posted some of my experience, some phrase
>>> might click and be useful when you are doing this, which
>>> I suppose is what you hope for initiation the discussion.
>>
>> What I am concerned about is developerA deciding on a use
>> for some set of objects/"names" at some point in time.  He
>> implicitly expects his use of them (the names *and* the objects)
>> to be "exclusive" ("Why would anyone ever want to use this
>> mumbledypeg object, 'foo'?  It shouldn't be of value to anyone
>> OTHER THAN me!").
>
> Ah I see now. I suppose in my vocabulary this would be "name
> reservation". Can you not simply prevent the developer from
> using names which have already been used? Like in any language,
> "duplicate symbol definition" etc. sort of thing error if he
> tries? May be I am still not getting what you are after.

You write a program that will, eventually, create a *named*
object and pass the *name* of that object on to some other
program (so that other program can reference that object
through the namespace instead of via a "bare handle"; for
example, if the other program is invoked some time LONG
*after* the original program has terminated!).

What if some other program (sharing that same namespace)
creates a named object having that same name *before*
your program has a chance to create it's object having
that particular name?  (remember, these are just objects;
the first example program may think it is creating a timer
called "deadline" -- while the *third* thinks it is
creating a "handler"/executable object that will be
invoked when some deadline is exceeded).

Or, the first program creates the "deadline" TIMER as
planned.  The other (third) program comes along and
tries to create an object having that same name.  But,
the name is already "in use".  Does it unlink the name
("delete") so that it can safely create it's "deadline
handler" and bind the handler to that name?  (what happens
when that *second* program is invoked and it goes looking
for the "deadline TIMER" -- which no longer exists even
though "deadline" still resolves to a live object!)
Or, does it abend claiming "name exists"?

Etc.

All of these "problems" go away with excellent documentation
and *responsible* developers.  But, that's true of *all* bugs!
Yet, we still encounter them!  :-/

What I'm trying to do is put in place mechanisms whereby the
developer can set up "contracts" through the RTOS that allow
his/her "programs" to operate reliably in light of past AND
future components in the system -- without having to research
every possible contingency (which makes the system increasingly
difficult to "scale")

A name is a resource just like space on a disk, bytes in memory
or cycles on a processor.  Ideally, I want a developer to "formally"
lay out the requirements/assumptions that his code RELIES UPON
and then have the RTOS ensure those guarantees are met.

So, if some *other* "program" starts to run, *when* it similarly
lays out it's requirements, it is told of conflicts that exist
IN THE CURRENT ENVIRONMENT and *it* can address them.  Much like
getting NULL returned by malloc() -- you can't assume that because
you *want* something you will *get* it.  Figure out how to recover
(e.g., decide to use a different name if the name you *want* is
already "reserved" -- and, include a mechanism by which you can
convey to that "other/second" program your choice of names!)

Have to get started on another cheesecake...

Hi Don,


On Wed, 20 May 2015 12:03:17 -0700, Don Y <this@is.not.me.com> wrote:

>The problem I'm addressing is conflict resolution in the *namespaces*.
>Note that T1 and T2 -- residing on different processors! -- may share
>all, part or *none* of their respective namespaces.  Their namespaces
>are just like any other object -- they can "physically" reside
>anywhere in the system (though "locally" -- i.e., on P1 for T1 and
>P2 for T2) obviously gives better performance if they are doing lots
>of namespace operations!).

You can't resolve multiple name conflicts without a global protocol.
However, you still haven't provided any convincing reason to be
worried about it.

If what you want to do is make the object and its name appear to be
one and the same, you have to transitively lock both before allowing
access to either.


>> However, this has implications - new directory entry type (so the
>> data gets deallocated only for one of the directory entries) or just
>> making a "link" type directory entry which may easily turn out
>> to point to nothing (user deletes the file it points to). Solvable
>> issues but adding complexity and no serious benefit.

That's why Unix indirects through the inode - the complexity is
necessary to correct operation.  [Yes, there are odd cases where Unix
has problems, but it got right virtually all of the common uses.  And
most of the real problems are caching/timing issues that have no good
solution.]


> :
>It wouldn't make sense for you to try to "fseek()" on a baudrate
>object, for example!

Why?  Consider it as seeking to a new frequency. 

Almost any function can be shoehorned into Unix's read/write/ioctl
interface ... believing otherwise indicates a lack of imagination 8-)


Both object based and function based APIs have their place. Too many
people have drunk the KoolAid and think objects always are the right
way to go.



>What if some other program (sharing that same namespace)
>creates a named object having that same name *before*
>your program has a chance to create it's object having
>that particular name?  

You can mitigate that slightly by supporting multiple versions if
there is something to distinguish them (object type?) but there is no
way to entirely prevent it.

The paired programs require a protocol for dealing with disruption to
their name scheme.  That's what UUIDs are supposed to handle.  UUIDs
don't entirely prevent collisions, but they are quite rare.  No number
of name bits will entirely prevent collisions, but you can make
collisions so rare as to not bother worrying about them


>What I'm trying to do is put in place mechanisms whereby the
>developer can set up "contracts" through the RTOS that allow
>his/her "programs" to operate reliably in light of past AND
>future components in the system -- without having to research
>every possible contingency (which makes the system increasingly
>difficult to "scale")

A program may contract with a particular service, but it must rely
transitively on further (sub)contracts which that service has with
others and over which the program has no control.

There is no satisfactory answer to this.


>A name is a resource just like space on a disk, bytes in memory
>or cycles on a processor.  Ideally, I want a developer to "formally"
>lay out the requirements/assumptions that his code RELIES UPON
>and then have the RTOS ensure those guarantees are met.
>
>So, if some *other* "program" starts to run, *when* it similarly
>lays out it's requirements, it is told of conflicts that exist
>IN THE CURRENT ENVIRONMENT and *it* can address them. 

That depends on the complexity of the dependency web:  the probability
of satisfying simultaneous dependencies multiplies both horizontally
in the number of dependencies and vertically in their depth.

If you're willing to wait, you can turn to transaction and queue
theory to estimate wait times.  However, the estimates may be too
broad to be worthwhile.


George

Hi George,

On 5/21/2015 4:11 AM, George Neuner wrote:
> On Wed, 20 May 2015 12:03:17 -0700, Don Y <this@is.not.me.com> wrote:
>
>> The problem I'm addressing is conflict resolution in the *namespaces*.
>> Note that T1 and T2 -- residing on different processors! -- may share
>> all, part or *none* of their respective namespaces.  Their namespaces
>> are just like any other object -- they can "physically" reside
>> anywhere in the system (though "locally" -- i.e., on P1 for T1 and
>> P2 for T2) obviously gives better performance if they are doing lots
>> of namespace operations!).
>
> You can't resolve multiple name conflicts without a global protocol.

Namespaces are self-contained objects -- administered by whichever
namespace server happens to be holding the object at the time.
If we limit the discussion (complexity) to just "directory objects"
(in which create/delete/rename options make obvious sense), then
N different tasks may have a particular directory object in their
shared namespace (or, some part thereof).  There may be cached
copies of that *single* directory object in various places throughout
the system.  *But*, operations that modify it (instead of just lookups
which are, by far, the most common operation) are forced to happen in
one copy at a time -- those changes then automagically propagated to
any other instances (DSM).

So, whichever server holds the write-lock on that object knows that
its copy of the cache is "gospel" (for the time being).

That server may (conceptually) be handling multiple requests on
multiple directory objects -- including multiple requests on this
particular object (multithreaded so actions for one client don't
tie up the service while those are resolved... like waiting for
an object lock, DSM update, etc).  But, the server wishing to
update that "directory object" (to add/remove/rename an entry)
has to take the (local) lock *for* that object -- which means
also having the (DSM) write lock -- in order to make that change.

So, any other threads in that server instance inherently block
awaiting access to that particular directory object *if* they
similarly want to modify that object's contents.  When they return,
the first thread will already have made its changes; any actions
that any other threads want to make will have to consider those
changes in their service:  "Hmmm... the name that I've been asked
to delete doesn't exist (anymore).  I'll have to return an error to
*my* client!"

> However, you still haven't provided any convincing reason to be
> worried about it.

Names are a limited resource.  Just like memory, CPU cycles, etc.
Names *typically* have significance -- we don't allow "The System"
to make up names for us:  "What's the 'console' called, today??"
That would just add another layer of indirection to the naming
system!

Imagine two processes use a named pipe to exchange information.
But, before they get to the point where they *need* to access
that pipe (e.g., perhaps before either has been *invoked*),
someone else creates an object (maybe a plain file, maybe a
"device", etc. [UNIX-speak]) having that same name.  Now,
when (either of) the aforementioned processes starts up and
tries to create that pipe, the name is already in use!

If those processes were written "early on" (when the namespace was
sparse and/or relatively unshared), the developers probably
didn't consider that possibility (shame on them!).  So, when
someone *diligently* (and corectly) uses that name at some future
date, the original use ends up manifesting as a run-time bug.

> If what you want to do is make the object and its name appear to be
> one and the same, you have to transitively lock both before allowing
> access to either.

No.  All I want to do is be able to place "reservations" on names
in much the same way that I can place reservations on memory, CPU time,
etc.  So, at the time you place the reservation, you *know* that it
will (eventually) be satisfied -- or not.  And, because "not" is
a possibility, you explicitly address how you will handle that case
*in* your coding.

>>> However, this has implications - new directory entry type (so the
>>> data gets deallocated only for one of the directory entries) or just
>>> making a "link" type directory entry which may easily turn out
>>> to point to nothing (user deletes the file it points to). Solvable
>>> issues but adding complexity and no serious benefit.
>
> That's why Unix indirects through the inode - the complexity is
> necessary to correct operation.  [Yes, there are odd cases where Unix
> has problems, but it got right virtually all of the common uses.  And
> most of the real problems are caching/timing issues that have no good
> solution.]
>
>
>> :
>> It wouldn't make sense for you to try to "fseek()" on a baudrate
>> object, for example!
>
> Why?  Consider it as seeking to a new frequency.
>
> Almost any function can be shoehorned into Unix's read/write/ioctl
> interface ... believing otherwise indicates a lack of imagination 8-)

<grin>  Forcing the file-paradigm on all "objects" similarly shows
lack of imagination.  This is why systems that do so have to force
"commands" (functions) to be "in-band": fwrite(uart, "set parity odd"...)

> Both object based and function based APIs have their place. Too many
> people have drunk the KoolAid and think objects always are the right
> way to go.

"Object" only is of significance to me as everything is RPC-based.
So, you need some way of indicating the "thing" (object) on which you
are operating.  Almost all of my syntax is function based simply
because the "object reference" (handle?) isn't something that you
can resolve with a traditional object syntax!

>> What if some other program (sharing that same namespace)
>> creates a named object having that same name *before*
>> your program has a chance to create it's object having
>> that particular name?
>
> You can mitigate that slightly by supporting multiple versions if
> there is something to distinguish them (object type?) but there is no
> way to entirely prevent it.

If you can place "reservations" (reserves) when a module is *installed*
(i.e., before it even executes), then installing a *new* module would
make the conflict apparent *then* -- instead of at some later "run-time".

> The paired programs require a protocol for dealing with disruption to
> their name scheme.  That's what UUIDs are supposed to handle.  UUIDs
> don't entirely prevent collisions, but they are quite rare.  No number
> of name bits will entirely prevent collisions, but you can make
> collisions so rare as to not bother worrying about them

I think just reserves gives you the same capability.  If a specific
name *is* required (because it is used to interact with <something>...
including the user!), that can be specified in the "IDL" for the
module.  When the module is installed into the system -- with its
companion(s) -- that shared requirement can be enforced at install
time (i.e., if the reservation can't be met, the "installer" can
rescript all components that rely on that reserve:  "OK, guys...
the 'console' will be called 'disc'!").

Having been successfully installed, this (e.g., "disc") reservation
now further constrains *future* installs ("Hmmm... 'disc' is in
use; let's call the disk object 'bob'!  This works because
installs are serial (and seldom).

>> What I'm trying to do is put in place mechanisms whereby the
>> developer can set up "contracts" through the RTOS that allow
>> his/her "programs" to operate reliably in light of past AND
>> future components in the system -- without having to research
>> every possible contingency (which makes the system increasingly
>> difficult to "scale")
>
> A program may contract with a particular service, but it must rely
> transitively on further (sub)contracts which that service has with
> others and over which the program has no control.
>
> There is no satisfactory answer to this.

See above.  Note how I deliberately chose name conflicts that
are bizarre.  The issue is "coercing" the developer to address
this problem instead of just BREAKING and claiming the problem
lies with software from some other "vendor" -- leaving the user
in the lurch.

I can currently handle this by creating private portions of
namespaces and advertising those to "interested parties" at
run time.  But, that just reduces the *number* of potential
conflicts (because you can bundle all the "names of interest"
into that single ISOLATED namespace) but you still need a
"well known name" that you can agree upon a priori for
the "advertisement".

E.g., like "\Program Files\<vendor name>" relying on the assumption
that "Hughe's Paradise" won't opt for the <vendor name> of "HP".

>> A name is a resource just like space on a disk, bytes in memory
>> or cycles on a processor.  Ideally, I want a developer to "formally"
>> lay out the requirements/assumptions that his code RELIES UPON
>> and then have the RTOS ensure those guarantees are met.
>>
>> So, if some *other* "program" starts to run, *when* it similarly
>> lays out it's requirements, it is told of conflicts that exist
>> IN THE CURRENT ENVIRONMENT and *it* can address them.
>
> That depends on the complexity of the dependency web:  the probability
> of satisfying simultaneous dependencies multiplies both horizontally
> in the number of dependencies and vertically in their depth.
>
> If you're willing to wait, you can turn to transaction and queue
> theory to estimate wait times.  However, the estimates may be too
> broad to be worthwhile.

If you can resolve those "conflicts" at install time, there is no
wait *in* the system:  it effectively looks like you bought/installed
the modules a millisecond -- or *day*! -- later than you actually did!
(this is a dodge but an effective one as it removes the RT component
from the action)

I am expecting such conflicts to be incredibly rare.  It's too easy
to create a new namespace and very little incentive to *share*
namespaces, broadly (except for "system objects" where a "well known
name" is inherent).

Because these conflicts are expected to be rare, I expect developers to
be lazy and *not* check for them as they should in their "best practices".
Just like developers expecting malloc() to always succeed!  For an even
more prominent example of how much developers take for granted, when
was the last time you saw the return value of printf() examined??  ;)

Hope you're making progress!  Last of the "cheesecake triad" done, this
morning.  Hopefully a break in the baking, for a while -- short as it
may be!  :<

--don

On 21.5.2015 г. 14:11, George Neuner wrote:
>
>>> However, this has implications - new directory entry type (so the
>>> data gets deallocated only for one of the directory entries) or just
>>> making a "link" type directory entry which may easily turn out
>>> to point to nothing (user deletes the file it points to). Solvable
>>> issues but adding complexity and no serious benefit.
>
> That's why Unix indirects through the inode - the complexity is
> necessary to correct operation.

Yes, like I said solvable issues - but the added expense just does not
justify the benefit this give you. The inode must link back to all
directory entries pointing to it for this to work - they probably do
that - which has further implications, e.g. do they store the path
name as text - this must then be resolved for every access - or do
they store medium specific (LBN etc.) data - this will take special
processing when copying (my guess is they have opted for the first).

In DPS the inode is called RIB (.... this is what it was called on the
first OS I had contact with - MDOS, Motorolas OS on their Exorciser,
Retrieve Information Block - so I just reused a name I remembered well).
It does not link back to the directory entry - and for files split
in up to 2 pieces it can be unused, thus saving one disk access per
file open - but I could easily expand that if I wanted to.
For an application it would be trivial which name was used so it
was invoked as iocb-s contain the directory entry name.
But I can't see how this buys me anything of real value.

> [Yes, there are odd cases where Unix
> has problems, but it got right virtually all of the common uses.  And
> most of the real problems are caching/timing issues that have no good
> solution.]

I agree, my contacts with unix have been only as a layman user trying
to do something but I never encountered any issue with the filesystem,
apart from the big one - they store names as bytes and expect the
user to convert them to text (i.e. name search is case dependent only).

Dimiter

------------------------------------------------------
Dimiter Popoff, TGI             http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/

On Thu, 21 May 2015 06:44:20 -0700, Don Y <this@is.not.me.com> wrote:

>Hi George,
>
>On 5/21/2015 4:11 AM, George Neuner wrote:
>> On Wed, 20 May 2015 12:03:17 -0700, Don Y <this@is.not.me.com> wrote:
>>
>>> The problem I'm addressing is conflict resolution in the *namespaces*.
>>> Note that T1 and T2 -- residing on different processors! -- may share
>>> all, part or *none* of their respective namespaces.  Their namespaces
>>> are just like any other object -- they can "physically" reside
>>> anywhere in the system (though "locally" -- i.e., on P1 for T1 and
>>> P2 for T2) obviously gives better performance if they are doing lots
>>> of namespace operations!).
>>
>> You can't resolve multiple name conflicts without a global protocol.
>
>Namespaces are self-contained objects -- administered by whichever
>namespace server happens to be holding the object at the time.
>If we limit the discussion (complexity) to just "directory objects"
>(in which create/delete/rename options make obvious sense), then
>N different tasks may have a particular directory object in their
>shared namespace (or, some part thereof).  There may be cached
>copies of that *single* directory object in various places throughout
>the system.  *But*, operations that modify it (instead of just lookups
>which are, by far, the most common operation) are forced to happen in
>one copy at a time -- those changes then automagically propagated to
>any other instances (DSM).
>
>So, whichever server holds the write-lock on that object knows that
>its copy of the cache is "gospel" (for the time being).

If you have a proper coherence protocol, then you can't have a
conflict unless the network is partitioned.  And if there is a
partition, you are screwed because "eventual" consistency doesn't work
unless you either can version or afford to lose updates.



>> However, you still haven't provided any convincing reason to be
>> worried about it.
>
>Names are a limited resource.  Just like memory, CPU cycles, etc.
>Names *typically* have significance -- we don't allow "The System"
>to make up names for us:  "What's the 'console' called, today??"
>That would just add another layer of indirection to the naming
>system!

You have to indirect through a global shared namespace: i.e. "console"
is a local alias for "231cacd6-0074-11e5-a322-1697f925ec7b", which
never changes.

Want to create a new file called "foo"?  Fine!  Call it whatever you
want but the system calls it "231cb834-0074-11e5-a322-1697f925ec7b".
It's up to you to keep the name mapping and, if need be, to transmit
the mapping to someone else.


>Imagine two processes use a named pipe to exchange information.
>But, before they get to the point where they *need* to access
>that pipe (e.g., perhaps before either has been *invoked*),
>someone else creates an object (maybe a plain file, maybe a
>"device", etc. [UNIX-speak]) having that same name.  Now,
>when (either of) the aforementioned processes starts up and
>tries to create that pipe, the name is already in use!

Same answer as above.  

For mail drop semantics, both sides have to agree on a suitably unique
name.  For rendezvous semantics, it suffices to agree on a unique
service key to create the object on demand when both processes are
ready.  

A unique name generally must be longer (more bits) than a unique key
because it usually needs long duration persistence.  A key OTOH needs
to be unique only when it's actually used - which may be both
predictable and short duration.  



>All I want to do is be able to place "reservations" on names
>in much the same way that I can place reservations on memory, CPU time,
>etc.  So, at the time you place the reservation, you *know* that it
>will (eventually) be satisfied -- or not.  And, because "not" is
>a possibility, you explicitly address how you will handle that case
>*in* your coding.
>
>If you can place "reservations" (reserves) when a module is *installed*
>(i.e., before it even executes), then installing a *new* module would
>make the conflict apparent *then* -- instead of at some later "run-time".

That may be inconveniently late: the module may have to be recompiled
if the conflict isn't resolvable by configuration.  What about cross
development without access to the target system?  What if a user
purchases a binary module that can't be installed due to a conflict?
What if there are several such modules because short sighted
developers picked the same names?

The time to reserve the name is when the module is compiled.


>> The paired programs require a protocol for dealing with disruption to
>> their name scheme.  That's what UUIDs are supposed to handle.  UUIDs
>> don't entirely prevent collisions, but they are quite rare.  No number
>> of name bits will entirely prevent collisions, but you can make
>> collisions so rare as to not bother worrying about them
>
>I think just reserves gives you the same capability.  If a specific
>name *is* required (because it is used to interact with <something>...
>including the user!), that can be specified in the "IDL" for the
>module.  When the module is installed into the system -- with its
>companion(s) -- that shared requirement can be enforced at install
>time (i.e., if the reservation can't be met, the "installer" can
>rescript all components that rely on that reserve:  "OK, guys...
>the 'console' will be called 'disc'!").

Reservation works in situ if the mappings are persistent, but it
doesn't solve the cross development problem.



>Having been successfully installed, this (e.g., "disc") reservation
>now further constrains *future* installs ("Hmmm... 'disc' is in
>use; let's call the disk object 'bob'!  This works because
>installs are serial (and seldom).

Maybe it works.  Maybe it doesn't.



>I am expecting such conflicts to be incredibly rare.  It's too easy
>to create a new namespace and very little incentive to *share*
>namespaces, broadly (except for "system objects" where a "well known
>name" is inherent).
>
>Because these conflicts are expected to be rare, I expect developers to
>be lazy and *not* check for them as they should in their "best practices".


You can force them to be incredibly rare regardless of what the
programmer does (or does not).


>when was the last time you saw the return value of printf() examined?? 

When was the last time you knew a priori what value to expect?

I used to do a lot of multiple sprintf into a fixed length buffer, so
I did check and use it to position the "cursor" for the next write.
But I haven't done much in C recently.



>Hope you're making progress! 

Somewhat.  Working on several things at once.

Anything simple loses the flavor.

I'm currently thinking the best approach is to compute Wilson mean for
each variable - substituting range median for nulls - and then treat
the ordered values as vector components of a polyline.

From there are a number interesting comparisons possible, but I think
the most meaningful is the ratio of the volume of 2 N-balls centered
on the point where all components are minimized: ball with "radius"
distance to endpoint of the polyline vs ball with "radius" the
distance to the point where all components are maximized.

Dunno. Noodles are still wet. 


>--don

George

On Fri, 22 May 2015 14:57:24 +0300, Dimiter_Popoff <dp@tgi-sci.com>
wrote:

>On 21.5.2015 ?. 14:11, George Neuner wrote:
>>
>>>> However, this has implications - new directory entry type (so the
>>>> data gets deallocated only for one of the directory entries) or just
>>>> making a "link" type directory entry which may easily turn out
>>>> to point to nothing (user deletes the file it points to). Solvable
>>>> issues but adding complexity and no serious benefit.
>>
>> That's why Unix indirects through the inode - the complexity is
>> necessary to correct operation.
>
>Yes, like I said solvable issues - but the added expense just does not
>justify the benefit this give you. The inode must link back to all
>directory entries pointing to it for this to work - they probably do
>that - which has further implications, e.g. do they store the path
>name as text - this must then be resolved for every access - or do
>they store medium specific (LBN etc.) data - this will take special
>processing when copying (my guess is they have opted for the first).

No.  Links are one way: from directory entries to the inode.  The
inode is reference counted and persists as long as there is at least
one directory entry which references it.  

The inode maintains the structure and security information for the
file.  Directory entries are just a name and an inode reference.

Inodes for open files are cached in memory and updates are lazily
written back to disk [unless you deliberately (f)sync].  There is a
lock on the in-memory cached inode, but it is used only for updates to
the inode itself to coordinate sync flushes.  File content locks are
handled separately through a filesystem service - content locks are
neither in nor on the inode.


>In DPS the inode is called RIB (.... this is what it was called on the
>first OS I had contact with - MDOS, Motorolas OS on their Exorciser,
>Retrieve Information Block - so I just reused a name I remembered well).
>It does not link back to the directory entry - and for files split
>in up to 2 pieces it can be unused, thus saving one disk access per
>file open - but I could easily expand that if I wanted to.
>For an application it would be trivial which name was used so it
>was invoked as iocb-s contain the directory entry name.
>But I can't see how this buys me anything of real value.

In Unix the inode directly links to some small number of data blocks
before indirecting through index blocks.  The size of a file that can
be addressed directly depends on the logical block size of the media,
but it typically is 40-64KB for a desktop filesystem and may be
megabytes in a server filesystem.

George

Hi George,

On 5/22/2015 6:57 AM, George Neuner wrote:
> On Thu, 21 May 2015 06:44:20 -0700, Don Y <this@is.not.me.com> wrote:
>> On 5/21/2015 4:11 AM, George Neuner wrote:
>>> On Wed, 20 May 2015 12:03:17 -0700, Don Y <this@is.not.me.com> wrote:
>>>
>>>> The problem I'm addressing is conflict resolution in the *namespaces*.
>>>> Note that T1 and T2 -- residing on different processors! -- may share
>>>> all, part or *none* of their respective namespaces.  Their namespaces
>>>> are just like any other object -- they can "physically" reside
>>>> anywhere in the system (though "locally" -- i.e., on P1 for T1 and
>>>> P2 for T2) obviously gives better performance if they are doing lots
>>>> of namespace operations!).
>>>
>>> You can't resolve multiple name conflicts without a global protocol.
>>
>> Namespaces are self-contained objects -- administered by whichever
>> namespace server happens to be holding the object at the time.
>> If we limit the discussion (complexity) to just "directory objects"
>> (in which create/delete/rename options make obvious sense), then
>> N different tasks may have a particular directory object in their
>> shared namespace (or, some part thereof).  There may be cached
>> copies of that *single* directory object in various places throughout
>> the system.  *But*, operations that modify it (instead of just lookups
>> which are, by far, the most common operation) are forced to happen in
>> one copy at a time -- those changes then automagically propagated to
>> any other instances (DSM).
>>
>> So, whichever server holds the write-lock on that object knows that
>> its copy of the cache is "gospel" (for the time being).
>
> If you have a proper coherence protocol, then you can't have a
> conflict unless the network is partitioned.  And if there is a
> partition, you are screwed because "eventual" consistency doesn't work
> unless you either can version or afford to lose updates.

Yes.  The shared memory protocol is reliable (though expensive if
you're doing lots of writes; the premis is that *shared* namespaces
will be rare(r) -- especially those that are cached in more than one
place.  I.e., caching expedites lookups at the expense of slowing
down *updates*.

>>> However, you still haven't provided any convincing reason to be
>>> worried about it.
>>
>> Names are a limited resource.  Just like memory, CPU cycles, etc.
>> Names *typically* have significance -- we don't allow "The System"
>> to make up names for us:  "What's the 'console' called, today??"
>> That would just add another layer of indirection to the naming
>> system!
>
> You have to indirect through a global shared namespace: i.e. "console"
> is a local alias for "231cacd6-0074-11e5-a322-1697f925ec7b", which
> never changes.

The *actual* handle can vary from node to node, task to task and
run to run.  The "system wide" objects are created at system start-up
as the initial "system" namespace is built.  Only "init" sees this
namespace; things that it spawns is given portions of the namespace
as appropriate to their respective needs.  And, of course, individual
nodes can create their own "dynamic" objects naming them as they
wish and *exporting* references to them by various means (injecting
specific names into other tasks' namespaces, passing "live handles"
to other tasks, etc.).

The goal is to restrict access to objects to only those entities that
(should) need that access.

> Want to create a new file called "foo"?  Fine!  Call it whatever you
> want but the system calls it "231cb834-0074-11e5-a322-1697f925ec7b".
> It's up to you to keep the name mapping and, if need be, to transmit
> the mapping to someone else.

If the object doesn't exist as part of the "system namespace", then
there is no need for a handle, there.  E.g., unlike the single
filesystem hierarchy in most machines, it is possible for a completely
*isolated* namespace to exist in my system -- where only those
entities having access to that namespace are even aware that the
objects named therein exist.

>> Imagine two processes use a named pipe to exchange information.
>> But, before they get to the point where they *need* to access
>> that pipe (e.g., perhaps before either has been *invoked*),
>> someone else creates an object (maybe a plain file, maybe a
>> "device", etc. [UNIX-speak]) having that same name.  Now,
>> when (either of) the aforementioned processes starts up and
>> tries to create that pipe, the name is already in use!
>
> Same answer as above.
>
> For mail drop semantics, both sides have to agree on a suitably unique
> name.  For rendezvous semantics, it suffices to agree on a unique
> service key to create the object on demand when both processes are
> ready.

But, in each case, there must be some guarantee that the agreed upon
"unique name" or "unique service key" are available for use by
those interested parties!  E.g., we can agree to use the name (key)
"fred".  But, if someone else INDEPENDENTLY decides to use "fred"
(for some purpose), things break.

E.g., it would be a sign of na&#4294967295;vet&#4294967295; to create "/tmp/log".  (actually,
you could probably get away with this -- because no one *else* would
be na&#4294967295;ve enough to try it!  :)

> A unique name generally must be longer (more bits) than a unique key
> because it usually needs long duration persistence.  A key OTOH needs
> to be unique only when it's actually used - which may be both
> predictable and short duration.

But, "in general", you can neither predict when nor how long a name
or key will be used.  It requires detailed knowledge of the application
and potential competitors.

>> All I want to do is be able to place "reservations" on names
>> in much the same way that I can place reservations on memory, CPU time,
>> etc.  So, at the time you place the reservation, you *know* that it
>> will (eventually) be satisfied -- or not.  And, because "not" is
>> a possibility, you explicitly address how you will handle that case
>> *in* your coding.
>>
>> If you can place "reservations" (reserves) when a module is *installed*
>> (i.e., before it even executes), then installing a *new* module would
>> make the conflict apparent *then* -- instead of at some later "run-time".
>
> That may be inconveniently late: the module may have to be recompiled
> if the conflict isn't resolvable by configuration.  What about cross
> development without access to the target system?  What if a user
> purchases a binary module that can't be installed due to a conflict?
> What if there are several such modules because short sighted
> developers picked the same names?

That's exactly the problem!  If a developer unilaterally picks a
name and *expects* it to be available for his/her use (without
taking measures to adapt when/if it proves not to be), then it
is "broken".

But, this is apparent when the application (name consumer) tries to
install the module -- not at some "random" later time when the
perfect storm of conflicts *happens* to occur (because the module(s)
with which the name conflicts *happen* to be "live" when the later
introduced module is also live).

Would you rather the module be installable -- leading to the
appearance that it *will* work (in all future instantiations
of the EXISTING modules) -- only to belatedly discover that
<something> (which may not be the module in question) STOPS
working?

[i.e., if it was OK for module #25 to unilaterally pick a name
that conflicts with one used by module #3, then it's just as
acceptable for #3 to expect it's use of the name to be OK.
So, if #25 runs before #3 -- even though #3 was purchased
and installed LONG before #25 -- then #3 looks to be "broken"
when, in fact, the introduction of #25 is the real source of
the problem]

> The time to reserve the name is when the module is compiled.

That requires everyone to cooperate on name choices.  Or,
everything to have independent namespaces (\Program Files\HP)
to make this possible.  Entities then can't effectively
share things.

>>> The paired programs require a protocol for dealing with disruption to
>>> their name scheme.  That's what UUIDs are supposed to handle.  UUIDs
>>> don't entirely prevent collisions, but they are quite rare.  No number
>>> of name bits will entirely prevent collisions, but you can make
>>> collisions so rare as to not bother worrying about them
>>
>> I think just reserves gives you the same capability.  If a specific
>> name *is* required (because it is used to interact with <something>...
>> including the user!), that can be specified in the "IDL" for the
>> module.  When the module is installed into the system -- with its
>> companion(s) -- that shared requirement can be enforced at install
>> time (i.e., if the reservation can't be met, the "installer" can
>> rescript all components that rely on that reserve:  "OK, guys...
>> the 'console' will be called 'disc'!").
>
> Reservation works in situ if the mappings are persistent, but it
> doesn't solve the cross development problem.

By putting the reservations in the IDL, I am hoping to force the
conflict issue to be more visible.  "Hello, Support Desk?  I
just purchased your ABC module and it won't install:  'name
conflict'..."

Like someone deciding to name their program "/kernel" and
*expecting* that name in their code!

>> Having been successfully installed, this (e.g., "disc") reservation
>> now further constrains *future* installs ("Hmmm... 'disc' is in
>> use; let's call the disk object 'bob'!  This works because
>> installs are serial (and seldom).
>
> Maybe it works.  Maybe it doesn't.

If it doesn't, it is because the developer hasn't played by the
rules.  E.g., you can unilaterally pick "/kernel" as your program name
on a NetBSD system.  (You can *possibly* also pick "/netbsd" -- the
actual kernel name -- as I don't think the kernel name is hardcoded into
any of the system utilities; even if so, there would usually be a
command line argument to allow those utilities to fetch names from a
different kernel image)

It's akin to a program *expecting* to be able to malloc() N bytes
of memory, regardless of what else may be running in the system
at any given time and, therefore, not *checking* to see if the
allocation was successful.  Then, crashing.

If, instead, that application was *guaranteed* M bytes of free memory
regardless of other modules installed in the system, then as long
as N<=M, the allocation *should* succeed (because all other modules
honor their contracts as well)

>> I am expecting such conflicts to be incredibly rare.  It's too easy
>> to create a new namespace and very little incentive to *share*
>> namespaces, broadly (except for "system objects" where a "well known
>> name" is inherent).
>>
>> Because these conflicts are expected to be rare, I expect developers to
>> be lazy and *not* check for them as they should in their "best practices".
>
> You can force them to be incredibly rare regardless of what the
> programmer does (or does not).
>
>> when was the last time you saw the return value of printf() examined??
>
> When was the last time you knew a priori what value to expect?

I always expect the return value to be non-negative!  Yet, how often do you
see:
     if (0 > printf(...)) {
        /* Oh, crap! */
     }

> I used to do a lot of multiple sprintf into a fixed length buffer, so
> I did check and use it to position the "cursor" for the next write.
> But I haven't done much in C recently.

I use it a lot in character based displays -- similar reason.

But, you still rarely (ever?) see testing for the *FAIL* case (above).

>> Hope you're making progress!
>
> Somewhat.  Working on several things at once.

<grin>  Join the club!

> Anything simple loses the flavor.
>
> I'm currently thinking the best approach is to compute Wilson mean for
> each variable - substituting range median for nulls - and then treat
> the ordered values as vector components of a polyline.

I think your biggest exposure will be if the "numbers" don't get/stay
large.  I.e., the "fad factor" (but, I can't see how you can work
around this -- you need some meat in order to make sense of anything).
At some point, even "incentives" lose their appeal.

>  From there are a number interesting comparisons possible, but I think
> the most meaningful is the ratio of the volume of 2 N-balls centered
> on the point where all components are minimized: ball with "radius"
> distance to endpoint of the polyline vs ball with "radius" the
> distance to the point where all components are maximized.

I'd have to think about whether volume or radius would be the better
comparative metric.  (too early in the day to do stuff like that!)

> Dunno. Noodles are still wet.

Throw against wall.  If they *stick*, consider them done!  :>
(but don't eat those samples, regardless!)

I think I'm supposed to be going to a party tonight.  I should
probably find some pants...  (or, start feigning an exotic illness!)

Hi Dimiter,

On 5/22/2015 4:57 AM, Dimiter_Popoff wrote:
> On 21.5.2015 г. 14:11, George Neuner wrote:
>>
>>>> However, this has implications - new directory entry type (so the
>>>> data gets deallocated only for one of the directory entries) or just
>>>> making a "link" type directory entry which may easily turn out
>>>> to point to nothing (user deletes the file it points to). Solvable
>>>> issues but adding complexity and no serious benefit.
>>
>> That's why Unix indirects through the inode - the complexity is
>> necessary to correct operation.
>
> Yes, like I said solvable issues - but the added expense just does not
> justify the benefit this give you. The inode must link back to all
> directory entries pointing to it for this to work - they probably do
> that - which has further implications, e.g. do they store the path
> name as text - this must then be resolved for every access - or do
> they store medium specific (LBN etc.) data - this will take special
> processing when copying (my guess is they have opted for the first).

No.  In UN*X (and in my scheme), names aren't intimately tied to
"files" (if you use the term 'file' to indicate the actual contents).
A "directory object" maps "names" to "files".  It's a pointer-style
reference:  you can have multiple pointers referencing the same
"thing" (just like you can store the address of a particular thing
in several different places!).

The (traditional) "lock" is applied to the "thing" (file contents),
not to the pointer through which it was accessed.  Otherwise, the
lock wouldn't work:  you could get around it and access the object
through a different pointer!

A *file* doesn't know how many references "point" to it.  Nor
which reference was used to access it in this instance!

[My scheme doesn't suffer from this limitation.  An object (by that,
I mean, "the server that implements an object") knows how many
references/handles exist for it at any given time because these are
more than just simple "pointers".  Also, an object knows how (and
who) it is being accessed because it can see which handle is being
used in that access.]

> In DPS the inode is called RIB (.... this is what it was called on the
> first OS I had contact with - MDOS, Motorolas OS on their Exorciser,
> Retrieve Information Block - so I just reused a name I remembered well).
> It does not link back to the directory entry - and for files split
> in up to 2 pieces it can be unused, thus saving one disk access per
> file open - but I could easily expand that if I wanted to.
> For an application it would be trivial which name was used so it
> was invoked as iocb-s contain the directory entry name.
> But I can't see how this buys me anything of real value.
>
>> [Yes, there are odd cases where Unix
>> has problems, but it got right virtually all of the common uses.  And
>> most of the real problems are caching/timing issues that have no good
>> solution.]
>
> I agree, my contacts with unix have been only as a layman user trying
> to do something but I never encountered any issue with the filesystem,
> apart from the big one - they store names as bytes and expect the
> user to convert them to text (i.e. name search is case dependent only).

In my case, names apply to *all* objects:  timers, devices, tasks,
nodes, processes, etc.  (though there are anonymous objects).  This
allows objects to be referenced by other objects, manipulated,
destroyed, etc.

E.g., I can build a "memory object" and, optionally, give it a name.
This would allow me to share/reference that object (read it, write it,
grow it, delete it, move it, etc.) within the system.

Further, I can fill it with sets of (name, handle) pairs.  I can then
call this a "directory object" and, optionally, name it.  It can now
be referenced by any task on any node in the system -- *if* a handle
to it is made available to that other task, etc.

Handles contain access permissions.  So, I can create *one* handle
for it that gives the "holder" permission to invoke the grow()
and write() methods, but denies access (for the holder of that handle)
to the read/delete/move/etc. methods.  And, create *another* handle
that grants permissions for the read() and write() methods, but no
others.

When the "directory server" is passed one of these handles in a
method invocation, it implicitly *knows* whether that method is
allowed on that handle.  So, trying to invoke grow() using that
second example handle results in a permission error.

With this sort of mechanism, I can provide handles to the *directory*
that allow lookup()'s, but no create()'s.  Or, create()'s but no
delete()'s.

Because the directory object exists in one (conceptual) place, a
suitably empowered client can install a "name" that the directory
server (for that object) will treat as "reserved" -- and only
allow a client having the "able to create a binding with this
particular name" to invoke the create() method with that name
as an argument!  Thus, implementing a "name reservation".

[The same sort of mechanism prevents certain clients from deleting
that particular name, etc.]

On 5/22/2015 11:06 AM, George Neuner wrote:
> On Fri, 22 May 2015 14:57:24 +0300, Dimiter_Popoff <dp@tgi-sci.com>
> wrote:
>
>> On 21.5.2015 ?. 14:11, George Neuner wrote:
>>>
>>>>> However, this has implications - new directory entry type (so the
>>>>> data gets deallocated only for one of the directory entries) or just
>>>>> making a "link" type directory entry which may easily turn out
>>>>> to point to nothing (user deletes the file it points to). Solvable
>>>>> issues but adding complexity and no serious benefit.
>>>
>>> That's why Unix indirects through the inode - the complexity is
>>> necessary to correct operation.
>>
>> Yes, like I said solvable issues - but the added expense just does not
>> justify the benefit this give you. The inode must link back to all
>> directory entries pointing to it for this to work - they probably do
>> that - which has further implications, e.g. do they store the path
>> name as text - this must then be resolved for every access - or do
>> they store medium specific (LBN etc.) data - this will take special
>> processing when copying (my guess is they have opted for the first).
>
> No.  Links are one way: from directory entries to the inode.  The
> inode is reference counted and persists as long as there is at least
> one directory entry which references it.

Subtle point:  "reference counted".  I.e., the inode has no idea
*where* the references happen to be!  Anything referring to that
inode (adding or removing a reference) is responsible for
correctly updating the reference count and ensuring its integrity.

> The inode maintains the structure and security information for the
> file.  Directory entries are just a name and an inode reference.

Another subtle point:  the metadata is tied to the *file*, not the
*name*.  I.e., names are functionally equivalent.

[In my world, names map to handles and handles to objects.  So,
I can create different capabilities (bad choice of words) for
different names (and different handles -- as I can subset
capabilities for any particular handle before passing it on,
anonymously)]

Hi George,

On 22.5.2015 г. 21:06, George Neuner wrote:
> On Fri, 22 May 2015 14:57:24 +0300, Dimiter_Popoff <dp@tgi-sci.com>
> wrote:
>
>> On 21.5.2015 ?. 14:11, George Neuner wrote:
>>>
>>>>> However, this has implications - new directory entry type (so the
>>>>> data gets deallocated only for one of the directory entries) or just
>>>>> making a "link" type directory entry which may easily turn out
>>>>> to point to nothing (user deletes the file it points to). Solvable
>>>>> issues but adding complexity and no serious benefit.
>>>
>>> That's why Unix indirects through the inode - the complexity is
>>> necessary to correct operation.
>>
>> Yes, like I said solvable issues - but the added expense just does not
>> justify the benefit this give you. The inode must link back to all
>> directory entries pointing to it for this to work - they probably do
>> that - which has further implications, e.g. do they store the path
>> name as text - this must then be resolved for every access - or do
>> they store medium specific (LBN etc.) data - this will take special
>> processing when copying (my guess is they have opted for the first).
>
> No.  Links are one way: from directory entries to the inode.  The
> inode is reference counted and persists as long as there is at least
> one directory entry which references it.

Thanks for the explanation. This way it would be impractical to have
directory entries from different directories pointing to one inode,
or are they doing it? Would take a lot of directory digging before
an inode is deleted so my guess is obviously "no", but it is only
a guess.

> The inode maintains the structure and security information for the
> file.  Directory entries are just a name and an inode reference.
>
> Inodes for open files are cached in memory and updates are lazily
> written back to disk [unless you deliberately (f)sync].  There is a
> lock on the in-memory cached inode, but it is used only for updates to
> the inode itself to coordinate sync flushes.  File content locks are
> handled separately through a filesystem service - content locks are
> neither in nor on the inode.

In DPS there is no separate lock for the RIB - since I do not allow
multiple directory references to the same data. Even if I would opt
to implement it, I would make a different directory entry type (the
entry type is 5 or 6 bits I think (it is a byte and I don't remember
how many bits of it I used for flags). This directory entry can easily
point to the "original" entry which points to the "data" (RIB or no
RIB), or could use some level of indirection etc. Not that I plan
to do it, I see no use for that sort of thing myself (I imagine they
implemented in in the 70-s to save space for the multiple commands,
which would have a negligible effect today if one just uses a file
per command - or, if space allocation conscious, would put
all the commands in a file with a "disk" image having a small cluster
size (say 32 bytes or even 1 byte)).

>
>> In DPS the inode is called RIB (.... this is what it was called on the
>> first OS I had contact with - MDOS, Motorolas OS on their Exorciser,
>> Retrieve Information Block - so I just reused a name I remembered well).
>> It does not link back to the directory entry - and for files split
>> in up to 2 pieces it can be unused, thus saving one disk access per
>> file open - but I could easily expand that if I wanted to.
>> For an application it would be trivial which name was used so it
>> was invoked as iocb-s contain the directory entry name.
>> But I can't see how this buys me anything of real value.
>
> In Unix the inode directly links to some small number of data blocks
> before indirecting through index blocks.  The size of a file that can
> be addressed directly depends on the logical block size of the media,
> but it typically is 40-64KB for a desktop filesystem and may be
> megabytes in a server filesystem.

Does that mean the inode is part of the directory file? That would
be very similar to the way I do it - only I have put
"segment descriptors" (two of them) in the directory entry, if there are
more than 2 segments the first one of the two points to the RIB (which
is not inside the directory file).

Dimiter