Portable Assembly

On 06/06/17 06:24, George Neuner wrote:
> On Mon, 5 Jun 2017 09:10:10 -0700, Don Y <blockedofcourse@foo.invalid>
> wrote:
> 
>> C's portability problem isn't with the language, per se, as much as it is
>> with the "practitioners".  It could benefit from much stricter type
>> checking and a lot fewer "undefined/implementation-defined behaviors"
>> (cuz it seems folks just get the code working on THEIR target and
>> never see how it fails to execute properly on any OTHER target!)

I don't think C would benefit from a /lot/ fewer undefined or
implementation-dependent behaviours.  Some could happily be removed
(IMHO), but most are fine.  However, I would like to see
/implementations/ working harder towards spotting this sort of thing in
user code - working to fix the /real/ problem of bad programmers, rather
than changing the language.

For example, some people seem to think that ints have two's complement
wrap-around behaviour on overflow in C, just because that is how the
underlying cpu handles it.  Some languages (like Java) avoid undefined
behaviour by giving this a definition - they say exactly how signed
overflow should be handled.  In my opinion, this is missing the point -
if your code has signed integers that overflow, you've got a bug.  There
is /no/ right answer - picking one and saying "we define the behaviour
/this/ way" does not make it right.  So allowing the compiler to assume
that it will never happen, and to optimise accordingly, is a good idea.

But compilers should do their best to spot such cases, and hit out hard
when they see it.  When the compiler sees "for (int i = 0; i >= 0;
i++)", it should throw a tantrum - it should not merely break off
compilation with an error message, it should send an email to the
programmer's boss.  (I'll settle for a warning message that is enabled
by default.)

Compilers /are/ getting better at warning on undefined behaviour, but
they could always be better.

> 
> The argument always has been that if implementation defined behaviors
> are locked down, then C would be inefficient on CPUs that don't have
> good support for <whatever>.
> 

Yes - and it is still a good argument.

It might be a nice idea to do a little bit of clean-up of some of the
options.  I don't think it would do much harm if future C standards
enforced two's complement signed integers without padding, for example -
one's complement and signed-magnitude machines are extremely rare.


> 
> There is no question that C could do much better type/value and
> pointer/index checking, but it likely would come at the cost of far
> more explicit casting (more verbose code), and likely many more
> runtime checks.

That is not necessarily the case - but to get much stronger type
checking in C, you would need to include so many features to the
language that you might as well use C++.  For example, it is quite
possible in C to define types "speed", "distance" and "time" so that you
can't simply add a "distance" and a "time", while still being able to
generate optimal code.  But you can't use normal operators in
expressions with the types - you can't write "v = d / t;", but need to
write "v = divDistanceTime(d, t);".

> 
> A more expressive type system would help [e.g., range integers, etc.],
> but that would constitute a significant change to the language.  

Yes, and yes.

Ranged integer types would be nice, and would give not just safer code,
but more efficient code.


There are many things that would be nice to add to the language (and to
C++), some of which are common as extensions in compilers but which
could usefully be standardised.  An example is gcc's
"__builtin_constant_p" feature.  This can be used to let the compiler do
compile-time checking where possible, but skip run-time checks for code
efficiency:

extern void __attribute__((error("Assume failed"))) assumeFailed(void);

// The compiler can assume that "x" is true, and optimise or warn
// accordingly
// If the compiler can see that the assume will fail, it gives an error
#define assume(x) \
	do { \
		if (__builtin_constant_p(x)) { \
			if (!(x)) { \
				assumeFailed(); \
			} \
		} \
		if (!(x)) __builtin_unreachable(); \
	} while (0)


If such features were standardised, they could be used in all code - not
just gcc-specific code.

> 
> Some people point to Ada as an example of a language that can be both
> "fast" and "safe", but many people (maybe not in this group, but many
> nonetheless) are unaware that quite a lot of Ada's type/value checks
> are done at runtime and throw exceptions if they fail.

They also involve a good deal more verbose code.

> 
> Obviously, a compiler could provide a way to disable the automated
> runtime checking, and even when enabled checks can be elided if the
> compiler can statically prove that a given operation will always be
> safe.  But even in Ada with its far more expressive types there are
> many situations in which the compiler simply can't do that.
> 
> More stringent languages like ML won't even compile if they can't
> statically type check the code.  In such languages, quite a lot of
> programmer effort goes toward clubbing the type checker into
> submission.
> 
> 
> TANSTAAFL,
> George
> 

Hi George,

Snow melt, yet?  :>  (105+ here, all week)

On 6/5/2017 9:24 PM, George Neuner wrote:
> On Mon, 5 Jun 2017 09:10:10 -0700, Don Y <blockedofcourse@foo.invalid>
> wrote:
>
>> C's portability problem isn't with the language, per se, as much as it is
>> with the "practitioners".  It could benefit from much stricter type
>> checking and a lot fewer "undefined/implementation-defined behaviors"
>> (cuz it seems folks just get the code working on THEIR target and
>> never see how it fails to execute properly on any OTHER target!)
>
> The argument always has been that if implementation defined behaviors
> are locked down, then C would be inefficient on CPUs that don't have
> good support for <whatever>.

Of course.  When you design a language, you seek to optimize some set
of *many* (often conflicting) design criteria.  Do you want it to be
portable?  Deterministic?  Require minimal keystrokes?  Lead to
pronounceable code?  etc.

Most "programmers" like to consider themselves "artists" -- in the same
vein as authors/novelists.  The fewest constraints on the way we practice
our craft (art?).  Imagine if all novels *had* to be written composed entirely
of simple sentences built in a subject-predicate order.  Or, if every subject
had to be proper noun, etc.

Alternatively, you can try to constrain the programmer (protect him from
himself) and *hope* he's compliant.

Of course, the range of applications is also significant.  A language
intended for scripting has a different set of goals than one that can
effectively implement an OS, etc.

> Look at the (historical) troubles resulting from Java (initially)
> requiring IEEE-754 compliance and that FP results be exactly
> reproducible *both* on the same platform *and* across platforms.
>
> No FP hardware fully implements any version of IEEE-754: every chip
> requires software fixups to achieve compliance, and most fixup suites
> are not even complete [e.g., ignoring unpopular rounding modes, etc.].
> Java FP code ran slower on chips that needed more fixups, and the
> requirements prevented even implementing a compliant Java on some
> chips despite their having FP support.
>
> Java ultimately had to entirely back away from its reproducibility
> guarantees.  It now requires only best consistency - not exact
> reproducibility - on the same platform.  If you want reproducible
> results, you have to use software floating point (BigFloat), and
> accept much slower code.  And by requiring consistency, it can only
> approximate the performance of C code which is likewise compiled. Most
> C compilers allow to eshew FP consistency for more speed ... Java does
> not.
>
> Of course, FP in general is somewhat less important to this crowd than
> to other groups, and C has a lot of implementation defined behavior
> unrelated to FP.  But the lesson of trying to lock down hardware
> (and/or OS) dependent behavior still is important.

Yes.  But this moves the languages design choices on the axis
AWAY from (inherent) "portability".

You can write "portable" code, in C -- but, it requires a conscious decision
to do so (and a fair bit of practice to do so WELL).

And, what does it mean to claim some piece of code is "portable"?  That
it produces the same results without concern for resource usage, execution
speed, code size, etc.?  (who decided that THOSE criteria were more/less
important?)

For example, My BigDecimal package will tailor itself to the size of the
largest integer data type in the target.  But, this is often NOT what you
would want (it tends to make larger BigDecimals more space efficient), esp
on smaller machines!  E.g., on a 16b machine, there might be support for
ulonglong's that my code will exploit... but, if the inherent data type
is "ushort" and all CPU operations are geared towards that size data,
there will be countless "helper routines" invoked just to let my code
use these "unnaturally large" data types, even if they do so inefficiently
(and the code could have just as easily tailor itself to a data size closer
to ushort)

> There is no question that C could do much better type/value and
> pointer/index checking, but it likely would come at the cost of far
> more explicit casting (more verbose code), and likely many more
> runtime checks.

And, more contortions from programmers trying to "work-around" those
checks ("Yeah, I *want* to dereference NULL!  I want to 'jump 0x0000'")

> A more expressive type system would help [e.g., range integers, etc.],
> but that would constitute a significant change to the language.
>
> Some people point to Ada as an example of a language that can be both
> "fast" and "safe", but many people (maybe not in this group, but many
> nonetheless) are unaware that quite a lot of Ada's type/value checks
> are done at runtime and throw exceptions if they fail.

IME, that's the only way to get the degree of checking you *want*
(i.e., you, as an individual programmer on a specific project coding
a particular algorithm).  But, that voids many tools designed to protect
you from these evils.

> Obviously, a compiler could provide a way to disable the automated
> runtime checking, and even when enabled checks can be elided if the
> compiler can statically prove that a given operation will always be
> safe.  But even in Ada with its far more expressive types there are
> many situations in which the compiler simply can't do that.
>
> More stringent languages like ML won't even compile if they can't
> statically type check the code.  In such languages, quite a lot of
> programmer effort goes toward clubbing the type checker into
> submission.

I'd considered coding my current project in C++ *just* for the
better type-checking.

E.g., I want new types to be syntactically treated as different
types, not just aliases:

typedef long int handle_t;        // reference for an OS object
typedef handle_t file_handle_t;   // reference for an OS *file* object
typedef handle_t lamp_handle_t;   // reference for an OS lamp object

extern turn_on(lamp_handle_t theLamp);
extern unlink(file_handle_t theFile);

main()
{
    file_handle_t aFile;
    lamp_handle_t aLamp;

    ...  // initialization

    turn_on( (lamp_handle_t) aFile);
    unlink( (file_handle_t) aLamp);
}

should raise eyebrows!

Add the potential source of ambiguity that the IDL can introduce
and its too easy for an "uncooperative" developer to craft code
that is way too cryptic and prone to errors.

Imagine poking bytes into a buffer called "stack_frame" and
then trying to wedge it "under" a function invocation...
sure, you can MAKE it work.  But, will you know WHY it works,
next week??

On 06/06/17 16:03, Don Y wrote:

> I'd considered coding my current project in C++ *just* for the
> better type-checking.
> 
> E.g., I want new types to be syntactically treated as different
> types, not just aliases:
> 
> typedef long int handle_t;        // reference for an OS object
> typedef handle_t file_handle_t;   // reference for an OS *file* object
> typedef handle_t lamp_handle_t;   // reference for an OS lamp object
> 
> extern turn_on(lamp_handle_t theLamp);
> extern unlink(file_handle_t theFile);
> 
> main()
> {
>    file_handle_t aFile;
>    lamp_handle_t aLamp;
> 
>    ...  // initialization
> 
>    turn_on( (lamp_handle_t) aFile);
>    unlink( (file_handle_t) aLamp);
> }
> 
> should raise eyebrows!
> 

You can get that in C - put your types in structs.  It's a pain for
arithmetic types, but works fine for handles.

typedef struct { long int h; } handle_t;
typedef struct { handle_t fh; } file_handle_t;
typedef struct { handle_t lh; } lamp_handle_t;

Now "turn_on" will not accept a lamp_handle_t object.

On 6.6.17 17:58, David Brown wrote:
> On 06/06/17 16:03, Don Y wrote:
> 
>> I'd considered coding my current project in C++ *just* for the
>> better type-checking.
>>
>> E.g., I want new types to be syntactically treated as different
>> types, not just aliases:
>>
>> typedef long int handle_t;        // reference for an OS object
>> typedef handle_t file_handle_t;   // reference for an OS *file* object
>> typedef handle_t lamp_handle_t;   // reference for an OS lamp object
>>
>> extern turn_on(lamp_handle_t theLamp);
>> extern unlink(file_handle_t theFile);
>>
>> main()
>> {
>>     file_handle_t aFile;
>>     lamp_handle_t aLamp;
>>
>>     ...  // initialization
>>
>>     turn_on( (lamp_handle_t) aFile);
>>     unlink( (file_handle_t) aLamp);
>> }
>>
>> should raise eyebrows!
>>
> 
> You can get that in C - put your types in structs.  It's a pain for
> arithmetic types, but works fine for handles.
> 
> typedef struct { long int h; } handle_t;
> typedef struct { handle_t fh; } file_handle_t;
> typedef struct { handle_t lh; } lamp_handle_t;
> 
> Now "turn_on" will not accept a lamp_handle_t object.


Don seems to invent objects without objects.

-- 

-TV

On Tue, 6 Jun 2017 07:03:38 -0700, Don Y <blockedofcourse@foo.invalid>
wrote:

>Hi George,
>
>Snow melt, yet?  :>  (105+ here, all week)

45 and raining.  It has rained at least some nearly every day for the
last 2 weeks.  Great for pollen counts, bad for mold spores.



>On 6/5/2017 9:24 PM, George Neuner wrote:
>> On Mon, 5 Jun 2017 09:10:10 -0700, Don Y <blockedofcourse@foo.invalid>
>> wrote:
>
>Most "programmers" like to consider themselves "artists" -- in the same
>vein as authors/novelists.  The fewest constraints on the way we practice
>our craft (art?).  

But that's the thing ... programming in and of itself is a skill that
can be taught, but software "engineering" *IS* an art.

It is exactly analogous to sculpting or piano playing ... almost
anyone can learn to wield hammer and chisel, or to play notes on keys
- but only some can produce beauty in statue or music.


>Imagine if all novels *had* to be written composed entirely
>of simple sentences built in a subject-predicate order.  Or, if every
>subject had to be proper noun, etc.

Iambic meter?  Limerick?  Words that rhyme with "orange".


>Alternatively, you can try to constrain the programmer (protect him from
>himself) and *hope* he's compliant.

Yes.  And my preference for a *general* purpose language is to default
to protecting the programmer, but to selectively permit use of more
dangerous constructs in marked "unsafe" code.


>Of course, the range of applications is also significant.  A language
>intended for scripting has a different set of goals than one that can
>effectively implement an OS, etc.

Absolutely.  A scripting or domain specific language often does not
need to be Turing powerful (or even anywhere close).



>> A more expressive type system would help [e.g., range integers, etc.],
>> but that would constitute a significant change to the language.
>>
>> Some people point to Ada as an example of a language that can be both
>> "fast" and "safe", but many people (maybe not in this group, but many
>> nonetheless) are unaware that quite a lot of Ada's type/value checks
>> are done at runtime and throw exceptions if they fail.
>
>IME, that's the only way to get the degree of checking you *want*
>(i.e., you, as an individual programmer on a specific project coding
>a particular algorithm).  But, that voids many tools designed to protect
>you from these evils.

Lisp is safe by default, and even without type declarations a good
compiler can produce quite good code through extensive type/value
propogation analyses.

But by selectively adding declarations and local annotations, a Lisp
programmer can improve performance - sometimes significantly.  In many
cases, carefully tuned Lisp code can approach C performance.  Type
annotation in Lisp effectively tells the compiler "trust me, I know
what I'm doing" and lets the compiler elide runtime checks that it
couldn't otherwise eliminate, and sometimes switch to (smaller,faster)
untagged data representations.

There's nothing special about Lisp that makes it particularly amenable
to that kind of tuning - Lisp simply can benefit more than some other
languages because it uses tagged data and defaults to perform (almost)
all type checking at runtime.

[Aside: BiBOP still *effectively* tags data even where tags are not
explicitly stored.  In BiBOP systems the type of a value is deducible
from its address in memory.]


Modern type inferencing allows a "safer than C" language to use C-like
raw data representations, and to rely *mostly* on static type checking
without the need for a whole lot of type declarations and casting. But
type inferencing has limitations: with current methods it is not
possible to completely eliminate the need for declarations.  And as
with Lisp, most type inferencing systems can be assisted to generate
better code by selective use of annotations.

In any case, regardless of what type system is used, it isn't possible
to completely eliminate runtime checking IFF you want a language to be
safe: e.g., pointers issues aside, there's no way to statically
guarantee that range reduction will produce a value that can be safely
stored into a type having a smaller (bit) width.  No matter how
sophicated the compiler becomes, there always will be cases where the
programmer knows better and should be able to override it.  


But even with these limitations, there are languages that are useful
now and do far more of what you want than does C.


YMMV,
George

On 17-06-06 07:24 , George Neuner wrote:
> On Mon, 5 Jun 2017 09:10:10 -0700, Don Y <blockedofcourse@foo.invalid>
> wrote:
>
>> C's portability problem isn't with the language, per se, as much as it is
>> with the "practitioners".  It could benefit from much stricter type
>> checking and a lot fewer "undefined/implementation-defined behaviors"
>> (cuz it seems folks just get the code working on THEIR target and
>> never see how it fails to execute properly on any OTHER target!)
>
> The argument always has been that if implementation defined behaviors
> are locked down, then C would be inefficient on CPUs that don't have
> good support for <whatever>.

     [snip]

> There is no question that C could do much better type/value and
> pointer/index checking, but it likely would come at the cost of far
> more explicit casting (more verbose code), and likely many more
> runtime checks.
>
> A more expressive type system would help [e.g., range integers, etc.],
> but that would constitute a significant change to the language.
>
> Some people point to Ada as an example of a language that can be both
> "fast" and "safe", but many people (maybe not in this group, but many
> nonetheless) are unaware that quite a lot of Ada's type/value checks
> are done at runtime and throw exceptions if they fail.

None of the Ada *type* checks are done at runtime. Only *value* checks 
are done at runtime.

> Obviously, a compiler could provide a way to disable the automated
> runtime checking,

All Ada compilers I have seen have an option to disable runtime checks.

> and even when enabled checks can be elided if the
> compiler can statically prove that a given operation will always be
> safe.  But even in Ada with its far more expressive types there are
> many situations in which the compiler simply can't do that.

There are other, more proof-oriented tools that can be used for that, 
for example the CodePeer tool from AdaCore, or the SPARK toolset.

It is not uncommon for real Ada programs to be proven exception-free 
with such tools, which means that it is safe to turn off the runtime checks.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

On Wed, 7 Jun 2017 00:30:40 +0300, Niklas Holsti
<niklas.holsti@tidorum.invalid> wrote:

>On 17-06-06 07:24 , George Neuner wrote:
>
>> Some people point to Ada as an example of a language that can be both
>> "fast" and "safe", but many people (maybe not in this group, but many
>> nonetheless) are unaware that quite a lot of Ada's type/value checks
>> are done at runtime and throw exceptions if they fail.
>
>None of the Ada *type* checks are done at runtime. Only *value* checks 
>are done at runtime.

Note that I said "type/value", not simply "type".

In any case, it's a distinction without a difference.  The value
checks that need to be performed at runtime are due mainly to use of
differing types that have overlapping range compatibility.

The remaining uses of runtime checks are due to I/O where input values
may be inconsistent with the types involved.


>> Obviously, a compiler could provide a way to disable the automated
>> runtime checking,
>
>All Ada compilers I have seen have an option to disable runtime checks.

Yes.  And if you were following the discussion, you would have noticed
that that comment was directed not at Ada, but toward runtime checking
in a hypothetical "safer" C.

George

On 6/6/2017 1:42 PM, George Neuner wrote:
> On Tue, 6 Jun 2017 07:03:38 -0700, Don Y <blockedofcourse@foo.invalid>
> wrote:
>
>> Snow melt, yet?  :>  (105+ here, all week)
>
> 45 and raining.  It has rained at least some nearly every day for the
> last 2 weeks.  Great for pollen counts, bad for mold spores.

Yeah, until it eases up a bit and all that stuff comes into BLOOM!  :<
We're paying the price for a Spring that came 4 weeks early...

>> On 6/5/2017 9:24 PM, George Neuner wrote:
>>> On Mon, 5 Jun 2017 09:10:10 -0700, Don Y <blockedofcourse@foo.invalid>
>>> wrote:
>>
>> Most "programmers" like to consider themselves "artists" -- in the same
>> vein as authors/novelists.  The fewest constraints on the way we practice
>> our craft (art?).
>
> But that's the thing ... programming in and of itself is a skill that
> can be taught, but software "engineering" *IS* an art.

Exactly.  But, there are a sh*tload of folks who THINK themselves
"artists" that really should see how the rest of the world views
their "art"!  :>

The problem is that you have to *design* systems for with these
folks in mind as their likely "maintainers" and/or "evolvers".

So, even if you're "divinely inspired", you have to manage to either
put in place a framework that effectively guides (constrains!) their
future work to remain consistent with that design...

Or, leave copious notes and HOPE they read them, understand them and
take them to heart...

Or, create mechanisms (tools) that cripple the budding artistry they
(think) possess!  :>

> It is exactly analogous to sculpting or piano playing ... almost
> anyone can learn to wield hammer and chisel, or to play notes on keys
> - but only some can produce beauty in statue or music.

Yes.  Now, imagine The David needing a 21st century "update".
Michelangelo is "unavailable" for the job.  <grin>

Do you find the current "contemporary master" (of that art form)
and hire him/her to perform the task?  Or, some run-of-the-mill
guy from Sculptors University, Class of 2016??

If you're only concerned with The David *and* have the resources that
such an asset would command, you can probably afford to have the
current Master tackle the job.

OTOH, if you've got a boatload of similar jobs (The David, The Rita,
The Bob, The Bethany, The Harold, The Gretchen, etc.), that one artist
may decide he's tired of being asked to "tweek" the works of past
artists and want a commission of his own!  Or, simply not have time
enough in his schedule to get to all of them at the pace you desire!

>> Alternatively, you can try to constrain the programmer (protect him from
>> himself) and *hope* he's compliant.
>
> Yes.  And my preference for a *general* purpose language is to default
> to protecting the programmer, but to selectively permit use of more
> dangerous constructs in marked "unsafe" code.

And, count on the DISCIPLINE of all these would-be Michelangelos to
understand (and admit!) their own personal limitations prior to enabling
those constructs?

Like telling a 16 year old "keep it under 35MPH"...  <grin>

>> Of course, the range of applications is also significant.  A language
>> intended for scripting has a different set of goals than one that can
>> effectively implement an OS, etc.
>
> Absolutely.  A scripting or domain specific language often does not
> need to be Turing powerful (or even anywhere close).

What you (ideally) want, is to be able to "set a knob" on the 'side' of
the language to limit its "potential for misuse".  But, to do so in a
way that the practitioner doesn't feel intimidated/chastened at its
apparent "setting".

How do you tell a (as yet, unseen!) developer "you're not capable of
safely using pointers to functions?  Or, recursive algorithms?  Or,
self-modifying code?  Or, ...  But, *I* am!"  :>

(Returning to "portability"...)

Even if I can craft something that is portable under some set of
conditions/criteria that I deem appropriate -- often by leveraging
particular features of the language of a given implementation
thereof -- how do I know the next guy will understand those issues?
How do I know he won't *break* that aspect (portability) -- and
only belatedly discover his error (two years down the road when the
code base moves to a Big Endian 36b processor)?

It's similar to trying to ensure "appropriate" documentation
accompanies each FUTURE change to the system -- who decides
what is "appropriate"?  (Ans:  the guy further into the future who
can't sort out the changes made by his predecessor!)

>>> A more expressive type system would help [e.g., range integers, etc.],
>>> but that would constitute a significant change to the language.
>>>
>>> Some people point to Ada as an example of a language that can be both
>>> "fast" and "safe", but many people (maybe not in this group, but many
>>> nonetheless) are unaware that quite a lot of Ada's type/value checks
>>> are done at runtime and throw exceptions if they fail.
>>
>> IME, that's the only way to get the degree of checking you *want*
>> (i.e., you, as an individual programmer on a specific project coding
>> a particular algorithm).  But, that voids many tools designed to protect
>> you from these evils.
>
> Lisp is safe by default, and even without type declarations a good
> compiler can produce quite good code through extensive type/value
> propogation analyses.
>
> But by selectively adding declarations and local annotations, a Lisp
> programmer can improve performance - sometimes significantly.  In many
> cases, carefully tuned Lisp code can approach C performance.  Type
> annotation in Lisp effectively tells the compiler "trust me, I know
> what I'm doing" and lets the compiler elide runtime checks that it
> couldn't otherwise eliminate, and sometimes switch to (smaller,faster)
> untagged data representations.

I.e., "trust the programmer" -- except in those cases where you can't!  :>

> There's nothing special about Lisp that makes it particularly amenable
> to that kind of tuning - Lisp simply can benefit more than some other
> languages because it uses tagged data and defaults to perform (almost)
> all type checking at runtime.
>
> [Aside: BiBOP still *effectively* tags data even where tags are not
> explicitly stored.  In BiBOP systems the type of a value is deducible
> from its address in memory.]
>
> Modern type inferencing allows a "safer than C" language to use C-like
> raw data representations, and to rely *mostly* on static type checking
> without the need for a whole lot of type declarations and casting. But
> type inferencing has limitations: with current methods it is not
> possible to completely eliminate the need for declarations.  And as
> with Lisp, most type inferencing systems can be assisted to generate
> better code by selective use of annotations.

But it still requires the programmer to know what he's doing; the compiler
takes its cues from the programmer's actions!

How often have you seen a bare int used as a pointer?  Or, vice versa?
("Yeah, I know what you *mean* -- but that's not what you *coded*!")

> In any case, regardless of what type system is used, it isn't possible
> to completely eliminate runtime checking IFF you want a language to be
> safe: e.g., pointers issues aside, there's no way to statically
> guarantee that range reduction will produce a value that can be safely
> stored into a type having a smaller (bit) width.  No matter how
> sophicated the compiler becomes, there always will be cases where the
> programmer knows better and should be able to override it.

Exactly.  Hence the contradictory issues at play:
- enable the competent
- protect the incompetent

> But even with these limitations, there are languages that are useful
> now and do far more of what you want than does C.

But, when designing (or choosing!) a language, one of the dimensions
in your decision matrix has to be availability of that language AND in
the existing skillsets of its practitioners.

Being "better" (in whatever set of criteria) isn't enough to ensure
acceptance or adoption (witness the Betamax).

ASM saw widespread use -- not because it was the BEST tool for the
job but, rather, because it was (essentially) the ONLY game in town
(in the early embedded world).  Amusing that we didn't repeat the same
evolution of languages that was the case in the "mainframe" world
(despite having comparable computational resources to those
ANCIENT machines!).

The (early) languages that we settled on were simple to implement
on the development platforms and with the target resources.   Its
only as targets have become more resource-rich that we're exploring
richer execution environments (and the attendant consequences of
that for the developer).

On 06/06/17 22:42, George Neuner wrote:

> 
> Modern type inferencing allows a "safer than C" language to use C-like
> raw data representations, and to rely *mostly* on static type checking
> without the need for a whole lot of type declarations and casting. But
> type inferencing has limitations: with current methods it is not
> possible to completely eliminate the need for declarations.  And as
> with Lisp, most type inferencing systems can be assisted to generate
> better code by selective use of annotations.
> 

Modern C++ has quite powerful type inference now, with C++11 "auto".
This allows you to have complex types, encoding lots of information in
compile-time checkable types, while often being able to use "auto" or
"decltype" to avoid messy and hard to maintain source code.

The next step up is "concepts", which are a sort of meta-type.  A
"Number" concept, for example, might describe a type that has arithmetic
operations.  Instead of writing your code specifying exactly which
concrete types are used, you describe the properties you need your types
to have.

As you say, however, concrete type declarations cannot be eliminated -
in C++ they are essential when importing or exporting functions and data
between units.

> In any case, regardless of what type system is used, it isn't possible
> to completely eliminate runtime checking IFF you want a language to be
> safe: e.g., pointers issues aside, there's no way to statically
> guarantee that range reduction will produce a value that can be safely
> stored into a type having a smaller (bit) width.  No matter how
> sophicated the compiler becomes, there always will be cases where the
> programmer knows better and should be able to override it.  
> 

Yes.  You can do /some/ of the checking at compile-time, but not all of
it.  And a sophisticated whole-program optimiser can eliminate some of
the logical run-time checks, but not all of them.

> 
> But even with these limitations, there are languages that are useful
> now and do far more of what you want than does C.
> 

On Tue, 6 Jun 2017 23:16:52 -0700, Don Y <blockedofcourse@foo.invalid>
wrote:

>>> On 6/5/2017 9:24 PM, George Neuner wrote:
>>
>> ... software "engineering" *IS* an art.
>
>Exactly.  But, there are a sh*tload of folks who THINK themselves
>"artists" that really should see how the rest of the world views
>their "art"!  :>
>
>The problem is that you have to *design* systems for with these
>folks in mind as their likely "maintainers" and/or "evolvers".
>
>So, even if you're "divinely inspired", you have to manage to either
>put in place a framework that effectively guides (constrains!) their
>future work to remain consistent with that design...
>
>Or, leave copious notes and HOPE they read them, understand them and
>take them to heart...

Or adopt a throw-away mentality: replace rather than maintain.  

That basically is the idea behind the whole agile/devops/SaaS
movement: if it doesn't work today, no problem - there will be a new
release tomorrow [or sooner].


>> It is exactly analogous to sculpting or piano playing ... almost
>> anyone can learn to wield hammer and chisel, or to play notes on keys
>> - but only some can produce beauty in statue or music.
>
>Yes.  Now, imagine The David needing a 21st century "update".
>Michelangelo is "unavailable" for the job.  <grin>
>
>Do you find the current "contemporary master" (of that art form)
>and hire him/her to perform the task?  Or, some run-of-the-mill
>guy from Sculptors University, Class of 2016??
>
>If you're only concerned with The David *and* have the resources that
>such an asset would command, you can probably afford to have the
>current Master tackle the job.

You know what they say: one decent Lisp programmer is worth 10,000
python monkeys.


>OTOH, if you've got a boatload of similar jobs (The David, The Rita,
>The Bob, The Bethany, The Harold, The Gretchen, etc.), that one artist
>may decide he's tired of being asked to "tweek" the works of past
>artists and want a commission of his own!  Or, simply not have time
>enough in his schedule to get to all of them at the pace you desire!

No problem: robots and 3-D printers will take care of that.  Just read
an article that predicts AI will best humans at *everything* within 50
years.



>>> Alternatively, you can try to constrain the programmer (protect him from
>>> himself) and *hope* he's compliant.
>>
>> Yes.  And my preference for a *general* purpose language is to default
>> to protecting the programmer, but to selectively permit use of more
>> dangerous constructs in marked "unsafe" code.
>
>And, count on the DISCIPLINE of all these would-be Michelangelos to
>understand (and admit!) their own personal limitations prior to enabling
>those constructs?

For the less experienced, fear, uncertainty and doubt are better
counter-motivators than is any amount of discipline.   When a person
believes (correctly or not) that something is hard to learn or hard to
use, he or she usually will avoid trying it for as long as possible.

The basic problem with C is that some of its hard to master concepts
are dangled right in the faces of new programmers.


For almost any non-system application, you can do without (explicit
source level) pointer arithmetic.  But pointers and the address
operator are fundamental to function argument passing and returning
values (note: not "value return"), and it's effectively impossible to
program in C without using them.

This pushes newbies to learn about pointers, machine addressing and
memory management before many are ready.  There is plenty else to
learn without *simultaneously* being burdoned with issues of object
location.

Learning about pointers then invariably leads to learning about
arithmetic on pointers because they are covered together in most
tutorials.

Keep in mind that the majority of people learning and using C (or C++)
today have no prior experience with hardware or even with programming
in assembler.  If C isn't their 1st (non-scripting) language then most
likely their prior experiences were with "safe", high level, GC'd
languages that do not expose object addressing: e.g., Java, Scheme,
Python, etc. ... the commonly used "teaching" languages.

For general application programming, there is no need for a language
to provide mutable pointers: initialized references, together with
array (or stream) indexing and struct/object member access are
sufficient for virtually any non-system programming use.  This has
been studied extensively and there is considerable literature on the
subject.

[Note also I am talking about what a programmer is permitted to do at
the source code level ... what a compiler does to implement object
addressing under the hood is beside the point.]


<frown>

Mutable pointers are just the tip of the iceberg: I could write a
treatise on the difficulties / frustrations of the *average*
programmer with respect to manual memory management, limited precision
floating point, differing logical "views" of the same data,
parallelism, etc. ...
... and how C's "defects" with regard to safe *application*
programming conspire to add to their misery.

But this already is long and off the "portability" topic.



>What you (ideally) want, is to be able to "set a knob" on the 'side' of
>the language to limit its "potential for misuse".  But, to do so in a
>way that the practitioner doesn't feel intimidated/chastened at its
>apparent "setting".

Look at Racket's suite of teaching and extension languages.  They all
are implemented over the same core language (an extended Scheme), but
they leverage the flexibility of the core langauge to offer different
syntaxes, different semantics, etc.   

In the case of the teaching languages, there is reduced functionality,
combined with more newbie friendly debugging output, etc.

http://racket-lang.org/
https://docs.racket-lang.org/htdp-langs/index.html

And, yeah, the programmer can change which language is in use with a
simple "#lang <_>" directive, but the point here is the flexibility of
the system to provide (more or less) what you are asking for.



>(Returning to "portability"...)
>
>Even if I can craft something that is portable under some set of
>conditions/criteria that I deem appropriate -- often by leveraging
>particular features of the language of a given implementation
>thereof -- how do I know the next guy will understand those issues?
>How do I know he won't *break* that aspect (portability) -- and
>only belatedly discover his error (two years down the road when the
>code base moves to a Big Endian 36b processor)?

You don't, and there is little you can do about it.  You can try to be
helpful - e.g., with documentation - but you can't be responsible for
what the next person will do.

No software truly is portable except that which runs on an abstract
virtual machine.  As long as the virtual machine can be realized on a
particular base platform, the software that runs on the VM is
"portable" to that platform.


>It's similar to trying to ensure "appropriate" documentation
>accompanies each FUTURE change to the system -- who decides
>what is "appropriate"?  (Ans:  the guy further into the future who
>can't sort out the changes made by his predecessor!)

Again, you are only responsible for what you do.



>> ... No matter how sophisticated the compiler becomes, there always 
>> will be cases where the programmer knows better and should be able
>> to override it.
>
>Exactly.  Hence the contradictory issues at play:
>- enable the competent
>- protect the incompetent
>
>> But even with these limitations, there are languages that are useful
>> now and do far more of what you want than does C.
>
>But, when designing (or choosing!) a language, one of the dimensions
>in your decision matrix has to be availability of that language AND in
>the existing skillsets of its practitioners.

The modern concept of availability is very different than when you had
to wait for a company to provide a turnkey solution, or engineer
something yourself from scratch.  Now, if the main distribution
doesn't run on your platform, you are likely to find source that you
can port yourself (if you are able), or if there's any significant
user base, you may find that somebody else already has done it.

Tutorials, reference materials, etc. are a different matter, but the
simpler and more uniform the syntax and semantics, the easier the
language is to learn and to master.

question: why in C is *p.q == p->q 
                  but *p   != p 
	          and  p.q != p->q

followup: given coincidental addresses and modulo a cast, 
          how is it that *p can == *p.q

Shit like this makes a student's head explode.


In Pascal, the pointer dereference operator '^' and the record
(struct) member access operator '.'  were separate and always used
consistently.  The type system guarantees that p and p^ and p^.q  can
never, ever be the same object.

This visual and logical consistency made Pascal easier to learn.  And
not any less functional.

My favorite dead horse - Modula 3 - takes a similar approach.  Modula
3 is both a competent bare metal system language AND a safe OO
application language.  It does a whole lot more than (extended) Pascal
- yet it isn't that much harder to learn.

It is possible to learn Modula 3 incrementally: leaving advanced
subjects such as where objects are located in memory and when it's
safe to delete() them - until you absolutely need to know.  

And if you stick to writing user applications in the safe subset of
the language, you may never need to learn it: Modula 3 uses GC by
default.


>Being "better" (in whatever set of criteria) isn't enough to ensure
>acceptance or adoption (witness the Betamax).

Unfortunately.

>ASM saw widespread use -- not because it was the BEST tool for the
>job but, rather, because it was (essentially) the ONLY game in town
>(in the early embedded world).  Amusing that we didn't repeat the same
>evolution of languages that was the case in the "mainframe" world
>(despite having comparable computational resources to those
>ANCIENT machines!).
>
>The (early) languages that we settled on were simple to implement
>on the development platforms and with the target resources.   Its
>only as targets have become more resource-rich that we're exploring
>richer execution environments (and the attendant consequences of
>that for the developer).


There never was any C compiler that ran on any really tiny machine.
Ritchies' technotes on the development of C stated that the original
1972 PDP-11 compiler had to run in ~6KB (all that was left after
loading Unix), required several passes, and really was not usable
until the machine was given a hard disk.  Note also that that 1st
compiler implemented only a subset of K&R1.

K&R1 - as described in the book - was 1st implemented in 1977 and I
have never seen any numbers on the size of that compiler.


The smallest K&R1 compiler I can remember that *ran* on an 8-bit micro
was circa 1983.  It was a few hundred KB of code.  It ran in 48KB
using overlays, needed 2 floppy drives or a hard disk, and required 2
compile passes per source file and a final link pass.  

It was quite functional (if glacially slow), and included program code
overlay support and emulated single precision FP (in VAX format IIRC).
Although it targeted a 16-bit virtual machine with 6 16-bit registers,
it produced native 8-bit code : i.e. the "16-bit VM" program was not
interpreted, but was emulated by 8-bit code.

As part of the pro package (and available separately for personal use)
there also was a bytecode compiler that allowed packing much larger
applications (or their data) into memory.  It had all the same
features as the native code compiler, but produced interpreted code
that ran much slower.  You could use both native and interpreted code
in the same application via overlays.


There existed various subset C compilers that could run in less than
48KB, but most of them were no more than expensive learning toys.



But even by the standard of "the compiler could run on the machine",
there were languages better suited than C for application programming.

Consider that in the late 70's there already were decent 8-bit
implementations of BASIC, BCPL Logo, SNOBOL, etc.  (Extended) Pascal,
Smalltalk, SNOBOL4, etc. became available in the early 80's for both 8
and 16-bit systems.  But C really wasn't useable on any micro prior to
~1985 when reasonably<?> priced hard disks appeared.

Undoubtedly, AT&T giving away Unix to colleges from 1975..1979 meant
that students in that time frame would have gained some familiarity
with C.   16-bit micros powerful enough to really be characterized as
useful "development" systems popped out in the early 80's as these
students would have been graduating (or shortly thereafter).

But they were extremely expensive: tens of thousands of dollars for a
usable system.  You'd have to mortage your home to afford one, which
is not something the newly working with looming college loans would do
lightly.  And sans hard disk (more $$$), you'd manage only one or two
compiles a day.

Turbo Pascal was the 1st really useable [in the modern sense]
developement system.  It did not need a hard disk and it hit the
market before commodity hard disks were widely available.


The question is not why C was adopted for system programming, or for
cross development from a capable system to a smaller target.  Rather
the question is why it was so widely adopted for ALL kinds of
programming on ALL platforms given that were many other reasonable
choices available.


YMMV. I remain perplexed.
George