Embedded Scripting -- Tcl? Lua? Thoughts? Suggestions?

On Fri, 13 Mar 2015 20:35:54 -0700, Don Y <this@is.not.me.com> wrote:

>Hi Clifford,
>
>On 3/13/2015 7:14 PM, Clifford Heath wrote:
>
>> The only real use for multiple assignment is swapping values without using an
>> explicit temporary:
>> x, y = y, x
>
>Yes -- saving a temporary.  But, it's a feature that can easily be
>abused (no direct consequences of doing so -- for the *abuser*).
>
>Limbo suffers from this same problem:
>     x, y := 3;
>sets *both* x and y to '3'.

Which is crazy.  Looks like a bastard cross between Pascal's
assignment and Modula's constant declaration syntax.

>  x, y = 2, 3 is an error.

That's more reasonable.  If you using := for assignment, then = is
equal and multiple value equal doesn't make a whole lot of sense
unless you're talking about ordered sets (tuples).  But I would assume
that Limbo has a syntax for constant tuples.


>To it's *credit*, the lua style of multiple assignment can be
>accomplished using tuples:
>    (x, y) := (2, 3);
>though tuples are far more useful for other things!

But a tuple semantically is different from multiple values.  A tuple
is a single container with multiple content - the container must be
"opened" to get at its content whereas the individual values
(typically) do not.  

Then there is the question of whether tuples can be frame allocated or
whether they end up on the heap. And whether a tuple has O(1), O(N) or
O(something else) access.

Depending on the implementation, it might actually be faster to
allocate a heap tuple than to return/set multiple values.  But because
it usually is slower to unpack the tuple, that may eat up any saving
from using it in the first place.

George

On Sat, 14 Mar 2015 13:14:12 +1100, Clifford Heath
<no.spam@please.net> wrote:

>The only real use for multiple assignment is swapping values without 
>using an explicit temporary:
>x, y = y, x

No, there are other uses - the most common to return both a result and
error/status from a function.

However, multiple assignment - or equivalently, multiple value return
- too often is abused by allowing some of the values to be elided.
There are times when all values aren't interesting, but IMO there
should be a syntactic placeholder that is not assigned to.


>> Likewise, the potential for confusion when arguments are omitted
>> from function invocations.  I.e., it doesn't do much to help
>> protect the programmer from "little (painful) mistakes":
>>     "Oh, crap!  Foo() takes *3* parameters!)
>
>Calling a 3-argument function with 2 arguments should return a function 
>that takes the remaining argument :) (I.e. currying)

Remember which forum you're in 8-)   

Currying creates closures and *may* additionally require GC.  Many
high level language features don't mix well with small devices.

George

On Fri, 13 Mar 2015 16:27:12 -0500, Tim Wescott
<seemywebsite@myfooter.really> wrote:

>Rather than just use some clunky cryptic and limited language that's 
>unique to the problem and to me (and charging my customer up the wazoo for 
>it), I'd like to use something that's open-source and available.  
>Something that uses modest resources is to be preferred -- my 
>understanding is that I can expect Lua to use about 100k of flash, which 
>is a bit much for me but bearable.  I don't know how Tcl compares -- I 
>just know that it's an alternative.

Javacard would fit depending on the libraries you include.  Javacard
is a Java VM, but pure interpreter and very small.  It does not
support dynamic load or verification of class files.  Be aware though
that Javacard also has no GC - applets have to request heap space at
startup and manage it (if they are so inclined) themselves.

I haven't used Lua myself (dose of salt) but my understanding is that
the VM is separable.  If you can cross compile the bytecode, it should
be workable.

TCL will look weird to many people.  IME (not recent) it also has
lousy feedback on script errors.  I have never tried to use it in a
really small space so I can't comment on it's minimum RAM needs.


Code space wise, any of these should have plenty of headroom with
100KB. The real question is how much RAM can they use?  


>So -- anyone have any direct experience with either of these, or with 
>something else?  We briefly considered Java, but it just looks entirely 
>too big -- it would basically drive us to use Linux or some other "real" 
>OS (even the little linuxes are too big for my tastes) -- I'd really 
>rather just have a scripting engine that gets called whenever a script 
>needs to run, and otherwise doesn't need much hand-holding from the 
>application code.
>
>I'm looking for opinions, so fire away!

If you have a little more [well, 3x] space I'd mention TCC.


Have you looked at PicoC ?  https://code.google.com/p/picoc/


George

George Neuner <gneuner2@comcast.net> writes:
> Javacard also has no GC - applets have to request heap space at
> startup and manage it (if they are so inclined) themselves.

GC is present in some Javacard devices.  Maybe more of a problem is that
I don't think Javacard is open source.

> TCL will look weird to many people.... I can't comment on it's minimum
> RAM needs.

I checked into this yesterday and tcl these days has gotten rather
bloated.  The "small footprint" version is 100k-200k of code:

  http://jim.tcl.tk/index.html/doc/www/www/index.html

I used an embedded tcl several versions back and it was smaller then,
but people probably expect the recent improvements.

> If you have a little more [well, 3x] space I'd mention TCC.
> Have you looked at PicoC ?  https://code.google.com/p/picoc/

It doesn't sound like there's much need to host the compiler on the
embedded board, and using C defeats a main benefit of scripting
languages, which is to supply automatic memory management and protect
the user from the usual C hazards like pointer errors causing memory
corruption.

I do see there are some Scheme implementations around with mighty small
ram requirements, e.g. http://armpit.sourceforge.net/ (scroll down the
page) discusses some targets with as little as 32k of flash and 8k of
ram.  It is written in ARM assembler though, making me wonder if it has
a reasonable C FFI.

https://github.com/ryansuchocki/microscheme might also be interesting:
someone just pointed it out to me and I haven't looked at it much yet.

For simple purposes like UI customization, Scheme is probably ok for
non-Schemers since they can probably get their stuff working by copying
and modifying existing examples.  There are also lots of books on
Scheme.  So it seems worth looking into these if Lua turns out to be too
large.

I looked at Tiny Scheme which is a well known small implementation
(about 40k), but its ram footprint looks rather large, maybe since it
uses a recursive S-expression evaluator instead of a VM.

I'm a fan of Hedgehog Lisp which has an offline compiler and a very
compact VM (about 20k), but it's a weird dialect with almost no users.
Sometimes I think of hacking it to be more compatible with Scheme, and
maybe adding alternative front end to it.  For a while I thought using
its VM as a runtime for a Haskell dialect for microcontrollers that I
wanted to call "Control-H", so that the file extension would be a
backspace character.  ;-)

On 3/14/2015 8:59 PM, George Neuner wrote:
> On Fri, 13 Mar 2015 20:35:54 -0700, Don Y <this@is.not.me.com> wrote:
>> On 3/13/2015 7:14 PM, Clifford Heath wrote:
>>
>>> The only real use for multiple assignment is swapping values without using an
>>> explicit temporary:
>>> x, y = y, x
>>
>> Yes -- saving a temporary.  But, it's a feature that can easily be
>> abused (no direct consequences of doing so -- for the *abuser*).
>>
>> Limbo suffers from this same problem:
>>      x, y := 3;
>> sets *both* x and y to '3'.
>
> Which is crazy.  Looks like a bastard cross between Pascal's
> assignment and Modula's constant declaration syntax.

It's effectively,
    x: int;
    y: int;
    x = 3;
    y = 3;
Or,
    x, y: int;
    x = y = 3;

>>   x, y = 2, 3 is an error.
>
> That's more reasonable.  If you using := for assignment, then = is

':=' declares and defines; ':' declares while '=' defines
So,
     x, y = 3;

> equal and multiple value equal doesn't make a whole lot of sense
> unless you're talking about ordered sets (tuples).  But I would assume
> that Limbo has a syntax for constant tuples.
>
>> To it's *credit*, the lua style of multiple assignment can be
>> accomplished using tuples:
>>     (x, y) := (2, 3);
>> though tuples are far more useful for other things!
>
> But a tuple semantically is different from multiple values.  A tuple
> is a single container with multiple content - the container must be
> "opened" to get at its content whereas the individual values
> (typically) do not.

That's not required in limbo.  E.g., it is common for a function to
return a tuple:  one member being a flag (success/fail) with the other(s)
being the expected result(s).

E.g., you could create a function that parses a (unsorted) list of values
comparing each to a "given" and returns:
- the count of values below the given value
- the count of values above the given value
- the count of values *at* the given value
- a (sorted?) list of all (unique) values below the given value
- a sorted list of all the unique values above the given value
- a summary code (like an error code: LIST_EMPTY, SUCCESS, etc.)
[Note that the number of entries in each of these lists may be less
than the *counts* returned as duplicates will be elided]

In the following, assume the "values" are strings:

check_list(
    given: string,                 // what we seek
    candidates: list of string     // where it is sought
) :
    (int,                           // count below
     int,                           // count at
     int,                           // count above
     list of string,                // sorted strings below sought
     list of string,                // sorted strings above sought
     int                            // result code
     )

it could then be invoked as:

   (below, at, above, before, after, result) := check_list("Penny", names);

Use of the ':=' operator saves me the effort of declaring each of the types
of the members of that tuple.  This is sort of an "anonymous" tuple (the
function could have returned it's values to a variable of the appropriate
tuple type)

Then, I could say:
     if (below < above) {
        sys->print("Of the " + string (len names) + "names examined, far"
                   + " more names preceded the name of interest than followed."
                   + "  There were %d different names preceding, lexically,"
                   + " and %d names following.", len before, len after);
        ...

[Highly contrived example]

Assuming these are the only return values used from that function
invocation, a smarter way of invoking it would have been:

(below, nil, above, before, after, nil) := check_list("george", names);

which makes it obvious which values are not used.

> Then there is the question of whether tuples can be frame allocated or
> whether they end up on the heap. And whether a tuple has O(1), O(N) or
> O(something else) access.
>
> Depending on the implementation, it might actually be faster to
> allocate a heap tuple than to return/set multiple values.  But because
> it usually is slower to unpack the tuple, that may eat up any saving
> from using it in the first place.

> The only real use for multiple assignment is swapping values without
> using an explicit temporary:
> x, y = y, x

IMO there are other use cases, like rteurning multiple values, and 
dissecting a tuple (you can argue that these are the same case):

    status, result = some_operation()
    name, address = person

Wouter

On 14/03/15 03:45, Paul Rubin wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> A disadvantage [of Lua] is that there is only one number type, which
>> is floating point by default, but you can change that to integer
>
> They finally added integers to Lua, as of version 5.3, released earlier
> this year.
>

Very nice!

And I see that bitwise operators are now official, rather than just a 
common patch.

That pretty much removes the two stumbling points Lua had as an embedded 
language for something like a Cortex M4.

On Sat, 14 Mar 2015 22:20:25 -0700, Paul Rubin
<no.email@nospam.invalid> wrote:

>George Neuner <gneuner2@comcast.net> writes:
>> Javacard also has no GC - applets have to request heap space at
>> startup and manage it (if they are so inclined) themselves.
>
>GC is present in some Javacard devices.

Javacard does not require GC - that's an extension.  The standard
manager allocates but does not recycle until the applet ends.


>Maybe more of a problem is that I don't think Javacard is open source.

Oracle's VM isn't *libre*, but it is an open reference platform. There
are FOSS versions available.


>For simple purposes like UI customization, Scheme is probably ok for
>non-Schemers since they can probably get their stuff working by copying
>and modifying existing examples.  There are also lots of books on
>Scheme.  So it seems worth looking into these if Lua turns out to be too
>large.

I also like Scheme, but it is a non-starter for a lot of people.
Notwithstanding that a lot of scripts could be accomplished by
modifying examples, the prefix notation just seems to turn off many
people.


>I'm a fan of Hedgehog Lisp which has an offline compiler and a very
>compact VM (about 20k), but it's a weird dialect with almost no users.
>Sometimes I think of hacking it to be more compatible with Scheme, and
>maybe adding alternative front end to it.  For a while I thought using
>its VM as a runtime for a Haskell dialect for microcontrollers that I
>wanted to call "Control-H", so that the file extension would be a
>backspace character.  ;-)

I have, on occasion, embedded Guile and mzScheme.  Bare bones, they
are small enough, but useful configurations grow quickly.

George

On Sat, 14 Mar 2015 22:30:30 -0700, Don Y <this@is.not.me.com> wrote:

>On 3/14/2015 8:59 PM, George Neuner wrote:
>
>> But a tuple semantically is different from multiple values.  A tuple
>> is a single container with multiple content - the container must be
>> "opened" to get at its content whereas the individual values
>> (typically) do not.
>
>That's not required in limbo.  E.g., it is common for a function to
>return a tuple:  one member being a flag (success/fail) with the other(s)
>being the expected result(s).

And how do you access those values in the tuple - surely it can't be
the same as values not in the tuple.

Multiple-value return is more complex than single-value return.  Many
languages allow functions to return a container type to preserve the
(relative) simplicity of the single-value return.  Ansi C does this
with structs.

I don't know what Limbo does, but tuple using languages typically
allocate the tuple and its contents separately and then return a
reference to the tuple.  Which goes back to what I said about the cost
of accessing the values.

George

George Neuner <gneuner2@comcast.net> writes:
>>GC is present in some Javacard devices.
> Javacard does not require GC - that's an extension.  

Right, fair enough, but some devices do include it.

> The standard manager allocates but does not recycle until the applet
> ends.

Sort of like Microscheme :)

> Oracle's [J2ME, Javacard?] VM isn't *libre*, but it is an open
> reference platform. There are FOSS versions available.

Oh cool, good to know.

> I also like Scheme, but...the prefix notation just seems to turn off
> many people.  I have, on occasion, embedded Guile and mzScheme.  Bare
> bones, they are small enough, but useful configurations grow quickly.

Hmm, I had been wondering about the memory footprint of Guile.  I know
that it has some front ends for other language syntaxes, particularly
Ecmascript a/k/a Javascript, with Lua supposedly in development.

I spent yesterday evening looking into very small ram-stingy Scheme
implementations and I wonder whether the Guile Ecmascript front end
could be used with them.  But, it probably increases the product's
support burden.

Picobit looks interesting: https://github.com/stamourv/picobit

Absolutely unsuitable for non-nerd user scripting, but maybe of interest
to language geeks: Purescript (purescript.org) is a Haskell dialect that
compiles to Javascript, and could be probably made to generate Scheme
instead.  I think someone is already working on retargeting it to Lua.
But I think the pure functional approach (no mutable data, lots of GC)
isn't a good fit for small embedded systems.