Which scripting for embedded automation?

I was automating my home LAN.
Need suggestion for automation scripting upon embedded Linux, to do some operations over LAN, such as scan and ping other devices, string like data transfer,etc. And there is need of some logic to parse received content. Perl, python, lua or simple shell scripting .. which one? There will be branch and loop operations, better to have more ready tools for network operations.

Parser and LAN are the key here.

abhishekkumardwivedi@gmail.com wrote:
> I was automating my home LAN. Need suggestion for automation
> scripting upon embedded Linux, to do some operations over LAN, such
> as scan and ping other devices, string like data transfer,etc. And
> there is need of some logic to parse received content. Perl, python,
> lua or simple shell scripting .. which one? There will be branch and
> loop operations, better to have more ready tools for network
> operations.
>
> Parser and LAN are the key here.
>

You can use any of those. I find Tcl still to be less cost
than the others, but I'm already past the learning curve.

If you tackle Tcl, you need the Brent Welch book.

Here is a rough schematic of a "ping" thing.


set pingresults ""

proc pingparser { s } {
     # will only partially wotk - proof left as exercise....
     if {[string first "eceive" $s] != -1} {
     	puts "got it <$s>"
     }
     variable pingresults
     set pingresults [ set pingresults ]\n[set s]
}

proc rd { args } {
	variable s
	if {[eof $s]} { puts "oh dear.\n" ; set ::FLAG 1  }
	set ib [ read $s ]
         #does not handle fragmentation.
         foreach a [ split $ib \n ] {
             pingparser $a
         }
}

set IP 127.0.0.1
set s [ open "| ping -n 3 -t 1 $IP " "r" ]
# may or may not want to use "-translation crlf"
fconfigure $s -translation binary -blocking 0 -buffering 1
fileevent $s readable ::rd

# enter the event loop
vwait ::FLAG
puts "Ping results for $IP are:"
puts $pingresults


-- END

A comparable Python program will be much larger.

There is also Expect but I use this sort of pattern in Tcl instead.

-- 
Les Cargill

On 7/23/2015 2:24 AM, abhishekkumardwivedi@gmail.com wrote:
> I was automating my home LAN. Need suggestion for automation scripting upon
> embedded Linux, to do some operations over LAN, such as scan and ping other
> devices, string like data transfer,etc. And there is need of some logic to
> parse received content. Perl, python, lua or simple shell scripting .. which
> one? There will be branch and loop operations, better to have more ready
> tools for network operations..
>
> Parser and LAN are the key here.

It depends on how many "other" tools you can call on to do any of the
"heavy-lifting" that you may require.

I've adopted Inferno/Limbo for use in my automation system.  It lets me
build real *applets* instead of just crude scripts.  So, for example,
my irrigation controller can be *coded* in Limbo -- instead of writing
something in, for example, C and dispatching that on a processor.

So, I can have a bunch of separate "processes" (multiple instances of
the same job spawned *in* the Limbo implementation) created to handle
each irrigation zone without being aware of the existence of the other
zones nor their current activities.

Each "job" queries the database service to determine the criteria for
it's irrigation zone.  It consults other services to qualify that
criteria (e.g., "how much rain have we had since I last irrigated *this*
zone?").  If it decides that irrigation is necessary, it makes a request
for some portion of the water resource (e.g., I can't turn on EVERY
irrigation zone concurrently as the supply wouldn't meet the load;  or,
there may be rules in place that require access to that resource to be
deferred:  "don't water during daylight hours", etc.).

[Think of this as requesting a portion of a LIMITED network bandwidth
from a daemon that allocates that bandwidth to clients based on preestablished
criteria]

Eventually, the daemon that administers the water supply will ensure the
master irrigation valve is open (else no water flows to the irrigation
valve manifolds) and grant the request to whichever client(s) it deems
appropriate.  It may notice that there is no water pressure from the
upstream supply (municipal water or harvested water stored on the premises)
and react accordingly.

*When* the resource is made available, it can meter the desired amount
of water to its zone.  If the resource is withdrawn for some reason
while being used, it can actively recover (re-request the resource,
throw an alarm, etc.) on behalf of the things that it is responsible
for irrigating.  And, when complete, it can signal the water daemon
of this fact (release it's allocation of that resource), close the
valve for this zone and update the database to record its actions.

Eventually, the water daemon will notice that all clients have released
their allocations and close the main irrigation valve.  It will then
signal appropriate zone controllers to briefly *open* their irrigation
valves to relieve any pressure in the irrigation line and ensure there
is no water standing in the lines above ground level (freeze protection).

With *just* a scripting language, much of this gets tedious to implement
(asynchronous comms between jobs, DBMS access, timers/timeouts, etc.).
So, you end up having to implement some of it in a HLL and some in a
scripting language -- worst of both worlds (especially if you want
folks to be able to write scripts like that without forcing them to
learn those languages).

http://www.vitanuovo.com

On 7/23/2015 5:24 AM, abhishekkumardwivedi@gmail.com wrote:
> I was automating my home LAN.
> Need suggestion for automation scripting upon embedded Linux, to do some operations over LAN, such as scan and ping other devices, string like data transfer,etc. And there is need of some logic to parse received content. Perl, python, lua or simple shell scripting .. which one? There will be branch and loop operations, better to have more ready tools for network operations..
>
> Parser and LAN are the key here.

The obvious choice for this is Forth.  You can commercial products from 
Forth, Inc in the US or MPE in Europe.  A good open source product is 
Gforth among many others.

-- 

Rick

abhishekkumardwivedi@gmail.com writes:
> like data transfer,etc. And there is need of some logic to parse
> received content. Perl, python, lua or simple shell scripting .. which
> one? There will be branch and loop operations, better to have more
> ready tools for network operations... Parser and LAN are the key here.

I'd suggest Python of the ones you mention, assuming you have enough ram
(a Raspberry pi would be more than plenty).  I use it all the time
myself.  Or if you want something really different and interesting, look
at Erlang.  

Perl is old-school by now, and shell scripts too inflexible.  Lua is
very elegantly implemented and compact but it lacks Python's creature
comforts.  Forth is likely to be too low level for this sort of
programming.  It's virtue is being able to run in very small memory
space, like under 10K of code space and a few hundred ram bytes in an
MCU.  But it makes programming very low level and painstaking, almost
like assembler.

On 7/24/2015 1:00 AM, Paul Rubin wrote:
> abhishekkumardwivedi@gmail.com writes:
>> like data transfer,etc. And there is need of some logic to parse
>> received content. Perl, python, lua or simple shell scripting .. which
>> one? There will be branch and loop operations, better to have more
>> ready tools for network operations... Parser and LAN are the key here.
>
> I'd suggest Python of the ones you mention, assuming you have enough ram
> (a Raspberry pi would be more than plenty).  I use it all the time
> myself.  Or if you want something really different and interesting, look
> at Erlang.
>
> Perl is old-school by now, and shell scripts too inflexible.  Lua is
> very elegantly implemented and compact but it lacks Python's creature
> comforts.  Forth is likely to be too low level for this sort of
> programming.  It's virtue is being able to run in very small memory
> space, like under 10K of code space and a few hundred ram bytes in an
> MCU.  But it makes programming very low level and painstaking, almost
> like assembler.

I find your analysis of Forth to be very naive.  Much the same as other 
languages like C, Forth provides a basic capability with more advanced 
capabilities provided through libraries.  Suggesting that Forth is only 
useful where the entire development system needs to reside in 10 KB of 
code is a red herring.  This capability comes with limitations and so is 
only used where it is a significant advantage.  Otherwise Forth can be 
used to advantage in the same environments where you would use C, Python 
or any others.  It typically comes with exactly the sort of capabilities 
needed for this project.  A parser is built in with full capability of 
being used in the application and most Forths running under an OS 
support networking.

Referring to Forth as being "like assembler" just shows how little you 
understand the language and appreciate its features.

-- 

Rick

rickman <gnuarm@gmail.com> writes:
[Ok, replying to your crosspost even though my own earlier post was only
on comp.arch.embedded where the original question was.  The question
regards scripting home automation stuff on an embedded Linux system].

>> Forth is likely to be too low level for this sort of programming.
>> It's virtue is being able to run in very small memory space, like
>> under 10K of code space and a few hundred ram bytes in an MCU.  
> I find your analysis of Forth to be very naive.  Much the same as
> other languages like C, Forth provides a basic capability with more
> advanced capabilities provided through libraries.  

Sure, I'd consider C to also be low level though.  I'd have to add that
Forth doesn't have nearly as many available libraries, because of the
smaller userbase, but I think partly from a minimalist "do everything
yourself" spirit in Forth culture.  

> Suggesting that Forth is only useful where the entire development
> system needs to reside in 10 KB of code is a red herring. 

I didn't mean to suggest a 10K maximum.  I meant Forth had the
capability of running in such small space and that this is IMHO a big
achievement and advantage that it has, but it dictated a lot of Forth's
design decisions.  Historically Forth was most successful in the era of
16-bit systems with up to 64k ram, or maybe on MSDOS PC's with up to
640k.  Once you have tens of MB available the tradeoffs in scripting
become a lot different.

Other Forth advantages include pauseless operation (necessary for
real-time programs), higher cpu efficiency than Python even in simple
interpreters, and very high efficiency (comparable to C) if you use an
optimizing compiler.  But, in the context of scripting, these are
usually not important.  Python burns more machine resources, but in
return it takes care of a lot of issues for you that you'd have to spend
your own time looking after in Forth or C.

> Otherwise Forth can be used to advantage in the same environments
> where you would use C, Python or any others.  

I didn't say anything about C and I'd consider it comparable to Forth in
most regards.  C would IMHO be a poor choice of scripting language in
this situation too.

Have you actually used Python?  It's not anything like C or Forth.
Lumping Python with C seems to miss the distinction.  If you haven't
actually used both, then making comparisons between them isn't
convincing.

> It typically comes with exactly the sort of capabilities needed for
> this project.  A parser is built in with full capability of being used
> in the application and most Forths running under an OS support
> networking.

If the OP has control over the network protocol then using the Forth
parser can be a simple and efficient solution, especially if the network
devices are all trusted.  If there are devices speaking protocols like
XML (some home automation stuff is like that) then Forth starts to be
painful.  I don't know if a Forth TLS stack even exists.

> Referring to Forth as being "like assembler" just shows how little you
> understand the language and appreciate its features.

I understand Forth pretty well at a technical level, though I'm not a
very fluent user in terms of being used to the idioms it takes to use it
productively.

On 7/23/2015 3:08 PM, rickman wrote:
> On 7/23/2015 5:24 AM, abhishekkumardwivedi@gmail.com wrote:
>> I was automating my home LAN.
>> Need suggestion for automation scripting upon embedded Linux, to do
>> some operations over LAN, such as scan and ping other devices, string
>> like data transfer,etc. And there is need of some logic to parse
>> received content. Perl, python, lua or simple shell scripting .. which
>> one? There will be branch and loop operations, better to have more
>> ready tools for network operations..
>>
>> Parser and LAN are the key here.
>
> The obvious choice for this is Forth.  You can commercial products from
> Forth, Inc in the US or MPE in Europe.  A good open source product is
> Gforth among many others.

I cross posted this here so the OP could get some insight into the Forth 
community.  However, many of the replies are only to the Forth group 
where they will never be seen by the OP.

-- 

Rick

Paul Rubin wrote:

> > Otherwise Forth can be used to advantage in the same environments
> > where you would use C, Python or any others.  
> 
> I didn't say anything about C and I'd consider it comparable to Forth in
> most regards.  C would IMHO be a poor choice of scripting language in
> this situation too.
> 
> Have you actually used Python?  It's not anything like C or Forth.
> Lumping Python with C seems to miss the distinction.  If you haven't
> actually used both, then making comparisons between them isn't
> convincing.

It's obvious that the reason that he is recommending Forth
over all other languages is that he knows no language that
is higher-level than Forth.  And he finds it too difficult
to learn another language.

If a hammer is the only tool that you have, then you use
a hammer for everything.

How would these tasks be done with a hammer?

Using Gauche Scheme:

Write a function that accepts a vector (1-dimensional array) of
16-bit numbers and replaces each number with its difference
from the average.

(use gauche.uvector)
(use gauche.collection)

(define (relativize vec)
  (let ((avg (round->exact (/ (fold + 0 vec) (s16vector-length vec)))))
    (s16vector-sub! vec avg)))

(define my-vec (s16vector 0 1 2 3 4 5 6 7 8 9))
(relativize my-vec)
my-vec
 ===>
#s16(-4 -3 -2 -1 0 1 2 3 4 5)


Take a string that contains numbers separated by semicolons or
commas and convert it to a list of numbers.

(map string->number (string-split "2,3.14;5.8,9" #/[,;]/))
 ===>
(2 3.14 5.8 9)

-- 
The report card by the American Society of Civil Engineers showed the national
infrastructure a single grade above failure, a step from declining to the point
where everyday things simply stop working the way people expect them to. ---
washingtonpost.com/local/trafficandcommuting/us-infrastructure-gets-d-in-annual-report/2013/03/19/c48cb010-900b-11e2-9cfd-36d6c9b5d7ad_story.html

On 7/24/2015 4:06 PM, WJ wrote:

> It's obvious that the reason that he is recommending Forth
> over all other languages is that he knows no language that
> is higher-level than Forth.  And he finds it too difficult
> to learn another language.
>
> If a hammer is the only tool that you have, then you use
> a hammer for everything.
>
> How would these tasks be done with a hammer?
>
> Using Gauche Scheme:
>
> Write a function that accepts a vector (1-dimensional array) of
> 16-bit numbers and replaces each number with its difference
> from the average.
>
> (use gauche.uvector)
> (use gauche.collection)
>
> (define (relativize vec)
>    (let ((avg (round->exact (/ (fold + 0 vec) (s16vector-length vec)))))
>      (s16vector-sub! vec avg)))
>
> (define my-vec (s16vector 0 1 2 3 4 5 6 7 8 9))
> (relativize my-vec)
> my-vec
>   ===>
> #s16(-4 -3 -2 -1 0 1 2 3 4 5)
>
>
> Take a string that contains numbers separated by semicolons or
> commas and convert it to a list of numbers.
>
> (map string->number (string-split "2,3.14;5.8,9" #/[,;]/))
>   ===>
> (2 3.14 5.8 9)

What role would these have in the scripting of an automation system?
IME, tasks in that domain tend to be more procedural oriented.