Forums

Which scripting for embedded automation?

Started by Unknown July 23, 2015
I was automating my home LAN.
Need suggestion for automation scripting upon embedded Linux, to do some operations over LAN, such as scan and ping other devices, string like data transfer,etc. And there is need of some logic to parse received content. Perl, python, lua or simple shell scripting .. which one? There will be branch and loop operations, better to have more ready tools for network operations.

Parser and LAN are the key here.
abhishekkumardwivedi@gmail.com wrote:
> I was automating my home LAN. Need suggestion for automation > scripting upon embedded Linux, to do some operations over LAN, such > as scan and ping other devices, string like data transfer,etc. And > there is need of some logic to parse received content. Perl, python, > lua or simple shell scripting .. which one? There will be branch and > loop operations, better to have more ready tools for network > operations. > > Parser and LAN are the key here. >
You can use any of those. I find Tcl still to be less cost than the others, but I'm already past the learning curve. If you tackle Tcl, you need the Brent Welch book. Here is a rough schematic of a "ping" thing. set pingresults "" proc pingparser { s } { # will only partially wotk - proof left as exercise.... if {[string first "eceive" $s] != -1} { puts "got it <$s>" } variable pingresults set pingresults [ set pingresults ]\n[set s] } proc rd { args } { variable s if {[eof $s]} { puts "oh dear.\n" ; set ::FLAG 1 } set ib [ read $s ] #does not handle fragmentation. foreach a [ split $ib \n ] { pingparser $a } } set IP 127.0.0.1 set s [ open "| ping -n 3 -t 1 $IP " "r" ] # may or may not want to use "-translation crlf" fconfigure $s -translation binary -blocking 0 -buffering 1 fileevent $s readable ::rd # enter the event loop vwait ::FLAG puts "Ping results for $IP are:" puts $pingresults -- END A comparable Python program will be much larger. There is also Expect but I use this sort of pattern in Tcl instead. -- Les Cargill
On 7/23/2015 2:24 AM, abhishekkumardwivedi@gmail.com wrote:
> I was automating my home LAN. Need suggestion for automation scripting upon > embedded Linux, to do some operations over LAN, such as scan and ping other > devices, string like data transfer,etc. And there is need of some logic to > parse received content. Perl, python, lua or simple shell scripting .. which > one? There will be branch and loop operations, better to have more ready > tools for network operations.. > > Parser and LAN are the key here.
It depends on how many "other" tools you can call on to do any of the "heavy-lifting" that you may require. I've adopted Inferno/Limbo for use in my automation system. It lets me build real *applets* instead of just crude scripts. So, for example, my irrigation controller can be *coded* in Limbo -- instead of writing something in, for example, C and dispatching that on a processor. So, I can have a bunch of separate "processes" (multiple instances of the same job spawned *in* the Limbo implementation) created to handle each irrigation zone without being aware of the existence of the other zones nor their current activities. Each "job" queries the database service to determine the criteria for it's irrigation zone. It consults other services to qualify that criteria (e.g., "how much rain have we had since I last irrigated *this* zone?"). If it decides that irrigation is necessary, it makes a request for some portion of the water resource (e.g., I can't turn on EVERY irrigation zone concurrently as the supply wouldn't meet the load; or, there may be rules in place that require access to that resource to be deferred: "don't water during daylight hours", etc.). [Think of this as requesting a portion of a LIMITED network bandwidth from a daemon that allocates that bandwidth to clients based on preestablished criteria] Eventually, the daemon that administers the water supply will ensure the master irrigation valve is open (else no water flows to the irrigation valve manifolds) and grant the request to whichever client(s) it deems appropriate. It may notice that there is no water pressure from the upstream supply (municipal water or harvested water stored on the premises) and react accordingly. *When* the resource is made available, it can meter the desired amount of water to its zone. If the resource is withdrawn for some reason while being used, it can actively recover (re-request the resource, throw an alarm, etc.) on behalf of the things that it is responsible for irrigating. And, when complete, it can signal the water daemon of this fact (release it's allocation of that resource), close the valve for this zone and update the database to record its actions. Eventually, the water daemon will notice that all clients have released their allocations and close the main irrigation valve. It will then signal appropriate zone controllers to briefly *open* their irrigation valves to relieve any pressure in the irrigation line and ensure there is no water standing in the lines above ground level (freeze protection). With *just* a scripting language, much of this gets tedious to implement (asynchronous comms between jobs, DBMS access, timers/timeouts, etc.). So, you end up having to implement some of it in a HLL and some in a scripting language -- worst of both worlds (especially if you want folks to be able to write scripts like that without forcing them to learn those languages). http://www.vitanuovo.com
On 7/23/2015 5:24 AM, abhishekkumardwivedi@gmail.com wrote:
> I was automating my home LAN. > Need suggestion for automation scripting upon embedded Linux, to do some operations over LAN, such as scan and ping other devices, string like data transfer,etc. And there is need of some logic to parse received content. Perl, python, lua or simple shell scripting .. which one? There will be branch and loop operations, better to have more ready tools for network operations.. > > Parser and LAN are the key here.
The obvious choice for this is Forth. You can commercial products from Forth, Inc in the US or MPE in Europe. A good open source product is Gforth among many others. -- Rick
abhishekkumardwivedi@gmail.com writes:
> like data transfer,etc. And there is need of some logic to parse > received content. Perl, python, lua or simple shell scripting .. which > one? There will be branch and loop operations, better to have more > ready tools for network operations... Parser and LAN are the key here.
I'd suggest Python of the ones you mention, assuming you have enough ram (a Raspberry pi would be more than plenty). I use it all the time myself. Or if you want something really different and interesting, look at Erlang. Perl is old-school by now, and shell scripts too inflexible. Lua is very elegantly implemented and compact but it lacks Python's creature comforts. Forth is likely to be too low level for this sort of programming. It's virtue is being able to run in very small memory space, like under 10K of code space and a few hundred ram bytes in an MCU. But it makes programming very low level and painstaking, almost like assembler.
On 7/24/2015 1:00 AM, Paul Rubin wrote:
> abhishekkumardwivedi@gmail.com writes: >> like data transfer,etc. And there is need of some logic to parse >> received content. Perl, python, lua or simple shell scripting .. which >> one? There will be branch and loop operations, better to have more >> ready tools for network operations... Parser and LAN are the key here. > > I'd suggest Python of the ones you mention, assuming you have enough ram > (a Raspberry pi would be more than plenty). I use it all the time > myself. Or if you want something really different and interesting, look > at Erlang. > > Perl is old-school by now, and shell scripts too inflexible. Lua is > very elegantly implemented and compact but it lacks Python's creature > comforts. Forth is likely to be too low level for this sort of > programming. It's virtue is being able to run in very small memory > space, like under 10K of code space and a few hundred ram bytes in an > MCU. But it makes programming very low level and painstaking, almost > like assembler.
I find your analysis of Forth to be very naive. Much the same as other languages like C, Forth provides a basic capability with more advanced capabilities provided through libraries. Suggesting that Forth is only useful where the entire development system needs to reside in 10 KB of code is a red herring. This capability comes with limitations and so is only used where it is a significant advantage. Otherwise Forth can be used to advantage in the same environments where you would use C, Python or any others. It typically comes with exactly the sort of capabilities needed for this project. A parser is built in with full capability of being used in the application and most Forths running under an OS support networking. Referring to Forth as being "like assembler" just shows how little you understand the language and appreciate its features. -- Rick
rickman <gnuarm@gmail.com> writes:
[Ok, replying to your crosspost even though my own earlier post was only
on comp.arch.embedded where the original question was.  The question
regards scripting home automation stuff on an embedded Linux system].

>> Forth is likely to be too low level for this sort of programming. >> It's virtue is being able to run in very small memory space, like >> under 10K of code space and a few hundred ram bytes in an MCU. > I find your analysis of Forth to be very naive. Much the same as > other languages like C, Forth provides a basic capability with more > advanced capabilities provided through libraries.
Sure, I'd consider C to also be low level though. I'd have to add that Forth doesn't have nearly as many available libraries, because of the smaller userbase, but I think partly from a minimalist "do everything yourself" spirit in Forth culture.
> Suggesting that Forth is only useful where the entire development > system needs to reside in 10 KB of code is a red herring.
I didn't mean to suggest a 10K maximum. I meant Forth had the capability of running in such small space and that this is IMHO a big achievement and advantage that it has, but it dictated a lot of Forth's design decisions. Historically Forth was most successful in the era of 16-bit systems with up to 64k ram, or maybe on MSDOS PC's with up to 640k. Once you have tens of MB available the tradeoffs in scripting become a lot different. Other Forth advantages include pauseless operation (necessary for real-time programs), higher cpu efficiency than Python even in simple interpreters, and very high efficiency (comparable to C) if you use an optimizing compiler. But, in the context of scripting, these are usually not important. Python burns more machine resources, but in return it takes care of a lot of issues for you that you'd have to spend your own time looking after in Forth or C.
> Otherwise Forth can be used to advantage in the same environments > where you would use C, Python or any others.
I didn't say anything about C and I'd consider it comparable to Forth in most regards. C would IMHO be a poor choice of scripting language in this situation too. Have you actually used Python? It's not anything like C or Forth. Lumping Python with C seems to miss the distinction. If you haven't actually used both, then making comparisons between them isn't convincing.
> It typically comes with exactly the sort of capabilities needed for > this project. A parser is built in with full capability of being used > in the application and most Forths running under an OS support > networking.
If the OP has control over the network protocol then using the Forth parser can be a simple and efficient solution, especially if the network devices are all trusted. If there are devices speaking protocols like XML (some home automation stuff is like that) then Forth starts to be painful. I don't know if a Forth TLS stack even exists.
> Referring to Forth as being "like assembler" just shows how little you > understand the language and appreciate its features.
I understand Forth pretty well at a technical level, though I'm not a very fluent user in terms of being used to the idioms it takes to use it productively.
On 7/23/2015 3:08 PM, rickman wrote:
> On 7/23/2015 5:24 AM, abhishekkumardwivedi@gmail.com wrote: >> I was automating my home LAN. >> Need suggestion for automation scripting upon embedded Linux, to do >> some operations over LAN, such as scan and ping other devices, string >> like data transfer,etc. And there is need of some logic to parse >> received content. Perl, python, lua or simple shell scripting .. which >> one? There will be branch and loop operations, better to have more >> ready tools for network operations.. >> >> Parser and LAN are the key here. > > The obvious choice for this is Forth. You can commercial products from > Forth, Inc in the US or MPE in Europe. A good open source product is > Gforth among many others.
I cross posted this here so the OP could get some insight into the Forth community. However, many of the replies are only to the Forth group where they will never be seen by the OP. -- Rick
Paul Rubin wrote:

> > Otherwise Forth can be used to advantage in the same environments > > where you would use C, Python or any others. > > I didn't say anything about C and I'd consider it comparable to Forth in > most regards. C would IMHO be a poor choice of scripting language in > this situation too. > > Have you actually used Python? It's not anything like C or Forth. > Lumping Python with C seems to miss the distinction. If you haven't > actually used both, then making comparisons between them isn't > convincing.
It's obvious that the reason that he is recommending Forth over all other languages is that he knows no language that is higher-level than Forth. And he finds it too difficult to learn another language. If a hammer is the only tool that you have, then you use a hammer for everything. How would these tasks be done with a hammer? Using Gauche Scheme: Write a function that accepts a vector (1-dimensional array) of 16-bit numbers and replaces each number with its difference from the average. (use gauche.uvector) (use gauche.collection) (define (relativize vec) (let ((avg (round->exact (/ (fold + 0 vec) (s16vector-length vec))))) (s16vector-sub! vec avg))) (define my-vec (s16vector 0 1 2 3 4 5 6 7 8 9)) (relativize my-vec) my-vec ===> #s16(-4 -3 -2 -1 0 1 2 3 4 5) Take a string that contains numbers separated by semicolons or commas and convert it to a list of numbers. (map string->number (string-split "2,3.14;5.8,9" #/[,;]/)) ===> (2 3.14 5.8 9) -- The report card by the American Society of Civil Engineers showed the national infrastructure a single grade above failure, a step from declining to the point where everyday things simply stop working the way people expect them to. --- washingtonpost.com/local/trafficandcommuting/us-infrastructure-gets-d-in-annual-report/2013/03/19/c48cb010-900b-11e2-9cfd-36d6c9b5d7ad_story.html
On 7/24/2015 4:06 PM, WJ wrote:

> It's obvious that the reason that he is recommending Forth > over all other languages is that he knows no language that > is higher-level than Forth. And he finds it too difficult > to learn another language. > > If a hammer is the only tool that you have, then you use > a hammer for everything. > > How would these tasks be done with a hammer? > > Using Gauche Scheme: > > Write a function that accepts a vector (1-dimensional array) of > 16-bit numbers and replaces each number with its difference > from the average. > > (use gauche.uvector) > (use gauche.collection) > > (define (relativize vec) > (let ((avg (round->exact (/ (fold + 0 vec) (s16vector-length vec))))) > (s16vector-sub! vec avg))) > > (define my-vec (s16vector 0 1 2 3 4 5 6 7 8 9)) > (relativize my-vec) > my-vec > ===> > #s16(-4 -3 -2 -1 0 1 2 3 4 5) > > > Take a string that contains numbers separated by semicolons or > commas and convert it to a list of numbers. > > (map string->number (string-split "2,3.14;5.8,9" #/[,;]/)) > ===> > (2 3.14 5.8 9)
What role would these have in the scripting of an automation system? IME, tasks in that domain tend to be more procedural oriented.