EmbeddedRelated.com
Forums

A real world benchmark + a review

Started by microbit November 17, 2003
Hi All,

Some of you may recall my enthusiasm for Rowley's CrossWorks for MSP430
since quite some time :-)

We all know that some C tool chain companies tend to produce *slightly* biased
benchmarks on their tools,
so I decided to do an actual REAL test, mainly comparing EW430 V2.10 and RAL
CW430 V1.2
(I am in the process of testing on HITECH-C's HITide and compiler V9.00)

I ported a project that was tested on IARV2.10A to CW430 V1.2.
The project involves a BASIC on-line interactive interpreter with an integrated
900 MHz ISM frequency hopping link
with multi-tasking capability. (RFBasic, which we hope to put into relase mode
sonner than later :-)

The C code is IDENTICAL on both platforms, except for syntax in declaring the
INT handlers, and does not
include the RTOS.

The timing test is dictated mainly by the following factors :

-    Floating point and integer performance
-    Procedure calls through arrays of function pointers
-    Switch statement efficiency
-    Stack frame manipulation efficiency
-    Register allocation efficiency
-    Handling linked lists efficiency
-    Array and structure pointer efficiency
-    Mapping of structure array on function stack frame (expression stack)
     and how fast a struct pointer operates on it
-    Casting between 8, 16 or 32 bit integers and floats in both directions

The test involved typing a simple 2 line BASIC program from a Terminal (Telix)
and then typing RUN

The program :

100    FOR X = 1 to 10000000 STEP 10
110    NEXT X

executed in :
- 71 Secs on EW430 V2.10
- 58 Secs on CW430 V1.2

The loop needs 1 Million iterations, 
so Rowley's code took 58 uS per iteration, and IAR's code took 71 uS
per iteration.

IAR was set for highest speed optimisation, Rowley was set optimisation ON.

Rowley's code was about 15% larger than IAR's, but given the
considerable speed increase it
seems insignificant to me.
"Run of the mill" C code tends to pretty much equal IAR's code.


For anyone interested in trying Rowley's product here is my overview of
some of CW430's strongest features :

1. Editor : A true pleasure to work with, smart indent actually means
"smart indent".
The context-sensitive HTML help tremendeously increases productivity and lets me
concentrate where I need to.
The Embedded Workbench Help -> Contents -> bla bla bla gets extremely
tiring after a while - is a nuisance and
a distraction.

2. Editor Clipboard ring : A history is kept of code lines, fragments or blocks
I've deleted.
If I need any of that code again somewhere else while I'm editing, I merely
click "paste" on the snippet and it
lands in my C source window.

3. Docking windows : If I had a dollar for each time I've repositioned a C
source window in EW430 the last 3 years
I'd be a Millionaire now. The workbook mode style on the sources is
infinitely superior.

4. Source navigator : allows me to instantly find "where did I declare this
object again ???"

5. Debugger / Core CPU Simulator : Verified to be cycle-accurate, contains
powerful extensions to bring up Warning/Error
Window Prompts
The symbol browser with its cycle grouping and the source navigator makes for an
extremely powerful debugger.
I've never found my bugs so quick :-)

6. Hardware debugger :
This one is invaluable.
Ever plugged the 14 pin JTAG of the FET tool into a JTAG of a target that's
already powered up, only to endlessly get
"No Target" like errors with your Kickstart ??? (hard-resetting the
target by breaking the LPT connection between FET interface and PC
seems to be the only solution).
It's even more woeful when you have a dongle, my PC would paralyse for
about 60 secs after pressing F9, and then I'd get a license error !!
(which is always _sooo_ nice after paying Thousands of Dollars).

CW430 connects INSTANTLY .. EVERYTIME ... to my targets.
Code downloading is so much faster too. (I haven't tried USB CrossConnect
yet)

There's still so much more, but these is my "review".
Anyone that wonders about this, download the demo and try it yourself !!!


It should be stressed that I have no financial gain whatsoever with any of this,
Rowley associates does not bribe me
or torture me to publish this !!!!!!
I just think more people out there should know that you can get 5 times the
quality for ONE THIRD of the price.


Best regards,
Kris
Microbit Systems 
www.microbit.com.au





Beginning Microcontrollers with the MSP430

microbit wrote:

> Hi All,
> 
> Some of you may recall my enthusiasm for Rowley's CrossWorks for
MSP430 since quite some time :-)
> 
> We all know that some C tool chain companies tend to produce *slightly*
biased benchmarks on their tools,
> so I decided to do an actual REAL test, mainly comparing EW430 V2.10 and
RAL CW430 V1.2
> (I am in the process of testing on HITECH-C's HITide and compiler
V9.00)
> 
> I ported a project that was tested on IARV2.10A to CW430 V1.2.
> The project involves a BASIC on-line interactive interpreter with an
integrated 900 MHz ISM frequency hopping link
> with multi-tasking capability. (RFBasic, which we hope to put into relase
mode sonner than later :-)
> 
> The C code is IDENTICAL on both platforms, except for syntax in declaring
the INT handlers, and does not
> include the RTOS.
> 
> The timing test is dictated mainly by the following factors :
> 
> -    Floating point and integer performance
> -    Procedure calls through arrays of function pointers
> -    Switch statement efficiency
> -    Stack frame manipulation efficiency
> -    Register allocation efficiency
> -    Handling linked lists efficiency
> -    Array and structure pointer efficiency
> -    Mapping of structure array on function stack frame (expression stack)
>      and how fast a struct pointer operates on it
> -    Casting between 8, 16 or 32 bit integers and floats in both directions
> 
> The test involved typing a simple 2 line BASIC program from a Terminal
(Telix) and then typing RUN
> 
> The program :
> 
> 100    FOR X = 1 to 10000000 STEP 10
> 110    NEXT X

Now a really smart compiler would notice that there were 1m iterations 
only, and no body to the code so what is it? a delay? hmm, poor when the 
delay period is a function of the compiler, so what could the compiler 
do ? we could cheat and do 1 - 1m STEP 1 using the constant generator to 
speed things up. It seems irrelevant that this is BASIC, the end 
function w9ould appear a simple software timing loop I assume.

Just stirring the @#$% as usual

Al

> 
> executed in :
> - 71 Secs on EW430 V2.10
> - 58 Secs on CW430 V1.2
> 
> The loop needs 1 Million iterations, 
> so Rowley's code took 58 uS per iteration, and IAR's code took 71
uS per iteration.
> 
> IAR was set for highest speed optimisation, Rowley was set optimisation ON.
> 
> Rowley's code was about 15% larger than IAR's, but given the
considerable speed increase it
> seems insignificant to me.
> "Run of the mill" C code tends to pretty much equal IAR's
code.
> 
> 
> For anyone interested in trying Rowley's product here is my overview
of some of CW430's strongest features :
> 
> 1. Editor : A true pleasure to work with, smart indent actually means
"smart indent".
> The context-sensitive HTML help tremendeously increases productivity and
lets me concentrate where I need to.
> The Embedded Workbench Help -> Contents -> bla bla bla gets extremely
tiring after a while - is a nuisance and
> a distraction.
> 
> 2. Editor Clipboard ring : A history is kept of code lines, fragments or
blocks I've deleted.
> If I need any of that code again somewhere else while I'm editing, I
merely click "paste" on the snippet and it
> lands in my C source window.
> 
> 3. Docking windows : If I had a dollar for each time I've repositioned
a C source window in EW430 the last 3 years
> I'd be a Millionaire now. The workbook mode style on the sources is
infinitely superior.
> 
> 4. Source navigator : allows me to instantly find "where did I declare
this object again ???"
> 
> 5. Debugger / Core CPU Simulator : Verified to be cycle-accurate, contains
powerful extensions to bring up Warning/Error
> Window Prompts
> The symbol browser with its cycle grouping and the source navigator makes
for an extremely powerful debugger.
> I've never found my bugs so quick :-)
> 
> 6. Hardware debugger :
> This one is invaluable.
> Ever plugged the 14 pin JTAG of the FET tool into a JTAG of a target
that's already powered up, only to endlessly get
> "No Target" like errors with your Kickstart ??? (hard-resetting
the target by breaking the LPT connection between FET interface and PC
> seems to be the only solution).
> It's even more woeful when you have a dongle, my PC would paralyse for
about 60 secs after pressing F9, and then I'd get a license error !!
> (which is always _sooo_ nice after paying Thousands of Dollars).
> 
> CW430 connects INSTANTLY .. EVERYTIME ... to my targets.
> Code downloading is so much faster too. (I haven't tried USB
CrossConnect yet)
> 
> There's still so much more, but these is my "review".
> Anyone that wonders about this, download the demo and try it yourself !!!
> 
> 
> It should be stressed that I have no financial gain whatsoever with any of
this, Rowley associates does not bribe me
> or torture me to publish this !!!!!!
> I just think more people out there should know that you can get 5 times the
quality for ONE THIRD of the price.
> 
> 
> Best regards,
> Kris
> Microbit Systems 
> www.microbit.com.au
> 
> 
> 
> 
> 
> 
> .
> 
>  
> 
> ">http://docs.yahoo.com/info/terms/ 
> 
> 
> 


 >  CW430 connects INSTANTLY .. EVERYTIME ... to my targets.
 > Code downloading is so much faster too. (I haven't tried USB
CrossConnect
yet)

With USB CrossConnect, you loose this nice feature.  You have to
"reconnect"
each time.  That'a bummer, but the speed is unmatched, the msp gets
programmed before your finger is off the F5 button!!!
 
 





Hi Al,

I didn't want to elaborate too much on RFBasic, as the product range has
suffered endless distractions.
RFBasic does a LOT more than that !

You can edit BASIC programs by simply connecting say HyperTenminal.
BTW : that for-next loop uses X as a float.
If I used for X% = 1 to 10 ..... It'd be an integer (much faster).

RFBasic is not compiled like Stamp, it is on-the-fly interpreted and stored
in RAM for direct execution
or in Flash if you're editing a program.
The Basic has numerous extensions.
Your average control application via a RF link, or an RF network is written
in 5-10 Minutes
NOT 3 Months !!!
No compilers, no JTAGs, no C, no ASM, no FET I/F

You can also transfer variables from one node to another, transfer programs
between nodes or
even debug a BASIC program through the RF link from any node on another node
!
Whatever you can do during runtime, you can also do directly.
RFBasic can also multiTask up to 4 tasks from within BASIC. (with 16 Events,
and 4 Timers)

Anyway, too much to go deeper.

No worries Al, the first proto boards will have a pair sent to you :-)

> Now a really smart compiler would notice that
there were 1m iterations
> only, and no body to the code so what is it? a delay? hmm, poor when the
> delay period is a function of the compiler, so what could the compiler
> do ? we could cheat and do 1 - 1m STEP 1 using the constant generator to
> speed things up. It seems irrelevant that this is BASIC, the end
> function w9ould appear a simple software timing loop I assume.
>
> Just stirring the @#$% as usual
>
> Al


Hi Omer,

>  >  CW430 connects INSTANTLY .. EVERYTIME ...
to my targets.
>  > Code downloading is so much faster too. (I haven't tried 
> USB CrossConnect
> yet)
> 
> With USB CrossConnect, you loose this nice feature.  You have 
> to "reconnect" each time.  That'a bummer, but the speed is 
> unmatched, the msp gets programmed before your finger is off 
> the F5 button!!!

You shouldn't need to disconnect and reconnect each time.  Are you
physically disconnecting?  If you rework your code and press F5, with a
target connection still in place, the program should be downloaded
without a reconnect needed.

Perhaps we can discuss this offline as it's a bit off topic.  However,
if you have problems, I'd like to get them sorted.  The CrossConnect is
field upgradeable, so it's not a problem to send new firmware.

Regards,

--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for MSP430 and ARM processors 

Sorry if I mislead anyone here.  Once the MSP is *phsyically* disconnected
and reconnected (the same msp) I seem to have to do a "reconnect" on
the
target window.  Whereas, with parallel port FET, I didn't have to.
 
I do not have to do a reconnect if I am just recompiling and re-programming.
 
Regards,
 





Hi Omer,

The latest software has a patch for this behaviour.  It's all to do with
the transients that are presented on the JTAG line when you connect a
FET or CrossConnect to the MSP--you might or noght not go through fuse
check and, depending on whether you did or not, the MSP will either work
or not.

The latest CrossWorks software detects this situation and goes through a
power cycle--it needs to *really* disconnect power from the target for
5s then connect power.  This should fix your problem.  Try RC4 out, from
our website, and see if this improves your situation.  Jon made this
modification.

--
Paul Curtis, Rowley Associates Ltd http://www.rowley.co.uk
CrossWorks for MSP430 and ARM processors 

> -----Original Message-----
> From: Omer YALHI [mailto:oyalhi@oyal...] 
> Sent: 17 November 2003 10:13
> To: msp430@msp4...
> Subject: RE: [msp430] A real world benchmark + a review
> 
> 
> Sorry if I mislead anyone here.  Once the MSP is *phsyically* 
> disconnected and reconnected (the same msp) I seem to have to 
> do a "reconnect" on the target window.  Whereas, with 
> parallel port FET, I didn't have to.
>  
> I do not have to do a reconnect if I am just recompiling and 
> re-programming.
>  
> Regards,
>  
> 
> 
> 
> 
> 
> ------------------------ Yahoo! Groups Sponsor 
> ---------------------~--> Buy Ink Cartridges or Refill Kits 
> for your HP, Epson, Canon or Lexmark Printer at MyInks.com. 
> Free s/h on orders $50 or more to the US & Canada. 
> http://www.c1tracking.com/l.asp?cidU11
> 
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/CFFolB/TM
---------------------------------~->

.

 

">http://docs.yahoo.com/info/terms/ 



Kris,

Thanks for your kind words.  I'll single out just one point, which may
make a difference to some people:

> 6. Hardware debugger :
> This one is invaluable.
> Ever plugged the 14 pin JTAG of the FET tool into a JTAG of a 
> target that's already powered up, only to endlessly get "No 
> Target" like errors with your Kickstart ??? (hard-resetting 
> the target by breaking the LPT connection between FET 
> interface and PC seems to be the only solution). It's even 
> more woeful when you have a dongle, my PC would paralyse for 
> about 60 secs after pressing F9, and then I'd get a license 
> error !! (which is always _sooo_ nice after paying Thousands 
> of Dollars).
> 
> CW430 connects INSTANTLY .. EVERYTIME ... to my targets.
> Code downloading is so much faster too. (I haven't tried USB 
> CrossConnect yet)

The code that drives the FET is now written completely by RAL using our
own driver for the parallel port and our own JTAG code above that.  We
don't use the TI MSP430 DLL to interface to the FET anymore, but you can
still install it and use it if you wish.  We needed to write the code
ourselves so that we could move it onto the USB CrossConnect.  During
the rewrite, we uncovered many things about the TI implementation of
JTAG and the MSP devices themselves, culminating in a stay at TI
Friesing to go through a number of issues we had remaining with one
member of the MSP430 family.

All those issues are now well behind us.  We understand intimately how
the MSP JTAG works and have put that to good use in the CrossConnect and
FET drivers.  The upside is that the FET software now downloads faster
and is much more reliable.

Getting a USB device to this state has cost a fair amount of time and
money, but I don't think that anybody could now doubt our commitment to
the MSP430.

Regards,

-- Paul.

microbit wrote:
> Hi Al,
> 
> I didn't want to elaborate too much on RFBasic, as the product range
has
> suffered endless distractions.
> RFBasic does a LOT more than that !
> 
> You can edit BASIC programs by simply connecting say HyperTenminal.
> BTW : that for-next loop uses X as a float.
> If I used for X% = 1 to 10 ..... It'd be an integer (much faster).

I was thinking more of the compiler at the end of your interpreter, 
since the effective function of this loop is to delay. There is no 
active code, and X is effectively local

> 
> RFBasic is not compiled like Stamp, it is on-the-fly interpreted and stored
> in RAM for direct execution
> or in Flash if you're editing a program.

Why have you chosen an interpreted BASIC instead of one of the more 
modern, structured compiled basics? Just curious. I was having a 
conversation here recently where I stated that I thought there would ba 
a very strong market for an MSP with a compiled basic built in 
(somethign guttsier than the STAMP.

> The Basic has numerous extensions.
> Your average control application via a RF link, or an RF network is written
> in 5-10 Minutes
> NOT 3 Months !!!

Who takes 3 months to write remote control applications ? Sack them NOW!!

> No compilers, no JTAGs, no C, no ASM, no FET I/F
> 
> You can also transfer variables from one node to another, transfer programs
> between nodes or
> even debug a BASIC program through the RF link from any node on another
node
> !
> Whatever you can do during runtime, you can also do directly.
> RFBasic can also multiTask up to 4 tasks from within BASIC. (with 16
Events,
> and 4 Timers)
> 
> Anyway, too much to go deeper.
> 
> No worries Al, the first proto boards will have a pair sent to you :-)

Thanks for that. Always appreciated.

Cheers

Al


At 09:22 PM 11/17/2003 +1030, onestone wrote:
>...Why have you chosen an interpreted BASIC instead
of one of the more
>modern, structured compiled basics? Just curious. I was having a
>conversation here recently where I stated that I thought there would ba
>a very strong market for an MSP with a compiled basic built in
>(somethign guttsier than the STAMP.
>...

My guess is that it is easier to write a BASIC interpreter. Any case, when 
you said Compiled BASIC, do you mean truly compiled to native code, or to 
some sort of token code, ala STAMP?


// richard (This email is for mailing lists. To reach me directly, please 
use richard@rich...)