In article <454B6A26.BD824095@yahoo.com>, CBFalconer 
<cbfalconer@yahoo.com> writes
>David Brown wrote:
>> Chris Hills wrote:
>>> Paul Curtis <plc@rowley.co.uk> writes
>>>
>... snip ...
>>>>
>>>> You can't argue with an BSI or ISO test certificate, no.  Can
>>>> you point me to such a certificate for the compilers you
>>>> advocate?  One of them?
>>>
>>> You can start with the Plum Hall and Perennial test suites?
>>
>> I tried a quick google for "Plum Hall IAR", and looked at the
>> first link:
>>
>>   http://www.plumhall.com/cqs/tr/trackrecord.html
>>
>> It would appear that IAR is a customer of Plum Hall, as is Red Hat.
>> That by no means implies that gcc is certified by Plum Hall, but it
>> does look like they take test suite testing seriously.
>>
>> Another interesting find from google:
>>
>>   http://gcc.gnu.org/ml/gcc/1998-07/msg00656.html
>>
>>> Off line I should be interested in a serious discussion on compiler
>>> testing.
>
>The problem with the gcc test suite is that it is geared to the gcc
>'standard', rather than the ISO standard.

Correct. Though as most compilers have extensions this is not a problem 
of itself.

> A test suite should be
>open-source,

Why?
Are you going to work for free and make it available to all the 
commercial compiler companies freely as well?

> and there should probably be several of them (with
>some commonality), one each for C90, C99, and C0X.  The tests
>should be clearly tied to the standard.

Yes.

>  There are several classes
>of tests needed, i.e. conformance, detection of errors, and

Yes. (So far AFAIK this is how al the test suites work)

>quality.  The last will be controversial.

And impossible. (Define Quality in less than 10000 words :-)  But I 
think I know what you mean. It is a nice goal.

>The unfortunate experience with the Pascal test-suite is a
>warning.  This was excellent, but handed over to some British firm
>(I forget the name) and basically lost.  It included portable means
>of selecting individual tests, and of running an overall check.
>Thus GPL licencing is essential.


>The results should be very useful, even when systems fail.

They are often most useful when things fail

>.  Maybe a sourceforge
>project would be appropriate?

Why sourceforge?

>  Such a suite would obviously be poor
>at the start, but should mature.  The first task would be a method
>of organizing the tests.  The Pascal suite is an excellent
>example.

SO who is going to do all this work free of charge?

>There should be no pressure to include C++ in the
>testing, that is an entirely separate issue (and language).

I agree. C !=  C++


-- 
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England     /\/\/\/\/
/\/\/ chris@phaedsys.org      www.phaedsys.org \/\/\
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

Greetings,

CBFalconer wrote:
> Paul Keinanen wrote:
> 
>>"Leon" <leon.heller@bulldoghome.com> wrote:
>>
>>>Walter Banks wrote:
>>>
>>>>CBFalconer wrote:
>>>>
>>>>
>>>>>For example, where a compiler is aimed at a PIC many things are
>>>>>simply not feasible.
>>>>
>>>>I am curious, what for example?
>>>
>>>Some PICs only have three levels of stack.
>>
>>The only problem I can think of is saving the (code space) return
>>address into the data address space RAM.
>>
>>Usually even Harvard architecture have some means of transferring
>>data between code and data space, if these PICs do not have such
>>feature, implementing C would be problematic. As long as an
>>indirect jump can be performed, the situation is not hopeless.
> 
> 
> Not a PIC.  The only access to the return stack is the return
> instruction.  As I set, not feasible.  And there is not enough
> memory to allow creating an interpreter.

Indirect calls and/or jumps are trivial on the 16 (& I think 12 & 14) 
series so I don't see the problem. It's also fairly trivial to ignore 
the call/return stack by passing the return address as a hidden 
parametert to a function.

About the only problem I see is with only a single indirect register 
allowing recursion is difficult.

> 


-- 
Kyle A. York
Sr. Subordinate Grunt
DSBU

Chris Hills wrote:
> In article <454b5221$0$8092$8404b019@news.wineasy.se>, David Brown 
> <david@westcontrol.removethisbit.com> writes
>> Walter Banks wrote:
>>>  David Brown wrote:
>>>
>>>> Another interesting find from google:
>>>>
>>>> http://gcc.gnu.org/ml/gcc/1998-07/msg00656.html
>>>  Interesting comments. Standard test suites are an eyopener
>>> when first run on any compiler for the first time. They find
>>> a lot of things, most come as a complete surprise.
>>>  w..
>>>
>>
>> The mail linked above is from 1998 - I have no idea how much test 
>> suite testing has been done on more modern gcc versions,
> 
> The problem with GCC is which version. You can not test Gcc per say. 
> Everyone's version is different. Also as it is supplied in source code 
> form you can only apply the test results to the actual binary you tested.
> 
> For commercial compilers this is not a problem as they distribute only 
> the binary and for any given compiler there is only one binary.
> 

You are exaggerating the differences in an attempt to make your point - 
it makes you sound fanatical.  I agree with many of the *real* reasons 
why commercial compilers are often a better choice for professional use 
than gcc, but your hand-waving and generalised comments lead "best tool 
for the job" advocates like myself to react against you.

Like most compilers, gcc regularly gets new versions released.  Before a 
version is released, it undergoes extensive testing for all the 
supported hosts, all the supported targets (but not every combination of 
both), and is run through its own "torture tests".  It is tried out by 
developers and users in a huge range of areas - far more widespread than 
any other compiler.  When the gcc managers are happy, the source is 
packed up, given a new version number, and released.  And as with any 
other compiler, bug fixes or updates are released as and when required.

What happens after the source release may vary.  Some people may take 
the source code, and patch it - it is then not exactly the same 
compiler.  But if you take the same source and compile it on an x86 
machine using Intel's compiler and on a PowerMac using gcc, using the 
same target configuration, you will have the same compiler.  If you take 
the same target source code and compile on each machine with the same 
flags, you will get the same binary.

Now, while the above is true and satisfactory for engineers, it won't 
satisfy PHBs and lawyers.  So if you want to use Plum Hall (or whatever) 
testing on your compiler, you'll have to re-test for each binary you 
build, or you will insist that whoever supplies your commercial support 
for gcc tests each binary they sell you.

Where exactly is that different from a commercial supplier of 
closed-source compilers?  If IAR releases version 7.00 of their 
compiler, and use Plum Hall for compliance testing, then they will run 
Plum Hall on that compiler.  If they release a service pack for version 
7.01, then they will have to test that too.  There is no difference, 
except that if you are building gcc yourself you must do your own 
testing.  If you are getting gcc in pre-built binaries from a commercial 
supplier, then there is no difference at all.

>> or how gcc's own test suite compares with the big commercial suites. 
>> I'd be curious to know,
> 
> That might be interesting. Quite a few people know the answer to that 
> but I doubt any will put the answer on here for many reasons.
> 
>> although I don't think it would influence my decisions on which 
>> compiler to use at any given time (I fully expect any compiler to work 
>> correctly for common code constructs,
> 
> Define "correctly"  I am serious here. What do you define as the correct 
> set of rules?
> 

I suppose it means that the generated code should have the effects 
described by the source code - details are in various standards 
documents.  I'm aware that the C virtual machine is not described fully, 
and is not entirely consistent in places - that's why I make the 
distinction between sensible code and IOCCC code (I except "foo(i, i)" 
to be generated correctly - and I don't care what is done with "foo(i++, 
i++)").

Walter Banks wrote:
> David,
> 
> Your points were well taken.
> 

As are yours.

> It has been a couple years since we have done a significant amount of comparative testing of GCC vs commercial compilers. The conclusion at that time in some of the large platforms GCC was competitive (Not necessarily best but competitive) in the small
> embedded processors GCC was losing ground against commercial offerings. We found that there was a large number of people that thought that GCC was good enough or used it because the educational institutions they attended used it. The tracked costs for GCC
> was significantly higher than perceived costs. Supported distributions aka Rowley CrossWorks have lower overall costs than build your own from sources route.
> 
> w..
> 

The field of compiler technology is somewhat different for small 8-bit 
microcontrollers and larger processors.  In particular, gcc is not 
suitable for 8-bit CISC processors with few registers.  I'm sure that it 
could be bullied into producing runnable code for the 8051 or the RS08, 
but there is no way it could compete with the sort of tricks the 
Bytecraft compiler can do without massively re-structuring gcc.  The AVR 
is about the smallest gcc can sensibly target - it is 8-bit, but has 
plenty of registers.

> 
> 
> David Brown wrote:
> 
>> Walter Banks wrote:
>>> David Brown wrote:
>>>
>>>> Another interesting find from google:
>>>>
>>>> http://gcc.gnu.org/ml/gcc/1998-07/msg00656.html
>>> Interesting comments. Standard test suites are an eyopener
>>> when first run on any compiler for the first time. They find
>>> a lot of things, most come as a complete surprise.
>>>
>>> w..
>>>
>> The mail linked above is from 1998 - I have no idea how much test suite
>> testing has been done on more modern gcc versions, or how gcc's own test
>> suite compares with the big commercial suites.  I'd be curious to know,
>> although I don't think it would influence my decisions on which compiler
>> to use at any given time (I fully expect any compiler to work correctly
>> for common code constructs, I manually check the assembly for awkward
>> code such as volatile accesses or interrupt code, and I don't care what
>> the compiler thinks of code worthy of the IOCCC, since I don't write
>> such code).
>

On Sun, 05 Nov 2006 16:27:41 -0500, Walter Banks
<walter@bytecraft.com> wrote:

>
>
>John Temples wrote:
>
>> > Good point combined with Richard's comment on fprintf.
>>
>> There is no requirement to implement fprintf on a freestanding
>> implementation.
>>
>> > It is possible
>> > to implement around this limit on processors like the PIC but I do not
>> > know of anyone (including us) who have done so..
>>
>> I just compiled a 4,000 character string with Hi-Tech's compiler for a
>> 16F877A.  It worked fine.
>
>Constant strings of that length are no problem they are stored in ROM.
>The point that Richard Heathfield made was strings that are created
>in the process of execution of functions like fprintf need to meet a
>particular length. 

Sorry about nitpicking, but printf/fprintf should not be a problem,
since you could call putc() etc. as you process the format string one
character at the time and send the generated format/parameter
characters one at the time to the output device (such as I2C or write
into a block buffer to written into flash).

OTOH, sprintf would be a problem if the output buffer would be larger
than available RAM or need to cross page boundaries. The page boundary
is not an issue, if the putc() etc. routine checks each time if the
RAM page needs to be switched.

Paul

John Temples wrote:

> > Good point combined with Richard's comment on fprintf.
>
> There is no requirement to implement fprintf on a freestanding
> implementation.
>
> > It is possible
> > to implement around this limit on processors like the PIC but I do not
> > know of anyone (including us) who have done so..
>
> I just compiled a 4,000 character string with Hi-Tech's compiler for a
> 16F877A.  It worked fine.

Constant strings of that length are no problem they are stored in ROM.
The point that Richard Heathfield made was strings that are created
in the process of execution of functions like fprintf need to meet a
particular length. Our compilers can support temp strings of this length
on processors without this much RAM only by adding more system
RAM through user defined additional memory on a for example a
I2C bus or the free buffer memory in Hitachi LCD controllers.

w..
--
Walter Banks Byte Craft Limited
http://www.bytecraft.com/

John Temples wrote:

> > Good point combined with Richard's comment on fprintf.
>
> There is no requirement to implement fprintf on a freestanding
> implementation.
>
> > It is possible
> > to implement around this limit on processors like the PIC but I do not
> > know of anyone (including us) who have done so..
>
> I just compiled a 4,000 character string with Hi-Tech's compiler for a
> 16F877A.  It worked fine.

Constant strings of that length are no problem they are stored in ROM.
The point that Richard Heathfield made was strings that are created
in the process of execution of functions like fprintf need to meet a
particular length. Our compilers can support temp strings of this length
on processors without this much RAM only by adding more system
RAM through user defined additional memory on a for example a
I2C bus or the free buffer memory in Hitachi LCD controllers.

w..
--
Walter Banks Byte Craft Limited
http://www.bytecraft.com/

On 2006-11-05, Walter Banks <walter@bytecraft.com> wrote:
> Robert Adsett wrote:
>
>> > CBFalconer wrote:
>> >
>> > > For example, where a compiler is aimed at a PIC many things are simply
>> > > not feasible.
>> >
>> > I am curious, what for example?
>>
>> I seem to recall some of the minimum translation requirements would be
>> an issue.  Isn't there a minimum maximal length string restriction?
>
> Good point combined with Richard's comment on fprintf. 

There is no requirement to implement fprintf on a freestanding
implementation.

> It is possible
> to implement around this limit on processors like the PIC but I do not
> know of anyone (including us) who have done so..

I just compiled a 4,000 character string with Hi-Tech's compiler for a
16F877A.  It worked fine.

-- 
John W. Temples, III