Hi everyone, assuming two differently implemented FPU, both validated against the IEEE754, and limiting operations to the golden 5 (+,-, *, /, sqr), can I be sure that provided the inputs operands are the same the result would be the same bit wise? BACKGROUND: our dear customer wanted to verify that the control law specified is correctly implemented by means of comparing output with a golden model. Now the golden model is running on a standard PC, with matlab, a pile of mathematical libraries and a processor that is humongous compared to our little embedded unit. I've already warned the customer that bitwise comparison might only be viable if precision wise the two calculations are equally configured (single precision, rounding, etc.). Are there any other elements to consider? Moreover, we are driving a PWM with the control law on a 8bit, or max 10bit precision, so most of that fine grain precision would not be used anyway. Any comment/pointer/suggestion is appreciated. Al -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail?
two fpu compared
Started by ●February 12, 2015
Reply by ●February 12, 20152015-02-12
On 12/02/15 09:30, alb wrote:> Hi everyone, > > assuming two differently implemented FPU, both validated against the > IEEE754, and limiting operations to the golden 5 (+,-, *, /, sqr), can I > be sure that provided the inputs operands are the same the result would > be the same bit wise? > > BACKGROUND: our dear customer wanted to verify that the control law > specified is correctly implemented by means of comparing output with a > golden model. Now the golden model is running on a standard PC, with > matlab, a pile of mathematical libraries and a processor that is > humongous compared to our little embedded unit. I've already warned > the customer that bitwise comparison might only be viable if precision > wise the two calculations are equally configured (single precision, > rounding, etc.). Are there any other elements to consider? > > Moreover, we are driving a PWM with the control law on a 8bit, or max > 10bit precision, so most of that fine grain precision would not be used > anyway. > > Any comment/pointer/suggestion is appreciated. > > Al >It is normal to view floating point operations as slightly imprecise - there is a rule that you never compare floating point data with ==. Even if you are careful to have the same precision and rounding in the calculations, there may be other factors - for at least some operations, I believe there can be different results from two IEEE754 compatible implementations. More importantly, however, is that normally you don't want full IEEE754 compliance - the overhead is too much. Typically, FPU hardware does not provide full support - you need to have extra software handling as well. In some aspects, non-compliant code can be significantly more efficient. For example, with "-ffast-math" in gcc, the compiler can turn "x = y + z - y" into "x = z", while in compliant mode it has to do the full calculation as stated, even though it is slower and gives less accurate results. And for square root, some FPU's may have a fast "approximate square root" function that is good enough and much faster. The only way to be really sure of bit-accurate results is to use the same software floating point library on each system. But since the "golden model" here is a system that you have no control of (matlab plus external libraries - and different "standard PC's" can have different details in their FPU's), it is basically impossible to guarantee bit-accurate copies of the results.
Reply by ●February 12, 20152015-02-12
alb <al.basili@gmail.com> wrote:> assuming two differently implemented FPU, both validated against the > IEEE754, and limiting operations to the golden 5 (+,-, *, /, sqr), can I > be sure that provided the inputs operands are the same the result would > be the same bit wise?The x87, which as well as I know follows the IEEE standard, holds intermediate results in extended precision. (64 bit significand). In that case, the results could easily be different from one that didn't use extra precision. For x86-64 code, it is less usual to use x87, and so less likely that will be a problem.> BACKGROUND: our dear customer wanted to verify that the control law > specified is correctly implemented by means of comparing output with a > golden model. Now the golden model is running on a standard PC, with > matlab, a pile of mathematical libraries and a processor that is > humongous compared to our little embedded unit. I've already warned > the customer that bitwise comparison might only be viable if precision > wise the two calculations are equally configured (single precision, > rounding, etc.). Are there any other elements to consider?-- glen
Reply by ●February 12, 20152015-02-12
Op Thu, 12 Feb 2015 09:30:32 +0100 schreef alb <al.basili@gmail.com>:> Hi everyone, > > assuming two differently implemented FPU, both validated against the > IEEE754, and limiting operations to the golden 5 (+,-, *, /, sqr), can I > be sure that provided the inputs operands are the same the result would > be the same bit wise?Iff the software on both ends *use* the FPU in accordance with the standard (i.e. strictfp, no -ffast-math) and in the same mode, then it should be the same.> BACKGROUND: our dear customer wanted to verify that the control law > specified is correctly implemented by means of comparing output with a > golden model. Now the golden model is running on a standard PC, with > matlab, a pile of mathematical libraries and a processor that is > humongous compared to our little embedded unit. I've already warned > the customer that bitwise comparison might only be viable if precision > wise the two calculations are equally configured (single precision, > rounding, etc.). Are there any other elements to consider? > > Moreover, we are driving a PWM with the control law on a 8bit, or max > 10bit precision, so most of that fine grain precision would not be used > anyway. > > Any comment/pointer/suggestion is appreciated.It is one thing to have a mathematical model that happens to give accurate results when ran on a PC, but it is quite another thing to perform numerical analysis and determine the required accuracy (exactness) for each step of the algorithm. Regardless, if your customer can determine whether a 'golden' implementation provides accurate results, then I would think they can do the same for your implementation. -- (Remove the obvious prefix to reply privately.) Gemaakt met Opera's e-mailprogramma: http://www.opera.com/mail/
Reply by ●February 12, 20152015-02-12
On 02/12/2015 02:30 AM, alb wrote:> Hi everyone, > > assuming two differently implemented FPU, both validated against the > IEEE754, and limiting operations to the golden 5 (+,-, *, /, sqr), can I > be sure that provided the inputs operands are the same the result would > be the same bit wise?No, you can't even get the same result on the same FP unit. Years ago a young colleague of mine came to me with a problem. He was using floating point and at one point was getting a mis-compare on equal after some floating point computations on identical inputs. The computations went down some different paths and were compared at the end. (In same situations the inputs would not be identical on the two paths.) This was on a server grade PPC (some of you can see where this is going) which has a merged multiply-add. Due to the way it was coded on one path separate operations were done, on the other they were merged. The difference between one rounding and two was enough to give different results. We had a discussion about how fp wasn't as easy as it looks, and why you need to be very careful with an equal compare using floating point.> > BACKGROUND: our dear customer wanted to verify that the control law > specified is correctly implemented by means of comparing output with a > golden model. Now the golden model is running on a standard PC, with > matlab, a pile of mathematical libraries and a processor that is > humongous compared to our little embedded unit. I've already warned > the customer that bitwise comparison might only be viable if precision > wise the two calculations are equally configured (single precision, > rounding, etc.). Are there any other elements to consider? > > Moreover, we are driving a PWM with the control law on a 8bit, or max > 10bit precision, so most of that fine grain precision would not be used > anyway. > > Any comment/pointer/suggestion is appreciated. > > Al >
Reply by ●February 13, 20152015-02-13
Hi David, David Brown <david.brown@hesbynett.no> wrote: []>> assuming two differently implemented FPU, both validated against the >> IEEE754, and limiting operations to the golden 5 (+,-, *, /, sqr), can I >> be sure that provided the inputs operands are the same the result would >> be the same bit wise?[]> > It is normal to view floating point operations as slightly imprecise - > there is a rule that you never compare floating point data with ==. > Even if you are careful to have the same precision and rounding in the > calculations, there may be other factors - for at least some operations, > I believe there can be different results from two IEEE754 compatible > implementations.the extra bit I got recently is that the result is converted to an integer of 11bit which, a priori, means that a lot of the low bit precision from the 32bit is anyhow lost. Indeed I could calculate what the integer LSB corresponds to in terms of floating point representation and be more confident on the comparison (i.e. if floating point results are not bitwise equal does not necessary mean the integer result is going to be different).> More importantly, however, is that normally you don't want full IEEE754 > compliance - the overhead is too much. Typically, FPU hardware does not > provide full support - you need to have extra software handling as well.AFAIK the implementation we have is 'full IEEE754' and floating points operations are using directly the hardware (unless gcc is configured otherwise).> In some aspects, non-compliant code can be significantly more > efficient. For example, with "-ffast-math" in gcc, the compiler can > turn "x = y + z - y" into "x = z", while in compliant mode it has to do > the full calculation as stated, even though it is slower and gives less > accurate results. And for square root, some FPU's may have a fast > "approximate square root" function that is good enough and much faster.I see your point, this is certainly an extra bit of info since our harwdare is not that performing either and a soft implementation might actually be faster.> The only way to be really sure of bit-accurate results is to use the > same software floating point library on each system. But since the > "golden model" here is a system that you have no control of (matlab plus > external libraries - and different "standard PC's" can have different > details in their FPU's), it is basically impossible to guarantee > bit-accurate copies of the results.Still can be a viable approach if the result is converted to integer with a much lower precision. It is a bet though, since we cannot be absolutely sure that it would be the case. Al
Reply by ●February 13, 20152015-02-13
On 13/02/15 15:14, alb wrote:> Hi David, > > David Brown <david.brown@hesbynett.no> wrote: > [] >>> assuming two differently implemented FPU, both validated against the >>> IEEE754, and limiting operations to the golden 5 (+,-, *, /, sqr), can I >>> be sure that provided the inputs operands are the same the result would >>> be the same bit wise? > [] >> >> It is normal to view floating point operations as slightly imprecise - >> there is a rule that you never compare floating point data with ==. >> Even if you are careful to have the same precision and rounding in the >> calculations, there may be other factors - for at least some operations, >> I believe there can be different results from two IEEE754 compatible >> implementations. > > the extra bit I got recently is that the result is converted to an > integer of 11bit which, a priori, means that a lot of the low bit > precision from the 32bit is anyhow lost. > > Indeed I could calculate what the integer LSB corresponds to in terms of > floating point representation and be more confident on the comparison > (i.e. if floating point results are not bitwise equal does not necessary > mean the integer result is going to be different). > >> More importantly, however, is that normally you don't want full IEEE754 >> compliance - the overhead is too much. Typically, FPU hardware does not >> provide full support - you need to have extra software handling as well. > > AFAIK the implementation we have is 'full IEEE754' and floating points > operations are using directly the hardware (unless gcc is configured > otherwise). > >> In some aspects, non-compliant code can be significantly more >> efficient. For example, with "-ffast-math" in gcc, the compiler can >> turn "x = y + z - y" into "x = z", while in compliant mode it has to do >> the full calculation as stated, even though it is slower and gives less >> accurate results. And for square root, some FPU's may have a fast >> "approximate square root" function that is good enough and much faster. > > I see your point, this is certainly an extra bit of info since our > harwdare is not that performing either and a soft implementation might > actually be faster. > >> The only way to be really sure of bit-accurate results is to use the >> same software floating point library on each system. But since the >> "golden model" here is a system that you have no control of (matlab plus >> external libraries - and different "standard PC's" can have different >> details in their FPU's), it is basically impossible to guarantee >> bit-accurate copies of the results. > > Still can be a viable approach if the result is converted to integer > with a much lower precision. It is a bet though, since we cannot be > absolutely sure that it would be the case. > > Al >What this all boils down to is that the client wants to be sure that the code is "correct" - but has an unreasonable and restrictive definition of "correct". It would be better to make this clear /now/, before you end up contracted to produce code that is impossible to make (since it must match the bit-exact results from matlab on "a standard PC", even though these results could vary between different "standard PC's"). Figure out what the client /actually/ needs, such as a variation of +/-3 on the 10-bit PWM output, and tell them that this is a sensible goal.
Reply by ●February 13, 20152015-02-13
On February 13th, 2015, David Brown explained: |-------------------------------------------------------------------------| |"What this all boils down to is that the client wants to be sure that the| |code is "correct" - but has an unreasonable and restrictive definition | |of "correct". It would be better to make this clear /now/, before you | |end up contracted to produce code that is impossible to make (since it | |must match the bit-exact results from matlab on "a standard PC", even | |though these results could vary between different "standard PC's"). | | | |Figure out what the client /actually/ needs, such as a variation of +/-3 | |on the 10-bit PWM output, and tell them that this is a sensible goal." | |-------------------------------------------------------------------------| This reminds me of Chapter 97 Your Customers Do Not Mean What They Say of Kevlin Henney (editor), "97 Things Every Programmer Should Know: Collective Wisdom from the Experts", O'Reilly Media, 2010, HTTP://Shop.OReilly.com/product/9780596809492.do With best regards, Colin Paul de Gloucester
Reply by ●February 13, 20152015-02-13
On 13 Feb 2015 14:14:06 GMT, al.basili@gmail.com (alb) wrote:>AFAIK the implementation we have is 'full IEEE754' and floating points >operations are using directly the hardware (unless gcc is configured >otherwise).It technically may be compliant, but not necessarily in hardware. I'm not aware of *any* implementation that's fully 754/854 compliant in hardware. Every chip I have ever seen leaves most rounding modes and NaN handling to software/firmware. George.
Reply by ●February 14, 20152015-02-14
alb <al.basili@gmail.com> wrote:> David Brown <david.brown@hesbynett.no> wrote:>>> assuming two differently implemented FPU, both validated against the >>> IEEE754, and limiting operations to the golden 5 (+,-, *, /, sqr), can I >>> be sure that provided the inputs operands are the same the result would >>> be the same bit wise?>> It is normal to view floating point operations as slightly imprecise - >> there is a rule that you never compare floating point data with ==.(snip)> the extra bit I got recently is that the result is converted to an > integer of 11bit which, a priori, means that a lot of the low bit > precision from the 32bit is anyhow lost.Are you sure the calculation should be done in floating point? Without more details, I suspect it could be done in fixed point, the results would not vary depending on implementations, and everyone would be happy. (There is only one question left, and that is the way division works when one operand is negative.)> Indeed I could calculate what the integer LSB corresponds to in terms of > floating point representation and be more confident on the comparison > (i.e. if floating point results are not bitwise equal does not necessary > mean the integer result is going to be different).Small rounding differences can easily lead to differences when converted to fixed point.>> More importantly, however, is that normally you don't want full IEEE754 >> compliance - the overhead is too much. Typically, FPU hardware does not >> provide full support - you need to have extra software handling as well.> AFAIK the implementation we have is 'full IEEE754' and floating points > operations are using directly the hardware (unless gcc is configured > otherwise).I forget now exactly what "full IEEE754" is, but denormals are one that you have to watch. Personally, I think denormals were a bad idea, though. (snip)>> The only way to be really sure of bit-accurate results is to use the >> same software floating point library on each system. But since the >> "golden model" here is a system that you have no control of (matlab plus >> external libraries - and different "standard PC's" can have different >> details in their FPU's), it is basically impossible to guarantee >> bit-accurate copies of the results.Or use fixed point.> Still can be a viable approach if the result is converted to integer > with a much lower precision. It is a bet though, since we cannot be > absolutely sure that it would be the case.-- glen
