Different Results of Floating Point Arithmetic in LabWindows/CVI and Visual C++ Primary Software: LabWindows/CVI Development Systems>>Full Development SystemPrimary Software Version: 8.0 Primary Software Fixed Version: N/A Secondary Software: LabWindows/CVI Development Systems>>Base Package
Problem: I get different results in LabWindows/CVI and Visual Studio. Why do I see this discrepancy? Solution: Both LabWindows/CVI and Visual Studio adhere to the ANSI C 89 standard. This standard states that floating point numbers must be represented in at least 15 degrees of precision. This requirement is a minimum requirement. Visual Studio uses an additional digit to represent floating point numbers, while CVI uses 15 digits. This fact can result in different values between the two environments, especially if a floating point calculation depends on the 16th digit of precision. For example, the number 1.6 is stored in CVI as 1.600000000000000. On the other hand, Visual Studio uses 16 digits to represent double precision floating point numbers. As a result, 1.6 is stored as 1.6000000000000001. If you edit the value of 1.6 in CVI to have the additional digit that Visual Studio includes via a data tool tip, you should get consistent behavior between the two environments. So that 16th digit provides just enough precision for the floating point arithmetic to return the results you expect. Related Links: KnowledgeBase 1BTCIC0W: Comparing Floating Points Does Not Seem to be Working Properly Attachments:
Report Date: 01/28/2006 Last Updated: 03/02/2006 Document ID: 3TRDC7K7 |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
