60001057 Precision of doubles saved in VDML/VDCL

Article 60001057
Type Wish
Product Engine
Version 6016
Date Added 1/5/2010
Fixed (1/5/2010)
Submitted by Adam White


Precision of doubles (like gPoint's X, Y, Z coordinates) saved in VDML/VDCL can be up to 10 decimal places, using DoublePrecision property. Can you extend this ?


Extended in 6017.

DoublePrecision property represents the maximum number of decimal digits used when saving double values in vdml format.Valid range 0 - 17
The following code set the decimal precision to the maximum value:

   int prec = 17;

   doc.FileProperties.DoublePrecision = prec
;//change the number of decimal digits when saved in vdml/vdcl
   Globals.DefaultPointEquality = 1.0d / Math.Pow(10, prec);
// change the default point equality that is used to compare two gPoints
   Globals.DefaultLinearEquality = 1.0d / Math.Pow(10, prec);// change the default distance equality that is used to compare two double values.

   doc.lunits.Precision = (short)prec
;//change the display of the decimal digits shown in VectorDraw property grid and command line prompts
   doc.lunits.ZerosSuppression = LUnits.LZeroSuppression.DecimalTrailing
;// Suppresses trailing zeros in decimal dimensions (for example, 12.5000 becomes 12.5)

The .NET FrameWork is using maximum 17 digits (includes integer and decimal part) for double numbers.
For example, if we have a double like 1234567890.123456789 and we set the doc.FileProperties.DoublePrecision = 17 then the result will be 1234567890.1234568 (10 places for the "integer" part and 7 places for the "fractional" part).
Also : a double like 1234567890123456789 (it only have a integral part) will become 123456789012345680 and  a double like 0.1234567890123456789 will become 0.12345678901234568

Send comments on this topic.