You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been playing around with your code for a while, and I've noticed some unexpected results. I was able to reduce de code to the quite simple excerpt:
The last one is the behavior that i was expecting in all cases. However, I cannot make sense to what happens on the first two lines (Decimal128 and Decimal64). And by the way, the same kind of behavior occurs when adding other values, such as 1.0, etc.
Moreover, I did the same experiment with the builtin type decimal, and as expected adding zero does not change the value:
varnumber=1.2336464646444M;varzero=0.0M;varresult=number+zero;// here result == number is true
I'm only slightly familiar with arbitrary precision arithmetic, so I may very well be missing something obvious, but I cannot see it.
Would you be kind enough to provide an explanation to my observations, or to give me some pointers?
Cheers
The text was updated successfully, but these errors were encountered:
EContexts store precisions in digits without specifying the base (such as binary or decimal).
The predetermined Decimal contexts should not be used in EFloat methods (except those that return an EDecimal). Likewise, the predetermined Binary contexts should not be used in EDecimal methods (except those that return an EFloat).
Thus, for example, when passing EContext.Decimal128 to EFloat.Add, the result is rounded to have 34 significant bits, not decimal digits.
Hi Peter,
I've been playing around with your code for a while, and I've noticed some unexpected results. I was able to reduce de code to the quite simple excerpt:
I would expect that adding zero, in any base, would leave
number
unchanged. However, it's not what I observe:context
number
result
Decimal128
1.233646464644400086996256504789926111698150634765625
1.233646464650519192218780517578125
Decimal64
1.233646464644400086996256504789926111698150634765625
1.233642578125000
Binary64
1.233646464644400086996256504789926111698150634765625
1.233646464644400086996256504789926111698150634765625
The last one is the behavior that i was expecting in all cases. However, I cannot make sense to what happens on the first two lines (
Decimal128
andDecimal64
). And by the way, the same kind of behavior occurs when adding other values, such as 1.0, etc.Moreover, I did the same experiment with the builtin type
decimal
, and as expected adding zero does not change the value:I'm only slightly familiar with arbitrary precision arithmetic, so I may very well be missing something obvious, but I cannot see it.
Would you be kind enough to provide an explanation to my observations, or to give me some pointers?
Cheers
The text was updated successfully, but these errors were encountered: