You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
IBM360Converter compares the IBM float value (as an integer) with IEMAXIB, returning IEEEMAX (+Inf) if the value is larger than IEMAXIB.
IEMAXIB is defined as: public static final int IEMAXIB = 0x611FFFFF;
The IBM value 0x61100000 corresponds to IEEE +Inf (0x7F800000, signifying exponent FF and mantissa 0). IBM values between 0x61100001 up to 0x611FFFFF get converted to IEEE values in the range 0x7F800008 - 0x7FFFFFF8. These are NaN values, where exponent = FF and mantissa is not 0. Then IBM values from 0x61200000 upwards are converted to +Inf again (due to the IEMAXIB test).
It would appear that IEMAXIB should be 0x61100000 rather than 0x611FFFFF so that all values above 0x61100000 are converted to +Inf.
The text was updated successfully, but these errors were encountered:
If 0x61100000 is used, then the comparison could be changed to >= IEMAXIB for clarity (although since val == IEMAXIB yields +inf anyway, it doesn't really matter). Alternatively, > 0x610FFFFF could be used.
IBM360Converter compares the IBM float value (as an integer) with IEMAXIB, returning IEEEMAX (+Inf) if the value is larger than IEMAXIB.
IEMAXIB is defined as:
public static final int IEMAXIB = 0x611FFFFF;
The IBM value 0x61100000 corresponds to IEEE +Inf (0x7F800000, signifying exponent FF and mantissa 0). IBM values between 0x61100001 up to 0x611FFFFF get converted to IEEE values in the range 0x7F800008 - 0x7FFFFFF8. These are NaN values, where exponent = FF and mantissa is not 0. Then IBM values from 0x61200000 upwards are converted to +Inf again (due to the IEMAXIB test).
It would appear that IEMAXIB should be 0x61100000 rather than 0x611FFFFF so that all values above 0x61100000 are converted to +Inf.
The text was updated successfully, but these errors were encountered: