Graal Forums

Graal Forums (https://forums.graalonline.com/forums/index.php)
-   Tech Support (https://forums.graalonline.com/forums/forumdisplay.php?f=7)
-   -   float comparison error (https://forums.graalonline.com/forums/showthread.php?t=64254)

jake13jake 02-21-2006 04:02 AM

float comparison error
 
I just made a player rank database for classic, and I ran into a little operator issue. I used a binary search method for maintaining sorted order, yet the equation I decided on ended up giving this high decimal float.

function onWeaponFired() {
player.chat = (0 < 0.0001);
}

returns false

Admins 02-21-2006 09:51 PM

It's using a precision of 0.0001 when comparing if two floating point values are the same. It might be weird it some situations, but since most comparisons are with integer values it is better to use a rather low precision. If you compare 7.9999 with 8 it will show that it is the same, since when you do a lot of math operations then a former integer value might show up as 7.99998 or similar. If you want higher precision you could multiplicate both values with 1000 or so before comparing them.

jake13jake 02-22-2006 12:29 AM

Quote:

Originally Posted by Stefan
It's using a precision of 0.0001 when comparing if two floating point values are the same. It might be weird it some situations, but since most comparisons are with integer values it is better to use a rather low precision. If you compare 7.9999 with 8 it will show that it is the same, since when you do a lot of math operations then a former integer value might show up as 7.99998 or similar. If you want higher precision you could multiplicate both values with 1000 or so before comparing them.

alright. It just made more sense for me to use a multiplier to increase the total point change anyway. Floats aren't my best friend. Treating them as a string, though, it would probably be possible for me to create a little function to compare them with higher precision.

Admins 02-22-2006 04:38 AM

When converting to string it is sometimes rounding to the next integer value though (using the same precision), so that it will not help a lot ;)

jake13jake 02-22-2006 05:05 PM

Quote:

Originally Posted by Stefan
When converting to string it is sometimes rounding to the next integer value though (using the same precision), so that it will not help a lot ;)

Interesting. Why can't things be more precise? *cries*


All times are GMT +2. The time now is 08:25 AM.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
Copyright (C) 1998-2019 Toonslab All Rights Reserved.