![]() |
float comparison error
I just made a player rank database for classic, and I ran into a little operator issue. I used a binary search method for maintaining sorted order, yet the equation I decided on ended up giving this high decimal float.
function onWeaponFired() { player.chat = (0 < 0.0001); } returns false |
It's using a precision of 0.0001 when comparing if two floating point values are the same. It might be weird it some situations, but since most comparisons are with integer values it is better to use a rather low precision. If you compare 7.9999 with 8 it will show that it is the same, since when you do a lot of math operations then a former integer value might show up as 7.99998 or similar. If you want higher precision you could multiplicate both values with 1000 or so before comparing them.
|
Quote:
|
When converting to string it is sometimes rounding to the next integer value though (using the same precision), so that it will not help a lot ;)
|
Quote:
|
| All times are GMT +2. The time now is 08:25 AM. |
Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
Copyright (C) 1998-2019 Toonslab All Rights Reserved.