Can LC do math with precision to the 9th decimal place?  Solved
Moderators: Klaus, FourthWorld, heatherlaine, kevinmiller
Can LC do math with precision to the 9th decimal place?  Solved
Can LC do math with precision to the 9th decimal place?
Last edited by DR White on Fri Jul 12, 2019 10:35 am, edited 2 times in total.

 VIP Livecode Opensource Backer
 Posts: 2196
 Joined: Thu Feb 28, 2013 11:52 pm
 Location: Göttingen, DE
Re: Can LC do math with precision to the 7th decimal place?
LC uses the 64bit IEEE precision which is specified in terms of significant digits rather than decimal places.
So in general: No.
But see the thread here
viewtopic.php?f=9&t=31793
Also LC has announced some time ago a "decimal number implementation" for 9.x, see
viewtopic.php?p=173418#p173418
So in general: No.
But see the thread here
viewtopic.php?f=9&t=31793
Also LC has announced some time ago a "decimal number implementation" for 9.x, see
viewtopic.php?p=173418#p173418
shiftLock happens
Re: Can LC do math with precision to the 7th decimal place?
Hi Dr.
please decide: 7 or 9 decimal places?
Yes, LC can, but you may need to use > format:
put 1/3 > 0.333333
put format("%1.7f",1/3) > 0.3333333
put format("%1.8f",1/3) > 0.33333333
put format("%1.9f",1/3) > 0.333333333
put format("%1.10f",1/3) > 0.3333333333
etc.
Hope that helps!
Best
Klaus
please decide: 7 or 9 decimal places?
Yes, LC can, but you may need to use > format:
put 1/3 > 0.333333
put format("%1.7f",1/3) > 0.3333333
put format("%1.8f",1/3) > 0.33333333
put format("%1.9f",1/3) > 0.333333333
put format("%1.10f",1/3) > 0.3333333333
etc.
Hope that helps!
Best
Klaus

 VIP Livecode Opensource Backer
 Posts: 2196
 Joined: Thu Feb 28, 2013 11:52 pm
 Location: Göttingen, DE
Re: Can LC do math with precision to the 7th decimal place?
Klaus,
the question is not whether LC can display 9 decimal places but whether doing real number math with LC yields results with correct 9 decimal places.
This depends on the size of numbers that are involved in the computations.
H.
For example try with 1234567890.000000100 + 1.000000200
or format("%1.9f", 1234567890.000000100 + 1.000000200)
the question is not whether LC can display 9 decimal places but whether doing real number math with LC yields results with correct 9 decimal places.
This depends on the size of numbers that are involved in the computations.
H.
For example try with 1234567890.000000100 + 1.000000200
or format("%1.9f", 1234567890.000000100 + 1.000000200)
shiftLock happens
Re: Can LC do math with precision to the 7th decimal place?
Ah, OK, get it!
I was always weak in math...
I was always weak in math...
Re: Can LC do math with precision to the 7th decimal place?
(Looking at your webstacks etc.) Ha ha!
Re: Can LC do math with precision to the 7th decimal place?
Klaus and [hh],
That works BEAUTIFULLY!!!
LC is really the best development software on the planet
AND the community support is "Off the Charts" Awesome.
THANKS,
David
That works BEAUTIFULLY!!!
LC is really the best development software on the planet
AND the community support is "Off the Charts" Awesome.
THANKS,
David

 VIP Livecode Opensource Backer
 Posts: 328
 Joined: Mon Jun 10, 2013 1:32 pm
Re: Can LC do math with precision to the 7th decimal place?  Solved
I'm wondering if the decimal number implementation has anything to do with the Livecode for FM project.
Filemaker has a default 16 digit precision but can go up to 400 digits. They won't be able to be compatible if
the precision stays as it is.
In fact if this is the case we should get a lot of new goodies very soon if the last two betas are anything to go by.
https://filemaker.livecode.com/category/blog/
Lagi
Filemaker has a default 16 digit precision but can go up to 400 digits. They won't be able to be compatible if
the precision stays as it is.
In fact if this is the case we should get a lot of new goodies very soon if the last two betas are anything to go by.
https://filemaker.livecode.com/category/blog/
Lagi