Can LC do math with precision to the 9th decimal place? - Solved

DR White · Post by **DR White** » Wed Jul 10, 2019 4:46 pm

Can LC do math with precision to the 9th decimal place?

[-hh] · Post by **[-hh]** » Wed Jul 10, 2019 5:15 pm

LC uses the 64-bit IEEE precision which is specified in terms of significant digits rather than decimal places.

So in general: No.

But see the thread here
viewtopic.php?f=9&t=31793

Also LC has announced some time ago a "decimal number implementation" for 9.x, see
viewtopic.php?p=173418#p173418

Klaus · Post by **Klaus** » Wed Jul 10, 2019 5:18 pm

Hi Dr.

please decide: 7 or 9 decimal places?

Yes, LC can, but you may need to use -> format:
put 1/3 -> 0.333333
put format("%1.7f",1/3) -> 0.3333333
put format("%1.8f",1/3) -> 0.33333333
put format("%1.9f",1/3) -> 0.333333333
put format("%1.10f",1/3) -> 0.3333333333
etc.

Hope that helps!

Best

Klaus

[-hh] · Post by **[-hh]** » Wed Jul 10, 2019 5:24 pm

Klaus,
the question is not whether LC can display 9 decimal places but whether doing real number math with LC yields results with correct 9 decimal places.
This depends on the size of numbers that are involved in the computations.

H.

For example try with 1234567890.000000100 + 1.000000200
or format("%1.9f", 1234567890.000000100 + 1.000000200)

Klaus · Post by **Klaus** » Wed Jul 10, 2019 5:55 pm

Ah, OK, get it!
I was always weak in math...

[-hh] · Post by **[-hh]** » Wed Jul 10, 2019 5:58 pm

Me too

Klaus · Post by **Klaus** » Wed Jul 10, 2019 6:01 pm

(Looking at your webstacks etc.) Ha ha!

DR White · Post by **DR White** » Wed Jul 10, 2019 6:05 pm

Klaus and [-hh],

That works BEAUTIFULLY!!!

LC is really the best development software on the planet
AND the community support is "Off the Charts" Awesome.

THANKS,

David

Lagi Pittas · Post by **Lagi Pittas** » Thu Jul 11, 2019 2:59 pm

I'm wondering if the decimal number implementation has anything to do with the Livecode for FM project.

Filemaker has a default 16 digit precision but can go up to 400 digits. They won't be able to be compatible if
the precision stays as it is.

In fact if this is the case we should get a lot of new goodies very soon if the last two betas are anything to go by.

https://filemaker.livecode.com/category/blog/

Lagi

LiveCode Forums

Can LC do math with precision to the 9th decimal place? - Solved

Can LC do math with precision to the 9th decimal place? - Solved

Re: Can LC do math with precision to the 7th decimal place?

Re: Can LC do math with precision to the 7th decimal place?

Re: Can LC do math with precision to the 7th decimal place?

Re: Can LC do math with precision to the 7th decimal place?

Re: Can LC do math with precision to the 7th decimal place?

Re: Can LC do math with precision to the 7th decimal place?

Re: Can LC do math with precision to the 7th decimal place?

Re: Can LC do math with precision to the 7th decimal place? - Solved