Can LC do math with precision to the 9th decimal place? - Solved

Got a LiveCode personal license? Are you a beginner, hobbyist or educator that's new to LiveCode? This forum is the place to go for help getting started. Welcome!

Moderators: Klaus, FourthWorld, heatherlaine, kevinmiller

Post Reply
DR White
Posts: 470
Joined: Fri Aug 23, 2013 12:29 pm
Location: Virginia, USA

Can LC do math with precision to the 9th decimal place? - Solved

Post by DR White » Wed Jul 10, 2019 4:46 pm

Can LC do math with precision to the 9th decimal place?
Last edited by DR White on Fri Jul 12, 2019 10:35 am, edited 2 times in total.

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2029
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Can LC do math with precision to the 7th decimal place?

Post by [-hh] » Wed Jul 10, 2019 5:15 pm

LC uses the 64-bit IEEE precision which is specified in terms of significant digits rather than decimal places.

So in general: No.

But see the thread here
viewtopic.php?f=9&t=31793

Also LC has announced some time ago a "decimal number implementation" for 9.x, see
viewtopic.php?p=173418#p173418
shiftLock happens

Klaus
Posts: 11009
Joined: Sat Apr 08, 2006 8:41 am
Location: Germany
Contact:

Re: Can LC do math with precision to the 7th decimal place?

Post by Klaus » Wed Jul 10, 2019 5:18 pm

Hi Dr.

please decide: 7 or 9 decimal places? :D

Yes, LC can, but you may need to use -> format:
put 1/3 -> 0.333333
put format("%1.7f",1/3) -> 0.3333333
put format("%1.8f",1/3) -> 0.33333333
put format("%1.9f",1/3) -> 0.333333333
put format("%1.10f",1/3) -> 0.3333333333
etc.

Hope that helps!


Best

Klaus

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2029
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Can LC do math with precision to the 7th decimal place?

Post by [-hh] » Wed Jul 10, 2019 5:24 pm

Klaus,
the question is not whether LC can display 9 decimal places but whether doing real number math with LC yields results with correct 9 decimal places.
This depends on the size of numbers that are involved in the computations.

H.

For example try with 1234567890.000000100 + 1.000000200
or format("%1.9f", 1234567890.000000100 + 1.000000200)
shiftLock happens

Klaus
Posts: 11009
Joined: Sat Apr 08, 2006 8:41 am
Location: Germany
Contact:

Re: Can LC do math with precision to the 7th decimal place?

Post by Klaus » Wed Jul 10, 2019 5:55 pm

Ah, OK, get it!
I was always weak in math... 8)

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2029
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Can LC do math with precision to the 7th decimal place?

Post by [-hh] » Wed Jul 10, 2019 5:58 pm

Me too ;-)
shiftLock happens

Klaus
Posts: 11009
Joined: Sat Apr 08, 2006 8:41 am
Location: Germany
Contact:

Re: Can LC do math with precision to the 7th decimal place?

Post by Klaus » Wed Jul 10, 2019 6:01 pm

(Looking at your webstacks etc.) Ha ha! 8)

DR White
Posts: 470
Joined: Fri Aug 23, 2013 12:29 pm
Location: Virginia, USA

Re: Can LC do math with precision to the 7th decimal place?

Post by DR White » Wed Jul 10, 2019 6:05 pm

Klaus and [-hh],

That works BEAUTIFULLY!!!

LC is really the best development software on the planet
AND the community support is "Off the Charts" Awesome.

THANKS,

David

Lagi Pittas
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 327
Joined: Mon Jun 10, 2013 1:32 pm

Re: Can LC do math with precision to the 7th decimal place? - Solved

Post by Lagi Pittas » Thu Jul 11, 2019 2:59 pm

I'm wondering if the decimal number implementation has anything to do with the Livecode for FM project.

Filemaker has a default 16 digit precision but can go up to 400 digits. They won't be able to be compatible if
the precision stays as it is.

In fact if this is the case we should get a lot of new goodies very soon if the last two betas are anything to go by.

https://filemaker.livecode.com/category/blog/


Lagi

Post Reply

Return to “Getting Started with LiveCode - Complete Beginners”