Page 1 of 2
Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 9:12 am
by mrcoollion
Hello LC devs,
I hope someone has a clever suggestion?
I can not get LiveCode to calculate an outcome with more than 6 figures after the decimal point.
Code: Select all
Set the numberformat to "0.############"
put 2/3.2514789 into tAnswer
the LiveCode outcome is 0.615105
However, the more digit outcome is 0,615104714350138
Any suggestions?
Regards,
Paul
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 9:20 am
by richmond62
Well, if it is any comfort I did this:
Code: Select all
on mouseUp
put (2000000000 / 3.2514789)
end mouseUp
and got
615104714.350138
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 10:25 am
by mrcoollion
Thanks for the suggestion Richmond
. For now I will go into this direction with building a small numbers DIV function.
It does not explain why the
does not show more numbers after the decimal.
The dictionary states that it calculates to eighteen decimal places. it would be nice to be able to show as much.
Note: Since LiveCode does not use decimal numbers for its internal calculations (for reasons of speed), the decimal representation of a number is sometimes slightly off the correct number. For example, 10^-1 is equal to 0.1, but is calculated (to eighteen decimal places) as 0.100000000000000006.
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 11:02 am
by richmond62
mrcoollion wrote: ↑Wed Apr 15, 2020 10:25 am
The dictionary states that it calculates to eighteen decimal places. it would be nice to be able to show as much.
The dictionary states many things . . .
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 11:07 am
by richmond62
This worked in
LC 7.1.4:
Code: Select all
on mouseUp
set the numberformat to ".##################"
put 2/3.2514789
end mouseUp
0.615104714350137693
AND it worked in
LC 8.1.8
Sorry, messing around on a machine running MacOS 10.7.5, so no higher version to test over here.
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 11:09 am
by mrcoollion
After some stupid goofing around with options how about this for a funky solution
Code: Select all
set the numberformat to "0.##################"
put 2/325147890 into tComputeAnswer
try
set the showValue of tComputeAnswer to true
catch errorParameter
put item 4 of line 1 of errorParameter into tComputeAnswer
end try
Also works fine with
put 0.00000532*2 into tComputeAnswer
put 2/3.2514789 into tComputeAnswer
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 11:57 am
by mrcoollion
Build into a ManyDigitsFormula function (might have some typos because I cannot copy paste from script in the LC IDE)
However, due to engine restriction, the max number of digits after the decimal is 18! Higher becomes 0.
Code: Select all
put ManyDigitsFormula_v1 (2/325147890) into tAnswer
function ManyDigitsFormula_v1 tInFormula
put "Put " & tInFormula & " into tComputeAnswer" into tComputeFormula
Do tComputeFormula
try
set the showValue of tComputeAnswer to true
catch errorParameter
put item 4 of line 1 of errorParameter into tComputeAnswer
end try
return tComputeAnswer
end ManyDigitsFormula_v1
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 12:00 pm
by richmond62
mrcoollion wrote: ↑Wed Apr 15, 2020 11:57 am
I cannot copy paste from script in the LC IDE
I have come across this problem once or twice.
The solution is to copy-paste from the IDE into a Text Editor, and then copy-paste from the text editor
into this thing.
OK, OK, I know that seems slightly mental: but it works.
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 12:07 pm
by mrcoollion
Hi Richmond.. tried copy and paste into notepad++ but no luck. There is also no paste option is if do this via the application menu.
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 12:16 pm
by bogs
*Answer moved to the correct thread....
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 12:32 pm
by mrcoollion
Hi B,
by bogs » Wed Apr 15, 2020 12:16 pm
Which vers. of Win and Lc Paul?
LC Indy 9.6 (dp3)
Windows 10 Pro ; Version 1903 ; build 18362720
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 1:01 pm
by bogs
* Ditto...
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 1:02 pm
by mrcoollion
Let me know. If you do not have issues with copy-paste on Windows 10 in p.6 (dp3) it must be something is changed...
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 1:35 pm
by bogs
Mind you, I don't have Win 10
Pro, so the test is incomplete in covering every aspect of this.
However, on just regular Win 10, with Lc 9.6 DP 3, I was unable to recreate a situation where I could not copy from Lc and paste into Notepad using any regular method, i.e. ctrl + c, edit menu -> copy, etc.
Here is a link to a few of the shennanigans I used to test it -
https://youtu.be/RZhEkwNsz0o
Re: Need more than 6 numbers after the decimal point
Posted: Wed Apr 15, 2020 2:28 pm
by dunbarx
Paul.
The engine does a lot of stuff behind the scenes so we do not have to type variables. Sometimes just setting the numberFormat don't cut it. Try this:
Code: Select all
on mouseup
set the numberformat to "0.##################"
put 2/3.2514789 into tAnswer
put "" & tAnswer into tAnswer
end mouseup
It was Jacque, with whom I had a similar discussion years ago, that made me appreciate this. You have to force the engine into transforming the number into a string before you get the format you want. Usually, adding "0" or multiplying by "1" does the trick. I just experimented with concatenating empty, and that works fine as well.
It is a small price to pay for typeless variables.
Craig