Sub stacks not saving data in standalones on macOS 10.15
Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller
Sub stacks not saving data in standalones on macOS 10.15
Hello
Lc bugs are driving me crazy…
I have a sub stack in a standalone that stores some data, and i use "save stack stackname" regularly to save the stack.
After saving as standalone (checking "Move substacks into individual stackfiles"), this substack works fine in old versions of macOS (10.9 to 10.12, not tried with Hs and Mojave) but not in macOS Catalina: no data is saved in the stack, when the app is in "Applications" folder or in any other home folder.
When the app is placed on a volume without permissions, it works fine under Catalina, so this seems to be a permission write issue.
Is there any special permissions to apply to the app in some versions of macOS ? (in this case i will need creating a special installer for this case), but having a doubt on the reliability of the app after it's installation is unacceptable).
Many thanks for your help.
Best regards, Joan.
Lc bugs are driving me crazy…
I have a sub stack in a standalone that stores some data, and i use "save stack stackname" regularly to save the stack.
After saving as standalone (checking "Move substacks into individual stackfiles"), this substack works fine in old versions of macOS (10.9 to 10.12, not tried with Hs and Mojave) but not in macOS Catalina: no data is saved in the stack, when the app is in "Applications" folder or in any other home folder.
When the app is placed on a volume without permissions, it works fine under Catalina, so this seems to be a permission write issue.
Is there any special permissions to apply to the app in some versions of macOS ? (in this case i will need creating a special installer for this case), but having a doubt on the reliability of the app after it's installation is unacceptable).
Many thanks for your help.
Best regards, Joan.
-
- VIP Livecode Opensource Backer
- Posts: 9867
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: Sub stacks not saving data in standalones on macOS 10.15
Which is it?
I'd put my money on the latter.
Modern OSes tend to protect their Applications folders, strongly recommending using user space for user-writable data.
The specialFolderPath function can help you find such folders.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
Re: Sub stacks not saving data in standalones on macOS 10.15
Thanks for the response.
My app must be installed in "Applications" folder and work "out of the box".
I work on it for 6 months, and at last moment, a stupid write issue, this is really very frustrating...
This kind of bad trip will probably make me abandon Lc for commercial projects and back to FileMaker runtimes: extremely heavy, but everything works in FMP.
--
I've tried chmod -R 775, and removing all extended attributes, no effect.
The strange thing is that if the app is placed and opened once on a volume without permissions, the sub stack can be modified, and this "modified" version works fine if placed in the Catalina "Applications" folder, and can be modified without any writing issue. This means that the "Applications" folder can perfectly receive apps able to write data inside their bundle.
If i find no solution, i will probably write text files in the bundle to store data, a deplorable method to work around the bug.
Sorry for my bad english ('im French), i'm extremely disappointed working so hard creating a nice app and planning to abandon it due to "bizarre" issues.
My app must be installed in "Applications" folder and work "out of the box".
I work on it for 6 months, and at last moment, a stupid write issue, this is really very frustrating...
This kind of bad trip will probably make me abandon Lc for commercial projects and back to FileMaker runtimes: extremely heavy, but everything works in FMP.
--
I've tried chmod -R 775, and removing all extended attributes, no effect.
The strange thing is that if the app is placed and opened once on a volume without permissions, the sub stack can be modified, and this "modified" version works fine if placed in the Catalina "Applications" folder, and can be modified without any writing issue. This means that the "Applications" folder can perfectly receive apps able to write data inside their bundle.
If i find no solution, i will probably write text files in the bundle to store data, a deplorable method to work around the bug.
Sorry for my bad english ('im French), i'm extremely disappointed working so hard creating a nice app and planning to abandon it due to "bizarre" issues.
-
- VIP Livecode Opensource Backer
- Posts: 9867
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: Sub stacks not saving data in standalones on macOS 10.15
How does FMP let you write to folders the OS doesn't want you to?ekami wrote: ↑Sun Feb 23, 2020 1:58 amThanks for the response.
My app must be installed in "Applications" folder and work "out of the box".
I work on it for 6 months, and at last moment, a stupid write issue, this is really very frustrating...
This kind of bad trip will probably make me abandon Lc for commercial projects and back to FileMaker runtimes: extremely heavy, but everything works in FMP.
It may be that they mask what's happening, perhaps moving the editable part into user space. I don't know; it's been years since I built EXEs with FMP.
But even as a subsidiary of Apple, FMP isn't changing the OS. It's doing something else. If you can describe what it's doing, or point me to a description of it, I can tell you how to do that with LC.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
-
- VIP Livecode Opensource Backer
- Posts: 9867
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: Sub stacks not saving data in standalones on macOS 10.15
Also, help me understand: when you write "substacks", are you referring to substacks that are part of the mainstack which becomes the executable, or to separate stack files?
No modern OS allows an application to modify itself, so if those are truly substacks you'll want to move user documents and data into files separate from the executable.
From the LC User Guide, "Building a Standalone Application" (p513):
No modern OS allows an application to modify itself, so if those are truly substacks you'll want to move user documents and data into files separate from the executable.
From the LC User Guide, "Building a Standalone Application" (p513):
LiveCode and FileMaker are indeed different, and each has its strengths. If you're willing to move forward with an open mind about learning, I'm confident we can move your work forward as powerfully as the thousands of other devs who deploy with LC every day.Note: A stack file directly attached to a standalone application cannot have changes saved to it. This stack is bound directly to the executable file that runs. The OS locks an executable file while it is running. If you want to save changes in your standalone application, split your stack up until multiple files. A common technique is to create a "splash screen" stack that contains a welcome screen and then loads the stacks that make up the rest of your application. These stacks are referenced as stackFiles on this pane in the standalone settings screen. It is thus possible to automatically update these component stacks, or to save changes to them. You may also want to consider creating preference files in the appropriate location on your end user's system (see the specialFolderPath function and query/setRegistry functions for more information).
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
Re: Sub stacks not saving data in standalones on macOS 10.15
Um, ekami said in the first post :FourthWorld wrote: ↑Sun Feb 23, 2020 4:49 amAlso, help me understand: when you write "substacks", are you referring to substacks that are part of the mainstack which becomes the executable, or to separate stack files?
I may not know a lot about Macs, but if it was working in the previous 10.x versions, doesn't that mean by default that it should be working in the current one? I.e. they are already separate from the main exe file.
I'd absolutely agree with your suggestion to
if, even though separate from the app exe they are still in the app package though. It is simple to do, and makes sense permissions wise."to move user documents and data into files separate from the executable."
-
- VIP Livecode Opensource Backer
- Posts: 9785
- Joined: Wed May 06, 2009 2:28 pm
- Location: New York, NY
Re: Sub stacks not saving data in standalones on macOS 10.15
Not a problem. Put the executable in the "Applications" folder.My app must be installed in "Applications" folder and work "out of the box".
Now put everything else in any folder where you have permission to change and save a file. You likely have no particular LC issue. You have an OS issue. This just takes a little practice.
Craig
-
- VIP Livecode Opensource Backer
- Posts: 9867
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: Sub stacks not saving data in standalones on macOS 10.15
The thread is titled as being about substacks, but instead seems about stack files. I see no harm in seeking clarification that the test in question does indeed separate the stacks into separate files.bogs wrote: ↑Sun Feb 23, 2020 10:13 amUm, ekami said in the first post :FourthWorld wrote: ↑Sun Feb 23, 2020 4:49 amAlso, help me understand: when you write "substacks", are you referring to substacks that are part of the mainstack which becomes the executable, or to separate stack files?
There's the rub. In a static world, what you suggest would be true. But in threat vector analysis as in the rest of life, the only constant is change.I may not know a lot about Macs, but if it was working in the previous 10.x versions, doesn't that mean by default that it should be working in the current one? I.e. they are already separate from the main exe file.
The situation is not helped by Apple's choice to abandon semantic versioning, using version numbers for branding instead. "10" is no longer a meaningful representation of the state of the OS. For that we have to look at the minor number, to the right of the decimal. The primary "10" has been in use for all versions since Apple transitioned from what we now call "Mac Classic" (v9 and earlier) to everything delivered since about the turn of the century.
Lots of changes over time. Here's an example from a while back of how Apple's efforts to stay one step ahead of the bad guys sometimes require changes to how devs do things:
https://www.mail-archive.com/use-liveco ... 58123.html
More interesting to me is that bundles were ever writable in any version of OS X/mac OS over the last several years. For a good many years most OSes require elevated privileges to write anywhere in the Applications folder, and I would assume that includes folders inside the app bundle.
A quick search for security changes in macOS 10.15 brings up this page, which raises a question with item #2:
https://www.sentinelone.com/blog/7-big- ... -catalina/In macOS Catalina, Apple have introduced a new filesystem partitioning architecture. In one partition is a dedicated, “read-only” system volume containing the OS itself. This is completely separate from all user data, and nothing – other than Apple signed code, such as updates – can overwrite operating system files. It’s basically System Integrity Protection taken to the next level. Instead of protecting locations in an unprotected partition, the entire partition is sealed off.
There’s potential for some initial confusion here as the mount paths are no longer quite as obvious. Where is user data located? Traditionally, it would be in /Users/<username>. On Catalina, it’s in /System/Volumes/Data/Users/<username>. Good-old fashioned root, / appears now to be a system-only volume.
Most devs may not notice this change because they adopted Apple's guidance long ago about separating user-writable data out of the bundle in user space.
This leaves open the question of how FMP handles this, but I wonder if it's similar to how SuperCard does it, moving files out of the bundle into appropriate user folders behind the scenes. If I can turn up anything on how FMP handles this I'll share it; if anyone has such a link please post it.
Since the turn of the century it's been the recommendation of Apple, Microsoft, and most Linux maintainers with their respective OSes. When everyone suggests the same thing, it may be worth heeding.I'd absolutely agree with your suggestion toif, even though separate from the app exe they are still in the app package though. It is simple to do, and makes sense permissions wise."to move user documents and data into files separate from the executable."
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
-
- VIP Livecode Opensource Backer
- Posts: 9867
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: Sub stacks not saving data in standalones on macOS 10.15
FWIW, here's what I could find about FMP runtimes - it seems the DB file itself isn't put inside the app bundle at all, but is a separate document file:
https://fmhelp.filemaker.com/help/16/fm ... me.html%23
So unless there's something Claris is allowed to do as a subsidiary of Apple that other companies aren't, I would imagine we'll be able to help Joan get set up for the time of deployment anticipated, similar to how FMP does it.
https://fmhelp.filemaker.com/help/16/fm ... me.html%23
So unless there's something Claris is allowed to do as a subsidiary of Apple that other companies aren't, I would imagine we'll be able to help Joan get set up for the time of deployment anticipated, similar to how FMP does it.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
Re: Sub stacks not saving data in standalones on macOS 10.15
Hello From France.
After two days fighting with this issue; i finally solved it.
I'll try to explain it : during the standalone creation, the individual sub-stacks are stored in the /Ressources/_MacOS folder of the app bundle. This is the right place, because /Ressources/_MacOS folder is ignored during the cosign process (if the sub-stacks where placed in the /MacOS folder they will be considered as in "wrong place" during cosigning and generate a notarization fail "invalid package")
On MacOS 10.9 to 10.12 (not tested on 10.13 and 10.14), during the first opening of the standalone, the sub-stacks are moved automatically (by the Lc runtime) to the /MacOS folder. This breaks the signature (see technote below), but the bundle keeps working and the moved sub-stacks can be normally saved, so data can be stored inside them.
On macOS Catalina, the sub-stacks are not moved when the standalone starts, explaining why they can't store data when saved ( in counterpart, the bundle keeps signed).
I used AppWrapper to notarize the standalone ( revsecurity.dylib must be deleted because it can't be signed),
Next i build a .pkg with a post install script that (after the installation of the bundle in the "Applications" folder) deletes extended attributes, set permissions, moves the sub-stacks, and finaly opens the app.
sleep 0.25
xattr -cr /Applications/MyStandalone.app
sleep 0.25
chmod 775 /Applications/MyStandalone.app
sleep 0.25
mv /Applications/MyStandalone.app/Contents/Resources/_MacOS/Preferences.rev /Applications/MyStandalone.app/Contents/MacOS
sleep 0.25
open /Applications/MyStandalone.app
I finaly build a notarized .dmg with DMG Canvas, to be used with OSX 10.9 to 10.15
I know that moving files inside the .app after it's installation will break the signature of the bundle, and i probably better do storing data in other place, but as i can use sub-stacks as databases i can save or purge, writing to sub-stacks is the most obvious option i can use for the moment.
---
Please let me know if you need more informations.
Best regards, Joan.
--
edit: i use the same post-install script (excepted mv command) to allow FileMaker Pro runtimes opening/read/write to their related databases (stored at same level of FMPro bundle)
--
https://developer.apple.com/library/arc ... index.html
After two days fighting with this issue; i finally solved it.
I'll try to explain it : during the standalone creation, the individual sub-stacks are stored in the /Ressources/_MacOS folder of the app bundle. This is the right place, because /Ressources/_MacOS folder is ignored during the cosign process (if the sub-stacks where placed in the /MacOS folder they will be considered as in "wrong place" during cosigning and generate a notarization fail "invalid package")
On MacOS 10.9 to 10.12 (not tested on 10.13 and 10.14), during the first opening of the standalone, the sub-stacks are moved automatically (by the Lc runtime) to the /MacOS folder. This breaks the signature (see technote below), but the bundle keeps working and the moved sub-stacks can be normally saved, so data can be stored inside them.
On macOS Catalina, the sub-stacks are not moved when the standalone starts, explaining why they can't store data when saved ( in counterpart, the bundle keeps signed).
I used AppWrapper to notarize the standalone ( revsecurity.dylib must be deleted because it can't be signed),
Next i build a .pkg with a post install script that (after the installation of the bundle in the "Applications" folder) deletes extended attributes, set permissions, moves the sub-stacks, and finaly opens the app.
sleep 0.25
xattr -cr /Applications/MyStandalone.app
sleep 0.25
chmod 775 /Applications/MyStandalone.app
sleep 0.25
mv /Applications/MyStandalone.app/Contents/Resources/_MacOS/Preferences.rev /Applications/MyStandalone.app/Contents/MacOS
sleep 0.25
open /Applications/MyStandalone.app
I finaly build a notarized .dmg with DMG Canvas, to be used with OSX 10.9 to 10.15
I know that moving files inside the .app after it's installation will break the signature of the bundle, and i probably better do storing data in other place, but as i can use sub-stacks as databases i can save or purge, writing to sub-stacks is the most obvious option i can use for the moment.
---
Please let me know if you need more informations.
Best regards, Joan.
--
edit: i use the same post-install script (excepted mv command) to allow FileMaker Pro runtimes opening/read/write to their related databases (stored at same level of FMPro bundle)
--
https://developer.apple.com/library/arc ... index.html
Re: Sub stacks not saving data in standalones on macOS 10.15
Hello
In case if the post-install script of my .pkg fails moving files, i added this script i placed in the "startup" script of the standalone to move sub stacks.
In case if the post-install script of my .pkg fails moving files, i added this script i placed in the "startup" script of the standalone to move sub stacks.
Code: Select all
if the platform = "MacOS" and the environment = "standalone application"
then
set the itemDelimiter to slash
put the effective filename of stack "Preferences" into OriginalStackPath -- bug in macOS Catalina wich returns bad stack path !
put item 1 to -3 of OriginalStackPath & "/Resources/_MacOS/Preferences.rev" into StackPath -- force creating the real stack path
if there is a file StackPath
then
put item 1 to -3 of OriginalStackPath &"/MacOS/Preferences.rev" into NewStackPath
rename file StackPath to NewStackPath
if the result is not empty
then
-- answer StackPath & return & NewStackPath -- compare paths in case of error -- during script tests
end if
end if
end if
-
- VIP Livecode Opensource Backer
- Posts: 9867
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: Sub stacks not saving data in standalones on macOS 10.15
Re:
Has that been reported?
Could it be related to macOS' folder virtualization?
What is the actual path and the reported path?
Code: Select all
put the effective filename of stack "Preferences" into OriginalStackPath -- bug in macOS Catalina wich returns bad stack path !
Could it be related to macOS' folder virtualization?
What is the actual path and the reported path?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
Re: Sub stacks not saving data in standalones on macOS 10.15
I've made a video with a fresh new stack to explain the two bugs i've found, i'll send it tomorrow to Lc quality.
You can download it here:
https://my.pcloud.com/publink/show?code ... KWaXCg2w0y
You can download it here:
https://my.pcloud.com/publink/show?code ... KWaXCg2w0y