writing a file while another process is reading it

Anything beyond the basics in using the LiveCode language. Share your handlers, functions and magic here.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
adventuresofgreg
Posts: 349
Joined: Tue Oct 28, 2008 1:23 am
Contact:

writing a file while another process is reading it

Post by adventuresofgreg » Mon Sep 24, 2018 11:40 am

Hi there: If I am running 2 separate LC apps, with App A over-writing a file at the exact same time as App B is reading from it, is it possible that App B could read a PARTIAL file due to it being written to disk at the same time as the read? And if so, is there a way to ensure that App B always reads the entire file even if App A is over-writing it simultaneously.

The write format I'm using is put thisthing into URL mypath

Is is safer to write a temp file, then change it's name to overwrite the prev file?

thanks,
Greg

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9579
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: writing a file while another process is reading it

Post by dunbarx » Mon Sep 24, 2018 2:34 pm

Hi,

Would it not be better, more robust, to have the two apps talk to each other, and figure out how to live in harmony? Perhaps a message could be sent, or a flag set, so that whichever app goes first...

Craig Newman

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: writing a file while another process is reading it

Post by bogs » Mon Sep 24, 2018 2:44 pm

Certainly it is possible.
adventuresofgreg wrote:
Mon Sep 24, 2018 11:40 am
is there a way to ensure that App B always reads the entire file even if App A is over-writing it simultaneously.
Hm. What about having 2 sets of files?

...fileName.original <- read/write this file out from App A, when the write is *finished*, copy it to ~
...fileName.finished <- use this for App B

Or what Craig said :D
Image

adventuresofgreg
Posts: 349
Joined: Tue Oct 28, 2008 1:23 am
Contact:

Re: writing a file while another process is reading it

Post by adventuresofgreg » Mon Sep 24, 2018 3:58 pm

Seems like writing a .temp file first, then changing it's name to overwrite the old file works.

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9579
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: writing a file while another process is reading it

Post by dunbarx » Mon Sep 24, 2018 4:25 pm

Greg.

Aren't you just kicking the can down the road? Does it matter if you change a file's name whilst another app is writing to it, or simply try to read from it during that process?

Craig

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7215
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: writing a file while another process is reading it

Post by jacque » Mon Sep 24, 2018 5:38 pm

This is called a race condition and the typical way to avoid it is to mark a file as busy during a write and flag it as not busy afterward. Usually the flag is a just a text file on the server, and you can check to see if it contains "true" or "false" or just whether it exists at all.

The app looks for the existence of the file, or reads its content, and if it's busy it checks again later until it's no longer busy. When it's okay to do the update, it marks the file as busy, does the write, and then marks the file as free again.

There is still a problem with overwriting changes made by another copy of the app. You may have to read the content and merge it with the new content depending on what the app needs to do with the file.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: writing a file while another process is reading it

Post by bogs » Mon Sep 24, 2018 6:00 pm

dunbarx wrote:
Mon Sep 24, 2018 4:25 pm
Does it matter if you change a file's name whilst another app is writing to it, or simply try to read from it during that process?
That isn't what is happening, at least that isn't what I suggested up there nor how I read what he wrote which looks like a variation of my suggestion.
adventuresofgreg wrote:
Mon Sep 24, 2018 3:58 pm
Seems like writing a .temp file first, then changing it's name to overwrite the old file works.
App A writes the information to a temp file, when the write is finished App A renames the temp file to the file app B needs to read is what I get out of it. I could be wrong though :P

The method I suggested takes care of a few things Jacque mentioned as well.
...You don't have to worry if App A is writing to the (temp) file or not, so you don't need to check for a flag.
...Checking for the most up to date file isn't necessary, when App A finishes the write it copies/renames it to the file App B is checking, so it is always the most up to date version.
jacque wrote:
Mon Sep 24, 2018 5:38 pm
There is still a problem with overwriting changes made by another copy of the app. You may have to read the content and merge it with the new content depending on what the app needs to do with the file.
Depending on whether there is more than one App A, though, this is a very valid concern.
Image

adventuresofgreg
Posts: 349
Joined: Tue Oct 28, 2008 1:23 am
Contact:

Re: writing a file while another process is reading it

Post by adventuresofgreg » Mon Sep 24, 2018 6:06 pm

the flag is the answer. I think the only reason a file name change might work 'better' is that less time is taken changing a file name than writing the entire file, therefore, there is a smaller chance that the reading app will read at exactly the time the file name is being changed.

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: writing a file while another process is reading it

Post by FourthWorld » Mon Sep 24, 2018 6:09 pm

Greg, if this is on your Linux server you can reduce latency even more by using /run/shm/ as your write location.

That's essentially a RAM disk built into the system. Fast, but of course if you need a failsafe in addition to speed you may also want to write a copy to physical storage.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: writing a file while another process is reading it

Post by bogs » Mon Sep 24, 2018 6:20 pm

OOOooo, nice bit of info there Richard! (Making note)
Image

adventuresofgreg
Posts: 349
Joined: Tue Oct 28, 2008 1:23 am
Contact:

Re: writing a file while another process is reading it

Post by adventuresofgreg » Mon Sep 24, 2018 7:18 pm

thanks Richard - no this one will run on a windows server. But great advice - thank you.

Post Reply

Return to “Talking LiveCode”