busy files

[adventuresofgreg]

Howdy. My stack reads in a URL text file every second and does a bunch of stuff. Another stack is writing the data to the text file. When my export stack is in the process of writing, while the import stack is reading it, a partial file is read-in. I can handle this ok, but for some unexplained reason, this handler crashes every hour or so. Not sure what the cause is. I thought it might have something to do with reading in a file while it is being written. Is there a way to restrict the "put URL text:bla bla" to wait until the file is not busy? Ideas?

Thanks,
Greg

[FourthWorld]

Crashes or throws a script execution error?

[adventuresofgreg]

I don't know because the error happens while running the standalone. The result is that the handler that reads the text files suddenly stops doing it, and only a restart will fix it. recreating the error while running a non-standalone would be difficult.

[FourthWorld]

A crash is when an application process dies and the OS reports the error. If the application is still running it's not a crash per se.

This is a valuable distinction as it broadens our opportunities for diagnosing the root cause. Have you considered adding logging to your app so you can see the last thing it does before it hangs?

[adventuresofgreg]

got it! there are a dozen instances of a standalone all writing data to one text file - many times per second. A 'server' app reads in the text file and does stuff with the data. If the data is in the process of being written by an app, then the file is only partially read by the server app - or actually, I should say that the file is fully read by the server app, but the data in it might be partial because it might be in the process of being written. This is fine because it's reading many times a second, so partial data doesn't matter. Except that I'm doing some math on the numbers.

Here is a challenge for you: what might cause my sum(apileofnumbers) function to cause the script to crash? There can only be numbers in apileofnumbers, so no, it isn't that there is a non-numeric item in there. It took me a while to figure this out

[FourthWorld]

Have you considered using a lock file so other processes can know when one of them is writing to the file?

[adventuresofgreg]

how do you lock a file?

[FourthWorld]

A lock file doesn't lock a file, but merely serves as a flag that another file is busy. Any process using the main file would first check for the presence of the lock file, and if the wait is anticipated to be long it could return an error or if short it could just wait in a loop, checking until the lock file has been removed:

http://en.wikipedia.org/wiki/Lock_file#Lock_files