How to get date and time from a file
Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller
-
- Livecode Opensource Backer
- Posts: 16
- Joined: Sun Apr 09, 2006 9:49 pm
- Location: Irvine, CA, USA
How to get date and time from a file
Hi all,
I need to get the modified date and time from a file that resides in a folder with thousands of files.
Using the "detailed files" is way too slow as it has to get all the files.
So is there a direct way, with the path and file name, to get the information from the file?
Thanx,
Mark Stuart
I need to get the modified date and time from a file that resides in a folder with thousands of files.
Using the "detailed files" is way too slow as it has to get all the files.
So is there a direct way, with the path and file name, to get the information from the file?
Thanx,
Mark Stuart
Re: How to get date and time from a file
In Win7 I use the shell.
The returned data is the last access time and date and file size.
Linux would use "ls" of course. I don't know the Mac shell command.
Code: Select all
local tFilePath, tCmd
put "C:\Users\Walt\output.txt" into tFilePath
put "dir" && tFilePath into tCmd
put return & shell(tCmd) after fld "fFileData"
Linux would use "ls" of course. I don't know the Mac shell command.
Walt Brown
Omnis traductor traditor
Omnis traductor traditor
-
- VIP Livecode Opensource Backer
- Posts: 9658
- Joined: Wed May 06, 2009 2:28 pm
- Location: New York, NY
Re: How to get date and time from a file
Hi.
Goodness, if you have the pathname, use the "read from file" command, and work it in LC directly. See the dictionary...
Craig Newman
Goodness, if you have the pathname, use the "read from file" command, and work it in LC directly. See the dictionary...
Craig Newman
-
- VIP Livecode Opensource Backer
- Posts: 9834
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: How to get date and time from a file
@mfstuart: What is the amount of time your routine takes for obtaining that info from the "detailed files" list, and what does your code look like? Lineoffset is usually pretty fast.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
-
- Livecode Opensource Backer
- Posts: 16
- Joined: Sun Apr 09, 2006 9:49 pm
- Location: Irvine, CA, USA
Re: How to get date and time from a file
Hi Craig,
I'm not sure how to get the modified date and time of the file from your suggestion of using "read from file".
Can you please elaborate?
Thanx,
Mark Stuart
I'm not sure how to get the modified date and time of the file from your suggestion of using "read from file".
Can you please elaborate?
Thanx,
Mark Stuart
Re: How to get date and time from a file
Yes Craig, we are waiting!
-
- Livecode Opensource Backer
- Posts: 16
- Joined: Sun Apr 09, 2006 9:49 pm
- Location: Irvine, CA, USA
Re: How to get date and time from a file
Hi Richard,
On my Win7 computer, it takes 30 seconds just to process the script line "put the detailed files into tFiles".
Here's the script:
Since I have the path and filename, I'll use revCopy to copy the file to a folder with no files in it and apply the "put the detailed files into tFiles" there.
It should process in no time.
Thanx,
Mark Stuart
On my Win7 computer, it takes 30 seconds just to process the script line "put the detailed files into tFiles".
Here's the script:
Code: Select all
command StartProcess pFolder, pFilename
put the defaultFolder into defFolder
set the defaultFolder to pFolder
put the detailed files into tFiles --this line takes 30 seconds to process with close to 15,000 files in the folder pFolder
filter tFiles with "inprocess.txt*" --I'm looking for this file to get the modified date and time of it
set the defaultFolder to defFolder
end StartProcess
It should process in no time.
Thanx,
Mark Stuart
-
- VIP Livecode Opensource Backer
- Posts: 9834
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: How to get date and time from a file
How many files are in that folder, and what is the average length of their file names?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
-
- Livecode Opensource Backer
- Posts: 16
- Joined: Sun Apr 09, 2006 9:49 pm
- Location: Irvine, CA, USA
Re: How to get date and time from a file
Hi Richard and Walt,
The folder that I need to read contains files that contain logging information about processes that run in batch oriented applications - fetch records and send email. When the application starts up it creates a text file called "inprocess.txt". This is so that another process cannot start if this file exists. When the application process is complete, it deletes the file.
Windows Task Scheduler launches the batch application executable every 15 minutes, starting at 6am until 10pm, 7 days a week.
The application creates and writes to a log file while processing. So there's 4 log files for each hour.
So that's 64 files per day to read with "put the detailed files into tFiles".
I haven't "archived" these files in some time so the file count is at 15,100.
The other day the application crashed leaving the "inprocess.txt" in the folder. So we didn't have any email going out for a week.
Yes, I hadn't created a watchdog for this situation, and I hadn't manually checked the folder for this file. So hence the need for an app to check for the existence of this inprocess.txt file. It will read the modified date and time and compare it with today's date.
If it's say, more than 3 hours difference then send myself an email of the issue.
Now that I'm writing this reply, I'm thinking I need to archive these files daily. That will certainly return a better performance with the LC script.
The average file name length is 45 characters. e.g.: ECS_ECS_OUTBOUND_MISSED_B_20140813_104701.log
Walt - I'm currently using your "shell" command to get and parse the date and time directly from the folder. Works great. Thanx for that.
Thanx,
Mark Stuart
The folder that I need to read contains files that contain logging information about processes that run in batch oriented applications - fetch records and send email. When the application starts up it creates a text file called "inprocess.txt". This is so that another process cannot start if this file exists. When the application process is complete, it deletes the file.
Windows Task Scheduler launches the batch application executable every 15 minutes, starting at 6am until 10pm, 7 days a week.
The application creates and writes to a log file while processing. So there's 4 log files for each hour.
So that's 64 files per day to read with "put the detailed files into tFiles".
I haven't "archived" these files in some time so the file count is at 15,100.
The other day the application crashed leaving the "inprocess.txt" in the folder. So we didn't have any email going out for a week.
Yes, I hadn't created a watchdog for this situation, and I hadn't manually checked the folder for this file. So hence the need for an app to check for the existence of this inprocess.txt file. It will read the modified date and time and compare it with today's date.
If it's say, more than 3 hours difference then send myself an email of the issue.
Now that I'm writing this reply, I'm thinking I need to archive these files daily. That will certainly return a better performance with the LC script.
The average file name length is 45 characters. e.g.: ECS_ECS_OUTBOUND_MISSED_B_20140813_104701.log
Walt - I'm currently using your "shell" command to get and parse the date and time directly from the folder. Works great. Thanx for that.
Thanx,
Mark Stuart
-
- VIP Livecode Opensource Backer
- Posts: 9834
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: How to get date and time from a file
No doubt, but it's an interesting problem just the same. I did a quick test on Mac, Linux, and Win, and Win took about 6 times as long as the others to obtain the mod dates on the same random selection of 100 files in a folder of 15,100 files. Some of the speed difference may be accountable by my Win 8.1 install running in a VM, but for most CPU-intensive tasks like filtering I don't see that much of a difference, so I'm wondering if there may be an opportunity to optimize your NTFS by turning off short file name support if possible, or defragging the directory index with contig.exe as describe here:mfstuart wrote:Now that I'm writing this reply, I'm thinking I need to archive these files daily. That will certainly return a better performance with the LC script.
http://stackoverflow.com/questions/1971 ... irectories
Either way, on all three platforms my notion of lineoffset being faster than filter didn't pan out - they took almost the same amount of time, with lineoffset being only about 5% faster.
Interestingly, my Win 8.1 system still only took about 6.5 seconds to run through all 100 file mod dates, or just 650 ms per file. Considering this was in a VM, this suggests there may be something going on with your NTFS if it takes roughly 50 times longer to do the task (I'm running a 3.0 GHz Haswell duel core, but it's just a Pentium so it's not exactly the fastest CPU out there). I'd be interested to know if any file system optimizations improve performance for your system.
Ultimately, you've hit the nail on the head in your OP: we could really use a way to obtain file info like this for a single file, so I just submitted a request for this:
http://quality.runrev.com/show_bug.cgi?id=13182
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
-
- VIP Livecode Opensource Backer
- Posts: 931
- Joined: Thu Nov 13, 2008 6:48 am
- Location: London, UK
Re: How to get date and time from a file
Hi Richard, that link doesn't work anymore... but just wondering (since I need to do it), was this ever addressed in LC?FourthWorld wrote: ↑Wed Aug 13, 2014 8:48 pmUltimately, you've hit the nail on the head in your OP: we could really use a way to obtain file info like this for a single file, so I just submitted a request for this:
http://quality.runrev.com/show_bug.cgi?id=13182
Thanks
Mark
macOS 12.6.5 (Monterey), Xcode 14.2, LC 10.0.0, iOS 15.6.1
Targets: Mac, iOS
Targets: Mac, iOS
Re: How to get date and time from a file
Hi,
tl;dr:
Text operations with huge lists are unbearably slow in the Windows versions of the contemporary LCs. This is acknowledged by the mothership, a crude workaround exists, and maybe it will be fixed some day.
Have fun!
It may be caused by this. I wrote about it here.FourthWorld wrote: ↑Wed Aug 13, 2014 8:48 pm[...] Win took about 6 times as long as the others to obtain the mod dates on the same random selection of 100 files in a folder of 15,100 files. [...]
tl;dr:
Text operations with huge lists are unbearably slow in the Windows versions of the contemporary LCs. This is acknowledged by the mothership, a crude workaround exists, and maybe it will be fixed some day.
Have fun!
All code published by me here was created with Community Editions of LC (thus is GPLv3).
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!