Page 1 of 1
download a lot of .png files
Posted: Fri Jul 20, 2018 9:05 am
by jmburnod
Hi All,
We need download 3500 .png files from a server,
All file name are urlEncoded because some have accented letters.
I tried several methods
1. download a .zip file (160 Mo) using liburldownloadToFile with callback
Since mai 2108 i have got a "request out timed" error for iOS version and OSX version stop before the end.
2. download each file by liburldownloadToFile with callback in a repeat loop
Many files, not ever the same are corrupted with a size 980 octets
3. Using url("binfile:" & tFileS) into url("binfile:" & tFileD)
All files are empty with a size 0 octet although tFileS works as filename for a referenced image.
Thanks in advance for your help
Best regards
Jean-Marc
Re: download a lot of .png files
Posted: Thu Jul 26, 2018 9:01 am
by jmburnod
After many tries, export snapshot from an image with filename = url distant file seems the safest way i found to download 3500 png in a loop.
Code: Select all
put "https://..." into tDistantFolder
repeat with i = 1 to 3500
put line i of fld "fListImgDL" into tOneImg
put tDistantFolder & "/" & tOneImg into tfileS
set the filename of img "MyImage" to tfileS
export snapshot from img "MyImage" to file tFileD as PNG
wait 1 milliseconds
end repeat
Best regards
Jean-Marc
Re: download a lot of .png files
Posted: Fri Jul 27, 2018 6:33 pm
by jiml
Jean-Marc
Code: Select all
put "https://..." into tDistantFolder
repeat with i = 1 to 3500
put line i of fld "fListImgDL" into tOneImg
put tDistantFolder & "/" & tOneImg into tfileS
set the filename of img "MyImage" to tfileS
export snapshot from img "MyImage" to file tFileD as PNG
wait 1 milliseconds
end repeat
Your posted code doesn't change the value of tFileD (your local destination file). So it is simply replacing the contents of that file 3500 times!
Also, you might try this:
Code: Select all
put "https://..." into tDistantFolder
PUT the ID of img "MyImage" into tID
repeat with i = 1 to 3500
put line i of fld "fListImgDL" into tOneImg
put tDistantFolder & "/" & tOneImg into tfileS
put url tfileS into img "MyImage"
export image id tID to file (tFileD & i) as PNG
wait 1 milliseconds
end repeat
Jim Lambert
Re: download a lot of .png files
Posted: Fri Jul 27, 2018 8:02 pm
by jiml
Also if fld "fListImgDL" only contains the filenames of the remote files then this might be slightly faster, as it uses a variable instead of repeatedly reading a field. And it is uses the actual number of file names in that variable instead of the fixed amount 3500:
Code: Select all
put "https://..." into tDistantFolder
put the ID of img "MyImage" into tID
put fld "fListImgDL" int tFileList
repeat with i = 1 to -1 of tFileList
put line i of tFileList into tOneImg
put tDistantFolder & "/" & tOneImg into tfileS
put url tfileS into img "MyImage"
wait 1 milliseconds
export image id tID to file (tFileD & i) as PNG
wait 1 milliseconds
end repeat
Re: download a lot of .png files
Posted: Sun Jul 29, 2018 12:10 am
by jmburnod
Hi Jim,
Thank you for your help.
You pointed why i was unable to set the text of one image form a remote file.
I was wrong when i used url("binfile:" & tFileS) into url("binfile:" & tFileD) to download remote files directly. I have to use
Code: Select all
put url tFileS into url("binfile:" & tFileD) -- 22 seconds for 100 img
which is the faster way.
Code: Select all
put url tFileS into img "MyImageP"
export img "myImageP" to file tFileD as PNG -- 25 seconds for 100 img
Code: Select all
set the filename of img "MyImageP" to tFileS
export snapshot from img "myImageP" to file tFileD as PNG-- 27 seconds for 100 img
So it is simply replacing the contents of that file 3500 times!
Sorry i have forgotten 2 lines to define tFileD in my second post
Unfortunately
Code: Select all
repeat with i = 1 to -1 of tFileList
doesn't work
Best regards
Jean-Marc
Re: download a lot of .png files
Posted: Sun Jul 29, 2018 8:10 am
by SparkOut
Code: Select all
repeat with i = 1 to the number of lines in tFileList
Re: download a lot of .png files
Posted: Sun Jul 29, 2018 9:16 am
by jmburnod
I was surprised that exporting an image changes its resolution ( from 300 dpi to 72 dpi) like export snapshot.
Is it what is expected ?
Using put url tFileS into url("binfile:" tFileD) keeps resolution to 300 dpi
@ Sparkout
Thank you for your post
That was just a feed-back about Jim's suggestion above
Jean-Marc
Re: download a lot of .png files
Posted: Sun Jul 29, 2018 9:29 pm
by jiml
Code: Select all
repeat with i = 1 to the number of lines in tFileList
Of course. Duh!
Clearly I type faster than I think.
Thanks,
JimL
Re: download a lot of .png files
Posted: Mon Jul 30, 2018 4:38 am
by PBH
jmburnod wrote: ↑Sun Jul 29, 2018 9:16 am
I was surprised that exporting an image changes its resolution ( from 300 dpi to 72 dpi) like export snapshot.
Is it what is expected ?
Using put url tFileS into url("binfile:" tFileD) keeps resolution to 300 dpi
I would think it is expected, when you use export as PNG for example, LC is writing the PNG header/file data and including the relevant imagedata, i.e. creating a new file, whereas using put url tFileS into url("binfile:" tFileD) doesn't re-write the file header or change the file type, so it will keep any embedded metadata.
If you want/need the density of an exported image to be 300 ppi then you're in luck, the one metadata entry that you can include is "density" (effectively, file resolution), but it has to be in an array format, e.g.:
Code: Select all
put "300" into highResPpi["density"] -- the metadata array
export image id tID with metadata highResPpi to file (tFileD & i) as PNG
This is explained in the dictionary entry for export snapshot, but the metadata array works the same for export image too.
Hopefully there will be more metaData options added in future.
Paul
Re: download a lot of .png files
Posted: Mon Jul 30, 2018 8:20 am
by jmburnod
Hi Paul,
Thanks for explanations. That is clearer for me
Best regards
Jean-Marc