download a lot of .png files
Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller
-
- VIP Livecode Opensource Backer
- Posts: 2718
- Joined: Sat Dec 22, 2007 5:35 pm
- Location: Genève
- Contact:
download a lot of .png files
Hi All,
We need download 3500 .png files from a server,
All file name are urlEncoded because some have accented letters.
I tried several methods
1. download a .zip file (160 Mo) using liburldownloadToFile with callback
Since mai 2108 i have got a "request out timed" error for iOS version and OSX version stop before the end.
2. download each file by liburldownloadToFile with callback in a repeat loop
Many files, not ever the same are corrupted with a size 980 octets
3. Using url("binfile:" & tFileS) into url("binfile:" & tFileD)
All files are empty with a size 0 octet although tFileS works as filename for a referenced image.
Thanks in advance for your help
Best regards
Jean-Marc
We need download 3500 .png files from a server,
All file name are urlEncoded because some have accented letters.
I tried several methods
1. download a .zip file (160 Mo) using liburldownloadToFile with callback
Since mai 2108 i have got a "request out timed" error for iOS version and OSX version stop before the end.
2. download each file by liburldownloadToFile with callback in a repeat loop
Many files, not ever the same are corrupted with a size 980 octets
3. Using url("binfile:" & tFileS) into url("binfile:" & tFileD)
All files are empty with a size 0 octet although tFileS works as filename for a referenced image.
Thanks in advance for your help
Best regards
Jean-Marc
https://alternatic.ch
-
- VIP Livecode Opensource Backer
- Posts: 2718
- Joined: Sat Dec 22, 2007 5:35 pm
- Location: Genève
- Contact:
Re: download a lot of .png files
After many tries, export snapshot from an image with filename = url distant file seems the safest way i found to download 3500 png in a loop.
Best regards
Jean-Marc
Code: Select all
put "https://..." into tDistantFolder
repeat with i = 1 to 3500
put line i of fld "fListImgDL" into tOneImg
put tDistantFolder & "/" & tOneImg into tfileS
set the filename of img "MyImage" to tfileS
export snapshot from img "MyImage" to file tFileD as PNG
wait 1 milliseconds
end repeat
Jean-Marc
https://alternatic.ch
Re: download a lot of .png files
Jean-Marc
Your posted code doesn't change the value of tFileD (your local destination file). So it is simply replacing the contents of that file 3500 times!
Also, you might try this:
Jim Lambert
Code: Select all
put "https://..." into tDistantFolder
repeat with i = 1 to 3500
put line i of fld "fListImgDL" into tOneImg
put tDistantFolder & "/" & tOneImg into tfileS
set the filename of img "MyImage" to tfileS
export snapshot from img "MyImage" to file tFileD as PNG
wait 1 milliseconds
end repeat
Also, you might try this:
Code: Select all
put "https://..." into tDistantFolder
PUT the ID of img "MyImage" into tID
repeat with i = 1 to 3500
put line i of fld "fListImgDL" into tOneImg
put tDistantFolder & "/" & tOneImg into tfileS
put url tfileS into img "MyImage"
export image id tID to file (tFileD & i) as PNG
wait 1 milliseconds
end repeat
Re: download a lot of .png files
Also if fld "fListImgDL" only contains the filenames of the remote files then this might be slightly faster, as it uses a variable instead of repeatedly reading a field. And it is uses the actual number of file names in that variable instead of the fixed amount 3500:
Code: Select all
put "https://..." into tDistantFolder
put the ID of img "MyImage" into tID
put fld "fListImgDL" int tFileList
repeat with i = 1 to -1 of tFileList
put line i of tFileList into tOneImg
put tDistantFolder & "/" & tOneImg into tfileS
put url tfileS into img "MyImage"
wait 1 milliseconds
export image id tID to file (tFileD & i) as PNG
wait 1 milliseconds
end repeat
-
- VIP Livecode Opensource Backer
- Posts: 2718
- Joined: Sat Dec 22, 2007 5:35 pm
- Location: Genève
- Contact:
Re: download a lot of .png files
Hi Jim,
Thank you for your help.
You pointed why i was unable to set the text of one image form a remote file.
I was wrong when i used url("binfile:" & tFileS) into url("binfile:" & tFileD) to download remote files directly. I have to use
which is the faster way.
Unfortunately
doesn't work
Best regards
Jean-Marc
Thank you for your help.
You pointed why i was unable to set the text of one image form a remote file.
I was wrong when i used url("binfile:" & tFileS) into url("binfile:" & tFileD) to download remote files directly. I have to use
Code: Select all
put url tFileS into url("binfile:" & tFileD) -- 22 seconds for 100 img
Code: Select all
put url tFileS into img "MyImageP"
export img "myImageP" to file tFileD as PNG -- 25 seconds for 100 img
Code: Select all
set the filename of img "MyImageP" to tFileS
export snapshot from img "myImageP" to file tFileD as PNG-- 27 seconds for 100 img
Sorry i have forgotten 2 lines to define tFileD in my second postSo it is simply replacing the contents of that file 3500 times!
Unfortunately
Code: Select all
repeat with i = 1 to -1 of tFileList
Best regards
Jean-Marc
https://alternatic.ch
Re: download a lot of .png files
Code: Select all
repeat with i = 1 to the number of lines in tFileList
-
- VIP Livecode Opensource Backer
- Posts: 2718
- Joined: Sat Dec 22, 2007 5:35 pm
- Location: Genève
- Contact:
Re: download a lot of .png files
I was surprised that exporting an image changes its resolution ( from 300 dpi to 72 dpi) like export snapshot.
Is it what is expected ?
Using put url tFileS into url("binfile:" tFileD) keeps resolution to 300 dpi
@ Sparkout
Thank you for your post
That was just a feed-back about Jim's suggestion above
Jean-Marc
Is it what is expected ?
Using put url tFileS into url("binfile:" tFileD) keeps resolution to 300 dpi
@ Sparkout
Thank you for your post
That was just a feed-back about Jim's suggestion above
Jean-Marc
https://alternatic.ch
Re: download a lot of .png files
Code: Select all
repeat with i = 1 to the number of lines in tFileList
Clearly I type faster than I think.
Thanks,
JimL
-
- VIP Livecode Opensource Backer
- Posts: 129
- Joined: Sun Feb 20, 2011 4:26 pm
- Location: Vancouver Island, BC, Canada. ex.UK
- Contact:
Re: download a lot of .png files
I would think it is expected, when you use export as PNG for example, LC is writing the PNG header/file data and including the relevant imagedata, i.e. creating a new file, whereas using put url tFileS into url("binfile:" tFileD) doesn't re-write the file header or change the file type, so it will keep any embedded metadata.
If you want/need the density of an exported image to be 300 ppi then you're in luck, the one metadata entry that you can include is "density" (effectively, file resolution), but it has to be in an array format, e.g.:
Code: Select all
put "300" into highResPpi["density"] -- the metadata array
export image id tID with metadata highResPpi to file (tFileD & i) as PNG
Hopefully there will be more metaData options added in future.
Paul
-
- VIP Livecode Opensource Backer
- Posts: 2718
- Joined: Sat Dec 22, 2007 5:35 pm
- Location: Genève
- Contact:
Re: download a lot of .png files
Hi Paul,
Thanks for explanations. That is clearer for me
Best regards
Jean-Marc
Thanks for explanations. That is clearer for me
Best regards
Jean-Marc
https://alternatic.ch