download a lot of .png files

LiveCode is the premier environment for creating multi-platform solutions for all major operating systems - Windows, Mac OS X, Linux, the Web, Server environments and Mobile platforms. Brand new to LiveCode? Welcome!

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
jmburnod
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2718
Joined: Sat Dec 22, 2007 5:35 pm
Location: Genève
Contact:

download a lot of .png files

Post by jmburnod » Fri Jul 20, 2018 9:05 am

Hi All,
We need download 3500 .png files from a server,
All file name are urlEncoded because some have accented letters.
I tried several methods

1. download a .zip file (160 Mo) using liburldownloadToFile with callback
Since mai 2108 i have got a "request out timed" error for iOS version and OSX version stop before the end.

2. download each file by liburldownloadToFile with callback in a repeat loop
Many files, not ever the same are corrupted with a size 980 octets

3. Using url("binfile:" & tFileS) into url("binfile:" & tFileD)
All files are empty with a size 0 octet although tFileS works as filename for a referenced image.

Thanks in advance for your help
Best regards
Jean-Marc
https://alternatic.ch

jmburnod
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2718
Joined: Sat Dec 22, 2007 5:35 pm
Location: Genève
Contact:

Re: download a lot of .png files

Post by jmburnod » Thu Jul 26, 2018 9:01 am

After many tries, export snapshot from an image with filename = url distant file seems the safest way i found to download 3500 png in a loop.

Code: Select all

put "https://..." into tDistantFolder
repeat with i = 1 to 3500
   put line i of fld "fListImgDL" into tOneImg
   put tDistantFolder & "/" & tOneImg into tfileS
   set the filename of img "MyImage" to tfileS
   export snapshot from img "MyImage" to file tFileD as PNG
   wait 1 milliseconds
end repeat
Best regards
Jean-Marc
https://alternatic.ch

jiml
Posts: 336
Joined: Sat Dec 09, 2006 1:27 am
Location: Los Angeles

Re: download a lot of .png files

Post by jiml » Fri Jul 27, 2018 6:33 pm

Jean-Marc

Code: Select all

put "https://..." into tDistantFolder
repeat with i = 1 to 3500
   put line i of fld "fListImgDL" into tOneImg
   put tDistantFolder & "/" & tOneImg into tfileS
   set the filename of img "MyImage" to tfileS
   export snapshot from img "MyImage" to file tFileD as PNG
   wait 1 milliseconds
end repeat
Your posted code doesn't change the value of tFileD (your local destination file). So it is simply replacing the contents of that file 3500 times!

Also, you might try this:

Code: Select all

 put "https://..." into tDistantFolder
PUT the ID of img "MyImage" into tID
 
repeat with i = 1 to 3500
   put line i of fld "fListImgDL" into tOneImg
   put tDistantFolder & "/" & tOneImg into tfileS
   put url tfileS into img "MyImage"	
   export image id tID to file (tFileD & i) as PNG
   wait 1 milliseconds
end repeat
 
Jim Lambert

jiml
Posts: 336
Joined: Sat Dec 09, 2006 1:27 am
Location: Los Angeles

Re: download a lot of .png files

Post by jiml » Fri Jul 27, 2018 8:02 pm

Also if fld "fListImgDL" only contains the filenames of the remote files then this might be slightly faster, as it uses a variable instead of repeatedly reading a field. And it is uses the actual number of file names in that variable instead of the fixed amount 3500:

Code: Select all

 put "https://..." into tDistantFolder
put the ID of img "MyImage" into tID
put fld "fListImgDL" int tFileList
repeat with i = 1 to -1 of tFileList
   put line i of tFileList into tOneImg
   put tDistantFolder & "/" & tOneImg into tfileS
   put url tfileS into img "MyImage"	
   wait 1 milliseconds
   export image id tID to file (tFileD & i) as PNG
   wait 1 milliseconds	
end repeat

 

jmburnod
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2718
Joined: Sat Dec 22, 2007 5:35 pm
Location: Genève
Contact:

Re: download a lot of .png files

Post by jmburnod » Sun Jul 29, 2018 12:10 am

Hi Jim,
Thank you for your help.
You pointed why i was unable to set the text of one image form a remote file.
I was wrong when i used url("binfile:" & tFileS) into url("binfile:" & tFileD) to download remote files directly. I have to use

Code: Select all

put url tFileS into url("binfile:" & tFileD) -- 22 seconds for 100 img
which is the faster way.

Code: Select all

put url tFileS into img "MyImageP" 
export img "myImageP" to file tFileD as PNG -- 25 seconds for 100 img

Code: Select all

set the filename of  img "MyImageP" to tFileS
export snapshot from img "myImageP" to file tFileD as PNG-- 27 seconds for 100 img
So it is simply replacing the contents of that file 3500 times!
Sorry i have forgotten 2 lines to define tFileD in my second post :oops:
Unfortunately

Code: Select all

repeat with i = 1 to -1 of tFileList 
doesn't work
Best regards
Jean-Marc
https://alternatic.ch

SparkOut
Posts: 2839
Joined: Sun Sep 23, 2007 4:58 pm

Re: download a lot of .png files

Post by SparkOut » Sun Jul 29, 2018 8:10 am

Code: Select all

repeat with i = 1 to the number of lines in tFileList

jmburnod
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2718
Joined: Sat Dec 22, 2007 5:35 pm
Location: Genève
Contact:

Re: download a lot of .png files

Post by jmburnod » Sun Jul 29, 2018 9:16 am

I was surprised that exporting an image changes its resolution ( from 300 dpi to 72 dpi) like export snapshot.
Is it what is expected ?
Using put url tFileS into url("binfile:" tFileD) keeps resolution to 300 dpi

@ Sparkout
Thank you for your post
That was just a feed-back about Jim's suggestion above
Jean-Marc
https://alternatic.ch

jiml
Posts: 336
Joined: Sat Dec 09, 2006 1:27 am
Location: Los Angeles

Re: download a lot of .png files

Post by jiml » Sun Jul 29, 2018 9:29 pm

Code: Select all

repeat with i = 1 to the number of lines in tFileList
Of course. Duh!

Clearly I type faster than I think.

Thanks,
JimL

PBH
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 129
Joined: Sun Feb 20, 2011 4:26 pm
Location: Vancouver Island, BC, Canada. ex.UK
Contact:

Re: download a lot of .png files

Post by PBH » Mon Jul 30, 2018 4:38 am

jmburnod wrote:
Sun Jul 29, 2018 9:16 am
I was surprised that exporting an image changes its resolution ( from 300 dpi to 72 dpi) like export snapshot.
Is it what is expected ?
Using put url tFileS into url("binfile:" tFileD) keeps resolution to 300 dpi
I would think it is expected, when you use export as PNG for example, LC is writing the PNG header/file data and including the relevant imagedata, i.e. creating a new file, whereas using put url tFileS into url("binfile:" tFileD) doesn't re-write the file header or change the file type, so it will keep any embedded metadata.

If you want/need the density of an exported image to be 300 ppi then you're in luck, the one metadata entry that you can include is "density" (effectively, file resolution), but it has to be in an array format, e.g.:

Code: Select all

put "300" into highResPpi["density"] -- the metadata array
export image id tID with metadata highResPpi to file (tFileD & i) as PNG
This is explained in the dictionary entry for export snapshot, but the metadata array works the same for export image too.

Hopefully there will be more metaData options added in future. :)

Paul

jmburnod
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2718
Joined: Sat Dec 22, 2007 5:35 pm
Location: Genève
Contact:

Re: download a lot of .png files

Post by jmburnod » Mon Jul 30, 2018 8:20 am

Hi Paul,
Thanks for explanations. That is clearer for me 8)
Best regards
Jean-Marc
https://alternatic.ch

Post Reply

Return to “Getting Started with LiveCode - Experienced Developers”