Page 1 of 1
Must be a faster way
Posted: Fri Mar 03, 2017 4:56 pm
by adventuresofgreg
Hi there: I'm parsing a large html file and replacing placeholders marked as text strings with different text content, and the way I'm doing it seems to be taking a very long time. It takes 2 seconds on my MacPro, but 19 seconds on my Linux container. Can you think of a more efficient way of doing this?
This is how I'm currently doing this:
Code: Select all
replace "##PNPL" WITH field "TWSprof" in dashboardhtml
replace "##PL" WITH field "accountPL" in dashboardhtml
replace "##CCTNAME" WITH field "anonName" in dashboardhtml
replace "##CLSDTRDS" WITH closedpos in dashboardhtml
... etc...
It's a very convenient way of customizing an html template page, but I think the replace string search is the part that is very inefficient.
Re: Must be a faster way
Posted: Sat Mar 04, 2017 4:34 am
by capellan
Could you post the content of each field in different variables?
It's faster to work with data in variables, for example:
put field "TWSprof" into tData
replace "##PNPL" WITH tData in dashboardhtml
Re: Must be a faster way
Posted: Sat Mar 04, 2017 6:04 am
by dunbarx
What Capellan said.
This:
Code: Select all
on mouseUp
put the ticks into temp
put "xyz" into var
repeat 1000000
replace "xyz" with "aaa" in var -- twice as fast
replace "xyz" with field 1 in var
end repeat
answer the ticks - temp
end mouseUp
Craig Newman
Re: Must be a faster way
Posted: Sat Mar 04, 2017 11:04 am
by Thierry
adventuresofgreg wrote:
I'm parsing a large html file and replacing placeholders marked as text strings with different text content
... etc...
Hi,
Could you be more precise about the size of your html file ,
the number of different placeholders and the average frequency of your placeholders?
Re: Must be a faster way
Posted: Sat Mar 04, 2017 5:33 pm
by jacque
This is what the "merge" command is for. It will do all the replacements at once.
Re: Must be a faster way
Posted: Sat Mar 04, 2017 7:25 pm
by Thierry
This is what the "merge" command is for. It will do all the replacements at once.
I was thinking of merge but I have some benchmark where merge is
200 times slower than an equivalent list of replace, thus my questions before....
Regards,
Thierry
Re: Must be a faster way
Posted: Sun Mar 05, 2017 12:27 am
by adventuresofgreg
Thanks. I think the most important bit of data that should be in a var is the main html file that contains all of the replacement placeholders (from my code below, "dashboardhtml"), which is quite large, and that is already in a variable. I could try to put all of the small bits of data into variables first, but I just don't see that being any faster (you have to place them into the var first anyhow and that means moving the data from the field into the variable.
I'm lookup merge and give that a try.
Code: Select all
put VERYlargeHTMLfile into dashboardhtml
replace "##PNPL" WITH field "TWSprof" & return in dashboardhtml
replace "##PL" WITH field "accountPL" in dashboardhtml
replace "##CCTNAME" WITH field "anonName" in dashboardhtml
replace "##CLSDTRDS" WITH closedpos in dashboardhtml
replace "##CCUNTVLU" WITH field "myvalue" in dashboardhtml
replace "##CCUNTTYP" WITH Atype in dashboardhtml
replace "##MS" WITH the label of button "multisysmenu" in dashboardhtml
replace "##SYMBLS" WITH loadedsymbs in dashboardhtml
replace "##FLTRS" WITH "no filters" in dashboardhtml
replace "##PS" WITH positions in dashboardhtml
replace "##CPTL" WITH field "capital" in dashboardhtml
replace "##RSK" WITH field "riskperbar" in dashboardhtml
replace "##indexembed" WITH theINDEX in dashboardhtml
Re: Must be a faster way
Posted: Sun Mar 05, 2017 4:45 am
by jacque
I'd be curious to know what you find out. Using replace, the engine has to parse the long html file 12 times. Using merge, I think it does it all in one pass.
Re: Must be a faster way
Posted: Sun Mar 05, 2017 4:51 am
by capellan
Probably I need to confirm this, but as far as I know
everytime that you access data inside a field, then
Livecode engine sends messages... but this could have
changed in recent Livecode versions.
Richard Gaskin have an utility that could confirm this:
4w Props or 4w umbrella.
Re: Must be a faster way
Posted: Sun Mar 05, 2017 2:33 pm
by AxWald
Hi,
dunno if it helps here, but a while ago I did some timings & optimizations with replace loops. Got some very nice speed improvements for large files. To be found
here.
Have fun!
Re: Must be a faster way
Posted: Wed Mar 08, 2017 3:27 pm
by adventuresofgreg
Merge is about 3 times faster! THANKS!