Browser Widget Timing

LiveCode is the premier environment for creating multi-platform solutions for all major operating systems - Windows, Mac OS X, Linux, the Web, Server environments and Mobile platforms. Brand new to LiveCode? Welcome!

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
lohill
Posts: 770
Joined: Tue Dec 08, 2009 6:37 pm
Location: San Diego, CA USA

Browser Widget Timing

Post by lohill » Fri Feb 19, 2021 7:52 pm

I want to use the Browser Widget to retrieve data from a collection of similar but different web pages. I want to get the data quickly and reliably. The URLs are contained in the field "URLS". The following code (abbreviated) seems to work.

Code: Select all

on mouseUp pMouseButton
   repeat for each line tUrl in field "URLS"
      set the url of widget "Browser" to tUrl
      wait 60 ticks
      put the htmltext of widget "Browser" into tText
      --parse tText to get the data I want
   end repeat
end mouseUp
The pause between setting the URL for the widget and getting the htmltext is necessary to give the widget time to finish the retrieval from the web page. I know that if I cut the wait to say 10 ticks that the retrieve will not be complete and some if the data I want will not be supplied. I suppose, depending in the web traffic. that my wait of 60 ticks may not be enough on some occasions.

Since I would like to have the retrievals of data be as close as possible (i.e. No wasted ticks) I tried a different approach. I have created a custom variable in the widget called cBusy and made some code modifications to now look like the following.

I added this code to the browser widget:

Code: Select all

on browserNavigateBegin pUrl
   set the cBusy of me to true
end browserNavigateBegin

on browserNavigateComplete pUrl
   set the cBusy of me to false
end browserNavigateComplete
My mouseUp code was changed to this;

Code: Select all

on mouseUp pMouseButton
   repeat for each line tUrl in field "URLS"
      if tUrl="ALL" then next repeat
      set the cBusy of widget "Browser" to false
      set the url of widget "Browser" to tUrl
      repeat
         wait 5 ticks
         put the cBusy of widget "Browser" into tFlag
         if tFlag is false then exit repeat
      end repeat
      put the htmltext of widget "Browser" into tText
      --parse the text to get the data I want
   end repeat
end mouseUp
This executes but only gives me the data for the last website. If I change the wait from 5 ticks to 60 ticks it does work. I put break points in the widgets code at the two places where I have "set the cBusy of me to" and, interestingly enough, in the 60 tick version I has all my answers before any stop. When It finally did break it was at "browserNavigateBegin".

Any ideas?

Thanks,
Larry

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Browser Widget Timing

Post by FourthWorld » Fri Feb 19, 2021 8:03 pm

Do you want to render the page for display to the user, or do download the page data?

The browser widget is great for rendering, but complex overkill for a simple download.

For the latter, see the GET url command.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

lohill
Posts: 770
Joined: Tue Dec 08, 2009 6:37 pm
Location: San Diego, CA USA

Re: Browser Widget Timing

Post by lohill » Sat Feb 20, 2021 7:19 pm

Thanks Richard,

I had no idea there was a get url command. I thought I had to render it first and then get the hymltext for the information I wanted. Will I get the htmltext basically instantly or will I have to do some waiting? There are times when I want to see the page but not for this case.

Larry

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7214
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Browser Widget Timing

Post by jacque » Sat Feb 20, 2021 7:47 pm

The repeat loop is blocking which is why you don't see browserNavigateComplete until the loop is finished, and at that point only the last url is being processed. Ideally you'd use "send" instead of the repeat, but you can fix the problem by adding "with messages" to the wait command inside the loop.

But using "get url" will be faster and preferable since no rendering is needed, and also has the advantage that the next line of the repeat loop won't run until the URL has been fetched or the timeout is expired. You don't need a wait, though it would be good to check if a timeout occurred so you can try again.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: Browser Widget Timing

Post by bogs » Sat Feb 20, 2021 10:08 pm

I would also suggest that if you have planned time to wait, then add "with messages" to the end of it, so house keeping can be done while your already waiting. It would look like -

Code: Select all

repeat
         wait 5 ticks with messages 
         -- continue on...
Image

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Browser Widget Timing

Post by FourthWorld » Sat Feb 20, 2021 11:35 pm

lohill wrote:
Sat Feb 20, 2021 7:19 pm
I had no idea there was a get url command.
There's also a POST command for sending data to a server.
Will I get the htmltext basically instantly or will I have to do some waiting?
That's the question every Internet user has. ;)

Round trip time from your sending the request until you receive the reply will vary, depending on your connection speed, your server's connection speed, server processing time to satisfy the request, server load, how many hops each route takes, and more.

Small static pages often take less than a second. IMNSHO if it takes longer than six seconds abandon that server and use anything else. ;)
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

Post Reply

Return to “Getting Started with LiveCode - Experienced Developers”