How can I emulate a Javascript dopostback in Livecode?

[Bernd]

Hi Folks

I am a newbie and fighting my way thru the learning curve of Livecode. I have set up a private project in order to dig into the details. I want to capture data from a web site. In order to do so my script reads a HTML page (works) and capture some variable data(works as well). Now, not all data are in the same HTML page. Therefor, I need to grab the links in HTML und proceed to the next page. So far so good. I think, I have that (more or less) under control.

Where I get lost is the fact that not all of the neccessary pages are linked by a "regular" HTML-href-Link but are called by a javascript __doPostBack function. Somewhere at the upper region of the HTML there is a code snippet that looks like follows:

Code: Select all

::
::
function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}
::
::

Later, in the HTML file I have several "function calls" like the following one:

Code: Select all

::
::
<td class="tableNIStdHL"><a id="ctl01_rpLigen_ctl01_lbRunde" href="javascript:[b]__doPostBack('ctl01$rpLigen$ctl01$lbRunde','')">Regional</a>

Is there a way to emulate that using livecode?

Your help is appreciated very much. Thank you.
Bernd

[SoapDog]

Bernd,

You can't process javascript like this with LiveCode. You could try using a RevBrowser window to load the given HTML and interact with it from that context but this probably too error prone. In summary it is really hard to scrap dynamic webpages full of javascript. I don't think it can be done with LiveCode alone, you would need a web context to run the thing and then process it.

[Bernd]

SoapDog

Thanks a lot for your reply. I don't want to "emulate" the whole Javascript stuff. For my "naive" viewpoint the "only thing I have to do is to generate this form.onsubmit thing passing the parameter "ctl01$rpLigen$ctl01$lbRunde" as the event target so the server (it is an ASP server, right). Since I know what your background is I thought this won't even challenge you .

In essence, I am not satisfied with this answer - even if this answer is much appreciated. So, please give it another try.

Bernd

P.S.

For accessing the web page I don't use the revBrowser window but the following code:
ask "Wie ist die URL für die Webseite?"
put it into tWebSite
put URL tURL into Field "Resultate"
::

[Bernd]

Sorry, the code snippet is:
ask "Wie ist die URL für die Webseite?"
put it into tWebSite
put URL tWebSite into Field "Resultate"

[MasterchiefJB]

In order to do so my script reads a HTML page (works) and capture some variable data(works as well). Now, not all data are in the same HTML page. Therefor, I need to grab the links in HTML und proceed to the next page. So far so good. I think, I have that (more or less) under control.

Hello Bernd,

could you explain to me how you managed to accomplish the following parts:
1. read a HTML page
2. capture some variable data
3. grab links in HTML and proceed to the next page

My solution for the first part "1. read a HTML page" is something like this:

Code: Select all

 
    put field "F_Link" into tWeb
    put url tWeb into theHTML
    put theHTML into fld "F_Source"

The problem with this method is that it just reads one part of the html page. It doesn´t read the whole html part and it doesn´t read the javascript part.

- Do you have a solution that reads the entire HTML part of a website and maybe including the javascript parts?

I would also be interested how you managed to do part 2. and 3. Could you give an example on how to capture variable data and on how to grab a link of an HTML chunk?

Your answer would be much appreciated!
Best regards,
Masterchief

[sturgis]

I agree with soapdog. Not easy to scrape. However, if you're willing to put up with the quirks you can do as soapdog mentioned and use revbrowser as a sort of active interactable page holder. It doesn't have to be visible even.
Once the page is loaded, I believe something as simple as

revbrowserexecute(browserId,"__doPostBack('ctl01$rpLigen$ctl01$lbRunde','')" will force a page load in revbrowser at which point you can get the htmltext of the browserinstance then revbrowserback and send it the next __doPostBack.