HTML source code within Web Browser?

Zax · Post by **Zax** » Sun Sep 15, 2019 11:07 am

Hello,

Is there a way to retrieve the source code of a web page - a distant URL - displayed in a LC 9 web browser (widget or revBrowserSet command)?

Thank you.

bogs · Post by **bogs** » Sun Sep 15, 2019 11:44 am

*IF* I understand the question correctly, all you'd need to do is get the page the browser is on, and put url before it, like this.

Code: Select all

put url "https://www.google.com" into field "myWebPageSourceCodeField"

Which results in the source code for the address, like this -

Code: Select all

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><head><meta content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for." name="description"><meta content="noodp" name="robots"><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/logos/doodles/2019/celebrating-ynes-mexia-6753651837108230-l.png" itemprop="image"><meta content="Celebrating Ynés Mexía" property="twitter:title"><meta content="Celebrating Ynés Mexía! #GoogleDoodle #HispanicHeritageMonth2019" property="twitter:description"><meta content="Celebrating Ynés Mexía! #GoogleDoodle #HispanicHeritageMonth2019" property="og:description"><meta content="summary_large_image" property="twitter:card"><meta content="@GoogleDoodles" property="twitter:site"><meta content="https://www.google.com/logos/doodles/2019/celebrating-ynes-mexia-6753651837108230-2x.jpg" property="twitter:image"><meta content="https://www.google.com/logos/doodles/2019/celebrating-ynes-mexia-6753651837108230-2x.jpg" property="og:image"><meta content="1000" property="og:image:width"><meta content="400" property="og:image:height"><title>Google</title><script nonce="+6yiUmesSL4i82NIbs1JaQ==">(function(){window.google={kEI:'bhR-XciTMsSJ5wLLhbHwCA',kEXPI:'0,1353746,5663,730,224,510,19,1046,2082,1069,378,207,1017,53,1431,577,2,124,10,631,82,226,112,233,305,212,32,2329570,280,329244,1294,12383,4855,32692,15247,867,12163,5281,1100,853,2482,2,2,6801,364,3319,5505,224,2218,5942,1119,2,1306,2432,1361,4323,3700,1268,773,2249,1408,4483,9,1747,6412,669,1050,1808,1478,7,1,2,488,2044,5766,1,2837,305,5071,226,2016,38,920,873,1217,1364,4347,1558,1503,2,631,1140,2,2098,1812,5634,622,1416,1466,20,317,1118,902,1153,975,1,369,2777,519,400,994,1285,6,109,941,1746,885,82,48,553,11,14,667,612,2213,201,324,5,67,1184,840,324,193,1466,8,48,158,662,2620,818,260,52,1135,1,3,2063,606,1839,186,593,82,1100,143,377,360,142,184,1261,747,61,158,98,112,44,207,1,1,894,52,275,1284,16,85,335,82,505,1254,524,142,44,100,684,811,607,430,44,1339,29,700,501,557,669,1555,45,1,824,133,521,252,181,849,187,331,29,495,7,504,224,592,523,764,287,1227,373,228,83,425,240,129,270,133,4,281,1690,367,59,331,626,3333,235,172,293,1,36,2,662,43,11,110,2,823,119,21,34,58,177,25,267,75,342,1035,27,26,80,157,291,229,133,190,490,18,56,441,312,111,102,129,390,30,70,498,344,993,3,112,136,24,500,836,116,322,182,5892262,6000372,282,2799686,4,1572,549,333,444,1,2,80,1,900,580,7,309,1,8,1,2,2132,1,1,1,1,1,414,1,748,141,59,726,3,7,563,1,3749,32,1,5,4,1,73,22303273,1662865',authuser:0,kscs:'c9c918f0_bhR-XciTMsSJ5wLLhbHwCA',kGL:'US',kBL:'7eXu'};google.sn='webhp';google.kHL='en';google.jsfs='Ffpdje';})();(function(){google.lc=[];google.li=0;google.getEI=function(a){for(var b;a&&(!a.getAttribute||!(b=a.getAttribute("eid")));)a=a.parentNode;return b||google.kEI};google.getLEI=function(a){for(var b=null;a&&(!a.getAttribute||!(b=a.getAttribute("leid")));)a=a.parentNode;return b};google.https=function(){return"https:"==window.location.protocol};google.ml=function(){return null};google.time=function(){return(new Date).getTime()};google.log=function(a,b,e,c,g){if(a=google.logUrl(a,b,e,c,g)){b=new Image;var d=google.lc,f=google.li;d[f]=b;b.onerror=b.onload=b.onabort=function(){delete d[f]};google.vel&&google.vel.lu&&google.vel.lu(a);b.src=a;google.li=f+1}};google.logUrl=function(a,b,e,c,g){var d="",f=google.ls||"";e||-1!=b.search("&ei=")||(d="&ei="+google.getEI(c),-1==b.search("&lei=")&&(c=google.getLEI(c))&&(d+="&lei="+c));c="";!e&&google.cshid&&-1==b.search("&cshid=")&&"slh"!=a&&(c="&cshid="+google.cshid);a=e||"/"+(g||"gen_204")+"?atyp=i&ct="+a+"&cad="+b+d+f+"&zx="+google.time()+c;/^http:/i.test(a)&&google.https()&&(google.ml(Error("a"),!1,{src:a,glmm:1}),a="");return a};}).call(this);(function(){google.y={};google.x=function(a,b){if(a)var c=a.id;else{do c=Math.random();while(google.y[c])}google.y[c]=[a,b];return!1};google.lm=[];google.plm=function(a){google.lm.push.apply(google.lm,a)};google.lq=[];google.load=function(a,b,c){google.lq.push([[a],b,c])};google.loadAll=function(a,b){google.lq.push([a,b])};}).call(this);google.f={};var a=window.location,b=a.href.indexOf("#");if(0<=b){var c=a.href.substring(b+1);/(^|&)q=/.test(c)&&-1==c.indexOf("#")&&a.replace("/search?"+c.replace(/(^|&)fp=[^&]*/g,"")+"&cad=h")};</script><style>#gbar,#guser{font-size:13px;padding-top:1px !important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:right}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;vertical-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{color:#900 !important}
</style><style>body,td,a,p,.h{font-family:arial,sans-serif}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac_m td{line-height:17px}form{margin-bottom:20px}.h{color:#36c}.q{color:#00c}.ts td{padding:0}.ts{border-collapse:collapse}em{font-weight:bold;font-style:normal}.lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17px arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;margin-left:4px}input{font-family:inherit}a.gb1,a.gb2,a.gb3,a.gb4{color:#11c !important}body{background:#fff;color:black}a{color:#11c;text-decoration:none}a:hover,a:active{text-decoration:underline}.fl a{color:#36c}a:visited{color:#551a8b}a.gb1,a.gb4{text-decoration:underline}a.gb3:hover{text-decoration:none}#ghead a.gb2:hover{color:#fff !important}.sblc{padding-top:5px}.sblc a{display:block;margin:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#eee;border:solid 1px;border-color:#ccc #999 #999 #ccc;height:30px}.lsbb{display:block}.ftl,#fll a{display:inline-block;margin:0 12px}.lsb{background:url(/images/nav_logo229.png) 0 -261px repeat-x;border:none;color:#000;cursor:pointer;height:30px;margin:0;outline:0;font:15px arial,sans-serif;vertical-align:top}.lsb:active{background:#ccc}.lst:focus{outline:none}</style><script nonce="+6yiUmesSL4i82NIbs1JaQ=="></script></head><body bgcolor="#fff"><script nonce="+6yiUmesSL4i82NIbs1JaQ==">(function(){var src='/images/nav_logo229.png';var iesg=false;document.body.onload = function(){window.n && window.n();if (document.images){new Image().src=src;}
if (!iesg){document.f&&document.f.q.focus();document.gbqf&&document.gbqf.q.focus();}
}
})();</script><div id="mngb"> <div id=gbar><nobr><b class=gb1>Search</b> <a class=gb1 href="https://www.google.com/imghp?hl=en&tab=wi">Images</a> <a class=gb1 href="https://maps.google.com/maps?hl=en&tab=wl">Maps</a> <a class=gb1 href="https://play.google.com/?hl=en&tab=w8">Play</a> <a class=gb1 href="https://www.youtube.com/?gl=US&tab=w1">YouTube</a> <a class=gb1 href="https://news.google.com/nwshp?hl=en&tab=wn">News</a> <a class=gb1 href="https://mail.google.com/mail/?tab=wm">Gmail</a> <a class=gb1 href="https://drive.google.com/?tab=wo">Drive</a> <a class=gb1 style="text-decoration:none" href="https://www.google.com/intl/en/about/products?tab=wh"><u>More</u> &raquo;</a></nobr></div><div id=guser width=100%><nobr><span id=gbn class=gbi></span><span id=gbf class=gbf></span><span id=gbe></span><a href="http://www.google.com/history/optout?hl=en" class=gb4>Web History</a> | <a  href="/preferences?hl=en" class=gb4>Settings</a> | <a target=_top id=gb_70 href="https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=https://www.google.com/" class=gb4>Sign in</a></nobr></div><div class=gbh style=left:0></div><div class=gbh style=right:0></div> </div><center><br clear="all" id="lgpd"><div id="lga"><a href="/search?ie=UTF-8&amp;q=Yn%C3%A9s+Mex%C3%ADa&amp;oi=ddle&amp;ct=139894537&amp;hl=en&amp;kgmid=/m/0785vz&amp;sa=X&amp;ved=0ahUKEwjIgqCa0dLkAhXExFkKHctCDI4QPQgD"><img alt="Celebrating Ynés Mexía" border="0" height="220" src="/logos/doodles/2019/celebrating-ynes-mexia-6753651837108230-l.png" title="Celebrating Ynés Mexía" width="550" id="hplogo"><br></a><br></div><form action="/search" name="f"><table cellpadding="0" cellspacing="0"><tr valign="top"><td width="25%">&nbsp;</td><td align="center" nowrap=""><input name="ie" value="ISO-8859-1" type="hidden"><input value="en" name="hl" type="hidden"><input name="source" type="hidden" value="hp"><input name="biw" type="hidden"><input name="bih" type="hidden"><div class="ds" style="height:32px;margin:4px 0"><input style="color:#000;margin:0;padding:5px 8px 0 6px;vertical-align:top" autocomplete="off" class="lst" value="" title="Google Search" maxlength="2048" name="q" size="57"></div><br style="line-height:0"><span class="ds"><span class="lsbb"><input class="lsb" value="Google Search" name="btnG" type="submit"></span></span><span class="ds"><span class="lsbb"><input class="lsb" id="tsuid1" value="I'm Feeling Lucky" name="btnI" type="submit"><script nonce="+6yiUmesSL4i82NIbs1JaQ==">(function(){var id='tsuid1';document.getElementById(id).onclick = function(){if (this.form.q.value){this.checked = 1;if (this.form.iflsig)this.form.iflsig.disabled = false;}
else top.location='/doodles/';};})();</script></span></span></td><td class="fl sblc" align="left" nowrap="" width="25%"><a href="/advanced_search?hl=en&amp;authuser=0">Advanced search</a><a href="/language_tools?hl=en&amp;authuser=0">Language tools</a></td></tr></table><input id="gbv" name="gbv" type="hidden" value="1"><script nonce="+6yiUmesSL4i82NIbs1JaQ==">(function(){var a,b="1";if(document&&document.getElementById)if("undefined"!=typeof XMLHttpRequest)b="2";else if("undefined"!=typeof ActiveXObject){var c,d,e=["MSXML2.XMLHTTP.6.0","MSXML2.XMLHTTP.3.0","MSXML2.XMLHTTP","Microsoft.XMLHTTP"];for(c=0;d=e[c++];)try{new ActiveXObject(d),b="2"}catch(h){}}a=b;if("2"==a&&-1==location.search.indexOf("&gbv=2")){var f=google.gbvu,g=document.getElementById("gbv");g&&(g.value=a);f&&window.setTimeout(function(){location.href=f},0)};}).call(this);</script></form><div id="gac_scont"></div><div style="font-size:83%;min-height:3.5em"><br></div><span id="footer"><div style="font-size:10pt"><div style="margin:19px auto;text-align:center" id="fll"><a href="/intl/en/ads/">Advertising Programs</a><a href="/services/">Business Solutions</a><a href="/intl/en/about.html">About Google</a></div></div><p style="color:#767676;font-size:8pt">&copy; 2019 - <a href="/intl/en/policies/privacy/">Privacy</a> - <a href="/intl/en/policies/terms/">Terms</a></p></span></center><script nonce="+6yiUmesSL4i82NIbs1JaQ==">(function(){window.google.cdo={height:0,width:0};(function(){var a=window.innerWidth,b=window.innerHeight;if(!a||!b){var c=window.document,d="CSS1Compat"==c.compatMode?c.documentElement:c.body;a=d.clientWidth;b=d.clientHeight}a&&b&&(a!=google.cdo.width||b!=google.cdo.height)&&google.log("","","/client_204?&atyp=i&biw="+a+"&bih="+b+"&ei="+google.kEI);}).call(this);})();(function(){var u='/xjs/_/js/k\x3dxjs.hp.en_US.CqKx7coWtKw.O/m\x3dsb_he,d/am\x3dHhZsAg/d\x3d1/rs\x3dACT90oFMjf3G2i6qraPPdavAZsbh53V5OQ';setTimeout(function(){var a=document.createElement("script");a.src=u;google.timers&&google.timers.load&&google.tick&&google.tick("load","xjsls");document.body.appendChild(a)},0);})();(function(){window.google.xjsu='/xjs/_/js/k\x3dxjs.hp.en_US.CqKx7coWtKw.O/m\x3dsb_he,d/am\x3dHhZsAg/d\x3d1/rs\x3dACT90oFMjf3G2i6qraPPdavAZsbh53V5OQ';})();function _DumpException(e){throw e;}
function _F_installCss(c){}
(function(){google.spjs=false;google.snet=true;google.em=[];google.emw=false;})();(function(){var pmc='{\x22JnTczA\x22:{},\x22Qnk92g\x22:{},\x22RWGcrA\x22:{},\x22U5B21g\x22:{},\x22YFCs/g\x22:{},\x22YQeDTA\x22:{},\x22ZI/YVQ\x22:{},\x22d\x22:{},\x22mVopag\x22:{},\x22sb_he\x22:{\x22agen\x22:true,\x22cgen\x22:true,\x22client\x22:\x22heirloom-hp\x22,\x22dh\x22:true,\x22dhqt\x22:true,\x22ds\x22:\x22\x22,\x22ffql\x22:\x22en\x22,\x22fl\x22:true,\x22host\x22:\x22google.com\x22,\x22isbh\x22:28,\x22jsonp\x22:true,\x22msgs\x22:{\x22cibl\x22:\x22Clear Search\x22,\x22dym\x22:\x22Did you mean:\x22,\x22lcky\x22:\x22I\\u0026#39;m Feeling Lucky\x22,\x22lml\x22:\x22Learn more\x22,\x22oskt\x22:\x22Input tools\x22,\x22psrc\x22:\x22This search was removed from your \\u003Ca href\x3d\\\x22/history\\\x22\\u003EWeb History\\u003C/a\\u003E\x22,\x22psrl\x22:\x22Remove\x22,\x22sbit\x22:\x22Search by image\x22,\x22srch\x22:\x22Google Search\x22},\x22ovr\x22:{},\x22pq\x22:\x22\x22,\x22refpd\x22:true,\x22rfs\x22:[],\x22sbpl\x22:24,\x22sbpr\x22:24,\x22scd\x22:10,\x22sce\x22:5,\x22stok\x22:\x22S1ROcP1UAQNCcvp5dvlzrcnWmJ8\x22,\x22uhde\x22:false}}';google.pmc=JSON.parse(pmc);})();</script>        </body></html>

: Get to the source of your problems...

Zax · Post by **Zax** » Sun Sep 15, 2019 12:20 pm

Simple and great !
Thanks bogs

bogs · Post by **bogs** » Sun Sep 15, 2019 12:37 pm

Glad it helped

jacque · Post by **jacque** » Sun Sep 15, 2019 4:18 pm

If the browser widget already is displaying the URL, you can also:

get the htmltext of widget "browser"

bogs · Post by **bogs** » Sun Sep 15, 2019 4:41 pm

I'm curious, since I can't actually conduct the experiment at the moment, would something like -

Code: Select all

put the htmlText of widget "browser" into {container}

work?

jacque · Post by **jacque** » Sun Sep 15, 2019 4:50 pm

Yes, that works

[-hh] · Post by **[-hh]** » Sun Sep 15, 2019 10:30 pm

Note that "the htmltext" is the original htmltext of the page, NOT the generated one (which is working). Can be quite different...

bogs · Post by **bogs** » Sun Sep 15, 2019 10:46 pm

Hmmm, lemme see if I got this right then.

If you do the first thing I said, "put url (page address) into (your container), that should be the original source, right?

If you "put the htmlText of widget "browser" into (your container), that is also the original source?

I would have thought the 2nd would be the page after loading, or the generated pages source.

jacque · Post by **jacque** » Sun Sep 15, 2019 11:26 pm

I didn't know the htmltext of a widget would be different, it's a web view so shouldn't it be whatever the URL sends?

The htmltext of a field is of course very different usually.

[-hh] · Post by **[-hh]** » Mon Sep 16, 2019 12:48 am

The htmltext of a widget is, as in ordinay browsers, exactly the 'original' source code of the page.
But working in the browser/widget is the page with its scripts, inherited styles and later content settings applied.

You don't see these 'additional' things in the htmltext, you need 'deeper' javascript/ a web-developer tool (as the main browsers have) for that.
The *generated* htmltext of a browser page (ordinary browsers and widget) is by that characteristic comparable to the htmltext of a field (which is a translation of the *current* text to (old) HTML).

For example, if you have a web page with a textarea and you type into it or set the content of the textarea by javascript as follows:

Code: Select all

on mouseUp -- button "Create"
  if there is no widget "browser" then
    create widget "browser" as "com.livecode.widget.browser"
    set rect of widget "browser" to (0,0,350,150)
    set topleft of widget "browser" to the botleft of me
    set htmltext of widget "browser" to "<html><head></head>" & \
          "<body style='background-color:#ccc'><textarea " & \
          "id='jlg' rows='9' cols='44'></textarea></body></html>"
    wait 10 ticks with messages -- widget startup needs some time
  end if
  do "var JLG=document.getElementById('jlg'); JLG.value='This " & \
        "is a demo text.\n"&(the internet date)&"';" in widget "browser"
end mouseUp

Code: Select all

on mouseUp -- button "Show"
  if there is no fld "out" then
    create fld "out"
    set width of fld "out" to 350
    set height of fld "out" to 80
    set topleft of fld "out" to the botleft of widget "browser"
  end if
  put the htmltext of widget "browser" into fld "out"
end mouseUp

then you can see the text in the browser but the htmltext of the widget is, as in ordinary browsers, the original one without your added text.

jacque · Post by **jacque** » Mon Sep 16, 2019 1:44 am

I understand what you mean now, that makes sense.

Zax · Post by **Zax** » Mon Sep 16, 2019 7:51 am

jacque wrote: ↑
Sun Sep 15, 2019 4:18 pm
If the browser widget already is displaying the URL, you can also:

get the htmltext of widget "browser"

Perfect! Now Javascript generated code appears in the "source" web page.
Thank you very much

To go further, is there a way to know when the web page displayed in the widget is fully loaded?
( something equivalent to JQuery $(document).ready(function () )

bogs · Post by **bogs** » Mon Sep 16, 2019 11:48 am

@ -hh -
that is fascinating, and something I don't think I ever even thought of before. Awesome!

@ Zax -
If you crack open the dictionary, and type 'browser' into the search, you'll see an entry that says ...

: Blowser...er, BROWSER!

...under which you will find all the messages and properties for the widget, such as the highlighted selection here...

: What a complete load!

In the case of your question, I suspect the answer is 2 above the highlighted one, "browserDocumentLoadComplete", however I've never actually used any of the newer stuff, so I'll leave you to test that on your own

Zax · Post by **Zax** » Mon Sep 16, 2019 12:16 pm

Thanks bogs, I'm not familiar with these "new" widgets (I started coding with Hypercard

).

My next question would be about parsing DOM with a pure LC solution and build an array in order to perform accurate node search. Maybe something like https://simplehtmldom.sourceforge.io/ for PHP.
But first, I have to play around with the Browser widget.

LiveCode Forums.

HTML source code within Web Browser?

HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?

Re: HTML source code within Web Browser?