Writing UTF/8 from a field?

Anything beyond the basics in using the LiveCode language. Share your handlers, functions and magic here.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
thatkeith
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 346
Joined: Mon Mar 01, 2010 7:13 pm
Location: London, UK
Contact:

Writing UTF/8 from a field?

Post by thatkeith » Wed May 23, 2018 12:10 pm

I have a quick-n-dirty tool for generating HTML and posting it, so I can keep a schedule in a web page up to date without mucking about writing HTML directly. In LC I use tabbed text in a field, and I parse/convert this content to a table structure (wrapped in the rest of a basic HTML page), and upload that by FTP. This file is shown in an iFrame in a WordPress page.

This all works absolutely fine – but I can't get my head around how to support characters such as ö and Japanese characters. If I generate the page manually as UTF/8 from BBEdit it works fine, but when I update the remote page using my LC stack like so:

Code: Select all

put field "code" into URL "ftp://NAME:PASSWORD@FULL/FTP/PATH/TO/PAGE.HTML"
the upload works fine, as does the iFrame, but the encoding fails; in a browser I see diamond blocks for the 'special' Western chars and strings of question marks for the Japanese content.

The HTML page code I'm writing starts with this:

Code: Select all

<html>
<head>
<meta charset="utf-8"/>
but this doesn't help with the LC-generated output. Any tips on how should I handle writing out text in a form that doesn't trip up over this kind of content?

k
Technical Writer, Meta
University Lecturer
Technical Editor, MacUser (1996-2015)
360 VR media specialist

MaxV
Posts: 1579
Joined: Tue May 28, 2013 2:20 pm
Location: Italy
Contact:

Re: Writing UTF/8 from a field?

Post by MaxV » Wed May 23, 2018 1:03 pm

Just use the htmlText property:

Code: Select all

put the htmlText of  field "code" into URL "ftp://NAME:PASSWORD@FULL/FTP/PATH/TO/PAGE.HTML"
Livecode Wiki: http://livecode.wikia.com
My blog: https://livecode-blogger.blogspot.com
To post code use this: http://tinyurl.com/ogp6d5w

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9801
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Writing UTF/8 from a field?

Post by FourthWorld » Wed May 23, 2018 4:34 pm

HtmlText will escape the angle brackets, so while it's sometimes useful for converting non-HTML to a form of HTML (which usually needs further massaging to turn into web-ready HTML), if you're starting with HTML it'll just produce a form that browsers will render with all the tags showing, rather than parsing the tags.

To convert text to an encoding other than LiveCode's internal format, use the textEncode function.

Also, FTP sends passwords in plain text, considered too dangerous to use outside one's internal network, given the hostile nature of the open Internet these days.

With all LiveCode editions you could use a CGI script on the server to receive data posted to it via HTTPS.

With the Business and Indy editions you can also use the tsNet external to send via ftps or sftp.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7210
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Writing UTF/8 from a field?

Post by jacque » Wed May 23, 2018 4:49 pm

Just to add : LC uses UTF 16, and without textEncode, that's what the browser will get. The tipoff is all the missing characters and the weird symbols. LC recommends using textEncode any time you move data in or out of LC, unless you are sure the external app works with UTF 16.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

MaxV
Posts: 1579
Joined: Tue May 28, 2013 2:20 pm
Location: Italy
Contact:

Re: Writing UTF/8 from a field?

Post by MaxV » Thu May 24, 2018 3:10 pm

FourthWorld wrote:
Wed May 23, 2018 4:34 pm
HtmlText will escape the angle brackets, so while it's sometimes useful for converting non-HTML to a form of HTML (which usually needs further massaging to turn into web-ready HTML), if you're starting with HTML it'll just produce a form that browsers will render with all the tags showing, rather than parsing the tags.
...
I don't think so, the htmlText is what he needs to avoid chars problems.
For example "å" is HTML correct just with "&aring;" other solutions are not really reliable. And "&aring;" is the output of htmlText.

Regarding FTP, it depends on the service provider. Here in Italy all service providers use just FTP, and nobody still has problems with it.
Livecode Wiki: http://livecode.wikia.com
My blog: https://livecode-blogger.blogspot.com
To post code use this: http://tinyurl.com/ogp6d5w

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9801
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Writing UTF/8 from a field?

Post by FourthWorld » Thu May 24, 2018 3:36 pm

MaxV wrote:
Thu May 24, 2018 3:10 pm
FourthWorld wrote:
Wed May 23, 2018 4:34 pm
HtmlText will escape the angle brackets, so while it's sometimes useful for converting non-HTML to a form of HTML (which usually needs further massaging to turn into web-ready HTML), if you're starting with HTML it'll just produce a form that browsers will render with all the tags showing, rather than parsing the tags.
...
I don't think so, the htmlText is what he needs to avoid chars problems.
For example "å" is HTML correct just with "&aring;" other solutions are not really reliable. And "&aring;" is the output of htmlText.
Go ahead and try using it to produce HTML when the field's contents are already HTML and let us know how that works out.
Regarding FTP, it depends on the service provider. Here in Italy all service providers use just FTP, and nobody still has problems with it.
The security risk with FTP is inherent in the protocol, independent of any ISP using it.

If you leave your house unlocked and have never had a burglary, that's not because leaving your door unlocked is a good security practice but that you've merely been lucky.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

thatkeith
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 346
Joined: Mon Mar 01, 2010 7:13 pm
Location: London, UK
Contact:

Re: Writing UTF/8 from a field?

Post by thatkeith » Wed May 30, 2018 11:03 am

FourthWorld wrote:
Wed May 23, 2018 4:34 pm
FTP sends passwords in plain text, considered too dangerous to use outside one's internal network, given the hostile nature of the open Internet these days.
Good advice, exactly what should always be said when old-school FTP is invoked in LC. Fortunately, for this specific case it's a tool made just for a colleague and me, and it'll be redundant (and the password changed just in case) after the weekend.

(Which I realise doesn't address the issue of plain text directly, but it's at least not something I'm proposing to release or even use more than half a dozen times.)

:)
Technical Writer, Meta
University Lecturer
Technical Editor, MacUser (1996-2015)
360 VR media specialist

Post Reply

Return to “Talking LiveCode”