Text Problems over Socket Connection

[aircooled76]

Hi Guys,

I am trying to use some Croatian text characters "Č Ć Ž Š Đ" from a LiveCode field into a variable and then write it to a socket connection.

As far as I have found out LiveCode only supports ANSII and I need to convert this to UniCode or UTF-8 format. Can anyone suggest how I can do this...

Do I need to convert the field to UTF-8 then put it into the variable... ??? Any help would be appreciated...

[BvG]

Unicode is possible, but you always need to convert fore and back at the right entry and exit points:

Code: Select all

function createUtf8Text theText
   return unidecode(uniencode(theText),"utf8")
end createUtf8Text

function getRevTextFromUTF8 theText
   return unidecode(uniencode(theText,"utf8"))
end getRevTextFromUTF8

I now it's weird right now, but they've promised to fix unicode really soon now

[aircooled76]

Thanks BvG...

I am struggling as to where to put it in my code...

Example Code: MyCommand is a command I must send with the field variable
UniCodeName is a name field that may contain special characters (Č Ć Ž Š Đ)

Code: Select all

put quote into xml
put "MyCommand" into xml
put field "UniCodeName" into xml
put quote into xml
send xml to socket localhost

[FourthWorld]

Maybe it's overkill, but when transferring data over sockets I usually run it through base64Encode first.

[BvG]

fourthworld: yes that's overkill, depending on the protocols robustness to deal with returns, zeros or other relevant chars.

aircooled:
i can't work with your example, it's gibberish? I added comments:

Code: Select all

put quote into xml
-- you put " into the variable xml
-- xml contains """
put "MyCommand" into xml
-- you put "mycommand" into xml
-- xml contains "mycommand"
put field "UniCodeName" into xml
-- you put field "unicodename" into xml
-- xml contains field "unicode"
put quote into xml
-- you put " into xml
-- xml contains """
send xml to socket localhost
-- that line will result in an error, it's not valid livecode

[aircooled76]

Hi Bvg...

I was trying to shorten my code... Ill give you a more detailed rundown...

I am sending commands to a graphics playout server casparcg (add dot com for domain) I need to include quotes and escapes in the XML to the Caspar Graphics server that I am connecting to via socket connection...

The command I need to send to the socket is in the format below:

Code: Select all

CG 1-1 ADD 0 "demo/test" 1 "<templateData><componentData id=\"f0\">
<data id=\"text\" value=\"Niklas P Andersson\">
</data> </componentData><componentData id=\"f1\">
<data id=\"text\" value=\"developer\">
</data> </componentData></templateData>"

I cannot put the code in a livecode script as above as the quotes are taken literally and create errors in the script. So I put all the segments into a variable xml... then send the xml to the server (I note a typo in my original code... should be "write xml to socket localhost"... not "send xml to socket localhost" The actual IP address is specified by a field in my app...

My actual code refers to field variables for the video layer that are specified by the livecode app: eg field introvl is the intro video layer (or 1) and introfl is the flash layer 0

Code: Select all

  put "CG 1-" after xml
         put field introvl of card settings after xml
         Put " ADD " after xml
         put field introfl of card settings && field intro of the card settings after xml
         put " 1 " after xml
           put quote after xml
   put "<templateData><componentData id=\" after xml
   put quote after xml
   put "f1\" after xml
   put quote after xml
   put "><data id=\" after xml
   put quote after xml
   put "text\" after xml
    put quote after xml
    put " value=\" after xml
    put quote after xml
    put field competition after xml
    put "\" after xml
    put quote after xml
    put " /></componentData></templateData>" after xml
    put quote after xml
    write xml & format("\r\n") to socket field IP of card settings

I realize there may be a much better way of doing this (I am very open to a better way) But I am more a video graphics guy than a livecode programmer so I did what I had to to make it work...

So I need to know how to make the text in the field "competition" into the correct format when I bring it into the xml variable so when i send it to the server via the socket it has the correct text format to include the special characters I need (Č Ć Ž Š Đ.)

[Simon]

Yay! I get to post my little stack again.
It will help you with the first problem (the quotes). Won't solve the overall problem.
It has no instructions, I leave that as an exercise for the reader.

Simon

[BvG]

I guess it depends on where you are getting your text from. Assuming you have an utf-8 file that you want to load into a field:

Code: Select all

on mouseUp
answer file ""
if it = "" then
   exit mouseUp
end if
put url ("binfile:" & it) into theUTF8
put getRevTextFromUTF8(theUTF8) into field 1
end muoseUp

function getRevTextFromUTF8 theText
   return unidecode(uniencode(theText,"utf8"))
end getRevTextFromUTF8

There's shorter ways to do that of course, but this makes it clearer what is happening.

[aircooled76]

I actually need the charactors to come from a field... be put into a variable then sent to the socket connection...

Ie the team name is entered into a field.. then put together in a variable with the grapgic servers commands and sent to the server over thd socket connection...

Is this possible... I am struggleling to see hiw to adapt your code to suit a field to variable appriach...

[BvG]

Well instead of coming from a text file, you can get your string from anything, a variable, a field, a socket....

That is how computers work:

Input -> Evaluation -> Output

in your case:

Field -> Convert to desired format -> Send over socket

[aircooled76]

Thanks for your suggestions... I have been Googling and reading stuff but still no success getting this to work... or really understanding how this is supposed to work...

Input -> Evaluation -> Output

in your case:

Field -> Convert to desired format -> Send over socket

So if I just changed the reference from a file to field it should work

Code: Select all

on mouseUp
   put the field testfield into theUTF8
put getRevTextFromUTF8(theUTF8) into testfield2
end mouseUp

function getRevTextFromUTF8 theText
   return unidecode(uniencode(theText,"utf8"))
end getRevTextFromUTF8

But no dice... it doesn't work...

I did have some succsess based on some reading I did at http://revolution.byu.edu/unicode/unicodeInRev.php

Code: Select all

on mouseUp
   set the unicodeText of the field "testfield2" to the unicodeText of fld "testfield"
end mouseUp

This is great... but I need to put the result into a variable or send it to the socket connection... not sure how to code that...

[Klaus]

Hi aircooled76,

ALWAYS put QUOTES around object names!

Since you do not do this and even left out the keyword "field",
you end up with the UTF8 text in a variable named "testfield2"!
Maybe not neccessarily what you exspected

And do only use the keyword "THE" for custom properties!

Code: Select all

on mouseUp
  ## put the field testfield into theUTF8
  put field "testfield" into theUTF8
  put getRevTextFromUTF8(theUTF8) into FIELD "testfield2"
  ## Should do the trick :-)
end mouseUp

function getRevTextFromUTF8 theText
   return unidecode(uniencode(theText,"utf8"))
end getRevTextFromUTF8

Best

Klaus

[aircooled76]

Hi Klaus,

Thank you for your reply... I fixed the errors in my code... (learning all the time thanks) But unfortunately the code gives me only ????? in response...

however... the other code works and gives me the correct output in the field... "Č Ć Ž Š Đ"

Code: Select all

on mouseUp
   set the unicodeText of field "testfield2" to the unicodeText of field "testfield"
end mouseUp

so I am closer than i was... but I need to send the unicode of "testfield" to a socket connection with a bunch of other code... is there a way to put the output of this into a variable rather than a field (I tried

Code: Select all

set the unicodeText of "avariable" to the unicodeText of field "testfield"

But that doesn't work....

[aircooled76]

Some success of sorts...

Code: Select all

on mouseUp
   set the unicodeText of field "testfield2" to the unicodeText of field "testfield"
   put the unicodetext of fld "testfield2" into tData
put uniDecode(tData,"utf16")  into field "testfield3"
end mouseUp

Works... I get Č Ć Ž Š Đ copied from the variable to testfield3... Awesome... however

This has just become more complex... I need to send the commands to my graphics playout server using utf8....

I tried using:

Code: Select all

write uniEncode(xml,"utf8") & format("\r\n") to socket localhost:5250

But the commands come out all corrupted when arriving at the other application. Every second character is a ? and the special characters I need Č Ć Ž Š Đ are still question marks... (I also tried the above with utf16 but that is all question marks...)

This is starting to do my head in!!!! Would love some assistance if anyone has the know how!

[Klaus]

Well, you need to PUT it
...
PUT the unicodeText of field "testfield" into aVariable
...