unicode (utf-8) over socket

[pajito]

Hi everybody,
I am facing a problem with unicode - unidecode etc, I know it has been a classic problem but still it gives me a hard time. So as you can understand from the uploaded file in the end of this post, I load a csv with unicode(utf-8) greek characters, in fld"1" , at first I couldn't even read the characters correct,then I found the following code, so the greek characters were properly presented in my fld.

Code: Select all

 answer file "Select a file"
   put it into tFile
   put url ("file:" & tFile) into tUnicodeText
   set the unicodeText of field 1 to uniEncode(tUnicodeText,"UTF8")
   
   put url("file:" & tFile) into fld 1

Then I had to split those data into fields so i can send them to a socket, I managed to do that, and managed to send them in greek with the following code

Code: Select all

 repeat with x=1 to the number of lines  of field "1"
      add 1 to counter
   end repeat
   repeat with x=1 to the number of flds of group "pl"
      add 1 to counter
      put textencode(item 2 of line x of fld 1,"UTF-8") into fld x of grp "pl"
   end repeat

The problem is that the data in the fields are again presented """chinese""" with all the respect to Chinese characters. I tried again to add the first code, and the "chinese" turned to greek, but only "question marks" come to the socket.

Any suggestions
Thanks in advance
Ioannis

https://ufile.io/spmql <---- Project
https://uploadfiles.io/kac60 <---- Csv FIle

[Klaus]

Hi Ioannis,

ouch, lots of bad things!

1. Do NOT and NEVER name your LC objects with a number! -> field "1"
The engine does not differ between numbers and strings in that case and think you will mean the field with LAYER 1 (first field on the card) which may be different from its actual layer!
I used -> field "f1" in my script

2. In the script of that field you have a "mouseover" handler.
This is not a valid message in LC, maybe you mean "mouseenter"?

3. "uniencode" etc. has been deperecated since version 7 in LC.

4. Your CSV file is in fact saved as UTF-16!

5. This does work:

Code: Select all

on mouseUp
   answer file "Select a file"
   put it into tFile

   ## We need to treat unicode stuff as BINARY data, which it is!
   put url ("binfile:" & tFile) into tUnicodeText
   
   ## Always lock screen when updating contents of field(s) to speed things up!
   lock screen
   
   ## See above...
   # set the unicodeText of field 1 to uniEncode(tUnicodeText,"UTF16")
   set the Text of field 1 to textdecode(tUnicodeText,"UTF16")
   
   ## We already have the content of that file in the field!?
   ## put url("file:" & tFile) into fld 1
   set the itemDelimiter to ";"
   delete line 1 of fld 1
   delete line 2 of fld 1
   
   ## Since you do not use the variable COUNTER in this script, we do not need it!?
   #   repeat with x=1 to the number of lines  of field "f1"
   #      add 1 to counter
   #   end repeat
   repeat with x=1 to the number of flds of group "pl"
      # add 1 to counter
      # put textencode(item 2 of line x of fld 1,"UTF-8") into fld x of grp "pl"

      ## The text in fld 1 already has the correct format, so we only need to distribute ist content as is!
      put item 2 of line x of fld "f1" into tText
      put tText into fld x of grp "pl"
   end repeat 
    
   unlock scfeen
end mouseUp

Tested and woirks!


Best

Klaus

[pajito]

Hi Klaus

first of all I need to apologize for the UTF-16 file and for forgetting to mention that my socket accepts only UTF-8 encoding. Your code of course works, I get the greek characters correct both in field"f1" and all fields in grp "pl", but when I send them over the socket again they are displayed as question marks.
I saved the file in UTF-8 encoding, made some changes both in my script and your, but still ???? come out of the socket. Just an idea, is there a way to encode the socket itself, so we can use the rest of the code as it is?
As it concerns fld 1, I made this mistake lot and lot of times, but still some times I do it again.

Thanks again
Ioannis

[Klaus]

Hi Ioannis,

try with urlencode and HTMLtext before sending over the socket like this.
Field "f1" contains the greek text:

Code: Select all

on mouseUp
   set itemdel to ";"
   put urlencode(the htmltext of item 2 of line 2 of fld "f1") into tData
   set the htmltext of fld "f2" to urldecode(tData)
end mouseup

This way you send simple ASCII over the sockets and that should work.


Best

Klaus

[pajito]

Hi again

Unfortunately no, still question marks are coming out. I will give it a try again by changing or merging the scripts, hoping fot the best.

Thanks for your time Klaus
Ioannis

[Klaus]

Can you plase post the script(s) for reading/writing to/from socket?

[pajito]

Here you are Klaus,
please note that the one we care about is fld "p1a" , t1 is the dynamic field from Adobe Animate.
Thanks again

Code: Select all

global casparServer
on mouseUp
   
   write "CG 1-81 ADD 1" & quote & "superiera/SW1" & quote & "1" & quote &  "<templateData><componentData id=\" & quote & "t1\"& quote & "><data id=\"& quote &"text\"&quote &" value=\"& quote & fld "p1a" &"\"& quote &"/></componentData></templateData>"& quote & format("\r\n") to socket casparServer
   
end mouseUp

 

bellow is the code that I use to connect to the socket

Code: Select all

on mouseUp
   global casparServer
  
   if field "Server2" is empty then
      set the backgroundcolor of button "Server2" to "Blue "
   else 
      put field "Server2" into casparServer
      if casparServer is among the lines of the opensockets then
         close socket casparServer
         set the backgroundcolor of button "Server2" to "Gray "
         set the label of button "Server2" to " Disconnected"
      else
         open socket to casparServer
        
         set the backgroundcolor of button "Server2" to "green"
         set the label of button "Server2" to "Connected"
      end if
   end if
end mouseUp
end mouseUp

[Klaus]

Hi Ioannis,

I mean the scripts where you write and receive (and put itno fields) data via the socket.

General hint:
In cases like this one you should use this little function to save a lot of typing:

Code: Select all

function q tString
  return QUOTE & tString & QUOTE
end q

Get the picture?


Best

Klaus

[pajito]

Hi Klaus

sorry for the late response, there are no scripts writing to the fields, the data are coming straight from the cvs field "1" or "f1", the other way I use is by typing myself into the fields. If I got something wrong please let me know and I will send you what you need.

Best Regards
Ioannis