Correct function to encode an array over sockets

[trevix]

Hello.
Are these correct, in order to send an array using sockets between standalones, even on different OS/mobile?

Code: Select all

function EncodeMyData pArray
  return textEncode(arrayEncode(pArray),"utf-8")
end EncodeMyData

function DecodeMyData pArray
  put TextDecode(pArray,"utf-8") into pData
     try
          put ArrayDecode(pData) into TheData
     catch errorVariable
          --is not an array answer errorVariable
          put pData into TheData
     end try
     return TheData
end DecodeMyData

For me they don't seems to work (the received data is not an array)
This works, but the string to send is much longer:

Code: Select all

function EncodeMyData pArray
  return TextEncode(base64Encode(ArrayEncode(pArray)))
end EncodeMyData

function DecodeMyData pArray
  put base64Decode(TextDecode(pArray)) into pData
     try
          put ArrayDecode(pData) into TheData
     catch errorVariable
          --is not an array answer errorVariable
          put pData into TheData
     end try
     return TheData
end DecodeMyData

[Klaus]

Hi Trevix,

no idea if that is the "correct" way, but since ARRAYENCODE() return BINARY data, you do in fact
need to turn this into "simple" text with base64en-/decode before sending over a socket..
Even LC recommends this (base64en-decode) in the dictionary!

But you can and probably should leave out the TEXTENCODE part:

Code: Select all

function EncodeMyData pArray
  ## return TextEncode(base64Encode(ArrayEncode(pArray)))
  return base64Encode(ArrayEncode(pArray))
end EncodeMyData

Best

Klaus

[FourthWorld]

How are you sending it? Images, PDFs, and Zip files are all binary, and go over sockets just fine. Depending on how your sending/receiving the data, it may be as simple as having a header value that sets the length so the recipient knows when to stop reading.

[trevix]

Hello and thanks.
I am sending arrays (which may contain image data and can have subkeys like MyArray["animali"]["Asini"]) and simple text (mostly used to confirm reception, like "CONF".
My encoding is now done like this (following Klaus raccomandation):

Code: Select all

function EncodeData tData
     if tData is empty then return empty
     if (tData is an array) then
          return the base64Encode of ArrayEncode(tData)
     else
          return the base64Encode of tData
     end if
end EncodeData

[FourthWorld]

Once run through arrayEncode it's suitable for sending. Even better you could also compress it. Base64 is great for those cases where the receiver may not handle binary data, but with a length indicator that's usually not a problem.

How are you sending it? HTTP, some other protocol?

[trevix]

I am using read/write socket on local network.

I write the length on the first item of send:

Code: Select all

put the length of TheMessage into tLength
put  trunc(tLength/72) && tLength mod 72 into tNum --number of 72 read + leftovers
write tNum & comma & TheMessage to socket gSocket

and read it in the other side like this:

Code: Select all

put word 1 of pMsg into tNum
put word 2 of item 1 of pMsg into tSubNum --so to avoid the comma
if tNum is an integer then
          repeat tNum
               read from socket pSocket for 72
               if it is empty then exit repeat --I could remove this probably
               put it after tRD
          end repeat
        
          if tSubNum <> 0 then --read leftovers so it does not hang on the repeat
               read from socket pSocket for tSubNum
               put it after tRD
          end if
          put tRD into pMsg
end if

As I said, I may include image data in the array. The dictionary says:

To send an encoded array to a remote process over TCP/IP, it should be encoded using the URLEncode function, as it may contain characters not suitable for use in URLs.

I don't know if it applies, but I tried with URLEncode(ArrayEncode(tArray)) and Arraydecode(URLdecode(tMsg)) but it does not seem to work on the receiver. All my testing now are on OSX, with 2 Mac, but will be used with iOS and Android devices.