Sending large arrays using sockets

LiveCode is the premier environment for creating multi-platform solutions for all major operating systems - Windows, Mac OS X, Linux, the Web, Server environments and Mobile platforms. Brand new to LiveCode? Welcome!

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
trevix
Posts: 950
Joined: Sat Feb 24, 2007 11:25 pm
Location: Italy
Contact:

Sending large arrays using sockets

Post by trevix » Mon Nov 28, 2022 6:14 pm

Hello.
I feel like I am back to step 1...
I had a socket exchange between two my Apps, on local network, on OSX,iOS and Android. They were mostly sending each other small arrays.
Then I had to solve the problem with larger arrays, since my procotol based on "read until x" did not work anymore.

I came up with what follows below and I am wondering if it is correct and it is the best/fastest way to have a solid socket communication:

Arrays are treated this way:

Code: Select all

function CodificaDati tData
     if tData is empty then return empty
     if tData is an array then
          return TextEncode(base64Encode(ArrayEncode(tData)),"UTF-8")
     else
          return TextEncode(base64Encode(tData),"UTF-8")
     end if
end CodificaDati

function DecodificaDati pData
     if pData is empty then return empty
     put base64Decode(TextDecode(pData,"UTF-8")) into pData
     try
          put ArrayDecode(pData) into TheData
     catch errorVariable
          put pData into TheData
     end try
     return TheData
end DecodificaDati
Thanks for any help.

--Sender

Code: Select all

function broadcastClientSend pCommand, pArray
          put false into sRePlayReceived --a local flag
          put CodificaDati(pArray) into TheMessage
          write length(TheMessage) & comma & TheMessage to socket gSocket
          wait 500 milliseconds with messages --give it some time to answer
          repeat 9
               if sRePlayReceived then 
                    put false into sRePlayReceived
                    read from socket pSocket for 1 items with message "ClientMessageReceived"
                    return true
               end if
               wait 500 milliseconds with messages
          end repeat
          ErrorComHandling 10
          return false
end broadcastClientSend

on ClientMessageReceived pSocket, pMsg
     put item 1 of pMsg into tSize
     if tSize is an integer then
          repeat until length(tRD) >= tSize
               read from socket pS for 10
               put it after tRD
          end repeat
          breakpoint
          put tRD into pMsg
          put item 1 of pMsg into tCommand
     end if
     delete item 1 of pMsg
     if tCommand = "CONF" then 
        put true into sRePlayReceived
     else
        --do other things
     end if
     read from socket pSocket for 1 items with message "ClientMessageReceived"
end ClientMessageReceived
--Receiver (Server)

Code: Select all

on ServerMessageReceived pSocket, pMsg
   put item 1 of pMsg into tSize
   if tSize is an integer then
     repeat until length(tRD) >= tSize
           read from socket p_Socket for 10
           put it after tRD
      end repeat
      put tRD into pMsg
      delete item 1 of pMsg --this leaves only the content
       write "CONF" & kEndOfFile  to socket p_Socket  
   end if
  ......... --do something with pMsg
   read from socket pSocket for 1 item with message "ServerMessageReceived"
end ServerMessageReceived
Trevix
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>

Emily-Elizabeth
Posts: 101
Joined: Mon Jan 03, 2022 7:10 pm

Re: Sending large arrays using sockets

Post by Emily-Elizabeth » Mon May 29, 2023 12:37 am

One thing you can try is to put a transmission delimiter at the end of your payload and then read until that delimiter.

Code: Select all

function broadcastClientSend pCommand, pArray
          put false into sRePlayReceived --a local flag
          put CodificaDati(pArray) into TheMessage
          write TheMessage & "|DELIMITER|" to socket gSocket with message "SocketWriteCompleted"
end broadcastClientSend

on SocketWriteCompleted socketPointer
   read from socket socketPointer until "|DELIMITER|" items with message "ClientMessageReceived"
end SocketWriteCompleted

on ClientMessageReceived pSocket, pMsg
     -- do what you need to do here, make sure you strip off the delimiter 
end ClientMessageReceived

stam
Posts: 2600
Joined: Sun Jun 04, 2006 9:39 pm
Location: London, UK

Re: Sending large arrays using sockets

Post by stam » Mon May 29, 2023 11:23 am

I have no idea if this would help or if even is correct but it struck me:
Instead of converting array to binary and then binary to text and then text encoding, why not remove some steps by converting the array to JSON and just transmitting that? A lot of the web does this so possibly more efficient.
Emily-Elizabeth’s suggestion sounds more likely to fix your issue having said that….

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9801
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Sending large arrays using sockets

Post by FourthWorld » Mon May 29, 2023 3:27 pm

How large is "large"?

And why was HTTP ruled out?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

trevix
Posts: 950
Joined: Sat Feb 24, 2007 11:25 pm
Location: Italy
Contact:

Re: Sending large arrays using sockets

Post by trevix » Mon May 29, 2023 7:24 pm

How large is "large"?
Let's suppose 1 MB
And why was HTTP ruled out?
The two apps are connected directly on a local network, using a socket (read/write), for other actions.
I am not sure I understand what you mean.
Trevix
Trevix
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9801
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Sending large arrays using sockets

Post by FourthWorld » Mon May 29, 2023 8:06 pm

trevix wrote:
Mon May 29, 2023 7:24 pm
And why was HTTP ruled out?
The two apps are connected directly on a local network, using a socket (read/write), for other actions.
I am not sure I understand what you mean.
HTTP is a protocol. You're designing your own protocol. HTTP has vast tooling and documentation available. You're expressing challenges with your custom protocol.

In my own work, everything is HTTP unless there's something specific about the workflow that prevents me from using it. Then I explore other relevant established protocols. And only when I find none that do what I need to I roll my own.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

stam
Posts: 2600
Joined: Sun Jun 04, 2006 9:39 pm
Location: London, UK

Re: Sending large arrays using sockets

Post by stam » Tue May 30, 2023 5:45 pm

I suspect a higher level guide is needed - in principle what would be needed to get to apps to speak to each other over http, etc.

trevix
Posts: 950
Joined: Sat Feb 24, 2007 11:25 pm
Location: Italy
Contact:

Re: Sending large arrays using sockets

Post by trevix » Tue May 30, 2023 5:56 pm

I guess my choice of using sockets was due to the fact that I didn't know any better.
To clear out:
The cell phone connects to a TvBox running the same standalone and acting as a server.
Every send from the cell, has a receipt confirmation and returns some data to the cell and viceversa.
Often the data exchange is started also from the server (after the connection has been established).
Data are mainly arrays, "do" actions (a sort of poor man remote control) and screenshots.
The title of this post refers to the need of exchanging full screen images (sponsor's logo) that should be displayed on a tv with a good resolution.
On top of this, in some case, the TvBox is set in a hotspot mode(no internet).
Regards
Trevix
Trevix
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>

Post Reply

Return to “Getting Started with LiveCode - Experienced Developers”