Timing over socket connection

[trevix]

- PCa is connected over local network and sockets to PCb.
- PCa send a message to PCB that must do something with the received message (make take 1 or 2 seconds)
- Upon completion, PCB send a CONF message to PCa, just to say that everything went well or to provide some data.
- After receiving the CONF msg, PCa can go on with its tasks.

In order to accomplish this, I scripted what you can find below. But I am having problems.
Is it the wrong way to do it or my scripts have bugs?
Thanks.

Code: Select all

function SendToServer
    write tLenght & comma & TheMessage to socket gSocket
    wait 500 milliseconds with messages
    --"ClientMessageReceived" read the size (first item) and do other reading pass
    read from socket gSocket for 1 items with message "ClientMessageReceived" 
    --the idea here is that "ClientMessageReceived" has time to set the sRePlayReceived flag
    repeat 9
               if sRePlayReceived then
                    put false into sRePlayReceived
                    return true
               end if
               wait 500 milliseconds with messages
   end repeat
   return false
end SendToServer

[FourthWorld]

But I am having problems.

What problems are you having?

[trevix]

Sometime, while on a "wait for 500 milliseconds with messages" I am not sure that the "read from socket" happens.
As I understand, LC only do things serially.
Thus, what happens if the "read" takes more time then the 500 milliseconds of the "send" loop? (the "read" also is on a loop, given the eventual size of the content).
Essentially there is a main wait loop (send) and a secondary wait loop (read) that I guess runs only on the wait moments of the main loop.
Or not?
Far out...that's why I think I may have it all wrong...

[trevix]

Also.. is this the only way to clear the socket buffer, while keeping the connection open?

Code: Select all

 read from socket TheSocket until "*" --*= some non existent char

[Cairoo]

Hi Trevix,

Sometime, while on a "wait for 500 milliseconds with messages" I am not sure that the "read from socket" happens. ...

Looking at your code, you're using a callback message to notify you when the server's response has arrived. If the callback message hasn't been received by the time the 500 milliseconds wait is over, then you can be sure that no data has arrived yet.

... Thus, what happens if the "read" takes more time then the 500 milliseconds of the "send" loop? (the "read" also is on a loop, given the eventual size of the content). ...

You gave it 9 intervals of 500 milliseconds each in which to finish reading the server's response.
If your ClientMessageReceived handler does not finish reading the data in that time, your SendToServer function would return false.
So if you really want the SendToServer function to return true only if the server's response was completely read within 4500 milliseconds, then your SendToServer function is good to go.

Also.. is this the only way to clear the socket buffer, while keeping the connection open?

Code: Select all

 read from socket TheSocket until "*" --*= some non existent char

The read command clears the socket buffer unless told to read only a specified amount of chunks or to read until it encounters a specified string. After clearing the read buffer, the socket normally remains open until closed explicitly on either end or until the app quits.

If the problems you're having only occurs with large amounts of data, it may well be that your ClientMessageReceived handler takes longer than 4500 milliseconds to finish reading the data.

If you want your SendToServer function to return true as soon as the server's response arrives, i.e. even before the server's response has been fully read, then consider setting the sRePlayReceived flag at the start of your ClientMessageReceived handler.

[Cairoo]

... Essentially there is a main wait loop (send) and a secondary wait loop (read) that I guess runs only on the wait moments of the main loop.
Or not? ...

The secondary read loop should happen in the ClientMessageReceived handler and unless something unusual happens, would complete independently from the main send loop. Your main send loop basically waits for your secondary read loop to complete but may not be giving it enough time if the server's response is large.

[trevix]

Hello and thanks for answering.
Everything you wrote is correct and this is how my script works.
Having the server send a confirmation before the read, or even removing the confirmation doesn't solve my problem when the size of the returned package is over...I would say around 120k.
Under that amount my client script reads every that has been sent correctly.
Over that amount, the received data are somehow scrambled.

My script is a little more complex of what I presented as an example, but essentially the client receives:

- a number to specify how much to read && a short command && comma
- a base64 encoded data,that gets read on a loop for 72 chars

Check the script below.
As I said, when the payload is too big, the data received is, for example:

- first item= 126 12 DOTHIS --correct
- empty -- not correct. as any subsequent reading

then if I do again a send/read, the client receives part of the binary, followed by & DOTHIS -- not correct
There must be a problem on socket reading big chunks.
The reading is incorrect for big chunks even if instead of using this method, I do a "read until char XX"

Code: Select all

 if pMsg is empty then
          read from socket p_Socket for 1 items with message "chatServerMessageReceived_new"
          exit chatServerMessageReceived_new
     else
          put word 1 of pMsg into tNum --number of 72 chars chunks
          put word 2 of item 1 of pMsg into tSubNum --the left over number of chars
          if tNum is an integer then
               put word 3 of item 1 of pMsg into tCommand
               repeat tNum
                    read from socket p_Socket for 72
                    if it is empty then exit repeat --on big payload this is empty
                    put it after tRD
               end repeat
               if tSubNum <> 0 then --read leftovers
                    read from socket p_Socket for tSubNum
                    put it after tRD
               end if
               put tRD into pMsg
          else
               put item 1 of pMsg into tCommand
          end if
----

As for clearying the socket buffer, the problem is that read until "NotExistingChar" takes forever, like10 seconds. There should be a command for fast doing this.

[Cairoo]

Trevix

- a base64 encoded data,that gets read on a loop for 72 chars

Reading the base64encoded data in chunks of 72 chars in a loop would most definitely slow things down.
Maybe you could consider splitting the binary on the sending end into large but less-than 90K sized chunks and send each chunk as a base64encoded string, and on the receiving end just read the whole chunk and base64decode it?

- Gerrie

[Cairoo]

I've attached a simple example of sending binary data in chunks over a socket.
I've used a similar approach with sending a header with every chunk, as you have.
Let me know if it helps.

- Gerrie

EDIT: Don't use the one attached here; see next post.

SocketsTestApp.zip

Send a binary file over a socket

(3.6 KiB) Downloaded 57 times

[Cairoo]

To my embarrassment, I haven't tested my sample stack properly. It worked well sending from my Windows PC to my Mac, but glitched out when sending from my Mac to my Windows PC.
I now think sending the chunks over in a repeat loop is the wrong way. I'm also learning.
Next I'll try a callback with the "write so socket" command and in the callback wait for a response from the server with another callback that sends the next chunk, and hopefully I can heal my reputation with a functional sample stack.

[Cairoo]

... hopefully I can heal my reputation with a functional sample stack.

Attached below is the more functional stack.

- Gerrie

SocketsTestApp.zip

Send a file over a socket v2

(3.86 KiB) Downloaded 61 times

[trevix]

Very well written Cairoo, thanks.
I noticed that you did not take in account socketTimeout. No need for it?

One more question:
I am from italy, where we use accented characters.
Loading a file to send, whose name is "Toni Carta Identità 2.jpg"

Code: Select all

the line urlEncode(textEncode(item -1 of it,"UTF-8")) into sUrlEncodedFileName -- return "Toni+Carta+Identita%CC%80+2.jpg"
--while 
put files(tFolder,"detailed-utf8") into tFilesList --in the list the file is returned as Toni+Carta+Identit%C3%A0+2.jpg,856519,38521,1210322211,etc

This way no file is listed on tFileList.
Probably there is an easy way to solve this, but I wonder why the function "files()" must return an UrlEcoded name...
Regards

[trevix]

I solved the second question like this:

Code: Select all

  put item -1 of it into tName
...
 -- get the file size
put files(tFolder,"detailed-utf8") into tFilesList
repeat for each line tLine in tFilesList
    put textDecode(URLDecode(item 1 of tLine), "utf8") into tNameFromList
    if tNameFromList = tName then --found
         put tLine into tFilesList
         exit repeat
    end if
end repeat
put item 2 of tFilesList into tFileSize

[FourthWorld]

I wonder why the function "files()" must return an UrlEcoded name...

The Mac Finder used to store folder metadata in a hidden file that use a return character at the end of its name, presumably to avoid being overwritten by any other file the user might create.

Since the value returned by "the files" is a return-delimited list, this of course made the list unusable.

Tho inconvenient, using urlEncode was a reasonably efficient solution to account for anything that might show up in a file name which could break the delimiting in the value returned from "the files".

[trevix]

Thanks FourthWorld.

One more question for Cairoo, since you seem very confident with this stuff:
Since in your stack you made extensive use of local variables, to store ChunkNumer, TotalChunks, etc, what happens if 2 connected clients send each a different file to the server in the same time?
Do the recursions between the commands in the server keep each var istance separated from the others or the all thing must be rethinked?
Regards
Trevix