Peer to peer networking with UDP

[WinstonJenks]

This is a similar situation to the Peer to Peer networking post already in this forum, but distills the question a bit more.

Here is a situation i think LC should be able to handle, but it does not. Imagine a UDP server that reads a line from port 5200, then sends a UDP response back to the client that sent the data. The idea is to send a UDP packet to the UDP server, then read the response from that server.

put "a.b.c.d:5200" into sckDef
open datagram socket to sckDef
write "Hello World" & lf to socket sckDef
read from socket sckDef until lf
put "The response is:" && it
close socket sckDef

In WireShark, I see the "Hello World" go from "<myLocalIP>:61304" to "a.b.c.d:5200" and then the response comes back 9ms later from "a.b.c.d:<otherRandomPort>" to "<myLocalIP>:61304". Each time I do the open socket, the local port, shown here as 61304, changes.

I suspect the problem is that
a) the "read from sckDef" line is looking for UDP data that comes back FROM socket 5200 (as it would if it were TCP data) instead of data that comes back TO the local socket used to send the data, and
b) there is no way to find out what the local socket is after the open, so that an "accept datagram connection on port xxx" cannot be used to read the data.

Has anyone else ever got such a simple UDP cllient to work?

[WinstonJenks]

Never mind. With the help of the http://lessons.livecode.com/m/4071/l/12 ... ng-sockets I got my little test UDP and TCP server working. The trick for me was to do the read with message "ReadComplete" before doing the write. And don't expect the "open datagram socket to sckDef with message "ClientConnected" to fire the "ClientConnected" message. That makes sense as there is no connection, although not mentioned in the docco. As I get the code boiled down to the minimum, I'll post it here.

[WinstonJenks]

Wait a minute! Further testing and wiresharking shows the situation is more complicated and possibly sheds light on the woes of others. It turns out that if you open a UDP socket to a remote endpoint (IP Address + port), then when you read from the socket, you will ONLY see data that comes back to your port (which is normal) AND came from the specific remote endpoint (IP Address + port) you "connected" the socket to in the open command. This could easily be a design decision, but in that case it should be mentioned in the documentation.

An example will help:
Clients A and B want to talk to server S over UDP. They know that server S lives at IP address X, and listens on port Y. The clients send requests to the server, and the server responds with one reply per request. So each client makes a socket (think socket == two endpoints, each of which has an IP Address + port), and with the livecode syntax,

Code: Select all

put X & ":" & Y into svrSocketDef
open datagram socket to svrSocketDef

they create a UDP socket to send data to the server.

After creating the socket, you can write the request, "Hello", and read the reply from the socket with

Code: Select all

read from socket svrSocketDef with message "ReadCompleted"
write "Hello" to socket svrSocketDef

on ReadCompleted pSocket, payload
   put the long time & "; Recd(" & pSocket & ")" & payload & return after field "ClientLog"
   read from socket svrSocketDef with message "ReadCompleted"
end ReadCompleted

When the socket is created in the "open socket" command, the IP Address of the local and remote endpoint are defined. So is the remote port, but the local port is not, so the OS just assigns one. Generally the user does not care what this is, and since the underlying protocol needs one, a free one is chosen. When the UDP message is transmitted from Client A and B, the description of the source endpoint is included in the message, so the receiver knows who sent the message. This is very useful if the receiver, Server S in this case, wants to send a reply to the right client. However, when the server sends the reply, it can either use the same socket (again think socket == two endpoints), or it can create a new socket which is connected to the source endpoint of the original request from the client. So that means that the reply message will go back to the right endpoint (IP Address + port) on the client side, but it might not come from the same endpoint that received the original request.

This seems to be the factor that determines if the "read from socket svrSocketDef..." command sees the reply data or not. If the server sends the reply back using the original socket it received the data on, then the livecode client sees it. If the server creates a new socket and uses that to send the data back to the original endpoint, then the livecode client does not see it. I just verified this with my little test stack and WireShark.

Specifically, on the Server side, there is the following code:

Code: Select all

on StartTheServer
   put field "ServerIPAddress" & ":" & field "ServerPort" into svrListenSock
   if (the hilite of button "chkUDP") then
      accept datagram connections on port (field "ServerPort") with message "Svr_Accept"
      put false into bUsingTCP
   else
      accept  connections on port (field "ServerPort") with message "Svr_Accept"
      put true into bUsingTCP
   end if
   if the result is empty then
      put the long time & " ServerStarted" & return after the field "ServerLog"
      put "Listening..." into field "ServerStatus"
   else
   end if
end StartTheServer

on Svr_Accept clientSocketDef, clientData
   put the long time & " Accepted(" & clientSocketDef & ")" && clientData & return after field "ServerLog"
   if (bUsingTCP) then 
      --this is a TCP socket
      put clientSocketDef & return after sConnectedClients
      read from socket clientSocketDef with message "Svr_Read"
      put "Connected (" & the number of lines of sConnectedClients & ")" into field "ServerStatus"
   else
      -- this is a UDP socket and we just got a datagram
      put toUpper(clientData) into upperData
      write upperData to socket clientSocketDef -- this writes the return back over the same socket it came in on
      -- TEST HERE now try to write to the same location from a new socket
      put clientSocketDef & "|new" into newSockDef
      open datagram socket to newSockDef
      write "COPY---" & clientData to socket newSockDef -- this does not show up on the rev client
      -- write "COPY---" & clientData to socket clientSocketDef -- but this does!
      close socket newSockDef
      -- END TEST HERE
      if the result is empty then
         put the long time & " Sent(" & clientSocketDef & ")" & return after field "ServerLog"
      else
         put the long time & " SendError(" & clientSocketDef &"}" && the result & return after field "ServerLog"
      end if
   end if
end Svr_Accept

So what? Is this a bug? Maybe. I have a client written in Perl that can receive messages that are addressed to the right endpoint but come from any endpoint. In fact, I was not even sure how to only take datagrams from a specific source endpoint. At the very least, it should be noted in the documentation. I think it is not uncommon for threaded-servers to receive requests, parcel them out to a worker thread and then have the thread send the reply--each thread with its own socket. The current implementation prevents an LC client from seeing that data from such a server. Of course, normal TCP sockets allow only the connected endpoint to talk back, but IMHO the connection-less UDP sockets should allow other sockets to "muscle-in" on the conversation.

And by the way, I was testing this with LC 6.6.2 on Windows 7 and it appears to be this way in LC 5.0.0 as well.

[lilRalph]

G'Day WinstonJenks,

That is interesting. I have never managed to get WireShark working to my satisfaction but I blame myself.
Like you I would have assumed that reading from a socket wouldn't care where the source was unless it is supposed to be a secure socket circuit and then the source must remain the same.

I haven't done much with direct socket communications for quite a while but I think that Python also would read from the socket it created without being concerned with the source port.
Perhaps it has to do with preventing MIM attacks but I'm sure that it would be possible to spoof the 'correct IP/port combination' for that type of attack.

A curious situation, good luck.

[FourthWorld]

Winston, it may be that you've run into one of the difficulties with P2P in LC, the inability to do UDP hole punching, as described in this thread from a while back:
http://forums.livecode.com/viewtopic.php?f=11&t=18594

If you're in a position to consider a hybrid approach that uses a server as an intermediary you should be able to work out many good solutions for sharing data, but without hole punching it may be difficult to do pure P2P.

[WinstonJenks]

Richard,

I looked at the suggested post. It is related, but not exactly the issue at hand. The post describes what looks like a nice feature for the sending socket, but I am stuck with the inability to receive datagrams that don't come from the specified socket, even though they are directed to the proper port on the receive side.