Peer to Peer Networking Stack

Bringing the internet highway into your project? Building FTP, HTTP, email, chat or other client solutions?

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

splash21
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 369
Joined: Sun Dec 19, 2010 1:10 am
Location: UK
Contact:

Re: Peer to Peer Networking Stack

Post by splash21 » Thu Jan 09, 2014 7:02 pm

townsend wrote:but the solution to being able to get my IP address in LiveCode without going out to the web, I have not found. IF a shell statement is required, I would rather open up a web page, that have my app flagged, as dangerous, by virus protection software.
If each client is going to register with a server for availability / ip updates / etc, then there is the possibility of sending the client their own ip address back from the server. The server will see the client's router ip address, which is what other peers will require and the server will obtain the information when a connection is made.
LiveCode Development & Training : http://splash21.com

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Peer to Peer Networking Stack

Post by FourthWorld » Thu Jan 09, 2014 7:48 pm

townsend wrote:As I stated early on, we might work together, to develop a reliable peer-to-peer stack that any LiveCode developer could use as a basis to develop their own unique vision of a peer-to-peer network.
I'm always attracted to the idea of a generalized solution, but when I look at the variety of P2P protocols here I have to wonder how well any one of those would server as a solution for all them:
http://en.wikipedia.org/wiki/List_of_P2P_protocols
Personally, I'm thinking in terms of a peer-to-peer social network. But IF each user has to set up port forwarding on their router, the potential market would be so small, it would not be worth developing.
Excellent! That's a very useful focus.

I was reviewing this list of distributed social networking systems a few weeks ago, pondering which might emerge as a sort of de facto standard we might rely on for future development:
http://en.wikipedia.org/wiki/Comparison ... networking

The more I looked into it, the more I began to come across references to web hooks:
http://en.wikipedia.org/wiki/Web_hook

I find myself excited by the ease of implementing web hooks, and indeed they've already found their way into popular solutions like WordPress. But that takes us back to servers.

One of the challenges of distributed social networking is that it's almost the opposite of P2P, more of a server-to-server, since any member of such a federated system would need reliable access to a machine hosting any other member's media, and given the range of time zones in what would hopefully be a global network, the machine would have to be available 24/7.

This has been one of the things that's held such federated/distributed social networking systems back, the need for server resources and the difficulty in managing them. While simpler than either attempting to do this from a laptop in a distributed P2P model or replicating Facebook in a centralized model, the resources needed (both servers and skills) would exclude many would-be participants. Diaspora is a good system, and I like the community it's gathering, but it's very much server-bound.

NNPT-like systems (which may apply somewhat to Gnutella I suppose, in an abstract way), which replicate the entire collection across multiple nodes, would obviate the need for all nodes to be always available, but at the cost of tremendous loads on the nodes that take on that role (one of the reasons the world was happy to let NNTP die).

My friend Mark Frazier started a thread peripherally related, on federated wikis:
http://forums.runrev.com/viewtopic.php? ... 98&p=83458

He would certainly have good thoughts on this, so I'll drop him a note letting him know of this discussion.
I know I can go out to the web and get my IP address http://whatismyipaddress.com/ but the solution to being able to get my IP address in LiveCode without going out to the web, I have not found. IF a shell statement is required, I would rather open up a web page, that have my app flagged, as dangerous, by virus protection software.
Yes, it would be much more desirable to have a built-in function to obtain that. But if we consider a directory server for this it effectively takes on the role that whatismyipaddress.com would provide, and since I don't think we really need the MAC address anyway that should cover it.
Yes-- coffee sounds good-- me too.
Indeed it is. I'm enjoying a cup of Fair Trade Certified coffee this morning - the great taste of social justice. :)
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

townsend
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 430
Joined: Sun Feb 13, 2011 8:43 pm
Location: Seattle, USA

Re: Peer to Peer Networking Stack

Post by townsend » Thu Jan 09, 2014 9:24 pm

Okay-- It sounds like having a directory server is the best solution.

One of the main selling points of p2p is that the network can't be taken down by blocking or shutting off the server. To maintain this resiliency of p2p networks, I have a suggestion, which I think would be possible.

Once a hosting account has be set up, you also need to define a name for a mySQL database. This does the initial creation. Once that is done, would it be possible to have the actual table structure created by the LiveCode stack? In this way, anyone could add a server to the network, and if one server when down, others could easily pick up the slack.

This could be complicated, as the LiveCode stack would need to have the correct permission to create the file structure. Also, a function would need to be written into the remote table, that simply returned the clients IP address.

Excellent research there Richard!
I'm always attracted to the idea of a generalized solution, but when I look at the variety of P2P protocols here I have to wonder how well any one of those would server as a solution for all them:
http://en.wikipedia.org/wiki/List_of_P2P_protocols
Yes, certainly, if possible. I guess the best choice would be the one that would work with LiveCode. Personally, I have no idea how to implement any of these. Nor difference between those protocols and the "web hooks" you mentioned.

Maybe if the one of these protocols or web hooks might someday be included in the LiveCode library, much the way SQLite is.

townsend
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 430
Joined: Sun Feb 13, 2011 8:43 pm
Location: Seattle, USA

Re: Peer to Peer Networking Stack

Post by townsend » Sun Jan 12, 2014 8:15 pm

I was thinking about the whole mySQL directory idea. MySQL, permissions, and database functions are complicated and could easily take many days to do in a way that retains the resiliency of p2p networks. If it could even be done at all. On the other hand, going out to the internet to get an IP address and then storing that in a drop box or goggle text file would take less than an hour to code.

Since it is not possible to get one's IP address without going out tot he web, I wrote a simple function here that returns the client's IP Address.

Code: Select all

on mouseUp
     answer "IP Address is" && ip.address()
end mouseUp

function ip.address
     local pos, temp
     put URL "http://whatismyipaddress.com/" into temp
     replace space with empty in temp
     put offset("script-->",temp) into pos
     delete char 1 to pos + 9 in temp
     put offset("<!--",temp) into pos
     delete char pos to 9999 in temp
     replace cr with empty in temp
     replace "&#46;" with "." in temp
     return temp
end ip.address
These web scraping routines are very easy for me to do. I can do others if necessary.

None the less-- I don't see this project moving forward any time soon. BUT-- I want to thank Richard and Splash for their participation in this brain storming thread. Anyone else interested in doing a peer-to-peer stack can now find a fantastic catalog of valuable information, thanks to their extensive experience, and a little prodding from myself.

Personally I've experimented with MANY development platforms. One of the most important aspects of any platform is the community. This is just another example of why the LiveCode Community is the BEST I've found, anywhere.

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Peer to Peer Networking Stack

Post by FourthWorld » Sun Jan 12, 2014 11:23 pm

townsend wrote:I was thinking about the whole mySQL directory idea. MySQL, permissions, and database functions are complicated and could easily take many days to do in a way that retains the resiliency of p2p networks. If it could even be done at all. On the other hand, going out to the internet to get an IP address and then storing that in a drop box or goggle text file would take less than an hour to code.
One of the things I love about POSIX systems is fstab, a simple word-delimited file that's critical for the OS to find the volumes it needs to mount in order to run: it reminds us that sometimes even important things can be handled through simple formats. :)
Personally I've experimented with MANY development platforms. One of the most important aspects of any platform is the community. This is just another example of why the LiveCode Community is the BEST I've found, anywhere.
[/quote]
Amen to that, brother.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

BvG
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1236
Joined: Sat Apr 08, 2006 1:10 pm
Location: Zurich
Contact:

Re: Peer to Peer Networking Stack

Post by BvG » Fri Jan 17, 2014 5:07 pm

This discussion made me try hole punching out. I followed the instructions given on the wikipedia tcp hole punching entry. Of note is that in theory a lot of router have uPnP enabled by default, and it'd be probably considered friendlier/less intrusive if that would be used... I have no idea how that would be done tho. So, to punch holes:
  1. connect to a master server that is reachable, let that server send the ip:port identifier back to the client
  2. connect again to the same server, and send the ip:port again to the client
  3. now there's three scenarios that need handleding:
    • port 1 is the same as port 2 -> remember the identifier as is
    • port 1 is one less then port 2 -> add one to port 2
    • other -> remember port 2 as is (wild guess, try anything at all)
    • there's acutally a third valid option, but wikipedia talks about reusing some tcp identifier which is (as far as i know) not settable with LC
  4. repeat the steps for a second client/node
  5. now connect to the newly acquired port from both nodes simultaneously
Now I had no problem with the first few steps, but that last one has me stumped. Usually opening a port is always single directional. There's a server and a client, and the client uses "open socket to <ip>:<port>" and the server uses "accept connections on port <port>"... Just using "open sockets" on both ends results in a socketError message saying "Error 10060 on socket" (no idea what that means). I did not test it with the whole process yet, instead using already forwarded ports that point to my win and mac machine respectively. Funnily the error is always on the windows side of my test setup, never on the mac. I am assuming I misunderstood the wikipedia page somehow. Any ideas?
Various teststacks and stuff:
http://bjoernke.com

Chat with other RunRev developers:
chat.freenode.net:6666 #livecode

splash21
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 369
Joined: Sun Dec 19, 2010 1:10 am
Location: UK
Contact:

Re: Peer to Peer Networking Stack

Post by splash21 » Sat Jan 18, 2014 11:49 am

BvG wrote:now there's three scenarios that need handleding:
port 1 is the same as port 2 -> remember the identifier as is
port 1 is one less then port 2 -> add one to port 2
other -> remember port 2 as is (wild guess, try anything at all)
there's acutally a third valid option, but wikipedia talks about reusing some tcp identifier which is (as far as i know) not settable with LC
Hi, Bjoernke. The last time I tried, I was getting 'other' - the port seemed to be random (OSX mavericks). Back in 2006/2007 I tried looping until I hit a specific target port (Linux and Win XP). That way, if the server was down, you could attempt to contact previously discovered peers on the predefined port. Each peer could do the same, and any that connect successfully could share their known peers - building all the possible connections while the server was down, eventually becoming self aware and launching a nuclear attack on mankind.....

I've not tried recently, but I remember windows was sometimes a bit flaky. What version are you trying it on?
LiveCode Development & Training : http://splash21.com

BvG
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1236
Joined: Sat Apr 08, 2006 1:10 pm
Location: Zurich
Contact:

Re: Peer to Peer Networking Stack

Post by BvG » Sat Jan 18, 2014 5:00 pm

Sadly, I'm not sure what you are asking about. As I said, step 1-4 where working for me easily, my router is doing sub-option 2, so i add one to port 2 for the "next" port. my problem is in the last step even when using known and forwarded ports. In other words, when I leave out the complexity of step 1-4, even then I can't get a working connection if i try to open it with "connect to port" from both both sides.

As I mentioned this is highly unusual way of trying to connect to a port, therefore I assume I have read/interpreted something wrong on the wikipedia page at http://en.wikipedia.org/wiki/TCP_hole_punching
Various teststacks and stuff:
http://bjoernke.com

Chat with other RunRev developers:
chat.freenode.net:6666 #livecode

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Peer to Peer Networking Stack

Post by FourthWorld » Fri Feb 07, 2014 8:11 am

@townsend:
Inspiration, a novel application of P2P:
https://www.kickstarter.com/projects/14 ... net-search
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

splash21
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 369
Joined: Sun Dec 19, 2010 1:10 am
Location: UK
Contact:

Re: Peer to Peer Networking Stack

Post by splash21 » Fri Mar 07, 2014 1:52 pm

I've had a couple of multiplayer ideas on the shelf for ages and since this thread sparked my interest in p2p again... I downloaded Corona and Gideros yesterday (both Lua based) and had a quick read over the Lua socket library. It took a very small amount of code to get two devices connected via p2p (Scotland => England). The mechanism discussed earlier in this thread allowed the hole punching : being able to specify the local port used for the connection. A server was used as the initial directory to discover each others ip address, then a direct link was established between devices. (The directory is a LiveCode app running on a Linux server). If it was possible to do with LiveCode (http://quality.runrev.com/show_bug.cgi?id=5097), it would open up the possibility of all sorts of multiplayer games working across all platforms.
LiveCode Development & Training : http://splash21.com

BvG
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1236
Joined: Sat Apr 08, 2006 1:10 pm
Location: Zurich
Contact:

Re: Peer to Peer Networking Stack

Post by BvG » Sat Mar 08, 2014 2:52 pm

It would help me if you code an example script, of how it should work in LC, and what would need to change for it to work. Because as far as I understood it, there's nothing missing in the socket implementation for LC?

Also you never clarified your question (further up in the thread), so I couldn't answer it?
Various teststacks and stuff:
http://bjoernke.com

Chat with other RunRev developers:
chat.freenode.net:6666 #livecode

splash21
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 369
Joined: Sun Dec 19, 2010 1:10 am
Location: UK
Contact:

Re: Peer to Peer Networking Stack

Post by splash21 » Mon Mar 10, 2014 11:22 am

From the bug report : "Rev. lacks the ability to set the LOCAL port used for the socket connection"

Code: Select all

open datagram socket to "a.b.c.d:x" from local port y
(we can't specify the port y)

I created a simple example that connected 2 iPhones - both on their own local WIFI connections. Each device can use it's own arbitrary local port, but to keep things simple, I used port 10300 on both.

Here's a simple example (that works ;))

Step 1
Each device registers with the server (which records their external ip addresses).

Step 2
Request from DeviceB => Server sends DeviceA's external IP (address of it's router) to DeviceB.
Request from DeviceA => Server sends DeviceB's external IP to DeviceA.

Step 3
Assuming DeviceA sends the first message;
DeviceA sends a message from LOCAL port 10300 to port 10300 on DeviceB's router - which is blocked - DeviceB's router has no knowledge of activity on port 10300.
DeviceB sends a message from LOCAL port 10300 to port 10300 on DeviceA's router. DeviceA's router sees it as a reply to the data originally sent out on port 10300 from DeviceA, so the data is accepted and passed to DeviceA.
DeviceA sends another message from LOCAL port 10300 to DeviceB's router. DeviceB's router sees this message as a reply to the message sent out on that same port from DeviceB : the data is accepted and passed to DeviceB.


Now that you have established direct 2 way communication on port 10300, both devices can talk with no need for a server in the middle. The important part is DeviceB being able to send a message to DeviceA on the same port that DeviceA has already opened and sent data out from - and vice versa.
LiveCode Development & Training : http://splash21.com

BvG
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1236
Joined: Sat Apr 08, 2006 1:10 pm
Location: Zurich
Contact:

Re: Peer to Peer Networking Stack

Post by BvG » Mon Mar 10, 2014 9:09 pm

Hmm.. so you're saying it needs to be the same originating port, something like:

Code: Select all

open socket to remote:8000 from local:8000
I think that it's impossible to specify the local (second in the example) port. It's given out randomly not by the engine, but by the OS. at least as far as I know, no other language can do that either? But I have really almost no experience with other languages, so maybe when creating a custom tcp/ip stack from scratch these things become possible.

For example, problems with random os port numbers:
http://stackoverflow.com/questions/1758 ... meral-port

As far as I understand what you said, there is no built in way to do what you want, that's why I asked for a specification. Instead I think we need to be able to force SYN as response to a SYN, instead of the usual ACK. See also section 3.4, figure 7 and 8 (page 31/32) of the rfc.

Either way, currently we can't do hole punching and something in the engine needs to change to enable it.

Said all that, maybe we'd be better served with an implementation of uPnP (respectively IGD) and/or STUN?
Various teststacks and stuff:
http://bjoernke.com

Chat with other RunRev developers:
chat.freenode.net:6666 #livecode

splash21
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 369
Joined: Sun Dec 19, 2010 1:10 am
Location: UK
Contact:

Re: Peer to Peer Networking Stack

Post by splash21 » Mon Mar 10, 2014 10:09 pm

RFC768 : User Datagram Protocol - https://www.ietf.org/rfc/rfc768.txt

Code: Select all

Format
------

                                    
                  0      7 8     15 16    23 24    31  
                 +--------+--------+--------+--------+ 
                 |     Source      |   Destination   | 
                 |      Port       |      Port       | 
                 +--------+--------+--------+--------+ 
                 |                 |                 | 
                 |     Length      |    Checksum     | 
                 +--------+--------+--------+--------+ 
                 |                                     
                 |          data octets ...            
                 +---------------- ...                 

                      User Datagram Header Format

Fields
------

Source Port is an optional field, when meaningful, it indicates the port
of the sending  process,  and may be assumed  to be the port  to which a
reply should  be addressed  in the absence of any other information.  If
not used, a value of zero is inserted.
(Used code block for fixed width font)

When zero is used for the source port then the OS supplies an ephemeral port number.
LiveCode just needs to allow binding of the UDP socket to the local port.
LiveCode Development & Training : http://splash21.com

BvG
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1236
Joined: Sat Apr 08, 2006 1:10 pm
Location: Zurich
Contact:

Re: Peer to Peer Networking Stack

Post by BvG » Tue Mar 11, 2014 12:00 am

You said that this is possible for other languages, which sounds like you've used it or seen it in a documentation somewhere?
Various teststacks and stuff:
http://bjoernke.com

Chat with other RunRev developers:
chat.freenode.net:6666 #livecode

Post Reply

Return to “Internet”