Incoming Serial data capture performance

Interested adding compiled externals from LiveCode and third parties to your LiveCode projects? This is the place to talk about them.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
dr_mumps
Posts: 23
Joined: Wed Dec 24, 2014 7:00 pm

Incoming Serial data capture performance

Post by dr_mumps » Thu Dec 03, 2015 2:39 am

I am writing an Arduino to LiveCode (v7.0.1 community edition running on a Windows 7 Enterprise platform on a fairly current i7 based PC) application. I have the basic serial communication working OK but it is rather flaky and unreliable. The issue is with the Livecode side (I’ve verified the reliability of the Arduino side with a real-time hardware protocol analyzer). Livecode just doesn’t seem to be fast enough to ALWAYS capture the serial stream being sent from the Arduino. I’d say about 80% of the time it works fine but the other 20%, the Livecode side either drops the incoming data (which is only 5 bytes long) or seems to go into some weird “suspended” state (with the whirling little blue busy icon).

The read section code snippet is written as “tight” as I can devise. As you can see, there are actually two reads. The Arduino sends the same “packet” twice to improve the reliability (yes, I’ve tried just one with a vast increase in packet loss). The packet itself consists of 5 bytes … the first is a “!” (ascii 33) and the last is a “*” (ascii 42) to frame the 3 data bytes in between. The “getData” command sets up the serial port after having acquired the actual port ID (variable PBPort) earlier in the script.

The call to “changeBar” within the “readData” command is to a small command call to update a do-it-yourself scrolling bar indicating the value being processed on the Arduino.

As I said, this works fine about 80% of the time but I need nearly 100% reliability and speed. I’ve tried many, many combinations of the timers and wait conditions and all of the other “read from driver” options available … this combination seems to provide the greatest overall success. I suspect I’ll need to go to an external code to improve the performance unless some kind soul from this forum can suggest an alternate approach of idea.

Yes, I know I could do a “full” ACK/NAK communication protocol but then I’d lose the desired processing speed on the Arduino side, which would then have to wait for an ACK or NAK and respond accordingly. I’m going to try that but I’d VERY much appreciate any feedback on this subject from the forum first.

And while you’re pondering that issue, also in the read loop, I want the read exited when I press the mouse button. HOWEVER, this ONLY works when I hold the mouse button down while a packet is arriving from the Arduino. That doesn’t make any sense to be but it is 100% consistent. So the question is: How do I interrupt this loop simply by pressing the mouse button? … whether or not a packet is coming in from the Arduino or not?

Code: Select all


on getData
     put "BAUD=9600 PARITY=N DATA=8 STOP=1 to=off xon=off" into setSerial
     set serialControlString to setSerial
     
     open driver PBPort for binary update
     put the result into y
     readData
     close driver PBPort
end getData

on readData
     repeat until the mouse is down
          put empty into xx
          read from driver PBPort for 5 uInt1 in 10 milliseconds
          put the result into z1
          put it into y1
          wait 5 milliseconds with messages
          read from driver PBPort for 5 uInt1 in 10 milliseconds
          put the result into z2
          put it into y2

          if (item 1 of y1 = 33) and (item 5 of y1 = 42) and (z1 is empty) then
               put y1 into xx
          else if (item 1 of y2 = 33) and (item 5 of y2 = 42) and (z2 is empty) then
               put y2 into xx
          end if
          
          if xx is not empty then
               changeBar ("bar" & (item 3 of xx)), ((item 2 of xx) * 4.54)
          end if
          
     end repeat
     
end readData


jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7210
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Incoming Serial data capture performance

Post by jacque » Fri Dec 04, 2015 12:40 am

I'm no expert with drivers, but I see something that would slow down performance and possibly miss any incoming data.

The main thing is the repeat loop that also checks for the mouse status. You really don't want that. Repeats are slow by themselves, but constantly checking the mouse makes it worse. In addition, the mouse will not be "down" unless it's down at exactly the moment when that line of script executes. If you click at any other moment, the "down" position may never be recognized.

The recommended way to execute loops like this is to set a script flag to true when the process begins and to false when it should end. The readData handler should check that flag and exit if it is false. If it's okay to continue, then it should read for 5 and then send a message to itself to execute readData again in 10 milliseconds.

I wouldn't put a cap on the read ("read for 5 in 10 milliseconds") -- let it complete on its own. The next read will occur in 10 milliseconds after the first one finishes.

The general (untested) revision is like this:

Code: Select all

local sReadFlag -- make sure you declare this

on getData
  put "BAUD=9600 PARITY=N DATA=8 STOP=1 to=off xon=off" into setSerial
  set serialControlString to setSerial
  
  open driver PBPort for binary update
  put the result into y
  put true into sReadFlag
  readData
end getData

on readData
  if sReadFlag = false then exit readData
  put empty into xx
  read from driver PBPort for 5 uInt1 
  put the result into z1
  put it into y1
  wait 5 milliseconds with messages
  read from driver PBPort for 5 uInt1 
  put the result into z2
  put it into y2
  
  if (item 1 of y1 = 33) and (item 5 of y1 = 42) and (z1 is empty) then
    put y1 into xx
  else if (item 1 of y2 = 33) and (item 5 of y2 = 42) and (z2 is empty) then
    put y2 into xx
  end if
  
  if xx is not empty then
    changeBar ("bar" & (item 3 of xx)), ((item 2 of xx) * 4.54)
  end if
  send "readData" to me in 10 milliseconds  -- calls itself to execute again
end readData

on stopRead -- sent from a button or some other method of control
  put false into sReadFlag
  close driver PBPort
end stopRead
You'd have a button somewhere that you click to stop the process, and it would call "stopRead" so that the flag is turned off and the driver is closed. The above assumes you've declared PBPort somewhere so that all handlers can use its value.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

dr_mumps
Posts: 23
Joined: Wed Dec 24, 2014 7:00 pm

Re: Incoming Serial data capture performance

Post by dr_mumps » Fri Dec 04, 2015 10:52 pm

Thank you so much for your reply! I had started this script with something very similar to your suggestion (which was great for you to have written an alternative for me!). I have inserted your approach into my app; however, it seems to make very little appreciable difference. Perhaps if I were to capture some actual timing stats, I might see a difference but, subjectively, I am still losing messages at an unacceptable frequency for my needs.

I put this call for help in this "externals" topic because I'm fairly convinced I'll need a tight C++ (etc.) external to acquire the reliability I'd really like to have. I'm wondering if anyone has or knows about any such real-time data acquisition external that I could try. I know that Monte Goulding writes externals for mobile device. Actually, I'm trying this approach to get a handle on eventually porting this code to an iPhone/iPad and using his Bluetooth library for the communication.

Unfortunately for me, I am not proficient enough in C++ or Java, etc to attempt writing such an external myself... so ANY help or suggestions would be hugely appreciated!

Again, thanks Jacque for your reply!

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7210
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Incoming Serial data capture performance

Post by jacque » Fri Dec 04, 2015 11:37 pm

The reads should be very quick, so I wonder if the "changebar" handler is the bottleneck. Just as a test, you might try commenting out that and the "if" tests for xx. Put the results of the read into a script-local variable, which you can display after the stop button triggers to see if you got the data you expect. That would tell you if the problem is in the communications, or in the side-track into another handler.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

dr_mumps
Posts: 23
Joined: Wed Dec 24, 2014 7:00 pm

Re: Incoming Serial data capture performance

Post by dr_mumps » Wed Dec 09, 2015 2:06 am

Thanks again for the interest. I tried your suggestion and, unfortunately, I still get about the same amount of lost or partially lost bytes as with the "if's" and "changeBar" calls.

I truly think this is only going to be solved by using an external call... now the challenge is to find someone who has one or would be willing to write one ... for pay, if necessary.

I've searched for an recent external SDK for Windows.... the only one I can find is one for the Mac

tcbcats
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 27
Joined: Tue May 06, 2008 5:33 am
Location: Las Vegas
Contact:

Re: Incoming Serial data capture performance

Post by tcbcats » Wed Dec 16, 2015 6:17 am

It looks like I see the same problem with totally different code. I'm still trying to find out why the serial reads are S o o o o slow.
Move your code to rev 5.5 and you will see it fun a lot faster.
I posted my problem under the "windows" section Yesterday.
Cheers,
Tcbcats
Las Vegas
tcbcats

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: Incoming Serial data capture performance

Post by mwieder » Tue Jun 21, 2016 11:06 pm

Late to the party here, but dr mumps:

It looks like you're trying to read two packets of five unsigned integers each, no matter whether there was an error in the first or not. You might want to check the error from the first (the result in z1), and use that to decide whether to grab the next packet.

On the other hand (it's hard to tell your intention from the posted code) you might be trying two reads in the hope that one succeeds. In that case the above still applies, and would translate to "only try a second time if the first fails".

In either case, that might account for some of the missing data.

Post Reply

Return to “Using Externals”