Speeding up text operations on Win

[AxWald]

Hi,

it has been reported repeatedly that operations with large amounts of text are unbearable slow when using new versions of LC on Windows, while not suffering this much on MacOS.

Now there is a thread on the mailing list (where the enlightened ones enjoy their neolithic quote hell undisturbed by us unwashed masses) targeting this, and it actually yielded interesting results. So for the education of "the rest of us" I did a few quick tests and will provide some code samples here.

A very common piece of code:

Code: Select all

   repeat for each line L in myData
      put L & CR after myVar
   end repeat
   delete char -1 of myVar

Usually we would have something here that filters, or changes the data - for this example we omit this. We just copy myData, line by line, into myVar.

myData here is 22,470,000 Bytes, in 502,000 lines (rounded).
Running this on LC 6.7.10/Win takes 559 millisecs.
Running this on LC 9.6.3/Win takes 25,724 millisecs. Ouch.

Mark Waddingham replies and, as usual, finds a scapegoat ("the windows heap manager not being very good at continually re-extending a buffer"). But shows a workaround:

Code: Select all

   put 1000000 into myBufferSize
   repeat for each line L in myNewData
      put L & CR after myBuffer
      if (the number of codeunits in myBuffer> myBufferSize) then
         put myBuffer after myVar
         delete codeunit 1 to -1 of myBuffer
      end if
   end repeat
   put myBuffer after myVar
   delete codeunit -1 of myVar

This reduces the time on LC 9.6.3/Win to tolerable 1,200 millisecs. Only 100% slower than 6.7.10 ;-)

Interesting here:

• "the number of codeunits in myBuffer" is much, much faster than the classic "len(myBuffer)"!
• The size of the buffer is important - in my case the "sweet spot" is ~ 1MB.

I hope this may help one of the remaining Win users of LC. Don't despair, Mark mentions: "This is an engine issue - we'll need to look into why there's such a difference with 6.7". So, maybe as early as even LC 12, we'll see better performance on Win, too :)

Have fun!

[richmond62]

maybe as early as even LC 12

Someone got out the wrong side of bed today.

It would be interesting to know how long this type of exercise takes on Linux.

And, just to be picky, this:

Code: Select all

repeat for each line L in myData
      put L & CR after myVar
   end repeat
   delete char -1 of myVar

might have a var myData contining lines and lines of numbers, and would
that make any difference . . . while LiveCode, on the surface, may not differentiate between numeric
and string variable I wonder what goes on "under the hood."

[AxWald]

Hi,

It would be interesting to know how long this type of exercise takes on Linux.

I'd expect it to be faster - but that's off topic here: it's about the dramatic degradation of LCs performance with some basic operations in the new versions, and it's explicitly about Windows (thus the "Windows" sub forum ;-) ).

It looks a lot like the mothership has spent few effort about the Win implementation & left an ugly "handbrake pulled" - after all, on MacOS it's faster meanwhile! Example from the list (for my first code snippet):

6.7.11 MacOS 8 seconds
6.7.11 Win32 7 seconds
9.6.3 MacOS 0 seconds
9.6.3 Win32 591 seconds

Further:

[...] contining lines and lines of numbers, and would that make any difference

At this point we're concerned about string variables only (even if it may be strings composed by numeric chars). It's about repeatedly concatenating strings (a thing where LC usually shines) that obviously got fraggled up in the new Windows versions ;-)

Anyway, at least there's a work-around that speeds up execution of such actions to a tolerable amount. Even if it's a rather clumsy one, and you'll have to fine-tune your buffer size carefully.
But it reduces the time in my example from 25,724 millisecs to 1,200 millisecs, an improvement that may save someones project - and that's why I posted it here.

maybe as early as even LC 12

Someone got out the wrong side of bed today. 8)

Well, when 9.6.1 was fresh, I posted a bug report: For some button types ("roundRect", "rectangle", ...) the "autohilite" is broken - it "flickers" when you move the mouse/ your finger a bit. You cannot use such in professional software!
But this "only" occurs on Windows & Android (Edit: and Linux, I forgot!), so who cares? It's confirmed, but still not corrected in 9.6.3, a 3/4 year later.
The button misbehavior may be a simple typo somewhere (other button types still work flawlessly!) & easy to fix. But what an amount of time will it take to fix this threads topic, where it is about "the rules about extending buffers [...] in string concatenation"? And where not even Android is hit, only Windows, and then only a few poor chaps that have to massage insanely huge variables? I'd not even bet for LC 12 ...

Have fun!

[richmond62]

Have fun!

I tend to mainly because I don't generally deploy to Windows.

If we want to ruminate on LiveCode's partiality towards operating systems we can do, and as
they do the "deep and dirty work" making LiveCode on Macintosh computers . . .

Those of us who regularly use Linux are "Effed off" re sound and video . . . from my point of view
a "right hoo-ha" as my Language School would be far "sexier" if the standalones I wrote for all
the Linux boxes there featured consistent sound and video.

My last "adventure" with sound and video for Linux was with Ubuntu in 2007, and I had to slow
down all my sound samples to half speed so that RR/LC would play them so they did not sound
like a squeaky mouse on my machines.

Linux apart, give. the market share that Windows enjoys globally it does behoove LiveCode to
get off their collective cough-cough-cough and sort the text problem out.