Corrupted stack?

[danielrr]

I have a very large stack of LiveCode and I suspect it (or a portion of it) may be corrupted. First let me explain the situation.
I use this stack to handle text in very different ways. Whenever the text I'm using is ASCII, I open it with LC 6.5 or 6.6, since it is much, much faster than LC7 (with large chunks of text It may be dozens of times faster). But when I have to deal with Unicode, I have to open it with LC7.
Well, I have come to learn the hard way that none of the DP versions are really stable (that is by definition, so no problem here). The ugly things come mainly when trying to process large chunks of text, or you request many repetitions in the same handler (as when you try to recode text). In such cases it may well happen that LC provides corrupted output (aleatory non UTF chars) randomly. With a little bit of patience (an depending on python for the chunks that LC definitely can't handle) I can go on.
Now the problem: when I try to process one big chunk of text inside this program (150.000 chars), most of the time one line at the time, LC7 crashes. For instance, the following code (using the whole text) will make LC crash some times, some times not

Code: Select all

repeat while "  " is in bigText
replace "  " with " " in bigText
end repeat

Crashes occur at different points at the code. If you use the debugger, you can see that the crash happens at different places and while executing different functions. LC7 doesn't like big variables of Unicode text, thats for sure. But I have tried processing the text sucking it one line at the time from a field, and crash will happen anyway.

The problem occurs with all versions of LC7, from 7 to 10. No message of corrupted stack appears. I'm using MacOS 10.9.4 on an iMac and a powerBook. I restarted the computer several times.
My question is threefold:
1. Is there a way to say whether a stack is corrupted? Is there such a thing as a temporary state of LC where a stack seems to be corrupted, but after the state has been corrected, the stack will return to normal functioning?
2. If one is to conclude that the stack is corrupted, is there a way to save a clean copy of the stack (and I don't mean just the scripts, but the whole stack). If the problem is located in one substack Is it enough to clone the substack?
3. Say the stack is clean? How do yo u deal with large Unicode texts to avoid crashes?

many thanks and best wishes

Daniel (praying every day for a stable version of LC7)

[FourthWorld]

If you can open the stack it's not corrupted. Crashes are bugs - please submit your recipe with a sample stack so the team can address it:
http://quality.runrev.com/

[atout66]

@ Daniel

I encounter the same problems as you. That's why I stay on 6.5 version too.
As I deal with the same kind of data (quiet big!), I wonder if you got a fix from the quality department ?

Thanks in advance, Jean-Paul.

[Simon]

Hi Daniel,
hmmm....

Code: Select all

repeat while "  " is in bigText
replace "  " with " " in bigText
end repeat

Replace doesn't need the repeat loop (not saying that will solve all your problems). With that, a change in thinking how memory is handled. Watch your memory and see if it ever pegs (hope you are from the US because I couldn't figure out a what a common term for "pegs" would be, "is full"?).

Simon

[dave.kilroy]

Hi danielrr

If you are working line-by-line are you by any chance updating the field as you go? If so then don't, or if you must at least use a 'lock screen' at the top of your script and finish off with a 'unlock screen'

Now the problem: when I try to process one big chunk of text inside this program (150.000 chars), most of the time one line at the time, LC7 crashes. For instance, the following code (using the whole text) will make LC crash some times, some times not

It does sound as if it might be a memory issue - try reporting values from the hasmemory() function at various parts of your code?

Note: The max number of chars you can have in each line of a field is 65,536

Dave

[Simon]

Rats, forgot to look at the original posting date...
August of last year.

[dave.kilroy]

Ah. I also didn't look. Current versions of LiveCode have probably sorted the issue he was having with the version he was using then.

[FourthWorld]

It probably has been resolved. 150k is very small compared to LC's limits. I have apps that handle several MBs of text, one of them is regularly used on data larger than 300 MBs, and it all flies swimmingly in LiveCode.

[MaxV]

However I think that to speed up the program, this code is more efficient because it avoids loops and searches inside text:

Code: Select all

repeat for each char tchar in bigText
   put tchar after newBigText
   if (char -2 to -1 of newBigText) is "  " then 
       delete char -1 in newBigText
    end if
end repeat
put NewBigText into BigText #optional