crash on repeated filters with large text

If you find an issue in LiveCode but are having difficulty pinning down a reliable recipe or want to sanity-check your findings with others, this is the place.

Please have one thread per issue, and try to summarize the issue concisely in the thread title so others can find related issues here.

Moderators: heatherlaine, Klaus, FourthWorld, robinmiller, kevinmiller

Post Reply
gmccarthy
Posts: 62
Joined: Sat Feb 17, 2007 4:56 am
Location: Australia

crash on repeated filters with large text

Post by gmccarthy » Mon Jun 13, 2016 11:56 am

I have a been getting crashes everytime in livecode 8 when repeating filters of a large text more than about 5 times.
The text is over 60000 lines with about 20 tabbed items per lines.
The first 2 tab delimited items of each line are a name and an ID.
Sometimes I need to get the line for a name or for an ID.
There may be repeated uses of the same name -with different IDs.
I need to filter the text for certain strings in item 1 or 2 of the text.
In certain situations I have to many strings to filter for, not just one (which seems to be stable).
When I need a repeat loop with the filtering more than about 5 times to gather information for more than 5 strings the repeated filters end up causing a crash.

I have reported this as a bug. http://quality.livecode.com/show_bug.cgi?id=17841

Any ideas what I might do while waiting for a fix?
Should I consider building arrays from the data instead, although it would have to be done after starting up the start to take into account the new data file which might is typically altered several times per day.
gmcrev

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 4303
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: crash on repeated filters with large text

Post by jacque » Mon Jun 13, 2016 9:40 pm

It sounds like a memory limitation, which wouldn't really be a bug. It's just that you're trying to stuff more data into RAM than the capacity of the computer will hold. I had the same issue when trying to load too many very large images at once, and altering my script fixed it. But we'd need to see your handler to know for sure.

If you aren't putting empty into variables the handler no longer needs, that would be the first thing to try. Re-using variables (replacing their contents) may do much the same thing. How is your repeat loop and filtering process structured?
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

AxWald
Posts: 309
Joined: Thu Mar 06, 2014 2:57 pm

Re: crash on repeated filters with large text

Post by AxWald » Wed Jun 15, 2016 3:42 pm

Hi,

never tried "filter" in such a situation - and why do you wait during the repeat? :shock:

Code: Select all

repeat with tCount = 1 to tRepeatFilterNum
      wait tRepeatFilterNum ticks
      put empty into tOutput
      put tRepeatFilterNum into fld "RepeatFilterNumCount"
      filter lines of tPlayers with wildcard pattern ("*" & tFilter & tab & "*") into tOutput
   end repeat
This code doesn't make sense to me. Care to explain?

Code: Select all

  put empty into tOutPut
    repeat for each line MyLine in tPlayers
        if offset(tFilter & tab,MyLine) is not "0" then
            put MyLine & return after tOutPut
        else next repeat
    end repeat
This gives you a variable containing all lines that match your search string - and I'm quite sure there'll be no crashes.

For really big tData I'd write the found lines to a file, this is even faster (see here).

But I assume your problem is different. To judge this more input is required ;-)

Have fun!
Livecode programming until the cat hits the fan ...

Post Reply

Return to “Bug Triage”