How I delete empty lines in a list ? [Solved]

[atout66]

Hi to all,

I need your help for coding a script which delete all empty lines in a list of more than 475000 index.
I've tried the code below but it doesn't work (infinit loop).

Code: Select all

on mouseUp-- delete empty lines
    put fld "texteBrut" into laListe -- contains > 475000 lines
    put the number of lines of laListe into nbrDeLignes   
    repeat with xxx = 1 to 25 --TEST until 25 not all nbrDeLignes --                   
        put line xxx of laListe into laLigneActive
        if laLigneActive is empty then
            delete line xxx of laListe      
            put empty into fld "texteBrut"   --   
            put laListe into fld "texteBrut"  -- update the fld        
            mouseUp -- we start again
        end if
    end repeat

Any idea much appreciated,
Regards, Jean-Paul.

[jmburnod]

Hi Jean-Paul,

I see two ways:

Code: Select all

on mouseUp-- delete empty lines
   put fld "texteBrut" into laListe -- contains > 475000 lines
   put empty into tTexteSansVide
      repeat for each line laLigneActive in laListe--TEST until 25 not all nbrDeLignes --         
         if laLigneActive is empty then 
            next repeat     
         end if
         put laLigneActive & cr after tTexteSansVide
      end repeat
      delete char -1 of tTexteSansVide
      put tTexteSansVide into fld "texteBrut" 
end mouseup

or

Code: Select all

on mouseUp-- delete empty lines
   put fld "texteBrut" into laListe -- contains > 475000 lines
   filter laListe without empty
   put laListe into fld "texteBrut" 
end mouseup

Best regards
Jean-Marc

[atout66]

Wouha ! This "filter" function is fantastic and your help also
Thanks a lot, Jean-Paul.

[jmburnod]

I tested the speed and it seems the repeat and filter way are equivalent 6700 milliseconds
for a list of 475000 lines
I guess someone will have some suggestions to do it faster

C'est toujours un plaisir de savoir qu'un truc fonctionne comme attendu par d'autres

[atout66]

You must have a very fast machin ! To me, it took 22s with the filter function.
Didn't try with the repeat because I suppose the internal functions are faster...

Regards, Jean-Paul.

[jmburnod]

You must have a very fast machin !

No. An old iMac IntelCore duo 2.66 GHz, 4Go
I wonder :
1. Each line of my TexteBrut are short
2. if LiveCode version explains this. (I used LC 6.7.0)

This script build the content of my fld "texteBrut" :

Code: Select all

on mouseUp
   put the milliseconds into tOld
   put empty into t
   repeat with i = 1 to 475000
      if random(5) = 1 then put empty into ajout
      else put i into ajout
      put ajout & cr after t
   end repeat
   put t into fld "texteBrut"
   put the milliseconds - tOld
end mouseUp

[atout66]

I wonder :
1. Each line of my TexteBrut are short

May be this could be the point.
The text file I deal with is 31105 Ko for 473495 lines cleaned...

Regards, Jean-Paul.

[jacque]

I changed the list creation handler to this:

Code: Select all

on mouseUp
  put the milliseconds into tOld
  put empty into t
  repeat with i = 1 to 475000
    if random(5) = 1 then put empty into ajout
    else put any line of the colornames into ajout
    put ajout & cr after t
  end repeat
  put t into fld 1
  put the milliseconds - tOld
end mouseUp

That made a list of colors (longer entries,) and took 8+ seconds to generate. Then I ran a test like this:

Code: Select all

on filtertest
  put the milliseconds into tStart
  get fld 1
  filter it without empty
  put the milliseconds - tStart
end filtertest

Using filter, this handler removed empty lines in 39 ms on 1-year-old iMac running Mavericks and LC 6.7.5. Then I tried the same handlers again in LC 7.0.5. It took 41+ seconds to generate the field content. It took 183 ms to filter it.

We know that LC 7 is much slower, but even so, 183 ms isn't very long, but older machines would be slower. The version of LC is the important thing though.

[atout66]

We know that LC 7 is much slower, but even so, 183 ms isn't very long, but older machines would be slower. The version of LC is the important thing though.

Very interesting test, thanks