Most Efficient Script question

[richardmac]

Here's the situation. I have a tab delimited text file that is 70,000 lines long - about 37 MB in size. Here's what I need my script to do:

1. Go through each line and read two items (first name and last name)
2. Create a new user name based on first name, last name, and a random number
3. Add a tab character and the new name at the end of each line

I've tried to different ways to loop through the file - when I do a regular repeat loop like the following, it takes a decade to run:

repeat with i = 1 to the number of lines of myVariable -- which contains the text file
do the code
end repeat

And I tried this type of loop, but it crashes LiveCode:

repeat for each line i of myVariable
do the code
end repeat

What is the best way to do this, where it won't take a decade to run and won't crash LiveCode? Any help greatly appreciated.

[Klaus]

HI Richard,

"repeat for each line tLine..." is READ ONLY!
That means you cannot modify the content of tLine!
If you do, this will have no effect or LC might even crash, as you experienced!

So you will need to put tLine into another variable and modify that!
This will take of course the double amount of RAM but is still very fast:

Code: Select all

...
set itemdel to TAB
repeat for each line tLine in tVariable
  put tLine into tLine2
  ## do this and that
  put TAB & item X of tLine after tLine2
  put tLine2 & CR after tVariable2
end repeat
## Get rid of trailing CR
delete char -1 of tVariable2
## Now write tVariable2 to file on disk if neccessary
...

Hope that helps.


Best

Klaus

[bn]

Hi Richard,

you may have a look at thread

http://forums.runrev.com/phpBB2/viewtop ... f=9&t=6226

You might even know the original poster in person

Kind regards

Bernd

[Klaus]