Page 1 of 1
faster way?
Posted: Fri Dec 24, 2010 10:11 pm
by adventuresofgreg
Hi: I'm processing a coma delimited data file and I need to issolate the second item of each line into it's own comma separated variable. Is there a faster way than:
put item 2 of line x of mydata & "," after itemTWOlist
put x+ 1 into x
Thanks,
Greg
Re: faster way?
Posted: Fri Dec 24, 2010 11:41 pm
by Dixie
Hi,
Using 'repeat for' is very quick...
Code: Select all
on mouseUp
put fld 1 into myData
repeat for each line thisLine in myData
put item 2 of thisLine & comma after myParsedData
end repeat
put myParsedData into fld 2
end mouseUp
be well
Dixie
Re: faster way?
Posted: Sat Dec 25, 2010 4:32 pm
by adventuresofgreg
WOW! That blew me away. With the x+1 method, it took 18 seconds to process a file with 20,000 lines. With the "repeat for each line thisline" method it took less than 1 second!
All these years I thought they were both the same, so x+1 became a habit.
I need to do some more speeding up. To run everything else I'm doing with these files, it's currently taking over 15 minutes! Other things that seem to take time is the lineoffset function. Is there a faster way of searching?
"put lineoffset(xdate,gZW) into y"
(gZW is a 20,000 line x 10 item comma delimited file in memory, and xdate is a date). I need to do this search 20,000 times on 10 different files of 20,000 lines x 10 items each.
Re: faster way?
Posted: Sat Dec 25, 2010 5:02 pm
by adventuresofgreg
I'm having one problem with "repeat for"
I was doing this:
put 1 into x
repeat for the number of lines of ES
put item z of line x+2 of ES into plast
Now, I do this and it doesn't work:
repeat for each line thisline in ES
put item z of thisline+2 into plast
it doesn't like "thisline+2" - so how do I reference a line offset from the current line?
Thanks,
Greg
Re: faster way?
Posted: Sun Dec 26, 2010 12:22 pm
by Dixie
Hi Greg...
You could try this...
Code: Select all
on mouseUp
local lineNumber
put fld 1 into ES
repeat for each line thisLine in ES
put item 2 of line (lineNumber + 2) after plast
add 1 to lineNumber
end repeat
put plast into fld 2
end mouseUp
be well
Dixie
Re: faster way?
Posted: Sun Dec 26, 2010 12:40 pm
by Klaus
Hi Greg,
merry christmas
I don't get the logics behind your script?
What you want is every second item of line 3 to last line of fld 1, right?
To me this seems to be identical to this:
Code: Select all
...
put fld 1 into tEs
delete line 1 to 2 of tEs
repeat for each line tLine in tEs
put item 2 of tLine & CR after tTwos
end repeat
delete char -1 ot tTwos
put tTwos
...
Or am I missing something obvious?
Best
Klaus
Re: faster way?
Posted: Sun Dec 26, 2010 4:13 pm
by adventuresofgreg
Hi Klaus and Dixie:
Here is a sample of the exact script that is too slow:
on mouseUp
put 1 into x
put fld "ES" into pricedata
repeat for the number of lines of pricedata
put item 5 of line x of pricedata + item 4 of line x-6 of pricedata & comma after xtheprice
add 1 to x
end repeat
put xtheprice into fld "console"
end mouseUp
The way to speed up these kinds of repeat functions is to use "repeat for each line thisline in Pricedata", but that only works on ONE line at a time. In other words, you can't reference another line which is offset from "thisline" - and therefore I have to revert back to the SLOW way of doing it. Any suggestions?
Thanks,
Greg
Re: faster way?
Posted: Sun Dec 26, 2010 5:26 pm
by FourthWorld
There are many ways to solve such problems, but I'm curious: why is it you want to compare data that's six lines apart? Will that ever change? Might there be some times when you want to compare things that are three lines apart, or five, or seven?
Re: faster way?
Posted: Sun Dec 26, 2010 10:38 pm
by adventuresofgreg
FourthWorld wrote:There are many ways to solve such problems, but I'm curious: why is it you want to compare data that's six lines apart? Will that ever change? Might there be some times when you want to compare things that are three lines apart, or five, or seven?
Yes - the number of lines apart, and the item numbers are all variables that may change in my script. My issue is that running a script that deals with actual line numbers runs VERY slow compared to a script that uses "repeat for each line"
So - to review my problem:
This method is SLOW:
repeat for the number of lines of holder
do anything with items in line x of holder
put x+1 into x
end repeat
This method is FAST:
repeat for each line thisline in holder
do anything with items in the line thisline
end repeat
The problem is that when using the FAST method, I cannot refer to any other line aside from the current THISLINE. If I keep track of the line number (like using a variable like x), then it runs SLOW like the first example.
Re: faster way?
Posted: Sun Dec 26, 2010 11:24 pm
by bn
Greg,
I dont get how you would get line number x - 6 if x is below x but that is what you wrote. I you would have posted just some lines of example data it would be easier to help you.
I propose to make an array for every line of your data which is actually quite fast. The key of the array is the 'linenumber'
Then you do your repeat for each line stuff on pricedata.
The code would be something like this. ( it is not tested since you did not provide testdata)
Code: Select all
on mouseUp
put field "ES" into priceData
-- first put every line of priceData into an array
-- here you could just as well put just the item of interest, that is up to you
put 0 into tCounter
repeat for each line aLine in priceData
add 1 to tCounter
put aLine into tArray[tCounter]
end repeat
-- now do the repeat for each line aLine in priceData
put 0 into x
repeat for each line aLine in priceData
add 1 to tCounter
-- how you would want to get at line x - 6 if x is lets say 1 is your secret, but that is what you
-- wrote in code. Adjust if I got it wrong.
put item 5 of aLine + item 4 of tArray[x - 6] & comma after xtheprice
end repeat
delete last char of xtheprice
put xtheprice into field "console"
end mouseUp
should be a lot faster than looking up the line number 20000 times and then get the item of interest.
regards
Bernd