How to read from a file every 3 lines until eof

[alemrantareq]

hi, I'm trying to make a filter from a certain list of texts in a text file. The texts are in this format:

word1-word2
word3-word4
word5-word6
......-......
wordn1-wordn2

I've 2 text fields to sort the texts separately. Field 1 will hold wordn1 and Field 2 will hold wordn2. Assume, there're 1000 lines in this format. I want to read only 3 lines each time and filter them and sort them in the 2 fields. I've made a script:

Code: Select all

on mouseUp
   answer file "select text file" with type "Text|txt"
   put it into tfile
   open file tfile
   set the itemdel to "-"
   repeat for each line tline in tfile
      read from file tfile for 3 line
      put item 1 of tline & cr after fld "f1"
      put item 2 of tline & cr after fld "f2"
   end repeat
   close file tfile
end mouseUp

My bad, it filters only the first 3 lines. Not repeating afterwards. I wonder how to read from file for 3 lines until eof? Can anyone help me out pls...

[dunbarx]

Hi.

I would read the entire file, and do all my three-line analyses within LC. This will be much faster, number one, and all the power of LC will be at your disposal. It should be incomparably more straightforward.

If you need more help, write back.

Craig Newman

[alemrantareq]

can you give me your code to help me out pls? thanks. Mine one is putting the text file location in field 1 instead of reading the contents of the text file...

[dunbarx]

OK. I have no idea what sort of filtering you intend, but try this simple experiment. On a new card, make one button and two fields. In the script of the button:

Code: Select all

on mouseUp
   put "" into fld 1
     repeat with y = 1 to 20
      put "aa" & y & "-" & "bb" & y into line y of fld 1
   end repeat
   wait 60
   get fld 1
   put 0 into counter
   set the itemDel to "-"
   repeat until counter > the number of lines of fld 1
      repeat 3
         add 1 to counter
         put item 2 of line counter of fld 1 & return after temp
      end repeat
      put temp into fld 2
      put "" into temp
      wait 30
   end repeat
end mouseUp

Now when you press the button, field 1 will load with data. After a second, the second portion of the handler will take successive groups of three lines and extract the second item in each line, placing that dataSet into fld 2. I suppose this could be considered "filtering", but perhaps you can provide your own. This is just an example of one method to do what I think you are asking for. If not, maybe it will help you along the way.

Craig

[jacque]

This will split the data into two columns and put each column into the fields:

Code: Select all

  answer file "Select the text file:"
  if it = "" then exit to top -- always check if the user cancels
  put url ("file:" & it) into tText
  set the columndelimiter to "-"
  split tText by column
  put tText[1] into fld "f1"
  put tText[2] into fld "f2"

I'm not sure why you need to process 3 lines at a time. Do you need to do more than just move the data into the fields?

[dunbarx]

Hi.

So is Jacque right that you only wish to separate the two parts of the raw text, separated by the dash? if so, why run in groups of three? I assume there is more going on here, though her example is a great introduction to arrays and the "split" command.

Craig

[alemrantareq]

thanks jacque and dunbarx for the replies. Script of jacque helped me solve the issue very easily. But there's a little more I want. As my text file is 26 MB in size, the process takes around 20-25 secs to show the result into the two fields. I want to show a progress bar in the mean time. Can anyone help me in this? thanks in advance..

[dunbarx]

Hi.

So the three-line thing was not required at all? I still am not sure. It does not seem so, since what you described as "filtering" was simply to separate the two values on each side of the dash in each line into two separate dataSets.

What part of your process takes all that time? The "split" command and the command to grab the URL will not lend themselves to the usual method of monitoring the progress of a loop. say, and scaling a progress bar to reflect the status through that loop.

Craig

[alemrantareq]

I also wonder split command can do the same that I was trying to do with repeat loop. But as I need to separate a large sized file, it takes time to complete splitting the two columns in the corresponding fields. Is that no way I can show a progress bar of the splitting process?

[dunbarx]

I made a field 1 with 16,000,000 lines, each line containing "aaa-bbb" I have two other fields, 2 and 3. I have a scrollbar with the endValue set to 1000. I have a button. In the button script:

Code: Select all

on mouseup
   put "" into fld 2
   put "" into fld 3
   put the ticks into tTime
   put fld 1 into tText
   set the itemDel to "-"
   put 1 into counter
   repeat for each line tLine in tText
      put item 1 of tLine & return after temp1
      put item 2 of tLine & return after temp2
      add 1 to counter
      if counter mod 16000 = 0 then set the thumbpos of scrollbar 1 to counter/2500
   end repeat
  
   put temp1 into fld 2
   put temp2 into fld 3
    answer the ticks - tTime
end mouseup

Now this is just to show you how it might work. The process took about 12 seconds. None of the scrollbar values are carefully set, so you may want to play with those.The split method that Jacque mentioned took about 7 seconds.

Craig Newman

[jacque]

I suspect the increased time required in the repeat loop is at least partly due to the progress bar itself. Updating a control is expensive, so there's a trade-off between letting the user know that something is happening (and adding to the execution speed) or not letting them know and allowing the handler to run faster.

If it is really taking many seconds to do the split, then I think I'd go ahead and use a progress bar along with Craig's "repeat" method. You can't track progress while the engine is working on the split command, there is no feedback provided. The progress bar will add some time to the script execution, so you'll need to see if it is worth it.

I recall a test I ran several years ago where I used a progress bar to show a lengthy process, and it took several seconds to run. When I commented out the progress bar, the handler ran in a few milliseconds. In that case the progress bar was taking more time than the script processing, but every case is different.

[dunbarx]

Jacque.

The repeat without the progress bar took just under six seconds. So that feature doubled the processing time.

alemrantareq:

Six seconds with no feedback or twelve with. As Jacque implied, this is your call. I would think that these two values are just about at the cusp of whether it is better to simply provide a watch cursor, or entertain the user with a progress bar.

Craig

[dunbarx]

Jacque.

Split? Forget that. Here is something for you.

Code: Select all

on mouseup
   --   put "" into fld 2  --YIPES
   --   put "" into fld 3
   put the ticks into tTime
   put fld 1 into tText --16,000,000 lines of "aaa-bbb"
   set the itemDel to "-"
   
   repeat for each line tLine in tText
      put item 1 of tLine & return after temp1
      put item 2 of tLine & return after temp2
   end repeat
   
   put temp1 into fld 2
   put temp2 into fld 3
   answer the ticks - tTime
end mouseup

The process runs 30% SLOWER if I do not empty the fields. I only did that for feedback. It actually takes a little time to empty them. I get about 315 ticks if I empty, and 400 if I do not

[alemrantareq]

thanks you both for replying me with your updates. You both are right; showing progress bar increases the filter time 6x. I made the following script which given me the result as expected but consumed too much time..

Code: Select all

on mouseUp
   answer file "Select the text file:"
   if it = "" then exit to top
   put url ("file:" & it) into tText
   filter tText without empty
   set the thumbpos of scrollbar "pbar" to 0
   set the endvalue of scrollbar "pbar" to 0
   set the endvalue of scrollbar "pbar" to the number of lines in tText
   put empty into tvar
   set the itemdel to "-"
   repeat for each line tLine in tText
      put item 1 of tLine & cr after fld "f1"
      put item 2 of tLine & cr after fld "f2"
      add 1 to tvar
      set the thumbpos of scrollbar "pbar" to tvar
   end repeat
end mouseUp

Now I'll try without progress bar. Hope I'll get the result much more efficiently

[jacque]

Putting text into a field is the slowest thing you can do in a script. Don't add each line to the field in the repeat loop, use a variable instead as Craig does. When the repeat loop is done, put the variable into the field all at once. If you only make one change to the script it should be that one.

Try it both ways to see the difference. You'll be surprised.