Filtering question

[Jellicle]

I have a list of words, one to a line. I want to filter that list to include only those that contain every letter in a string. So if my string = "aefg" I want to find only those words in my list that contain those characters (in any order).

How do I that? Is the filter command my best option?

Thanks for you help!

[SWEdeAndy]

Here's what I would do:

Code: Select all

function runFilter pString,pList
   filter lines of pList where myFilter(pString,each) ## 'Each' automatically refers to each line
   return pList
end runFilter

function myFilter pString,pLine
   repeat for each char tChar in pString
      if tChar is not in pLine then return false ## It found a letter not present in the line
   end repeat
   
   return true ## This only happens if all the letters were found in the line
end myFilter

Now, if you have e.g. your string of letters in field 1 and your word list in field 2, and want the filtered list to go in field 3, you would call:

Code: Select all

put runFilter(field 1,field 2) into field 3

[bn]

Hi Gerry,

I would use the filter command. I append a stack that does what I think you want.
I use the very handy (each) version of the filter command. I find it easier to maintain than a regualar expression which would probably also work but which I do not feel comfortable with.

filterWords.livecode.zip

(1.78 KiB) Downloaded 99 times

You put your characters that should occur in a line (in any order) into the indicated field and press the "Filter Words" button.
Should be easily adaptable to your needs.

Kind regards
Bernd

Edit: I just saw than Andreas also posted a similar solution.

[Jellicle]

Thanks!

I'm getting an error - I'm on the community version of LiveCode (9.0.2) so the form of the filter command you are using may not be available for me. The error is in this line:

Code: Select all

 filter lines of tData where checkLine(each) into tDest

The error messages says the compiler was expecting "with"...

[bn]

I'm getting an error - I'm on the community version of LiveCode (9.0.2) so the form of the filter command you are using may not be available for me. The error is in this line:

Code: Select all

 filter lines of tData where checkLine(each) into tDest

The error messages says the compiler was expecting "with"...

From the dictionary

The where form of the filter command was added in version 9.5.

Could you install at least LC 9.5?

Kind regards
Bernd

[bn]

Gerry,

you could try to replace the offending line (the one containing "where") with

Code: Select all

filter lines of tData matching "*[" & quote & sFilter & quote &"]" into tDest

in the sample stack above. In short testing it worked for me. Note that regex is case sensitive.

But the where form is a lot easier for Non-Regexers like me.

You should be able to find LC 9.5 community somewhere. I don't know where but there are some repositories somewhere.

Kind regards
Bernd

[bn]

Gerry,

On further testing my change to regex did not really work as expected.
Unless you know regex well I am afraid the stack is of little use without "where" of LC 9.5 and up.

Kind regards
Bernd

[SWEdeAndy]

Here's a slightly modified version of my first function, which should work in any version of LC:

Code: Select all

function runFilter pString,pList
   repeat for each line tLine in pList
      if myFilter(pString,tLine) then put tLine & cr after tFilteredList
   end repeat
   
   return tFilteredList
end runFilter

I guess it's slower than the filter function, but I don't know how many thousands of lines pList would have to be for the delay to be noticeable.

Are you building a Wordle solver?

[Jellicle]

Here's a slightly modified version of my first function, which should work in any version of LC:

Yes, that's roughly the method I came up with myself, but on a list with 20,000 or more lines...slow...

I can't find an installable version of 9.5 or higher

No, not a Wordle solver!

[rkriesel]

Hi, Jellicle. Here's a technique that runs over twenty times as fast as the nested-repeat technique (as coded by SWEdeAndy):

Code: Select all

function filterWordsWithChars pWords, pChars
   repeat for each char tChar in pChars
      filter pWords with "*" & tChar & "*"
   end repeat
   return pWords
end filterWordsWithChars

Here's a test to compare the techniques. If you execute it in the Script Editor, you can see the comparison in the Variables pane.

Code: Select all

command test
   local tWordCount=20000, tWordLength=8, tChars="aegh"
   local tWords, tTest, tHits, tCount, tMilliseconds, tRatio
   
   repeat tWordCount
      repeat tWordLength
         put any char of "abcdefghijklmnopqrstuvwxyz" after tWords
      end repeat
      put cr after tWords
   end repeat
   
   put "single repeat" into tTest
   put - the long milliseconds into tMilliseconds[tTest]
   put filterWordsWithChars( tWords, tChars ) into tHits[tTest]
   add the long milliseconds to tMilliseconds[tTest]
   put the number of lines in tHits[tTest] into tCount[tTest]
   
   put "nested repeat" into tTest
   put - the long milliseconds into tMilliseconds[tTest]
   put runFilter( tChars, tWords ) into tHits[tTest]
   add the long milliseconds to tMilliseconds[tTest]
   put the number of lines in tHits["nested repeat"] into tCount[tTest]
   
   put round( tMilliseconds["single repeat"]/tMilliseconds["nested repeat"], 2 ) into tRatio["single/nested"]
   put round( tMilliseconds["nested repeat"]/tMilliseconds["single repeat"], 2 ) into tRatio["nested/single"]
   
   breakpoint
end test

function runFilter pString,pList
   local tLine, tFilteredList
   repeat for each line tLine in pList
      if myFilter(pString,tLine) then put tLine & cr after tFilteredList
   end repeat
   
   return tFilteredList
end runFilter

function myFilter pString,pLine
   repeat for each char tChar in pString
      if tChar is not in pLine then return false ## It found a letter not present in the line
   end repeat
   
   return true ## This only happens if all the letters were found in the line
end myFilter

Is that fast enough for you?
-- Dick

[ClipArtGuy]

I can't find an installable version of 9.5 or higher

The community installers are still up here:

https://community.livecode.com

And once those are gone, you can find them on archive.org by searching "livecode,inc"

[bn]

Hi, Jellicle. Here's a technique that runs over twenty times as fast as the nested-repeat technique
...
Is that fast enough for you?
-- Dick

Hi Dick,

Thank you very much again for this beautiful piece of code. I did not realize you could run "filter" repeatedly on the same variable nor did I realize how much that increases execution speed. (in hindsight it rather drastically reduces the number of lines to filter after each consecutive filter operation and the number of iterations needed)

Kind regards
Bernd

[Jellicle]

The community installers are still up here:

https://community.livecode.com

And once those are gone, you can find them on archive.org by searching "livecode,inc"

Wow, thanks!

g

[rkriesel]

...
Thank you very much again for this beautiful piece of code.
...

You're very welcome, Bernd. Thank you for your sharing code and explanations on many topics. I especially like helping people I see as teachers.
-- Dick

[rkriesel]

...
Are you building a Wordle solver?

@all:

Do you think that a Wordle solver would be useful for teaching and learning and aggrandizing LC?

If you like the idea, what do you see as the requirements?

-- Dick