Page 1 of 2
Filtering question
Posted: Tue Jan 18, 2022 5:06 am
by Jellicle
I have a list of words, one to a line. I want to filter that list to include only those that contain every letter in a string. So if my string = "aefg" I want to find only those words in my list that contain those characters (in any order).
How do I that? Is the filter command my best option?
Thanks for you help!
Re: Filtering question
Posted: Tue Jan 18, 2022 8:45 am
by SWEdeAndy
Here's what I would do:
Code: Select all
function runFilter pString,pList
filter lines of pList where myFilter(pString,each) ## 'Each' automatically refers to each line
return pList
end runFilter
function myFilter pString,pLine
repeat for each char tChar in pString
if tChar is not in pLine then return false ## It found a letter not present in the line
end repeat
return true ## This only happens if all the letters were found in the line
end myFilter
Now, if you have e.g. your string of letters in field 1 and your word list in field 2, and want the filtered list to go in field 3, you would call:
Code: Select all
put runFilter(field 1,field 2) into field 3
Re: Filtering question
Posted: Tue Jan 18, 2022 9:45 am
by bn
Hi Gerry,
I would use the filter command. I append a stack that does what I think you want.
I use the very handy (each) version of the filter command. I find it easier to maintain than a regualar expression which would probably also work but which I do not feel comfortable with.
You put your characters that should occur in a line (in any order) into the indicated field and press the "Filter Words" button.
Should be easily adaptable to your needs.
Kind regards
Bernd
Edit: I just saw than Andreas also posted a similar solution.
Re: Filtering question
Posted: Tue Jan 18, 2022 11:50 am
by Jellicle
Thanks!
I'm getting an error - I'm on the community version of LiveCode (9.0.2) so the form of the filter command you are using may not be available for me. The error is in this line:
Code: Select all
filter lines of tData where checkLine(each) into tDest
The error messages says the compiler was expecting "with"...
Re: Filtering question
Posted: Tue Jan 18, 2022 11:55 am
by bn
Jellicle wrote: ↑Tue Jan 18, 2022 11:50 am
I'm getting an error - I'm on the community version of LiveCode (9.0.2) so the form of the filter command you are using may not be available for me. The error is in this line:
Code: Select all
filter lines of tData where checkLine(each) into tDest
The error messages says the compiler was expecting "with"...
From the dictionary
The where form of the filter command was added in version 9.5.
Could you install at least LC 9.5?
Kind regards
Bernd
Re: Filtering question
Posted: Tue Jan 18, 2022 12:15 pm
by bn
Gerry,
you could try to replace the offending line (the one containing "where") with
Code: Select all
filter lines of tData matching "*[" & quote & sFilter & quote &"]" into tDest
in the sample stack above. In short testing it worked for me. Note that regex is case sensitive.
But the where form is a lot easier for Non-Regexers like me.
You should be able to find LC 9.5 community somewhere. I don't know where but there are some repositories somewhere.
Kind regards
Bernd
Re: Filtering question
Posted: Tue Jan 18, 2022 1:40 pm
by bn
Gerry,
On further testing my change to regex did not really work as expected.
Unless you know regex well I am afraid the stack is of little use without "where" of LC 9.5 and up.
Kind regards
Bernd
Re: Filtering question
Posted: Tue Jan 18, 2022 2:14 pm
by SWEdeAndy
Here's a slightly modified version of my first function, which should work in any version of LC:
Code: Select all
function runFilter pString,pList
repeat for each line tLine in pList
if myFilter(pString,tLine) then put tLine & cr after tFilteredList
end repeat
return tFilteredList
end runFilter
I guess it's slower than the filter function, but I don't know how many thousands of lines pList would have to be for the delay to be noticeable.
Are you building a Wordle solver?
Re: Filtering question
Posted: Wed Jan 19, 2022 12:47 am
by Jellicle
SWEdeAndy wrote: ↑Tue Jan 18, 2022 2:14 pm
Here's a slightly modified version of my first function, which should work in any version of LC:
Yes, that's roughly the method I came up with myself, but on a list with 20,000 or more lines...slow...
I can't find an installable version of 9.5 or higher
No, not a Wordle solver!
Re: Filtering question
Posted: Wed Jan 19, 2022 8:07 am
by rkriesel
Hi, Jellicle. Here's a technique that runs over twenty times as fast as the nested-repeat technique (as coded by SWEdeAndy):
Code: Select all
function filterWordsWithChars pWords, pChars
repeat for each char tChar in pChars
filter pWords with "*" & tChar & "*"
end repeat
return pWords
end filterWordsWithChars
Here's a test to compare the techniques. If you execute it in the Script Editor, you can see the comparison in the Variables pane.
Code: Select all
command test
local tWordCount=20000, tWordLength=8, tChars="aegh"
local tWords, tTest, tHits, tCount, tMilliseconds, tRatio
repeat tWordCount
repeat tWordLength
put any char of "abcdefghijklmnopqrstuvwxyz" after tWords
end repeat
put cr after tWords
end repeat
put "single repeat" into tTest
put - the long milliseconds into tMilliseconds[tTest]
put filterWordsWithChars( tWords, tChars ) into tHits[tTest]
add the long milliseconds to tMilliseconds[tTest]
put the number of lines in tHits[tTest] into tCount[tTest]
put "nested repeat" into tTest
put - the long milliseconds into tMilliseconds[tTest]
put runFilter( tChars, tWords ) into tHits[tTest]
add the long milliseconds to tMilliseconds[tTest]
put the number of lines in tHits["nested repeat"] into tCount[tTest]
put round( tMilliseconds["single repeat"]/tMilliseconds["nested repeat"], 2 ) into tRatio["single/nested"]
put round( tMilliseconds["nested repeat"]/tMilliseconds["single repeat"], 2 ) into tRatio["nested/single"]
breakpoint
end test
function runFilter pString,pList
local tLine, tFilteredList
repeat for each line tLine in pList
if myFilter(pString,tLine) then put tLine & cr after tFilteredList
end repeat
return tFilteredList
end runFilter
function myFilter pString,pLine
repeat for each char tChar in pString
if tChar is not in pLine then return false ## It found a letter not present in the line
end repeat
return true ## This only happens if all the letters were found in the line
end myFilter
Is that fast enough for you?
-- Dick
Re: Filtering question
Posted: Wed Jan 19, 2022 7:18 pm
by ClipArtGuy
Jellicle wrote: ↑Wed Jan 19, 2022 12:47 am
I can't find an installable version of 9.5 or higher
The community installers are still up here:
https://community.livecode.com
And once those are gone, you can find them on archive.org by searching "livecode,inc"
Re: Filtering question
Posted: Wed Jan 19, 2022 8:52 pm
by bn
rkriesel wrote: ↑Wed Jan 19, 2022 8:07 am
Hi, Jellicle. Here's a technique that runs over twenty times as fast as the nested-repeat technique
...
Is that fast enough for you?
-- Dick
Hi Dick,
Thank you very much again for this beautiful piece of code. I did not realize you could run "filter" repeatedly on the same variable nor did I realize how much that increases execution speed. (in hindsight it rather drastically reduces the number of lines to filter after each consecutive filter operation and the number of iterations needed)
Kind regards
Bernd
Re: Filtering question
Posted: Wed Jan 19, 2022 9:35 pm
by Jellicle
Re: Filtering question
Posted: Wed Jan 19, 2022 11:24 pm
by rkriesel
bn wrote: ↑Wed Jan 19, 2022 8:52 pm
...
Thank you very much again for this beautiful piece of code.
...
You're very welcome, Bernd. Thank you for your sharing code and explanations on many topics. I especially like helping people I see as teachers.
-- Dick
Re: Filtering question
Posted: Wed Jan 19, 2022 11:38 pm
by rkriesel
SWEdeAndy wrote: ↑Tue Jan 18, 2022 2:14 pm
...
Are you building a Wordle solver?
@all:
Do you think that a Wordle solver would be useful for teaching and learning and aggrandizing LC?
If you like the idea, what do you see as the requirements?
-- Dick