Page 2 of 6
Re: Sort multidimensional array
Posted: Mon Mar 01, 2021 12:42 pm
by stam
Thierry wrote: Mon Mar 01, 2021 12:06 pm
Mmm, actually I warned you about this...
Yep, you did indeed. As always i'm ever so grateful for all your advice and contributions which i've found absolutely invaluable during my short time on this forum (especially with regex!) - didn't mean anything by my comment, just that that was needed for multidimensional array sorting.
On some cursory testing, i now see why you left it out

- you can't use this for 1-dimensional arrays as Merge() doesn't evaluate anything in brackets it seems...
Based on this i've revised the code to cater for both 1- and multi-dimensional arrays. Includes a conditional (without pKey as per your code, but also with pKey when needed)... can't get away from at least 1
if statement i guess...
Code: Select all
function sortArray @pArray, pDirection, pSortType, pKey
local tNextIndex, tSortedArray, tSortText
if (pDirection is not in "ascending,descending" and pDirection is not empty) or \
( pSortType is not in "international,numeric,datetime,text,binary" and pSortType is not empty) then
answer "Malformed parameters - sortDirection must be either Ascending, Descending or empty" & return & \
"sortType must be either 'international', 'numeric', 'datetime', 'text', 'binary' or empty"
return pArray
end if
get the keys of pArray
if pKey is empty then
put merge("sort lines of IT [[pDirection]] [[pSortType]] by pArray[each]") into tSortText //1-dimensional array
else
put merge("sort lines of IT [[pDirection]] [[pSortType]] by pArray[each][pKey]") into tSortText //multidimensional array
end if
do tSortText
repeat for each line tKey in IT
add 1 to tNextIndex
put pArray[tKey] into tSortedArray[tNextIndex]
end repeat
return tSortedArray
end sortArray
Usage: pArray is mandatory, all other parameters can be passed as empty or ""
- Empty pSortDIrection will default to ascending
- Empty pSortType will default to text
- Empty pKey will treat array as 1-dimensional
Thanks once again
@Thierry for helping improve my coding

Re: Sort multidimensional array
Posted: Mon Mar 01, 2021 12:54 pm
by Thierry
stam wrote: Mon Mar 01, 2021 12:42 pm
Yep, you did indeed. As always i'm ever so grateful for all your advice and contributions which i've found absolutely invaluable during my short time on this forum (especially with regex!) - didn't mean anything by my comment, just that that was needed for multidimensional array sorting.
You're welcome
On some cursory testing, i now see why you left it out

- you can't use this for 1-dimensional arrays as Merge() doesn't evaluate anything in brackets it seems...
Based on this i've revised the code to cater for both 1- and multi-dimensional arrays. Includes a conditional (without pKey as per your code, but also with pKey when needed)... can't get away from at least 1
if statement i guess...
Code: Select all
function sortArray @pArray, pDirection, pSortType, pKey
....
if pKey is empty then
put merge("sort lines of IT [[pDirection]] [[pSortType]] by pArray[each]") into tSortText //1-dimensional array
else
put merge("sort lines of IT [[pDirection]] [[pSortType]] by pArray[each][pKey]") into tSortText //multidimensional array
end if
.....
OR as it seems we have kind of same concerns with simple and short code:
Code: Select all
function sortArray @pArray, pDirection, pSortType, pKey
....
local sortCmd = "sort lines of IT [[pDirection]] [[pSortType]] by pArray[each]"
if pKey is not empty then put "[pkey]" after sortCmd
put merge( sortCmd) into tSortText
.....
Warning: you have to check/test everything I'm saying
Regards,
Thierry
Re: Sort multidimensional array
Posted: Mon Mar 01, 2021 1:10 pm
by stam
Excellent - 3 lines less
Final code (so far!) tested with one 1-dimensional and one multidimensional array:
Code: Select all
function sortArray @pArray, pDirection, pSortType, pKey
local tNextIndex, tSortedArray, tSortText
if (pDirection is not in "ascending,descending" and pDirection is not empty) or \
( pSortType is not in "international,numeric,datetime,text,binary" and pSortType is not empty) then
answer "Malformed parameters - sortDirection must be either Ascending, Descending or empty." & return & \
"sortType must be either 'international', 'numeric', 'datetime', 'text', 'binary' or empty."
return pArray
end if
get the keys of pArray
put "sort lines of IT [[pDirection]] [[pSortType]] by pArray[each]" into tSortText //1-dimensional array
if pKey is not empty then put "[pKey]" after tSortText //mutidimensional array
do merge(tSortText)
repeat for each line tKey in IT
add 1 to tNextIndex
put pArray[tKey] into tSortedArray[tNextIndex]
end repeat
return tSortedArray
end sortArray
Again - feedback welcome

Re: Sort multidimensional array
Posted: Mon Mar 01, 2021 3:59 pm
by Simon Knight
Hi all,
Crikey how this thread has grown! All good stuff. In my recent project I have been reading file details into an array. The files are image files and the folder structure contains something like 180,000 files. Two lessons I have learnt to speed processing are the use of the pass by reference @ qualifier as included in the code examples above and the use of the lock screen command.
Without locking the screen my handlers processed the file names in just under two minutes which I thought was o.k. then I had a "what would Bernd do" moment? So I added the lock screen commands and the file collection is now processed in about 0.36 seconds.
Re: Sort multidimensional array
Posted: Mon Mar 01, 2021 6:01 pm
by stam
I often enjoy these randomly/organically growing discussions

Good to know about lock screen - i've used it but haven't seen a massive increase in speed but maybe i've not been doing stuff that was taxing enough to make it obvious (and haven't bothered time-logging)
I guess it will (almost?) never hurt to use lock screen though

Re: Sort multidimensional array
Posted: Mon Mar 01, 2021 6:04 pm
by FourthWorld
Where appropriate, the combination of lockScreen with lockMessages can be quite dramatic.
Re: Sort multidimensional array
Posted: Mon Mar 01, 2021 6:22 pm
by Thierry
stam wrote: Mon Mar 01, 2021 6:01 pm
Good to know about lock screen
i've used it but haven't seen a massive increase in speed but maybe i've not been doing stuff that was taxing enough to make it obvious (and haven't bothered time-logging)
Well, I don't believe this will change the speed at all in this use case,
plus I think using or not the @ prefix in your parameter won't change much too...
I guess it will (almost?) never hurt to use lock screen though
It could
Regards,
Thierry
Re: Sort multidimensional array
Posted: Tue Mar 02, 2021 12:54 am
by Simon Knight
plus I think using or not the @ prefix in your parameter won't change much too...
it depends on the size of the data being passed: somewhere on this forum I posted about a problem where the routine was taking hours and hours to complete, adding the prefix stopped the duplication of data and reduced the processing to tens of seconds. I'll dig it out later but its getting late.
best wishes
Simon
Re: Sort multidimensional array
Posted: Tue Mar 02, 2021 2:11 am
by FourthWorld
How many records are in the returned data?
Re: Sort multidimensional array
Posted: Tue Mar 02, 2021 4:25 am
by bobcole
This may not be the topic to which Simon Knight referred but it is recent and, I think, relevant to the @ question:
viewtopic.php?f=7&t=35369
FYI,
Bob
Re: Sort multidimensional array
Posted: Tue Mar 02, 2021 4:31 am
by bobcole
Also, I seem to recall that there was an issue with using the "do" command.
If I remember correctly, it has somehow related to Apple not liking programs that have potentially self-altering code in their App Store.
Anyone familiar with that issue?
Sorry, it is only a vague recollection.
Bob
Re: Sort multidimensional array
Posted: Tue Mar 02, 2021 8:45 am
by Simon Knight
Hi
Here is a link to my post from 2011
viewtopic.php?f=32&t=7069&p=32207&hilit ... nce#p32207
In it I estimate that an operation would take 56 hours, this was reduced to 70 seconds mostly by switching to referenced variable passing. Mind you the variable was quite large at 70Mbytes.
Thanks to Bernd the estimated 56 hours for processing a datafile is down to a real 70 seconds. The main speed problem was caused by passing a copy of the complete data variable to a function that was being called over a million times. Livecode just choked. The solution was to add the @ character before the parameter name in the function declaration, this makes livecode pass a reference to the data rather than the default of sending a copy. the data is stored in theBinarydata e.g.
Re: Sort multidimensional array
Posted: Tue Mar 02, 2021 9:22 am
by stam
bobcole wrote: Tue Mar 02, 2021 4:31 am ...there was an issue with using the "do" command....
...somehow related to Apple not liking programs that have potentially self-altering code in their App Store....
I must say, that seems unlikely.... I’d be surprised if Apple did a code-level review of apps (but really have
no idea).
Plus there’s a ton of iOS apps (mainly games) that introduce substantial changes by altering themselves (downloading stuff after install into the app) or by acting as a “player” for essentially “unauthorised” apps
In this instance the do command is just replacing a ton of conditional statements. And not sure Do does anything that can’t be replicated with with other code structures?
Can anyone shed more light on this?
Re: Sort multidimensional array
Posted: Tue Mar 02, 2021 9:47 am
by stam
Yes but they have optimised as of v8 to always use a pass by reference when possible - Bob links to an interesting discussion above, in which Richard explains:
FourthWorld wrote: Sat Feb 06, 2021 7:35 pm
In fact, it's so useful to pass by reference that the engine now does that automatically by default (as of v8 or so).
It uses "copy on write" by default, so even if you don't explicitly declare a param as reference ("@"), it won't actually make a copy into you perform some action which would alter it. If you want that alteration to be reflected in the calling handler, use @ before the arg name in the definition handler.
In your case, the only operation is read, so you should see no difference in performance whether you use @ or not, since only a pointer to the original data is what's being used by default.
@Richard: I presume this means of moving around large data and writing to it would still benefit from passing by reference?
Re: Sort multidimensional array
Posted: Tue Mar 02, 2021 9:52 am
by Simon Knight
I didn't know about the changes in V8 so thanks for the link and quotes.
Simon