Large number of images in a data grid

Got a LiveCode personal license? Are you a beginner, hobbyist or educator that's new to LiveCode? This forum is the place to go for help getting started. Welcome!

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller

hopkins
Posts: 41
Joined: Wed Nov 28, 2018 11:09 am

Large number of images in a data grid

Post by hopkins » Sun Dec 23, 2018 7:40 pm

I am writing an application to display my albums in my music collection. I have set up a data grid with a form to display thumbnail (150x150) images of the album covers. The images are stored in an SQLite database.

For the time being, I just populate the entire grid with search results, however, depending on the screen size I only have 14 to 28 images displayed on the screen in one page.

I have over 1500 albums. When I display everything, it is, à little time consuming.

What strategy could I adopt to optimize the display? Is it possible, simply, to fill up the array according to what is being displayed? Are there any other ways of doing this?

Any suggestions would be greatly appreciated.

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9580
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: Large number of images in a data grid

Post by dunbarx » Sun Dec 23, 2018 8:08 pm

Hi.

Display everything? Do you mean that you load everything, and do a lot of scrolling? Is it the scrolling itself that is "time consuming", or the loading?

Nothing special about a dataGrid here. Any VERY long list that has to display its contents in a normally sized control ought to have some means of paging those contents. The method of doing that is up to you; it is usually a fun project. Binary searches are my favorite, but there are other live interface gadgets you can create as well. Do you need help with that?

Of course, you could buy a very, very tall monitor, and an extension ladder.

Craig Newman

hopkins
Posts: 41
Joined: Wed Nov 28, 2018 11:09 am

Re: Large number of images in a data grid

Post by hopkins » Sun Dec 23, 2018 8:40 pm

The loading takes some time, not the scrolling.

What do you mean by "paging methods"? Yes, if you could offer some help with specific methods that would be great.

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9580
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: Large number of images in a data grid

Post by dunbarx » Sun Dec 23, 2018 11:48 pm

Here is an experiment for you. Make, on a new card, a field of about 20 lines, a button and a scrollbar. Set the width and height of the scrollbar to make it vertical, say 16 wide and the height of the field. Put the scrollbar to the right of the field. Set the textHeight of the field explicitly to 12.

In the script of the button:

Code: Select all

on mouseUp
   repeat with y = 1 to 1500
   put "Album" && y into line y of temp
end repeat
set the rawData of fld 1 to temp
end mouseup
In the script of the scrollBar:

Code: Select all

local linesPerPage,numPages

on scrollBarDrag
   put round((the thumbPos of me + the thumbSize of me) / the endValue of me,2) into scrollRatio
put trunc(scrollRatio * numPages) into currentScrollPage
   put line currentScrollPage * linesPerPage to (currentScrollPage + 1) * linesPerPage of the rawData of fld 1 into fld 1
end scrollBarDrag

on mouseEnter   
   put round((the height of  fld 1 - 32) / (the textheight of  fld 1)) into linesPerPage
   put  round(the number of lines of the the rawData of fld 1 / linesPerPage)  into numPages
end mouseEnter
A little wordy, but that is for clarity. The numbers in the first line of the "mouseEnter" handler are kluges to account for margins and other unknown field properties.

The point is that as you move the thumb in the scrollbar, the field is loaded with full pages taken from the custom property "rawData", one at a time, and should be very fast.

With 1500 lines, there is an inherent coarseness of fine tuning in that it is hard to stop at a particular page with a reasonably sized field. Perhaps another bit of gadgetry, say holding the optionKey down, might take the current page number and allow you to scroll more finely.

You now have to modify this to use in your DG.

But does each page load quickly? If it does, you will have to configure some of these numbers to make it fit your needs.

Another approach, assuming your list of albums is sorted in one or more ways, or has properties assigned to them so that they can be grouped, is to enter one or more search terms, that will load only those albums that meet that criteria.

If you really want to pursue this, write back.

Craig Newman

hopkins
Posts: 41
Joined: Wed Nov 28, 2018 11:09 am

Re: Large number of images in a data grid

Post by hopkins » Mon Dec 24, 2018 12:07 am

Great, thanks I will test this out.

I do have search terms , but cannot rule out the possibility that either I (or anyone else using the app - as I want to make it available to others) will just display all albums. Some people have quite large collections...

I could also limit the number of albums displayed, but I rather have an efficient way of dealing with larger volumes. Some people have quite extensive album collections (> 10.000).

quailcreek
Posts: 746
Joined: Sun Feb 04, 2007 11:01 pm
Location: McKenna, WA

Re: Large number of images in a data grid

Post by quailcreek » Mon Dec 24, 2018 12:10 am

What OS are you running?
What version of LC?
Is this for a desktop app or mobile?
Do you have cache controls of the DG on or off?
Tom
MacBook Pro OS Mojave 10.14

hopkins
Posts: 41
Joined: Wed Nov 28, 2018 11:09 am

Re: Large number of images in a data grid

Post by hopkins » Mon Dec 24, 2018 12:18 am

Hi,

This is a desktop app so far (windows or OSX). I don't have cache control set on the DG. I am using LC 9.0.1. I may do a web version at some point, but not immediately, and a mobile version perhaps at a later date. I am working on a second app to manage music metadata, which will be using LC Server, but that is a seperate project, and will integrate the two together.

Here is what it looks like so far (far from finalized) just to give you an idea. The albums are selectable and can be dragged to the icons or various tabs.
Capture.JPG

sphere
Posts: 1145
Joined: Sat Sep 27, 2014 10:32 am
Location: Earth, Except when i Jump

Re: Large number of images in a data grid

Post by sphere » Mon Dec 24, 2018 9:07 am

Very nice!
I like the clear layout.

hopkins
Posts: 41
Joined: Wed Nov 28, 2018 11:09 am

Re: Large number of images in a data grid

Post by hopkins » Mon Dec 24, 2018 12:32 pm

I played around with several options:

- feeding the data grid as you scroll (as suggested). That seems promising, but I need to find a way to make it work "smoothly". I do not want to have the entire screen change when scrolling (as if switching from one page to another). I will experiment further

- setting the "cache control" flag makes the scrolling a little smoother, but there is a long delay between the time the albums are displayed on the screen and the time the page becomes responsive, which is in my opinion not acceptable.

- Instead of putting the images in the SQLite database, i tested creating them directly in the stack, and copying them into the grid from there. I also added a "prepare image" for all the album images on startup, but am not sure yet that makes much of a difference, will need to experiment more.

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9580
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: Large number of images in a data grid

Post by dunbarx » Mon Dec 24, 2018 6:36 pm

Just a thought.

Is it possible to display the thumbnails as the imageSource of characters somewhere in the DG? These might scroll along with everything else without issue.

Craig

hopkins
Posts: 41
Joined: Wed Nov 28, 2018 11:09 am

Re: Large number of images in a data grid

Post by hopkins » Mon Dec 24, 2018 7:52 pm

I just tried it. It seems to work, but scrolling is noticeably longer. Another problem is I cannot seem to reference an image of another card as a an "imagesource", which means the images would have to be duplicated if I were to use them in different cards.

Perhaps the optimum way would be to cache the data grid but to limit the initial display to X lines, then when scrolling to the bottom load another X lines ( somewhat like the "more results" when you display images on Goolge). When the "cached controls" options of the data grid is set scrolling is perfectly "smooth" but it introduces a delay if the number of images is too high - will try to experiment with this. Perhaps there is a way of filling the data grid by adding lines as a background task ? Or perhpas just a spinning cursor will suffice to indicate that the system is "working" at refreshing the data.

When setting "cached controls", what happens if the system memory is not sufficient ? Can this setting be detrimental to the overall performance of the app ?

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9580
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: Large number of images in a data grid

Post by dunbarx » Mon Dec 24, 2018 9:26 pm

what happens if the system memory is not sufficient ?
It has been a long time since this was an issue, since everybody has gigaBytes of RAM these days. Have you seen this, or are you just concerned?

Craig

hopkins
Posts: 41
Joined: Wed Nov 28, 2018 11:09 am

Re: Large number of images in a data grid

Post by hopkins » Mon Dec 24, 2018 9:37 pm

No I have not experienced any issues myself. I have 1500 albums in my collection, so 1500 images loading, no issues, it just takes a little time for the app to be responsive. As I metioned, some people have 10 times more... so just wondering.

Unless I find a mechanism of updating the datagrid in the background (am new to this) I will for the moment set a limit on the number of images displayed at one time, and set up a mechanism to jump to the next batch of images.

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9580
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: Large number of images in a data grid

Post by dunbarx » Mon Dec 24, 2018 10:48 pm

So I just placed 1500 lines of text into a scrolling field. I set the imageSource of the first char of each line to a 60x60 pixel image. The field scrolls normally.

Not a DG, to be sure, but just to test. I am interested in finding out why a DG scrolls more slowly. Perhaps this is just a matter of its internal overhead.

Craig

hopkins
Posts: 41
Joined: Wed Nov 28, 2018 11:09 am

Re: Large number of images in a data grid

Post by hopkins » Mon Dec 24, 2018 10:52 pm

Thanks. Could you try with a data grid ? Perhaps I am doing something wrong... Would be curious to find out. If you do try, could you try with and without "cache controls" ? P.S. My images are 150 by 150, and am working on a laptop with 8GB of memory, but it is not a racehorse (AMD 1.8GHz)... One other thing, in my DG I have 7 images per row.

In the script of the grid, I fill in the images + album titles and artists, and a key in a custom property, in 2 fields. I also manage selection of the albums changing the color of the title-artist text. Here is my script (the idea is to have a variable number of images per row - at a later stage):

repeat with tCount = 1 to 7
put "Key" & tCount into tKey
if not (pDataArray[tKey] is empty) then

put "Thumb" & tCount into tField
put "Album " & pDataArray[tKey] into tImageName
put image tImageName of card id 238133 into image tField of me

put "Thumb" & tCount into tField
set the AlbumKey of image tField of me to pDataArray[tKey]

put "TITLE" & tCount into tKey
put "Title" & tCount into tField
put pDataArray[tKey] & return into field tField of me

put "ALBUMARTIST" & tCount into tKey
put "Title" & tCount into tField
put pDataArray[tKey] after field tField of me

set the textstyle of line 1 of field tField of me to bold
set the textstyle of line 2 of field tField of me to normal

-- change background text color if the album is selected
put "Select" & tCount into tKey
put "Title" & tCount into tField
if pDataArray[tKey] = true then
set the opaque of field tField of me to true
else
set the opaque of field tField of me to false
end if

end if
end repeat

Post Reply

Return to “Getting Started with LiveCode - Complete Beginners”