LiveCode Forums

Posted: **Thu Aug 20, 2009 8:32 pm**

I have csv files with more than 6K lines of data and when I use chartsengine to create a chart nothing appears. I do not have any problems on smaller files, just the larger ones (above 3K lines). What is the upper limit of chartsengine to process data? Is there a way to overcome this limit?

I'm running RunRev Media 3.5 on a MBP 2.53 GHz, 4 Mb RAM, ChartsEngine 1.04

Thanks!

Posted: **Thu Aug 20, 2009 9:11 pm**

Dear t-rex22,

As far as I know, ChartsEngine's limits depend on the amount of memory available. I know for sure that 6000 lines of data is peanuts for ChartsEngine. Obviously, you're doing something wrong. You might want to give a precise description of what you're doing and post your scripts here. Maybe someone can come up with a solution to help you.

Best regards,

Mark

Posted: **Fri Aug 21, 2009 3:26 pm**

Thanks Mark, I'm sure you're right that I'm doing something wrong. I keep trying!

Posted: **Wed Aug 26, 2009 10:12 am**

Dear t-rex,

which chart type do you want to draw?

All the best,

Malte

Posted: **Wed Aug 26, 2009 10:36 am**

[quote="malte"]Dear t-rex,

which chart type do you want to draw?

All the best,

Malte[/quote]

Hi Malte,

I'm attempting to draw line charts for large (varying) sets of CSV data. I believe that, as Mark suggested, I'm running out of memory. Smaller (3K lines) data sets plot without problem.

Is there a way to create a plot without storing the data set in memory?

Thanks for your response!

Posted: **Wed Aug 26, 2009 2:11 pm**

For lines type it should work. could it be that the data is corrupt in the 6k lines? any blanks in there the parser could choke on? I would like to see the data (and the stack if possible) To see how you create the chart.

All the best,

Malte

Posted: **Wed Aug 26, 2009 9:18 pm**

Malte looked at my data file and found that there was a problem. He added the following code to my script:

-- create chart
replace "time, Probe Accel" with empty in tChartdata
replace "milliseconds, m/s^2" with empty in tChartData
set the charts["data"] of tChart to tChartData

My graphing script now works perfectly even with files that are 16K lines.

Thanks Malte!!

Posted: **Wed Aug 26, 2009 9:22 pm**

Glad to hear it works now. In case you see your charts not update again, check the contents of the result right after the chartsRefresh command. If that is not empty something went wrong. Most of the time it is data that can not be parsed by chartsEngine.

All the best,

Malte

Posted: **Wed Aug 26, 2009 9:27 pm**

One (more) question I have is how I might use this same script with files that have different headers?

In other words, can I write code to check for this kind of problem?

Thanks!

Posted: **Wed Aug 26, 2009 9:32 pm**

Your savest bet is to check the data, if any item is a number. Set the itemDel to the same itemdelimiter as the one you use for the chart (comma by default)

Code: Select all

repeat for each item theItem in tChartData
  if theItem is not a number then
    -- there is a problem
  end if
end repeat

The fastest routine to strip out any non numeric items would take me some thinking. However, if one item on a line is not a number then the whole line should be deleted.

All the best,

Malte

Posted: **Wed Aug 26, 2009 9:38 pm**

Okay, thank you very much for your help!!

Posted: **Wed Aug 26, 2009 10:17 pm**

As a fix to my non-numeric character problem in the first two lines of my data I inserted the following:

put empty into line 1 of tChartData
put empty into line 2 of tChartData

immediately before I,

set the charts["data"] of tChart to tChartData

Posted: **Wed Aug 26, 2009 10:21 pm**

There were a couple more header lines in the data. Where do you get the data from?

Cheers,

Malte

Posted: **Wed Aug 26, 2009 10:24 pm**

This data is exported from a proprietary data collection software. The tests are dynamic impacts collected through a data acquisition system.

LiveCode Forums

data limits

data limits