Writing Multiple Encoded Arrays to a File

Moderators: LCNeil, heatherlaine, kevinmiller, elanorb

Post Reply
dhurtt
Posts: 42
Joined: Sat Nov 09, 2013 7:37 pm
Location: Huachuca City, AZ

Writing Multiple Encoded Arrays to a File

Post by dhurtt » Mon Jan 13, 2014 5:13 pm

My application has several arrays of data and I would like to write it out to a file and read it back in when I open the application back up. I understand the basics of encoding arrays with arrayEncode(), reading and writing binary files, and putting up dialogs to prompt the user to pick a file or provide a file name. The question is more about the file's structure for reading and writing. I want to read and write more than one array to a single file, then read the data back into their respective arrays.

Conceptually the idea is something like (not real Livecode):

Code: Select all

put arrayEncode(array1) into binarray1
write binarray1 to file outputFile
put arrayEncode(array2) into binarray2
write binarray2 to file outputFile at ... [end? some byte position?]
It seems like there needs to be a separator between the first encoded array and the second, or else I need to know where each array begins and ends in the file. I see three possible approaches:

1. Get the size in bytes of each array and structure the file as something like:

[integer: number of arrays in file][long integer: length of first encoded array][long integer: length of second encoded array]...[binary blob: start of first encoded array][binary blob: start of second encoded array]

So the writing handler would need to encode each array, calculate the byte length of each encoded array, and then write to the file: the number of arrays, the byte size of each array, then each array. The reading handler would then need to read in the same order, extracting out each blob and decoding it.

Q: If I use this method how do I:

A. Ensure that I write a number out as an integer?
B. Determine the exact size of an encoded array?

2. Write each encoded array followed by a marker, which in turn can be found when reading. This seems problematic as any character you choose as a marker could theoretically be in the encoding of the array.

Q: Is there any reliable marker to use to make this method work?

3. Put all of the arrays into a single array before encoding.

Code: Select all

put array1 into megaArray[1]
put array2 into megaArray[2]
...
put encodeArray(megaArray) into megaBinArray
I have tested this method and it works. The issue I (think I) see with this is when the arrays get big enough copying the data from each array to the mega-array takes time and memory. If it does not do a copy, however, this might be the cleanest way.

Q: Do the Livecode statements above copy data from array1 and array2 to megaArray, or are they simply pointers?

(I suppose I could test by updating array1 and seeing if the data in megaArray is also updated, but that would lead to an implied answer. I was hoping for a definitive one.)

So, my question for the Wed Q&A is to see an example of something like this or have a discussion about the concept. I figure that there is some function, command, or keyword I am unaware of that makes this tedious exercise in other languages trivial in Livecode. :)

Regards,

Dale

LCNeil
Livecode Staff Member
Livecode Staff Member
Posts: 1223
Joined: Wed Oct 03, 2012 4:07 pm

Re: Writing Multiple Encoded Arrays to a File

Post by LCNeil » Tue Jan 14, 2014 5:52 pm

Dear Dale,

There have been some fantastic answers to this question on your summer academy post here with many of these suggestions being things we would also suggest-

http://forums.runrev.com/phpBB2/viewtop ... 70&t=18733

Jacque gives a great response to your first query -
The engine is forgiving about whether a character is an integer or text and it's very rare that you need to worry about it. In this case, if you are writing it to a text file, then it's going to be text anyway. A script can add 10 to an integer or a text representation without any trouble, it will know what you mean. But if you really, really want to force an integer just add 0 to the number.

The size of an encoded array will be its length: put len(tEncodedArray)
In regard to your second query, I am not familiar with a character that could be used as marker because, as you have mentioned, any character could be used in the encoding of the array

Method 3 seems the most viable.

The statement-
put array1 into megaArray[1]
put array2 into megaArray[2]
...
put encodeArray(megaArray) into megaBinArray
will copy data from both arrays to the megaArray.

I would recommend investigating other potential methods of storing your arrays. The suggestions of writing to a LiveCode stack file or SQLite database are ones that we would also recommend. Some great tutorials on how to use/implement some of there features can be found below-

http://lessons.runrev.com/s/lessons/m/4 ... pplication

http://lessons.runrev.com/s/lessons/m/4 ... e-database

http://lessons.runrev.com/s/lessons/m/4 ... e-it-again

I hope the above gives you some leads.

Kind Regards,

Neil Roger
--
RunRev Support Team ~ http://www.runrev.com

Post Reply

Return to “idea2app and Coding School”