Writing Multiple Encoded Arrays to a File
Posted: Sat Jan 11, 2014 11:43 pm
I realize this is not exactly a "Complete Beginners" question, but there is no "Intermediate" board (why is that?), so...
My application has several arrays of data and I would like to write it out to a file and read it back in when I open the application back up. I understand the basics of encoding arrays with arrayEncode(), reading and writing binary files, and putting up dialogs to prompt the user to pick a file or provide a file name. The problem is more about the file's structure for reading and writing. I want to read and write more than one array to a single file, then read the data back into their respective arrays.
Conceptually the idea is something like (not real Livecode):
It seems like there needs to be a separator between the first encoded array and the second, or else I need to know where each array begins and ends in the file. I see three possible approaches:
1. Get the size in bytes of each array and structure the file as something like:
[integer: number of arrays in file][long integer: length of first encoded array][long integer: length of second encoded array]...[binary blob: start of first encoded array][binary blob: start of second encoded array]
So the writing handler would need to encode each array, calculate the byte length of each encoded array, and then write to the file: the number of arrays, the byte size of each array, then each array. The reading handler would then need to read in the same order, extracting out each blob and decoding it.
If I use this method how do I:
A. Ensure that I write a number out as an integer?
B. Determine the exact size of an encoded array?
2. Write each encoded array followed by a marker, which in turn can be found when reading. This seems problematic as any character you choose as a marker could theoretically be in the encoding of the array.
3. Put all of the arrays into a single array before encoding.
The issue I (think I) see with this is when the arrays get big enough copying the data from each array to the mega-array takes time and memory. If it does not do a copy, however, this might be the cleanest way.
Any help, suggestions, or pointers you can provide would be greatly appreciated. Thanks in advance.
Regards,
Dale
My application has several arrays of data and I would like to write it out to a file and read it back in when I open the application back up. I understand the basics of encoding arrays with arrayEncode(), reading and writing binary files, and putting up dialogs to prompt the user to pick a file or provide a file name. The problem is more about the file's structure for reading and writing. I want to read and write more than one array to a single file, then read the data back into their respective arrays.
Conceptually the idea is something like (not real Livecode):
Code: Select all
put arrayEncode(array1) into binarray1
write binarray1 to file outputFile
put arrayEncode(array2) into binarray2
write binarray2 to file outputFile at ... [end? some byte position?]1. Get the size in bytes of each array and structure the file as something like:
[integer: number of arrays in file][long integer: length of first encoded array][long integer: length of second encoded array]...[binary blob: start of first encoded array][binary blob: start of second encoded array]
So the writing handler would need to encode each array, calculate the byte length of each encoded array, and then write to the file: the number of arrays, the byte size of each array, then each array. The reading handler would then need to read in the same order, extracting out each blob and decoding it.
If I use this method how do I:
A. Ensure that I write a number out as an integer?
B. Determine the exact size of an encoded array?
2. Write each encoded array followed by a marker, which in turn can be found when reading. This seems problematic as any character you choose as a marker could theoretically be in the encoding of the array.
3. Put all of the arrays into a single array before encoding.
Code: Select all
put array1 into megaArray[1]
put array2 into megaArray[2]
...
put encodeArray(megaArray) into megaBinArrayAny help, suggestions, or pointers you can provide would be greatly appreciated. Thanks in advance.
Regards,
Dale
