Wrapping DEFLATED data

Anything beyond the basics in using the LiveCode language. Share your handlers, functions and magic here.

Moderators: Klaus, FourthWorld, heatherlaine, robinmiller, kevinmiller

Post Reply
daryl
Posts: 47
Joined: Mon Apr 29, 2013 11:43 pm

Wrapping DEFLATED data

Post by daryl » Sat Jun 15, 2013 12:57 am

Hi Folks,

I have recently posted a question to the beginners forum titled "Inflating ZIP Deflated data string ", which I won't repeat here, but I have another angle I am trying on this problem, and was hoping maybe someone with more experience might be able to help. The issue is this: I have to read some deflated compressed data from a network socket and then inflate it back to the original. I discovered that LiveCode has the revZip functions to deal with zipped data, but they appear to only work on file archives. So created a file archive, added my data using the revZipAddUncompressedItemWithData function as I didn't want the data compressed yet again, closed the archive, opened it and used the revZipExtractItemToVariable function to get the data back from the archive and into a variable. That all worked, except the data was still deflated.

So having read the compress definition in the dictionary and about how to wrap deflated data with a header and a trailer, I thought I would try that, and I had partial success. That is, I can write my wrapped data to file and running the unix "file" command it tells me it's "file.gz: gzip compressed data, from Unix" but if I try to gunzip it I get "gzip: gzip.gz: invalid compressed data--format violated". I have done a hex dump to make sure the file has the 10 byte header and 8 byte trailer with my data in the middle as described in the dictionary and it looks like what I expect.

It is my understanding that both zip and gzip use the DEFLATE algorithm to compress data, and that the difference between the two is in the header and footer, so I am not sure what I'm doing wrong, as my data is deflated data, and I can read the same data and deflate it by using a perl script and their Compress::Zlib module.

Does anyone know if there are any plans on extending the revZip functions to include deflating a deflated binary string from a variable?

Any help, thoughts, pointers to documentation, etc. would be greatly appreciated.

Thanks,

Daryl

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2839
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: Wrapping DEFLATED data

Post by mwieder » Sat Jun 15, 2013 4:05 am

Daryl- I cross-linked this to the Engine Contributors forum.

http://forums.runrev.com/viewtopic.php?f=66&t=15576

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: Wrapping DEFLATED data

Post by monte » Sat Jun 15, 2013 4:16 am

You tried just using the decompress function I assume?
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2839
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: Wrapping DEFLATED data

Post by mwieder » Sat Jun 15, 2013 4:27 am

Daryl-

Both zip and gzip *can* use the DEFLATE algorithm, but there are about a dozen different compression algorithms that *could* be used.
Do you have control over the sending mechanism as well? i.e., can you force it to use a different compression algorithm?
I'm reasonably sure that the revZip functions use only one method of compression, and I don't know which that is.

daryl
Posts: 47
Joined: Mon Apr 29, 2013 11:43 pm

Re: Wrapping DEFLATED data

Post by daryl » Sun Jun 16, 2013 3:42 pm

Thanks Guys for your replies,

> ... Do you have control over the sending mechanism as well? i.e., can you force it to use a different compression algorithm?

Yes, but it's not my code so I will consult tomorrow with the developer to get more detailed information. It is my understanding that the code uses zip to compress the data, but I will get more information and details on Monday.

> You tried just using the decompress function I assume?

Yes, I've have tried using the decompress function, without success. It's what led me to examine the compress function in the dictionary.

> Daryl- I cross-linked this to the Engine Contributors forum.

Thanks, when I get more information/details I will also post it there.

Again, thanks for all your replies, I really appreciate it.

Regards,

Daryl

Janschenkel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 977
Joined: Sat Apr 08, 2006 7:47 am
Location: Aalst, Belgium
Contact:

Re: Wrapping DEFLATED data

Post by Janschenkel » Mon Jun 17, 2013 6:02 pm

Well, there is a way to trick the 'decompress' function into inflating the data, but you'll need to know the length of the uncompressed data.
It involves tampering with the header and trailer of the compressed data; but the point is moot if you don't know the uncompressed length.

Jan Schenkel.
Quartam Reports & PDF Library for LiveCode
www.quartam.com

daryl
Posts: 47
Joined: Mon Apr 29, 2013 11:43 pm

Re: Wrapping DEFLATED data

Post by daryl » Mon Jun 17, 2013 8:57 pm

Folks,

For anyone wanting to follow this issue, and to not post the same thing twice, I have posted my followup to the Engine Contributors forum at:

http://forums.runrev.com/viewtopic.php? ... 769#p78769

as I am hoping to see if it might be possible to add a LiveCode interface to the zlib in memory inflate and deflate functions, as described in RFC 1950, available at: http://www.ietf.org/rfc/rfc1950.txt.

Thank you everybody for your help.

Regards,

Daryl

Post Reply

Return to “Talking LiveCode”