"new" JSON library fastJson

[bhall2001]

phaworth,

The JSON you have there is valid. I'm going to use this snippet to figure out what's going on.

I'm suspecting the RTF is the issue somehow. As was mentioned. I've also been concerned with the libJson Unicode handling and will looking into that section as well.

Why did I use libJson? I am in the Create It with Livecode course and this was the library that was used in the class. I figured if it's good enough for Livecode, Ltd. to use, it must be the JSON library of choice. I also saw in the class that others used easyJson so those were the 2 libraries I focused on.

I'm going to look at the other library mentioned to see if there's something that I can use there if the license allows it.
Bob

[mwieder]

Brahmanathaswami-

Here's revOnline - it's available from the 'Sample Stacks' item from the Help menu on the menubar.
http://revonline.runrev.com/

[phaworth]

I couldn't find it online but did download it directly from http://www.rozek.mobi/LiveCode/JSONlib/ ... cr_en.html.

I'm happy to say that it parsed the LC8 dictionary JSON file with no problems, not quite so happy that it took roughly 32 seconds to do it. But bottom line is that it provided me with the info I need and since it will only be used once the first time an lcStackbrowser user runs LC8, I can live with the execution time.

Nones of this is meant as a criticism of fastJSON. It seems it is based on older libraries that fail for whatever reason on some json structures but I think probably work a very large percentage of the time.

Pete

[mwieder]

Bob-

Your unicodeEscapes() function is tripping on embedded backslashes in the RTF object: things like

'\\ul' and '\\u' are triggering inEscape instead of converting them to '\ul' and '\u'.

[phaworth]

Thanks for that Mark. Sounds like I may still be able to get the dictionary parsed with fastJSON.
Pete

[bhall2001]

Thanks mwieder! I had just figured that too and came here to post my findings to see your post. Great minds....

The route of the issue I believe is how libJson (and thus fastJson) handle Unicode. For now, I'm going to comment out the code for Unicode until I learn more about Unicode (I'm clueless on it at the moment).

With my "fix" for "\u", fastJson parses the file from JSON to a livecode array on average (on my machine wish is old and slow -- 2007 iMac) 35 milliseconds. Yes, milliseconds. Wow. Now I can claim fastJason is 1000s of times faster than other libraries

Now, with that said, I need to check the "correctness" of the array produced by the code. Here's a copy of the resulting array converted to text:

Code: Select all

Array
.	[associations] => Array
.	.	[1] => field
.	[OS] => Array
.	.	[5] => android
.	.	[1] => mac
.	.	[2] => windows
.	.	[3] => linux
.	.	[4] => ios
.	[introduced] => 2.0
.	[tags] => Array
.	.	[1] => text processing
.	[description] => Use the <RTFText> property to import and export text in RTF format.

The <RTFText> property is a representation of the styled text of the field in RTF format.

Setting the <RTFText> of a field (or a chunk of a field) sets both the text contents and the font, size, style, and color attributes corresponding to the information in the <RTFString>. Any other formatting controls in the <RTFstring> are ignored.

The <RTFText> property interprets the following RTF formatting controls:
Colors:\cf, \colortbl, \red, \green, \blue, \highlight
Styles:\plain, \b, \i, \ul, \ulnon, \strike, \dn, \up
Font and character set: \fcharset, \ansi, \mac, \pc, \pca, \u, \fonttbl, \f, \fs
Others:\header, \par, \line, \tab

Getting the <RTFText> property reports a string consisting of the text of the field (or chunk of a field), with any font, size, style, and color properties embedded in the text in the form of RTF control words. Unicode text is supported.

Paragraphs with a non-empty listStyle are appropriately marked in <rtfText> using both the (legacy) pn family of paragraph numbering tags and also with the new listtable tags.

By using both sets of tags a reasonable degree of interoperability is achieved with both TextEdit (and other Cocoa applications) on Mac, and Word and WordPad on Windows. 
>*Note:* Unfortunately, OpenOffice does not have particularly good rtf import / export capabilities (it doesn't even round-trip correctly through itself!) and thus copying / pasting of lists between LiveCode and OpenOffice will not work reliably or correctly.
>*Important:* Because the RTF standard does not include the box, threeDbox, and link styles supported by LiveCode, the <RTFText> property does not necessarily include all information necessary to reproduce the style information in a chunk. To export and re-import field information without losing any style information, use the htmlTextproperty instead.

For technical information about the RTF format, see the article at <http://msdn.microsoft.com/library/en-us/dnrtfspec/html/rtfspec.asp>.
.	[type] => property
.	[syntax] => Array
.	.	[1] => set the RTFText of [<chunk> of] <field> to <RTFString>
.	[references] => Array
.	.	[property] => Array
.	.	.	[3] => dragData
.	.	.	[1] => HTMLText
.	.	.	[2] => foregroundColor
.	.	[keyword] => Array
.	.	.	[1] => field
.	.	[function] => Array
.	.	.	[1] => charToNum
.	.	[glossary] => Array
.	.	.	[1] => format
.	.	.	[2] => RTF
.	[display name] => RTFText
.	[display syntax] => Array
.	.	[1] => set the RTFText of [<i>chunk</i> of] <i>field</i> to <i>RTFString</i>
.	[summary] => Specifies the contents of a <field>, with its text <format|formatting> represented in <RTF|RTF format>.
.	[platforms] => Array
.	.	[1] => desktop
.	.	[2] => server
.	.	[3] => web
.	.	[4] => mobile
.	[examples] => Array
.	.	[1] => Array
.	.	.	[script] => set the RTFText of field "Stuff" to URL "file:New Stuff.rtf"
.	.	[2] => Array
.	.	.	[script] => put the RTFText of field "Destination" into URL myURL
.	[value] => Array
.	.	[1] => Array
.	.	.	[type] => 
.	.	.	[description] => The <RTFText> of a field is a string.
.	.	.	[name] => value
.	[name] => rtftext
.	[id] => 708

I'm particularly interested in the RTF data elements. Is this what you'd expect to see in the array from the JSON provided? (verify the escape codes and let me know if this is correct or how it should be).

Bob

[bhall2001]

I just submitted the temporary fix for the RTF code on Github. Try out the 0.3.0 branch of fastJson and let me know if i the Dictionary works now. NOTE: Unicode will not work in this current branch.

Bob

PS. I can't seem to find this javascript file containing the dictionary JSON in the LC app folder. Is there someplace else I can find it?

[bn]

Hi Bob,

Peter wrote:

On my Mac, the file is in Applications/Contents/Tools/Documentation/html_viewer/resources/data/api/built_api.js

it should be similar for other operating systems.
Kind regards
Bernd

[FourthWorld]

Curious. I can appreciate the value of using JSON as an intermediary format, but for use within LiveCode itself I wonder why they don't just import that into an encoded array at build time.

[phaworth]

I wondered exactly the same thing Richard! I doubt anyone outside of Livecode would need or want to access the dictionary info. The only thing I can think is that it's for consistency with the way the widget documentation is recorded, so that the dictionary only needs one method to display it. Then again, who else but Livecoders would need access to the widget info.
Pete

[phaworth]

Thanks Bob, I will give that a whirl and report back. Don't know if there's any unicode in the file or not, probably not for the English version of it, not sure if there will be translations of it in the future.
Pete

[phaworth]

Bob,
You'll have to forgive my complete ignorance of gitHub but I now realize I don't know how to download a specific version, please enlighten me
Pete

[mwieder]

Pete-

Go here https://github.com/bhall2001/fastjson/t ... r/Livecode
Click on "JSON Speed Test.livecode"
You'll see a link named 'View Raw"
Keep your clothes on.
Instead, right- or control-click as you prefer, and select "Save Link As".. and use the default filename.

That will get you the current binary.
If you really need an earlier version, you'll have to clone the repository, but I don't think that's what you're asking.

[phaworth]

Thanks Mark. Worked great even after I took my clothes off!

[bhall2001]

I'm committing changes to the 0.3.1 branch in Git. If you right click the fastJson.lc link and download to a file, this is the most recent version of the development branch. These changes are not in the "master" as of yet. You can also click the link then copy/paste the code into your Library stack if that is easier.

What's new in this? I've spent some time working on the Unicode part. I think it's working but I have a limited data file to test with (also included in the Livecode folder). Next up in the 0.3.1 branch is an update to my silly little test stack. Coming shortly to Github.

https://github.com/bhall2001/fastjson/tree/0.3.1

Bob