revxml

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, LCMark

Locked
malte
Posts: 1098
Joined: Thu Feb 23, 2006 8:34 pm
Location: Ostenfeld germany
Contact:

Re: revxml

Post by malte » Fri May 02, 2014 8:06 pm

I think we are using almost everything in the external. Including move and copy and moveRemote and copyRemote. Append too, yes. In my case it does seem to put weird stuff (the same stuff) in multiple nodes. The larger the tree, the more likely to hit...

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: revxml

Post by LCMark » Thu May 08, 2014 4:33 pm

Okay - so I've had a look at the stack @malte constructed and have run it through valgrind... There do indeed seem to be issues :)

The problem I believe is due to Copy/MoveRemoteNode - the implementation of these worked fine in the older libxml, however obviously things have changed quite significantly in the new version. Some data amongst nodes in a tree is now shared - so you can't just take an xmlNodePtr from one, unlink it and shove it in another document.

Now, the DOM spec does support Copying / Moving remote nodes and there so appear to be some functions in libxml to perform those functions (xmlDOMWrapAdoptNode / xmlDOMWrapCloneNode) so we'll have to look into reworking the current CopyRemote/MoveRemote operations using those.

@trevordevore: Just to check to see if yours is the same issue - do you use any of the Remote calls in Clarify?

trevordevore
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1005
Joined: Sat Apr 08, 2006 3:06 pm
Location: Overland Park, Kansas
Contact:

Re: revxml

Post by trevordevore » Thu May 08, 2014 4:42 pm

@runrevmark - Not that I can tell. The issue I'm seeing occurs around the spot where I use revXMLRPC_AddParam which in turn calls revAppendXML. Any chance that could cause the problem?
Trevor DeVore
ScreenSteps - https://www.screensteps.com

LiveCode Repos - https://github.com/search?q=user%3Atrevordevore+topic:livecode
LiveCode Builder Repos - https://github.com/search?q=user%3Atrevordevore+topic:livecode-builder

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: revxml

Post by mwieder » Tue May 13, 2014 6:56 am

I got this notice today when updating my ruby gems:

IMPORTANT! Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

bundle config build.nokogiri --use-system-libraries
bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: revxml

Post by mwieder » Tue May 13, 2014 7:00 pm

Looks like maybe some significant patches to libxml2 have been done:

https://git.gnome.org/browse/libxml2/

and in particular, things like this:

https://git.gnome.org/browse/libxml2/diff/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: revxml

Post by LCMark » Wed May 14, 2014 10:49 am

@mwieder: Hmmm - I wonder if part of the problem is that in 2.9.0 they've added some optimizations and other such things to do with the way XML trees are stored which are not backwards-compatible with apps which perhaps did not use appropriate APIs to do certain things or did certain things under the assumption that it was fine as there was no API to do them previously. Certainly there's an issue with remote node operations in libXML as you can't just move nodes between XML documents anymore (the nodes in the tree share data with other nodes and the document it seems) - however, @trevordevore's issue seems to be different as 'AppendXML' doesn't look like its doing anything 'bad'.

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: revxml

Post by mwieder » Wed May 14, 2014 7:42 pm

I am still muchly confused by this.

Following up the links, it looks like the officially released versions (aside from grabbing the nightlies from the git tree) of libxml2 are 2.8.0 from May of 2012 and 2.9.1 from April 2013. The ruby nokogiri gem now has a specially patched version of libxml2-2.8.0 that is not part of the "regular" libxml distribution.

I'm tempted to grab the latest nightly build and try compiling from that rather than using 2.9.1 from a year ago with known problems.

malte
Posts: 1098
Joined: Thu Feb 23, 2006 8:34 pm
Location: Ostenfeld germany
Contact:

Re: revxml

Post by malte » Wed May 14, 2014 8:54 pm

I'd happiely test that if you did Mark!

Cheers,

Malte

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: revxml

Post by mwieder » Wed May 14, 2014 10:09 pm

...on further talking with my coworkers, it appears that there are lots of different versions of libxml out there: the "official" gnome/libxml.org release isn't updated very often, and it seems that each linux distro has its own tweaked version that somewhat corresponds to one of the released versions, usually with a bunch of subsequent bugfixes applied. So nokogirir isn't alone in crafting a special version, and the paucity of official releases helps contribute to the balkanization of unique libraries.

malte
Posts: 1098
Joined: Thu Feb 23, 2006 8:34 pm
Location: Ostenfeld germany
Contact:

Re: revxml

Post by malte » Mon Jun 09, 2014 9:27 am

I just testetd 6.6.2 and XML is still misbehaving. Anybody got an idea how we can move this forward?

Best,

Malte

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: revxml

Post by LCMark » Thu Jun 12, 2014 11:01 am

@malte: I'm taking a look at this today - I've moved the implementation of the RemoteNode operations to use the appropriate APIs in libxml2 and things still seem to work...

The problem with AppendXML is the same as that with Copy/MoveRemoteNode. The implementation of AppendXML creates a new document from the XML fragment, then moves the root node from the new document to be under the specified node in the original document - i.e. it is a MoveRemove operation which is doing things in an incompatible way with the new version of libxml2. I've changed this too and don't seem to be able to reproduce the problem @trevordevore was having in Clarify anymore.

Anyway, I think these problems should be fixed now - I've filed them as Bug 12628 - the fixes should be in both the first build of 6.6.3 and the next build of 6.7.

malte
Posts: 1098
Joined: Thu Feb 23, 2006 8:34 pm
Location: Ostenfeld germany
Contact:

Re: revxml

Post by malte » Fri Jun 13, 2014 8:57 am

Hey Mark,

this sounds awesome. If you want me to test the external before you put out the next DP, you can just mail it to me and I will give it a shot.

All the best,

Malte

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: revxml

Post by LCMark » Fri Jun 13, 2014 9:16 am

@malte: I've sent you a build for Mac via email - fingers-crossed it solves the issues!

malte
Posts: 1098
Joined: Thu Feb 23, 2006 8:34 pm
Location: Ostenfeld germany
Contact:

Re: revxml

Post by malte » Fri Jun 13, 2014 11:24 am

Awesome! Played a little with it. The issue that was always reproducable in the standalone and IDE is gone now!!!

Thumbs up and thank you!!

Malte

Locked

Return to “Engine Contributors”