It scares me more than a little how few comments there are in the engine. From the looks of things pre-Mark Waddingham there were next to no comments.
I think the docs integration will help a bit here and I'd really like that to be bumped to one of the first things that gets done. At the very least it would be good to have a sample of the docs style you want so if someone adds a function etc they will be able to document it in the engine code... even if for the time being someone needs to copy those docs out to the dictionary manually.
Integrating the docs
Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, LCMark
Integrating the docs
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/
-
mwieder
- VIP Livecode Opensource Backer
- Posts: 3581
- Joined: Mon Jan 22, 2007 7:36 am
- Location: Berkeley, CA, US
- Contact:
Re: Integrating the docs
...and that there are function arguments named "the"... as inIt scares me more than a little how few comments there are in the engine.
Code: Select all
if (!the)PowerDebug http://powerdebug.ahsoftware.net
PowerTools http://www.ahsoftware.net/PowerTools/PowerTools.irev
PowerTools http://www.ahsoftware.net/PowerTools/PowerTools.irev
Re: Integrating the docs
lol... and no docs about what the means.... It seems it's related to the difference between:
the keys of myArray and keys(myArray)... which kind of makes sense that it's named the
the keys of myArray and keys(myArray)... which kind of makes sense that it's named the
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/
Re: Integrating the docs
Welcome to our world - don't get scared, think of it as a challenge 
The engine source-code is quite opaque until you start to get your head around how it all works - although this can take some time (I've been working with it for nigh on 10 years...).
In terms of docs - then we are working on doing something like this as part of the refactoring project. At that point all syntax will be defined in specification files, which is the natural point for the (reference) docs to be placed.
Of course, describing what a function is doing is always a good thing next to the source, but I'm not sure the engine source-code is the right place for a dictionary entry type description. We'll be pushing the dictionary source into the public repo shortly (we didn't quite have time for Tues!), so that will mean you'll be able to create a dictionary entry for any new functions and such too.
The engine source-code is quite opaque until you start to get your head around how it all works - although this can take some time (I've been working with it for nigh on 10 years...).
In terms of docs - then we are working on doing something like this as part of the refactoring project. At that point all syntax will be defined in specification files, which is the natural point for the (reference) docs to be placed.
Of course, describing what a function is doing is always a good thing next to the source, but I'm not sure the engine source-code is the right place for a dictionary entry type description. We'll be pushing the dictionary source into the public repo shortly (we didn't quite have time for Tues!), so that will mean you'll be able to create a dictionary entry for any new functions and such too.
Re: Integrating the docs
Ah... ok, I had thought it would be in the form of JavaDoc style docs... for now I'm following your lead and putting a one liner with initials, date, and brief description although it occurs to me that git-blame would cover all those and not mess up the code...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/
Re: Integrating the docs
It will be something like JavaDoc
At the moment the problem is that things like commands exec functions contain code for all the different commands that start with the same word. Functions tend not to be overloaded in this way so it would work okay for those.
The refactoring project solves this by moving each command / function / property etc. implementation into a distinct function so at that point we might find in engine-source would work better - we can compare when we move to syntax specs and determine which works best.
In terms of git-blame - the form of comments we use at the moment was motivated by the fact it makes it easy to see all the changes in the source that came from a single bug fix / feature implementation. We've migrated from using SVN which is quite different from git - we'll see how things go.
At the moment the problem is that things like commands exec functions contain code for all the different commands that start with the same word. Functions tend not to be overloaded in this way so it would work okay for those.
The refactoring project solves this by moving each command / function / property etc. implementation into a distinct function so at that point we might find in engine-source would work better - we can compare when we move to syntax specs and determine which works best.
In terms of git-blame - the form of comments we use at the moment was motivated by the fact it makes it easy to see all the changes in the source that came from a single bug fix / feature implementation. We've migrated from using SVN which is quite different from git - we'll see how things go.
-
mwieder
- VIP Livecode Opensource Backer
- Posts: 3581
- Joined: Mon Jan 22, 2007 7:36 am
- Location: Berkeley, CA, US
- Contact:
Re: Integrating the docs
My guess is you're going to enjoy the differences.We've migrated from using SVN which is quite different from git - we'll see how things go.
They won't really become apparent until merge time. Creating branches is easy. Merging in svn (or cvs)...
My experience with svn has been that the team stops working for a week while the merge conflicts get worked out, then work can resume again. Git makes that as painless as possible.
PowerDebug http://powerdebug.ahsoftware.net
PowerTools http://www.ahsoftware.net/PowerTools/PowerTools.irev
PowerTools http://www.ahsoftware.net/PowerTools/PowerTools.irev
Re: Integrating the docs
Oh I'm enjoying the difference already - we've had a few little 'eh?' moments in the office (particularly with git submodules) but stackoverflow has always come to the rescue in a timely fashion
-
mwieder
- VIP Livecode Opensource Backer
- Posts: 3581
- Joined: Mon Jan 22, 2007 7:36 am
- Location: Berkeley, CA, US
- Contact:
Re: Integrating the docs
Yeah, I ran into the submodule thing earlier on github and Monte set me straight.
Always something new to learn about git.
Always something new to learn about git.
PowerDebug http://powerdebug.ahsoftware.net
PowerTools http://www.ahsoftware.net/PowerTools/PowerTools.irev
PowerTools http://www.ahsoftware.net/PowerTools/PowerTools.irev
Re: Integrating the docs
I must confess I'm still not entirely convinced about the submodule thing... I structured the repository in the way I did to ensure that the principal repo (livecode) didn't become bloated with lots of stuff (particular binary blobs) that is derivable from elsewhere.
Ideally the 'thirdparty' folder wouldn't be a submodule, but simple scripts and such which fetch and patch the appropriate versions of the thirdparty folders; and 'prebuilt' would get generated by running a script that builds those components (or downloads them from something that isn't git).
Anyway, it works quite well for now, even if managing branches and such in a repo with submodules is a little more fiddly than just having the single one
Ideally the 'thirdparty' folder wouldn't be a submodule, but simple scripts and such which fetch and patch the appropriate versions of the thirdparty folders; and 'prebuilt' would get generated by running a script that builds those components (or downloads them from something that isn't git).
Anyway, it works quite well for now, even if managing branches and such in a repo with submodules is a little more fiddly than just having the single one
Re: Integrating the docs
I think you haven't gone far enough and that may cause problems. One of the difficulties is we can't use our own repo as a submodule without it making a mess. What I think would be good is if the livecode repo were just a super project that contained a readme and submodules. Then we could all create our own version that used our own forks of each submodule we wanted to work on but left things we weren't interested in the same as you have. I particularly think libexternal should be in it's own repo so we can all use it as a submodule in our own projects... although it will need a different license for that.I must confess I'm still not entirely convinced about the submodule thing... I structured the repository in the way I did to ensure that the principal repo (livecode) didn't become bloated with lots of stuff (particular binary blobs) that is derivable from elsewhere.
Can't you have it as a folder that contains submodules pointing to the head of the master branch of the appropriate third party library?Ideally the 'thirdparty' folder wouldn't be a submodule, but simple scripts and such which fetch and patch the appropriate versions of the thirdparty folders; and 'prebuilt' would get generated by running a script that builds those components (or downloads them from something that isn't git).
Hmm... if you want to manage the branches together (all submodules have the same branches as the super project) then yeah you probably are better off skipping the submodules. If you treat them as independent repos with the super project tying it all together then submodules are good. You've got to remember to update after you checkout or things will get messy... also if you commit a change to a submodule and you push the super project but don't push the submodule git can't find the submodule like with the IDE the other day.Anyway, it works quite well for now, even if managing branches and such in a repo with submodules is a little more fiddly than just having the single one
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/
-
mwieder
- VIP Livecode Opensource Backer
- Posts: 3581
- Joined: Mon Jan 22, 2007 7:36 am
- Location: Berkeley, CA, US
- Contact:
Re: Integrating the docs
I may be hijacking this thread, but I was about to start one with much the same topic title, so let's see where this goes...
I'm trying to put together some xml files for the docs repository and there are several things I've run into:
1. I have no idea how the various xml files get lumped together into the released doc files, so it's impossible to test them.
2. The format of the docs xml files is *atrocious*. There is data in xml elements that shouldn't be there at all. There's html code embedded in xml tags, and no standard xml tool will deal with unraveling that. There are "<" and ">" chars embedded in other xml elements as well, in things like example and syntax tags. These should either be encoded and decoded properly or in CDATA tags.
3. There's no tool to deal with editing this format. I've been working with Bjornke on an editing tool, and I've got something that minimally does the job, but that leads to...
4. There are two ways to go here.
A: The easy way out is to extend the technical debt and just use the existing format. I really hate to do this, and I'm reluctant to release the doc editing tool because of that. This is just going to get harder and harder to maintain as time goes on.
B: The hard way is to use proper xml formatting and fiddle with the mechanics of converting the xml files into the doc files. This is a much better solution all around, although I think it's going to be a lot more work than I want to put in.
I'm trying to put together some xml files for the docs repository and there are several things I've run into:
1. I have no idea how the various xml files get lumped together into the released doc files, so it's impossible to test them.
2. The format of the docs xml files is *atrocious*. There is data in xml elements that shouldn't be there at all. There's html code embedded in xml tags, and no standard xml tool will deal with unraveling that. There are "<" and ">" chars embedded in other xml elements as well, in things like example and syntax tags. These should either be encoded and decoded properly or in CDATA tags.
3. There's no tool to deal with editing this format. I've been working with Bjornke on an editing tool, and I've got something that minimally does the job, but that leads to...
4. There are two ways to go here.
A: The easy way out is to extend the technical debt and just use the existing format. I really hate to do this, and I'm reluctant to release the doc editing tool because of that. This is just going to get harder and harder to maintain as time goes on.
B: The hard way is to use proper xml formatting and fiddle with the mechanics of converting the xml files into the doc files. This is a much better solution all around, although I think it's going to be a lot more work than I want to put in.
PowerDebug http://powerdebug.ahsoftware.net
PowerTools http://www.ahsoftware.net/PowerTools/PowerTools.irev
PowerTools http://www.ahsoftware.net/PowerTools/PowerTools.irev
Re: Integrating the docs
It may not be worthwhile. I believe documentation is meant to be integrated into lcidl files in the future...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/
Re: Integrating the docs
@monte is correct - the goal is to (at the point syntax gets integrated into the spec files the refactored engine needs) move reference documentation (the dictionary) into those spec files. It seems the natural place for it - next to the syntax it is describing.
In terms of where we are at the moment - the xml files were originally converted from the old documentation stack, and have been processed a couple of times since then. The current schema that's in the contributors guide is what new docs should be formatted to - and this is processed to generate the docs you see in the dictionary.
In regards to format of some of them - then, yes, some of the XML is still not particularly well formed (although I think newly written docs / docs that have been edited) should be in a better state than others. (Compare the 'mobile' ones - which are all new - with the older ones which haven't been touched in an age - revXML).
The point here is that it would be possible to cleanup the XML files as they stand - correcting the problems you have observed - and the docs system is fine with that (it's also fine with the slightly malformed ones too).
The docs 'bundles' that the dictionary uses are built using the 'builder/docs_builder.rev' stack in the github repo - although it doesn't (at the moment) have a way to just build the docs, it's tied into the process for building an installer.
From what I can see - the schema we have is sufficient for what the docs currently display (but that is not to say it couldn't be refined further) and the main issue is the formatting of the source XML files themselves. Thus I (personally) don't really see the need to 'throw the baby out with the bathwater' now so to speak - especially since the information within them will be migrated (and in many cases rewritten/updated) later on in the refactoring project. That being said, cleaning up the current source XML files (pretty printing, correcting the flaws in some of them) would certainly be a helpful thing - it is a task an automated tool could do (the XML files we have are happily read by revXML, so it's just a case of reading them in and then writing them out in a formatted way that makes them easier to edit).
In regards to a docs editing tool - then we do have one that we use to update docs - although it is not the most robust beast (not having included it in the repo yet was probably a case of being overly conservative - the code in it might well be useful, even if using it requires some knowledge of its flaws)... I'll add it to the github repo so you can take a look. From what I recall it has a way of previewing the documentation entries as they appear in the dictionary - it uses the same underlying library to do so.
In terms of where we are at the moment - the xml files were originally converted from the old documentation stack, and have been processed a couple of times since then. The current schema that's in the contributors guide is what new docs should be formatted to - and this is processed to generate the docs you see in the dictionary.
In regards to format of some of them - then, yes, some of the XML is still not particularly well formed (although I think newly written docs / docs that have been edited) should be in a better state than others. (Compare the 'mobile' ones - which are all new - with the older ones which haven't been touched in an age - revXML).
The point here is that it would be possible to cleanup the XML files as they stand - correcting the problems you have observed - and the docs system is fine with that (it's also fine with the slightly malformed ones too).
The docs 'bundles' that the dictionary uses are built using the 'builder/docs_builder.rev' stack in the github repo - although it doesn't (at the moment) have a way to just build the docs, it's tied into the process for building an installer.
From what I can see - the schema we have is sufficient for what the docs currently display (but that is not to say it couldn't be refined further) and the main issue is the formatting of the source XML files themselves. Thus I (personally) don't really see the need to 'throw the baby out with the bathwater' now so to speak - especially since the information within them will be migrated (and in many cases rewritten/updated) later on in the refactoring project. That being said, cleaning up the current source XML files (pretty printing, correcting the flaws in some of them) would certainly be a helpful thing - it is a task an automated tool could do (the XML files we have are happily read by revXML, so it's just a case of reading them in and then writing them out in a formatted way that makes them easier to edit).
In regards to a docs editing tool - then we do have one that we use to update docs - although it is not the most robust beast (not having included it in the repo yet was probably a case of being overly conservative - the code in it might well be useful, even if using it requires some knowledge of its flaws)... I'll add it to the github repo so you can take a look. From what I recall it has a way of previewing the documentation entries as they appear in the dictionary - it uses the same underlying library to do so.
Re: Integrating the docs
Pushed our docs editor into the 'tools' folder on 'develop'. I think it's relatively straightforward to use (I must confess I haven't ever used it) - it allows you to load a dictionary XML file then edit / preview / look at the XML source.