ANN: Script Tracker

A place to discuss Version Control in LiveCode

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller

Post Reply
bwmilby
Posts: 438
Joined: Wed Jun 07, 2017 5:37 am
Location: Henrico, VA
Contact:

ANN: Script Tracker

Post by bwmilby » Sun May 27, 2018 4:45 am

Script Tracker
https://github.com/bwmilby/lc-misc/tree ... iptTracker
Also available on RevOnline

scripttracker.png
The goal of this tool is to expose the scripts of a binary stack in a fashion that allows them to be tracked via a version control system like GitHub. The repo linked above contains the output of the stack. Even though GitHub can't provide useful information about the changes in binary files, each time I perform a commit I include the corresponding binary stack. This allows easy download of the stack for use but also puts the scripts in a convenient place for review online.

Each script in the stack is exported as a valid .livecodescript file, but the stack is not modified to make it a behavior. A custom property set is created for each object to track a hash of the script and the modification date/time of the exported file. On each sync, those properties are used to determine if the script needs to be exported or if the file needs to be imported. Whenever a script changes, a diff file is created that contains all of the changes in that particular sync operation. If a file is no longer needed (i.e. the script was removed), then the extra files are moved to a separate directory and renamed to ensure no name collisions. I have a .gitignore file configured to skip those files when considering changes.

The stack features an automatic import feature which watches the script directory looking for files that change. If a file changes, then a sync is performed on that individual file (presumably importing some changes). There is a stack level preference to allow a choice of how to handle the situation when a change is detected in both the script and the file (script {keep stack script/overwrite file}, file {import file}, ask {present options in a dialog}, skip {keep stack script, but do not update the hash nor overwrite the file}).

If automatic import is enabled, there is also an option to have scripts opened in an external editor. When enabled, whenever the IDE wants to edit a script it will also get opened in the chosen external editor. If an object doesn't have a script yet, then a temporary comment is added to create a file for editing.

There is an option is to have a sync performed before saving the stack file. This is done with a front script that intercepts the SaveStackRequest message to initiate the sync if required.

The system is designed to be non-destructive, so the way to completely remove a file is to clear the script in the IDE and perform a sync. Deleting a file from the operating system and performing a sync will just regenerate the file. If you remove everything after the header, the file will get removed on the second sync (the first will import the blank script into the IDE, but not remove the file).

The stack currently handles a single mainstack at one time. There is a drop down menu with all non-IDE mainstacks to quickly switch between different ones. Mode 0 stacks are not included in the list. IDE stacks or mode 0 stacks can be specified by typing the stack name in the field.

The Prefs substack provides all of the options available for the currently tracked stack and the Script Tracker overall. Changes are saved when the tab is changed or the window is closed. The overall preferences are saved in the same location as the LiveCode prefs (but in a separate file). Stack level preferences are stored in each mainstack as a custom property set.

The Log will provide the time taken for an export and list each file that was touched. Some error messages will get displayed there.

Edited to add screen shot image.
Brian Milby

Script Tracker https://github.com/bwmilby/scriptTracker

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: ANN: Script Tracker

Post by bogs » Sun May 27, 2018 2:13 pm

Wow, very nice Brian!
Image

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7210
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: ANN: Script Tracker

Post by jacque » Mon May 28, 2018 9:51 pm

Brian, you're such an asset to our community. Thank you for this. I know someone who really, really needs this. His current app is a maze of libraries and stack-only scripts that disrupt the message path to the point where it's difficult to find where errors occur. This will allow him to go back to a more standard message hierarchy in the binary stack while still tracking the project in git.

As someone who occasionally needs to advise on this project, I salute you. :)
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

bwmilby
Posts: 438
Joined: Wed Jun 07, 2017 5:37 am
Location: Henrico, VA
Contact:

Re: ANN: Script Tracker

Post by bwmilby » Thu May 31, 2018 5:45 am

Thanks very much @jacque and @bogs!

I just posted an update that uses textEncode/textDecode to ensure scripts are saved in UTF-8.
I also modified the script menu so that the alt key can be used to include IDE stacks (still hides mode 0 stacks which should keep the script only stacks off the list).
Brian Milby

Script Tracker https://github.com/bwmilby/scriptTracker

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7210
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: ANN: Script Tracker

Post by jacque » Sat Jun 02, 2018 6:49 am

@Brian, I just got a moment to try Script Tracker. I have a stack I've been working on for a long time. I made some changes to it recently. Suppose I want to see how the changes I just made compare to the way it was before I had Script Tracker. Is there a way to grandfather in an older version so I can compare it to the newer one?

I'm not a git guru so I'd like to compare two text files in BBEdit. If I export scripts from the old copy, and then export scripts from the new copy, will the second set of files overwrite the original set? I don't want to import any files, only see the differences. You mentioned that syncing is automatic, but I don't want my current version to change.

Edit: I guess I could just move the first set of files out of the folder before creating a second set. I'm also not clear on how diff files work or how much info they contain. Are they cumulative over several updates? (Git novice here.)
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

sphere
Posts: 1145
Joined: Sat Sep 27, 2014 10:32 am
Location: Earth, Except when i Jump

Re: ANN: Script Tracker

Post by sphere » Sat Jun 02, 2018 1:12 pm

Thanks Brian, i had to look first to get the idea.
It's an automatic version of what i do manually, which costs more time.
I mostly copy every piece of script in a text file with notepad++, and save it seperately, so i have to open every used element which contains a script and copy it.
I like to have it as a text file so i can view/copy it quickly in notepad++ to livecode again if i need it.

This can surely help :)

bwmilby
Posts: 438
Joined: Wed Jun 07, 2017 5:37 am
Location: Henrico, VA
Contact:

Re: ANN: Script Tracker

Post by bwmilby » Sat Jun 02, 2018 7:14 pm

jacque wrote:
Sat Jun 02, 2018 6:49 am
Is there a way to grandfather in an older version so I can compare it to the newer one?
There isn't a built in easy way to do a compare like you are asking, but working on copies it would not be very difficult to do.

If you just need a one-time diff between the code it two stack versions, it will be pretty easy.
  1. Create a folder and place a copy of the latest version of the stack inside
  2. Use Script Tracker to export the scripts ("Sync Scripts")
  3. Replace the stack with a copy of the old version
  4. Open the old stack in LC
  5. In the Script Tracker prefs, Change the stack collision policy to "Script" (won't make changes to the stack scripts)
  6. In the Script Tracker prefs, Change the options/diff context the the number of lines desired (this will be the number of lines before/after any change that are included in the diff file)
  7. Close the prefs window to be sure they are changed (switching to another tab also saves)
  8. Click the "Sync Scripts" button
This will generate a single diff file (in the diff folder) that contains all of the differences between the two stacks. It doesn't list new scripts or removed scripts though (I suppose I could add that). If you want to flip the + and -, then do the same process but use "File" for the collision policy.
jacque wrote:
Sat Jun 02, 2018 6:49 am
If I export scripts from the old copy, and then export scripts from the new copy, will the second set of files overwrite the original set?
On every export, Script Tracker will update the list of files and remove extras. So if you export from the old stack it will "update" everything that is different. But it will also see everything as a collision since it will think the script changed and the file is newer.

I have a Mac program called "DeltaWalker" that makes it easy to compare 2 or 3 files to see how they have changed. It can also compare 2 folders and let you then look at the individual changes. Using something like that, you would just need to put the old stack into a fresh directory and export the scripts. Then you could compare the 2 script folders (current and old).
jacque wrote:
Sat Jun 02, 2018 6:49 am
I'm also not clear on how diff files work or how much info they contain. Are they cumulative over several updates? (Git novice here.)
Script Tracker creates a diff for each sync operation (named with the seconds). Not nearly as useful as Git, but is mainly there to allow a quick check of what changed on a particular sync.

If you are going to actually use Git, then it gets easier in the future. Every time you perform a commit in Git, a full snapshot it stored. At any time you can obtain a delta between any 2 points in the history. Works great for text files (can tell exactly what changed in each file), not so much for binary (where you can just see that there are different files, but not what changed).

Going backward to before you started tracking the project in Git would be difficult. But you could build a history with a little bit of effort when you start tracking. Basic process would be to take the oldest version and export the scripts and do an initial commit with a note for that particular version. Then, wipe the scripts directory and do the same for the next version. Git will be able to detect changes so you just select everything and do a commit each time. It would be tedious, but you could end up with a history like that. Even though Git can't provide useful information about a binary stack, you could still commit it with each version to have an archive copy (that's what I'm doing). You don't get any of the benefits of Git's compression though, so it will make the repository grow quicker.
Brian Milby

Script Tracker https://github.com/bwmilby/scriptTracker

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7210
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: ANN: Script Tracker

Post by jacque » Sat Jun 02, 2018 8:39 pm

Thanks Brian. A one-time diff is all I need right now. I made some changes that don't all work, so I want to remove some of them. They are scattered around the stack so a diff file will be useful. I'll give your instructions a try. I'm storing all your posts for reference.

Some day I'll try to learn git, but the effort is more than I have time for right now. I know it's a powerful and useful tool, but it's also complex and nerdy. I even got lost in GitHub which is supposed to be the "dummies" interface. :)
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

bwmilby
Posts: 438
Joined: Wed Jun 07, 2017 5:37 am
Location: Henrico, VA
Contact:

Re: ANN: Script Tracker

Post by bwmilby » Sat Jun 23, 2018 11:18 pm

Are you talking about the consolidated diff? That is still in there but only is generated based on differences between successive exports.
Brian Milby

Script Tracker https://github.com/bwmilby/scriptTracker

sphere
Posts: 1145
Joined: Sat Sep 27, 2014 10:32 am
Location: Earth, Except when i Jump

Re: ANN: Script Tracker

Post by sphere » Wed Dec 26, 2018 11:24 pm

Hi Brian,

how can i get the name of the cadr and the name of a field or button in the filename?
It's working great further, but it is difficult for me to recoqnize on a button nr where it is used for.
Of course i can open each file one by one in Notepad++ as the info is contained within the generated livecodescript file.

Would be great (and easier) if the generated filename would contain the button or field name from this cardname.
like --> btn-buttonname_oncard-cardname.livecodescript
card-cardname.livecodescript
stack-stackname.livecodescript
dg-datagridname.livecodescript
btn-buttonname_ondg-datagridname.livecodescript

if it is too difficult or to time-cosuming i could add a button to the stack that runs over the files and read out the names in the file, and then rewrites the file name.

Thanks!

bwmilby
Posts: 438
Joined: Wed Jun 07, 2017 5:37 am
Location: Henrico, VA
Contact:

Re: ANN: Script Tracker

Post by bwmilby » Thu Dec 27, 2018 12:04 am

I've posted an updated version on GitHub that adds an index file. Here is the index for ScriptTracker:
https://github.com/bwmilby/lc-misc/blob ... Tracker.md

I do plan on revising the index a bit (it is inefficient currently since it gets re-written every update), but it does at least provide a single file index of object IDs to their names. Currently it is in layer order, but I want to sort it differently (which will happen when I batch the output).

Hopefully this provides part of what you are looking for.

The primary reason that I’m using the ID is so that the script can easily ensure unique file names without having to track anything since the ID is guaranteed to be unique in a given stack (within a given object type for HC).

Changing the file name like you mention would currently be a bit involved. I think it is just generated in one spot, but at the point where it is created the script does not have the information needed to ensure unique file names without including the stack name and object ID.

Going from file to object is not bad since the full object ID is pulled from the file. Going from object to file requires knowing the unique file name from the object ID (or name).
Brian Milby

Script Tracker https://github.com/bwmilby/scriptTracker

sphere
Posts: 1145
Joined: Sat Sep 27, 2014 10:32 am
Location: Earth, Except when i Jump

Re: ANN: Script Tracker

Post by sphere » Thu Dec 27, 2018 7:11 pm

The primary reason that I’m using the ID is so that the script can easily ensure unique file names without having to track anything since the ID is guaranteed to be unique in a given stack (within a given object type for HC).
Yes true, i was just thinking how i like to create a filename manually when i copy the script into a txt file, which i did previously.

But this is working for everybody just great, and real fast too.
I will adjust myself working with this cool tool.

Thanks!

bwmilby
Posts: 438
Joined: Wed Jun 07, 2017 5:37 am
Location: Henrico, VA
Contact:

Re: ANN: Script Tracker

Post by bwmilby » Sat Apr 27, 2019 4:27 am

Just a quick note that I've separated SciptTracker out into its own repository:
https://github.com/bwmilby/scriptTracker

I also added a brief description of how to initially set up a stack for GitHub (but not the actual details on how to use GitHub). That is in the README which you will see if you click the above link.
Brian Milby

Script Tracker https://github.com/bwmilby/scriptTracker

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: ANN: Script Tracker

Post by bogs » Sat Apr 27, 2019 12:03 pm

Nice, was easy enough for me to understand.
Image

Post Reply

Return to “Version Control”