Using a remote stack instead of a database to share data

Anything beyond the basics in using the LiveCode language. Share your handlers, functions and magic here.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
kaveh1000
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 508
Joined: Sun Dec 18, 2011 7:23 pm
Location: London
Contact:

Using a remote stack instead of a database to share data

Post by kaveh1000 » Mon Aug 12, 2019 3:11 pm

I have a stack that is used by several people and has some central data that needs to be updated for all users from a central location. I have (painfully) been using MySQL to hold that data. So an authorized user would export the data from LiveCode and others would read it.

Following a discussion yesterday here:

https://forums.livecode.com/viewtopic.php?f=9&t=32971

I wonder if I can use a stack that is read directly by all users and that has the data needed. This would make it much simpler as the data is natively in a LiveCode stack and no painful import/export is needed.

The size of data is small so it can easily be held in a script in a shared stack.

One question is security. When a stack is opened using a standalone local stack, is the user able to read the script of the remote stack? and if so how can I protect that?

Any thoughts welcome.
Kaveh

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: Using a remote stack instead of a database to share data

Post by bogs » Mon Aug 12, 2019 3:24 pm

My first thought is I hope Andy chime in, since this sounds very similar to his group editing work.

My second thought is that anything can be read if you have the (choose 2 or more)...
* skills needed
* time needed
* determination needed

Stack files are binary, but plain text in them can be read in an appropriate text editor, although I sure don't recommend trying to modify it that way.

In a standalone situation, though, my guess is it would be a little trickier to read a stack if you don't expressly code the program to be able to. You should also be able to (I think) password the stack, or encrypt it.
Image

kaveh1000
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 508
Joined: Sun Dec 18, 2011 7:23 pm
Location: London
Contact:

Re: Using a remote stack instead of a database to share data

Post by kaveh1000 » Mon Aug 12, 2019 4:21 pm

Thank you Bogs. I have indeed opened a stack once with a text editor, in desperation!

Putting security aside, suppose I have a main stack that is distributed to several users. What I want is that the stack automatically opens the remote stack, then that secondary stack is used in the background. so I can say in the main stack:

Code: Select all

get table 1 of card 1 of stack "remoteStack"
and ideally I don't want that remote stack to show at all but be usable by the main stack. Can that be done?
Kaveh

AndyP
Posts: 614
Joined: Wed Aug 27, 2008 12:57 pm
Location: Seeheim, Germany (ex UK)
Contact:

Re: Using a remote stack instead of a database to share data

Post by AndyP » Mon Aug 12, 2019 4:46 pm

First off, if there is any chance, no matter how remote that more than one person will be updating the data at the same time, stick with MySQL otherwise you would have to build some file locking and queuing system which is at some time in the future likely to bite you in the Ar**!

I think we need a little more info.. you talk about exporting data to MySQL, so are you replacing all the database data or just updating the tables with the new/amended data?

Is the data coming from fields or table or a datagrid?

How many users are likely to access the data?

What format is your data?
What format would you like it to be?

It may be that the obvious and therefor simplest solution would be for us to help you with problems you may be having getting data in and out of MySQL. There are a lot of DB experts on the forum who can help, so maybe a breakdown on what you are struggling with in LiveCode/MySQL would be the way forward.
Andy Piddock
https://livecode1001.blogspot.com Built with LiveCode
https://github.com/AndyPiddock/TinyIDE Mini IDE alternative
https://github.com/AndyPiddock/Seth Editor color theming
http://livecodeshare.runrev.com/stack/897/ LiveCode-Multi-Search

kaveh1000
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 508
Joined: Sun Dec 18, 2011 7:23 pm
Location: London
Contact:

Re: Using a remote stack instead of a database to share data

Post by kaveh1000 » Mon Aug 12, 2019 5:27 pm

Thank you Andy. Good point about more than one person updating. So what if only one person is updating? So only one read permission. It is a small team so very unlikely we need more than one.

I can certainly stick with MySQL. Just a few lines of data and one data table.

In that case my question to you and others is can I just use the remote stack in place of a database?
Kaveh

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Using a remote stack instead of a database to share data

Post by FourthWorld » Mon Aug 12, 2019 8:56 pm

Any file can be good for read-only multi-user storage. When a stack file is a good fit for the structure of the data, it can work well.

Writes can be mitigated with locks. For small teams in which concurrent writes are never needed, I use that often.

The only downside to stack files over any other file type is local memory management. With flat or LSON (encoded array) files, knowing what's in RAM is more straightforward than when working with stack files.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

kaveh1000
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 508
Joined: Sun Dec 18, 2011 7:23 pm
Location: London
Contact:

Re: Using a remote stack instead of a database to share data

Post by kaveh1000 » Mon Aug 12, 2019 10:27 pm

Thank you Richard. That is encouraging. Now probably a newbie question. Can I open a stack without it showing up? So user does not even know it has fetched a remote stack.
Kaveh

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: Using a remote stack instead of a database to share data

Post by bogs » Mon Aug 12, 2019 10:49 pm

I think the -ui switch when calling it is what your looking for, but for sure, Richard would know better.
Image

AndyP
Posts: 614
Joined: Wed Aug 27, 2008 12:57 pm
Location: Seeheim, Germany (ex UK)
Contact:

Re: Using a remote stack instead of a database to share data

Post by AndyP » Tue Aug 13, 2019 8:00 am

As for security if using the remote stack for storage, just encrypt the data before storing and decrypt when reading, make sure that you are passing data over ssl.
Andy Piddock
https://livecode1001.blogspot.com Built with LiveCode
https://github.com/AndyPiddock/TinyIDE Mini IDE alternative
https://github.com/AndyPiddock/Seth Editor color theming
http://livecodeshare.runrev.com/stack/897/ LiveCode-Multi-Search

AxWald
Posts: 578
Joined: Thu Mar 06, 2014 2:57 pm

Re: Using a remote stack instead of a database to share data

Post by AxWald » Tue Aug 13, 2019 9:15 am

Hi,

if you open a remote stack you're actually creating a snapshot of this stack in your local RAM. As soon as you have done this, someone other can change or even delete this stack, without you noticing.

To change data in this remote stack, you must save your local copy to it, actually replacing it. All changes made meanwhile by other users will be lost, and if someone else is trying the same at the same time, you'll get conflicts.

To circumvent this, you need to refresh your local copy very often while carefully maintaining anything you've done with it already. Further, you need to implement record locks at least, and to write code that resolves conflicts without creating inconsistencies.

Basically, you'll have to write a crude version of a database engine. This doesn't sound like a very good idea.

Have fun!
All code published by me here was created with Community Editions of LC (thus is GPLv3).
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!

AndyP
Posts: 614
Joined: Wed Aug 27, 2008 12:57 pm
Location: Seeheim, Germany (ex UK)
Contact:

Re: Using a remote stack instead of a database to share data

Post by AndyP » Tue Aug 13, 2019 10:05 am

Ok, I've put together a quick demo of working with data in a remote stack.
This assumes you have access to a lc server.

edit: the key to this is the start using stack command which makes all of the stack available to the server script.

The setup consists of 2 stacks and 1 lc server script file.

RemoteTransfer.livecode
Get Button

Code: Select all

on mouseUp
   put "option=" & urlEncode("getMyData") into tMyPostOption
   --change to your server address
   post tMyPostOption to url "https://andypiddock.co.uk/code/remotedatastack/remotefetch.lc"
   put empty into fld "DataTable"
   put it into fld "DataTable"
end mouseUp
Update Button

Code: Select all

on mouseUp
   put "option=" & "updateMyData" into tMyPostOption
   put "mydata=" & urlEncode(fld "DataTable") into tMyData
   answer tMyPostOption & "&" & tMyData
   --change to your server address
   post tMyPostOption&"&"&tMyData to url "https://andypiddock.co.uk/code/remotedatastack/remotefetch.lc"
end mouseUp
this posts commands and data to a lc server script remotefetch.lc

Code: Select all

<?lc
-- turn off error reporting, helps increase security
set the errorMode to "quiet"

put $_POST["option"] into tOption
put $_POST["mydata"] into tMyData

switch tOption

case "getMyData"
start using stack "remotedata.livecode"
put field "DataTable" of stack "remotedata.livecode"
break

case  "updateMyData"
start using stack "remotedata.livecode"
put urlDecode (tMyData) into field "DataTable" of stack "remotedata.livecode"
save stack "remotedata.livecode"
break

end switch
?>
which reads and writes to a stack remotedata.livecode sitting in the same server directory.
which is just a stack with a table field.
There is no error checking or encryption in this demo.

Ill leave the lc serverscript and remotedata.livecode up on my server so that you can test.
Attachments
remotedata.PNG
remotedata.PNG (4.2 KiB) Viewed 6428 times
remotedatastack.PNG
remotedatastack.PNG (4.93 KiB) Viewed 6428 times
RemoteStackTransfer.zip
(2.15 KiB) Downloaded 214 times
Last edited by AndyP on Tue Aug 13, 2019 4:53 pm, edited 1 time in total.
Andy Piddock
https://livecode1001.blogspot.com Built with LiveCode
https://github.com/AndyPiddock/TinyIDE Mini IDE alternative
https://github.com/AndyPiddock/Seth Editor color theming
http://livecodeshare.runrev.com/stack/897/ LiveCode-Multi-Search

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Using a remote stack instead of a database to share data

Post by FourthWorld » Tue Aug 13, 2019 4:52 pm

bogs wrote:
Mon Aug 12, 2019 10:49 pm
I think the -ui switch when calling it is what your looking for, but for sure, Richard would know better.
The -ui option is used on the command line to tell a LC standalone to run exclusively as a command-line app for the session, without attempting to initialize a GUI.

Here all that's happening is moving a data file over the network and back again. The file happens to be in stack file format, but the way we manage it is no different from any other file being read from and written to a remote server.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: Using a remote stack instead of a database to share data

Post by bogs » Tue Aug 13, 2019 5:02 pm

Ahhh, my bad, thanks for the clarification!
Image

Post Reply

Return to “Talking LiveCode”