Page 1 of 1
Using a remote stack instead of a database to share data
Posted: Mon Aug 12, 2019 3:11 pm
by kaveh1000
I have a stack that is used by several people and has some central data that needs to be updated for all users from a central location. I have (painfully) been using MySQL to hold that data. So an authorized user would export the data from LiveCode and others would read it.
Following a discussion yesterday here:
https://forums.livecode.com/viewtopic.php?f=9&t=32971
I wonder if I can use a stack that is read directly by all users and that has the data needed. This would make it much simpler as the data is natively in a LiveCode stack and no painful import/export is needed.
The size of data is small so it can easily be held in a script in a shared stack.
One question is security. When a stack is opened using a standalone local stack, is the user able to read the script of the remote stack? and if so how can I protect that?
Any thoughts welcome.
Re: Using a remote stack instead of a database to share data
Posted: Mon Aug 12, 2019 3:24 pm
by bogs
My first thought is I hope Andy chime in, since this sounds very similar to his group editing work.
My second thought is that anything can be read if you have the (choose 2 or more)...
* skills needed
* time needed
* determination needed
Stack files are binary, but plain text in them can be read in an appropriate text editor, although I sure don't recommend trying to modify it that way.
In a standalone situation, though, my guess is it would be a little trickier to read a stack if you don't expressly code the program to be able to. You should also be able to (I think) password the stack, or encrypt it.
Re: Using a remote stack instead of a database to share data
Posted: Mon Aug 12, 2019 4:21 pm
by kaveh1000
Thank you Bogs. I have indeed opened a stack once with a text editor, in desperation!
Putting security aside, suppose I have a main stack that is distributed to several users. What I want is that the stack automatically opens the remote stack, then that secondary stack is used in the background. so I can say in the main stack:
Code: Select all
get table 1 of card 1 of stack "remoteStack"
and ideally I don't want that remote stack to show at all but be usable by the main stack. Can that be done?
Re: Using a remote stack instead of a database to share data
Posted: Mon Aug 12, 2019 4:46 pm
by AndyP
First off, if there is any chance, no matter how remote that more than one person will be updating the data at the same time, stick with MySQL otherwise you would have to build some file locking and queuing system which is at some time in the future likely to bite you in the Ar**!
I think we need a little more info.. you talk about exporting data to MySQL, so are you replacing all the database data or just updating the tables with the new/amended data?
Is the data coming from fields or table or a datagrid?
How many users are likely to access the data?
What format is your data?
What format would you like it to be?
It may be that the obvious and therefor simplest solution would be for us to help you with problems you may be having getting data in and out of MySQL. There are a lot of DB experts on the forum who can help, so maybe a breakdown on what you are struggling with in LiveCode/MySQL would be the way forward.
Re: Using a remote stack instead of a database to share data
Posted: Mon Aug 12, 2019 5:27 pm
by kaveh1000
Thank you Andy. Good point about more than one person updating. So what if only one person is updating? So only one read permission. It is a small team so very unlikely we need more than one.
I can certainly stick with MySQL. Just a few lines of data and one data table.
In that case my question to you and others is can I just use the remote stack in place of a database?
Re: Using a remote stack instead of a database to share data
Posted: Mon Aug 12, 2019 8:56 pm
by FourthWorld
Any file can be good for read-only multi-user storage. When a stack file is a good fit for the structure of the data, it can work well.
Writes can be mitigated with locks. For small teams in which concurrent writes are never needed, I use that often.
The only downside to stack files over any other file type is local memory management. With flat or LSON (encoded array) files, knowing what's in RAM is more straightforward than when working with stack files.
Re: Using a remote stack instead of a database to share data
Posted: Mon Aug 12, 2019 10:27 pm
by kaveh1000
Thank you Richard. That is encouraging. Now probably a newbie question. Can I open a stack without it showing up? So user does not even know it has fetched a remote stack.
Re: Using a remote stack instead of a database to share data
Posted: Mon Aug 12, 2019 10:49 pm
by bogs
I think the -ui switch when calling it is what your looking for, but for sure, Richard would know better.
Re: Using a remote stack instead of a database to share data
Posted: Tue Aug 13, 2019 8:00 am
by AndyP
As for security if using the remote stack for storage, just encrypt the data before storing and decrypt when reading, make sure that you are passing data over ssl.
Re: Using a remote stack instead of a database to share data
Posted: Tue Aug 13, 2019 9:15 am
by AxWald
Hi,
if you open a remote stack you're actually creating a snapshot of this stack in your local RAM. As soon as you have done this, someone other can change or even delete this stack, without you noticing.
To change data in this remote stack, you must save your local copy to it, actually replacing it. All changes made meanwhile by other users will be lost, and if someone else is trying the same at the same time, you'll get conflicts.
To circumvent this, you need to refresh your local copy very often while carefully maintaining anything you've done with it already. Further, you need to implement record locks at least, and to write code that resolves conflicts without creating inconsistencies.
Basically, you'll have to write a crude version of a database engine. This doesn't sound like a very good idea.
Have fun!
Re: Using a remote stack instead of a database to share data
Posted: Tue Aug 13, 2019 10:05 am
by AndyP
Ok, I've put together a quick demo of working with data in a remote stack.
This assumes you have access to a lc server.
edit: the key to this is the start using stack command which makes all of the stack available to the server script.
The setup consists of 2 stacks and 1 lc server script file.
RemoteTransfer.livecode
Get Button
Code: Select all
on mouseUp
put "option=" & urlEncode("getMyData") into tMyPostOption
--change to your server address
post tMyPostOption to url "https://andypiddock.co.uk/code/remotedatastack/remotefetch.lc"
put empty into fld "DataTable"
put it into fld "DataTable"
end mouseUp
Update Button
Code: Select all
on mouseUp
put "option=" & "updateMyData" into tMyPostOption
put "mydata=" & urlEncode(fld "DataTable") into tMyData
answer tMyPostOption & "&" & tMyData
--change to your server address
post tMyPostOption&"&"&tMyData to url "https://andypiddock.co.uk/code/remotedatastack/remotefetch.lc"
end mouseUp
this posts commands and data to a lc server script remotefetch.lc
Code: Select all
<?lc
-- turn off error reporting, helps increase security
set the errorMode to "quiet"
put $_POST["option"] into tOption
put $_POST["mydata"] into tMyData
switch tOption
case "getMyData"
start using stack "remotedata.livecode"
put field "DataTable" of stack "remotedata.livecode"
break
case "updateMyData"
start using stack "remotedata.livecode"
put urlDecode (tMyData) into field "DataTable" of stack "remotedata.livecode"
save stack "remotedata.livecode"
break
end switch
?>
which reads and writes to a stack remotedata.livecode sitting in the same server directory.
which is just a stack with a table field.
There is no error checking or encryption in this demo.
Ill leave the lc serverscript and remotedata.livecode up on my server so that you can test.
Re: Using a remote stack instead of a database to share data
Posted: Tue Aug 13, 2019 4:52 pm
by FourthWorld
bogs wrote: ↑Mon Aug 12, 2019 10:49 pm
I think the -ui switch when calling it is what your looking for, but for sure, Richard would know better.
The -ui option is used on the command line to tell a LC standalone to run exclusively as a command-line app for the session, without attempting to initialize a GUI.
Here all that's happening is moving a data file over the network and back again. The file happens to be in stack file format, but the way we manage it is no different from any other file being read from and written to a remote server.
Re: Using a remote stack instead of a database to share data
Posted: Tue Aug 13, 2019 5:02 pm
by bogs
Ahhh, my bad, thanks for the clarification!