Some basic questions about LC server

Are you using LiveCode to create server scripts or CGIs?

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
AxWald
Posts: 578
Joined: Thu Mar 06, 2014 2:57 pm

Some basic questions about LC server

Post by AxWald » Wed Mar 17, 2021 9:56 pm

Hi,

starting to play a bit more with LC Server I've come to some questions I'd like to ask you who are more experienced here.
For clarification, it's about CGI mode - I want to talk to the server via

Code: Select all

   put URL (myServerURL & "test.lc") into myVar  --  returns "Hello world!"
or

Code: Select all

   put URL (myServerURL & "aFunction.lc?data=42") into myVar  --  returns a permutation of 42
(Using GET method here for easy reading. Real data may be POSTed/encrypted.)
The server is contacted from my LC programs on Desktop or Mobile (maybe even from PHP ...), it's NOT meant as a web server.

So far, so fine. Works like a charm. Let's go further. I want to use a lib that I can call from another .lc script. The lib be "alib.lc" & contain:

Code: Select all

   <?lc
   function upme what
   return upper(what)
   end upme
   ?>
I'd call it from "upperme.lc", which is:

Code: Select all

   <?lc
   require alib.lc
   put upme($_GET["data"])
   ?>
This gives a correct result:

Code: Select all

   put URL (myServerURL & "upperme.lc?data=axwald") into myVar  -->  AXWALD
Now the questions.
Q1.) In "upperme.lc" I use "require alib.lc" to specify the lib where my function resides, but "include alib.lc" works as well. I don't really understand the differences - what should I use?

Thinking about specifying the use case leads to another, related question:
Q1.a) I'll have a lot of functions (string/date/table formatting, db access, crypto etc.). In my projects these usually live in libraries - mainly a "base lib" (pure script size ~100 KB) and a "database lib" (pure script size ~20 KB). For good measure let's throw in 30 KB of yet unknown code, makes 150 KB of library code that needs to be read, loaded, kept in memory, unloaded - for each single call to the server!
Assuming frequent calls from any single instance of my project, and quite a number of such instances running concurrently, doesn't this smell like a classic server strangler? (There's other stuff on the server, too, and I don't mean to get it choking ...)

So, how to distribute these functions in libraries to get best speed & least wasted resources?
To keep 'em in lots of tiny script files, or group 'em thematically as best possible, or just throw all in 1 big "lib chunk"? Guess Apache caching will play a role here, too. Any advises?

Q2.) I read about hashBangs. I don't see a real difference - either using "#! /path/to/livecode" or "<?lc ?>" - to me it looks cosmetic. Is there a penalty using "<?lc ?>"? (Additional call to get this info from .htaccess maybe?)
Having "/path/to/executable" hard coded in every single script file looks rather strange to me, when I can have a single entry in .htaccess - but I admit, I'm not a Linux guy & know these to be fond of quite strange things ;-)

Q3.) Important: Will I find the parameters passed around between the scripts (as the "axwald"/ "AXWALD" in my last example above) somewhere in the logs of a plain vanilla Linux Debian web server?
These may be sensitive data and, even if not leaving the web server, shouldn't be logged in plain text anywhere. Anybody knows?

Thx a lot for any answers! Have fun!
All code published by me here was created with Community Editions of LC (thus is GPLv3).
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!

liveme
Posts: 240
Joined: Thu Aug 27, 2015 5:22 pm
Location: down under

Re: Some basic questions about LC server

Post by liveme » Thu Mar 18, 2021 7:33 pm

Hi,
Linux user and DB lover here : hmm..dont know what is your target use case since you mentioned...:
The server is contacted from my LC programs on Desktop or Mobile (maybe even from PHP ...),
it's NOT meant as a web server.
then...
somewhere in the logs
of a plain vanilla Linux Debian web server?
:?: :wink:
so, it will be a web/cloud server ?
to use LC server you are likely to be needing a non shared private server, which cost more than a shared one plan, that usualy cost a "bit" more, did you include this "a bit more" aspect of things in your biz plan beside just the optimizing aspect of things - which I find really worth the time ?

As for security, most ppl likely will want to have a strong back up of their hosted DB from time to time, its likely that these backup - stored who knows where by your hoster - might contains sensitive logs copies - so its hard to avoid this unless you go for no backup at all...weird.

There will always be some risk and if your hacker has found "some way" to acces your DB's logs on your server directory - mmm - he probably also can access your CGI files scripts too... and then read from some DB query directly, That would worry me more than him trying to read some logs since all your DB users PW/info are probably also stored in that DB. 8) Anyway...making sure your server folders dont "leak into public access" is a must.

As for benchmarking queries, only you can find out with real case, put the expected size DB on a server, build your queries, read the test, it will all depend of your HW. configuration (Ram Cpus) and queries (Number of users / weight in Ram), its hard to get a real idea from someone else that might be running on a very dif hardware, dif queries and DB size and Data types...it can always be optimized latter on I d say but try to get as close as possible to a real life case and not just a perfect laboratory test case of a small almost empty DB running...in-house with a single user connected ! :P :P

(*Wont be able to help you on the LC CGI type conex. since I'm not using that. - sorry)

PS: a Non-SQL (MongoDB, Cassandra, etc) vs an SQL type (Postgres or MariaDB) use also brings some speed dif. in queries, which type were you planning on to use ?

Heads up !
Terii

AxWald
Posts: 578
Joined: Thu Mar 06, 2014 2:57 pm

Re: Some basic questions about LC server

Post by AxWald » Sat Mar 20, 2021 3:00 pm

Hi,
liveme wrote:
Thu Mar 18, 2021 7:33 pm
so, it will be a web/cloud server ?
It's the servers of my customers. Shared servers, hosted @ able & trustworthy hoster. The customer has limited SSH access (no real root).
Server backup is outside of our accessible server space, btw.. Restore is done by hoster, on demand.

These servers all run a bare bone PHP ERP with mySQL (soon MariaDB).
Connecting to & extending this ERP/mySQL is what my software does - it's basically an alternative GUI, with vastly improved functionality, done in LC, used by the employees of my customers, on desktop or mobile.
So, a shared "plain vanilla Linux Debian web server" is what I have to work with. And it's my LC Server try-out, that runs on the former, that's "NOT meant as a web server".

A reason that I took it on to set up LC server (besides sheer curiosity ...) was that I wanted to get some stuff automated (backup, cleanup etc.). Throwing shell calls at the server is much more easy when able to use LC instead of PHP, for me.
For sure, I have found other uses immediately :) - but to follow these possibilities I'd like to know more, before starting to chase wild geese.

Some more:
I assume I'll be able to harden my cgi-bin & my dbs sufficiently. Wouldn't make much sense else even starting, right?

And you'll never find any user credentials in any db of mine - at least not in a form that can be guessed in my remaining lifetime.

As for benchmarking, I'm not concerned about the db. It's about the LC server - I have tested a bit vs. PHP 7.2 - LC 8.1 is slower, but not much. For simple tasks (as shown in my prev post) it's in the range of a few (= 1-digit) millisecs per call. Tolerable.

NoSQL dbs? I have structured table based data that already prosper in a SQL db. I don't use XML, JSON or [insert preferred buzzword]. I see no use case for NoSQL.


Due to the overwhelming amount of responses by the obviously huge numbers of LC server developers I'll repeat a tl;dr version of my questions:

Q1.) "require" vs. "include" when binding libs - what to use when?
Q1a.) How to organize these libs? (Regarding server load)

Q2.) HashBangs - any penalty not using 'em?

Q3.) Does a standard Linux/ Apache web server log the communication with/ between CGIs
and if, where? *)

Have fun!

*) A possible answer would be a grep cmd doing this:
- search all files the SSH user can see (= up to 100GB)
- find every file containing "[searchTerm]", even within archives.
- list these files (with path) in a "results" file
- and while doing this, ignore anything that matches *.sql or *.pdf
assuming grep wouldn't eat up all server resources and freeze the server for some hours ...
All code published by me here was created with Community Editions of LC (thus is GPLv3).
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!

Post Reply

Return to “CGIs and the Server”