Best settings for a dedicated server

[andyh1234]

Ive installed the Livecode server on a dedicated server, everything runs great 'most of the time'.

However, every so often when I ask a bit more of the engine it appears to lock up, the rest of the server works fine but anything requested via a .lc webpage just stalls.

As an example, I wrote a small script to check a user has a cookie set, then if they have load a jpeg from the disk, set the appropriate header and send it to the user. For 1 or 2 images this works, but if I load 10 images onto a page, the web page calls 10 instances of the server at once all of which try to access the disk to send the image back and everything locks.

Are there any settings I can make on the server to give the engine more power, like with php you can set timeouts and the number of concurrent sessions allowed etc?

Thanks

[andyh1234]

Just to add, from the log files the error seems to be..

[warn] [client x.x.x.x] Timeout waiting for output from CGI script /home/xxxxxx/public_html/cgi-bin/livecode-server

[bangkok]

Fascinating. I've never experienced this problem, but I never used LC server script with image files.

What kind of "dedicated server" are you using ?

Perhaps you should publish your script, so we could make test.

I have 2 LC Server : Linode machine (dedicated), and mutual hosting server.

[andyh1234]

As you say, until right now ive had no problems at all, and find the lc server is great.

Im running this script on my dedcated server, I also have an on-rev account but I dont want to try it on that in case it takes down that too (how to not be popular in 5 minutes flat!)

No problem with the script, the bit that will take livecode down on my server is...

Code: Select all

put $_GET["img"] into tFilename

put tBasePath & "/img-review/" before tFileName

if there is a file tFilename then
	put header "Content-Type: image/jpeg" 
	put "binfile:" & tFileName into tURL
	put URL (tURL) into tBinData
	put tBinData 
else
        put "No image"
end if

If you then add a html page with say 10 image requests that call the server, livecode will server the first couple and then timeout. I can kill the processes on the server (about 8 start) and it will then recover, but unless I kill them all the livecode-server requests will no longer work at all.

The server is a dedicated hosted at cloudweb, ive been with them for years as they are brilliant at responding to any issues, but this time I dont think they problem is with the hardware but maybe the settings in apache that is not giving livecode enough room to work when put under pressure. The actual server loads are close to 0, its got very little else on it at the moment.

[sturgis]

My gut feeling is that there is a memory issue. (lack of memory)

Each instance loads the entire image into memory and then holds it there while being sent.

I'm not sure how it should be constructed, but you might try using open file instead, and read the files in chunks rather than slurping the whole image file up at once.

Worth a check at least.

[andyh1234]

I think you have it!

There is plently of memory in the server, so I thought it must be a config settig.

I found a topic at:
http://forums.cpanel.net/f5/disabling-r ... 67726.html

Ive disabled the RLimitMEM option in apache, and so far so good, a page with 20 large image files is loading in super speedy time.

Ill push it to the limit and see how many images it now takes to crash it, but the current setting enables everything ill need for now.

Thanks for pointer in the right directly.

[sturgis]

Very cool!

Depending on how hard you expect your server to get hit you might still consider using a read loop instead, but as always it depends completely on your needs.

Do you mind if I ask what you're developing? I hope not, cause I just did. *grin*

EDIT:

Just realized. You can change your current script slightly to improve things while keeping the basic method the same.

Code: Select all

put $_GET["img"] into tFilename

put tBasePath & "/img-review/" before tFileName

if there is a file tFilename then
   put header "Content-Type: image/jpeg" 
   put "binfile:" & tFileName into tURL

   #put URL (tURL) into tBinData -- change this line so you don't use a holding var
   put URL (tUrl) -- just put the url directly, no need for an intermediary variable

   #put tBinData -- remove this line entirely
else
        put "No image"
end if

[andyh1234]

Thanks mate, I appreciate that.

I cant really say on a public forum what im working on at the moment, but its my biggest web project to date and its looking good so far. Id hope to have it all live in around 6 weeks from now.

This particular routine was needed so I can allow users to create a listing on the site and preview it before publishing, but I wanted to ensure any images that were uploaded were only available to the user uploading and all public published content is manually reviewed. By using the sessions I can check the user is logged into the admin system and feed the images via a livecode script, if the session is not found the user gets a placeholder image instead.

Once approved and on the public server there is no issue as the images are direct referenced so the major load is not an issue, there is just the load while the user is editing the images in their listing.

Im really pleased with what ive managed to get done and how little time everything has taken. Its just the odd road bumps like this that throw a wobble in there, but touch wood everything is good now.

[sturgis]

EDIT: Hmm I answered the 'incredible disappearing message!' If you figured it out, let me know and I'll remove this response to no question.

Well, my next question is this.. Bandwidth is not an issue? Meaning when you hit overload on the server, do you have enough bandwidth or are they timing out and hanging?

Next question.. You say you're doing this so that sessions etc either get a placeholder graphic or actual graphics. And you have the filenames of the graphics available. rather than start up 20 sessions of livecode each taking up mem, hitting the cpu, can you just have your main lc page loop through and create <img src=> tags based on your requirements?

Or are you not displaying them using img tags? (but you see what I mean)

so if you have 10 images named 1.jpg to 10.jpg (just an example)

Can you loop 10 times and just build the html to display them? If you're trying to obfuscate the actual files then yeah I understand why you're doing things this way.

Hard to know exactly how to help without a better idea of the goal. As for 20 instances crashing it, still gotta think there is something strange going on here. 20 separate instances shouldn't be an issue, so it must have something to do with timeouts, bandwidth, memory, or cpu locking (it seems to me), but you've checked those things so i'm still at a loss.

In the past I have managed to drive a server to its knees using backend lc server for multi-processing. At least thats what I thought was happening. The problem actually turned out to be a problem with how I was handling the returned data. There was a tremendous amount and I was running both the server and app that was sending requests to it on the same machine (to take advantage of extra cores since lc is single threaded) I ended up with SO much stored in memory that I caused some "minor" difficulties.

So I guess what i'm saying is, check your logs, check everything you can, try different methods and see if you can find the actual fail point. (I'm still with memory, maybe you have a bad stick and push hard enough to reach the trouble area?) Either way, time to start looking at logs, and as a test run you might try the read loop method just to see if it makes a diff. I have a server set up here also, if you don't mind throwing together a quick sample of JUST the code that causes this so that I can run it on my server here I'll see if I can make it fail, or succeed. (think I already have your code snippet here somewhere, but it has yet to fail on me, so a more complete example would help (including a basic calling page)

Either way, you're correct. With enough mem and power in the machine, sending 20 images should be a non-issue assuming things are set up correctly. I haven't messed with apache setup recently enough to remember many of the options, but i'll dig into those again too and see if anything clicks.

If you do 20 requests without the whole image thing (IE just start 20 random scripts that say.. generate a list of numbers or whatever) do they lock up then too?

[andyh1234]

Im sorry about that, I have not got to the bottom of it yet but pulled the other message just after posting it as I thought Id try to get a lot more info before wasting anyones time in case it was something I was doing wrong, or it was a config issue on my server.

Yes, im trying to ensure users cannot see these particular images unless logged in. The reason for this is that users can upload the pics to create a listing on the site, but I was concerned they might upload some unsavoury content and if this was placed in a public area it could be shared everywhere.

By using a script to check a login cookie has been set for the listing, and have livecode read the files and output them directy when called from within an img tag only the poster of the images would be able to see them until the images had been reviewed.

The users are limited to 20 images, and there are unlikely to be more than 1 or 2 users logged in at any one time so this really should not be a performance issue at all, but its flooring livecode on the server (everything else continues fine).

Its odd as everything else is working really well, even on busy areas of the site, it seems to be something to do with a concurrent and instant call to over 20 live codes sessions which all want to read data from the drive and return it, one will stall and all the others then stack up behind it.