Techniques for Developing for Server

[edgore]

I am curious to hear what people are doing when it comes to developing for Server.

Since I am not using On-Rev for my stuff I have been doing all my development in the desktop IDE and then copying and pasting things into server . lc scripts once I am done. However, I keep getting strange errors and differences in the output between the two, so I am hoping that by talking about some of this from a technique/process standpoint we might identify and document some of the traps.

Some things I do today:

I generally start with a blank stack and create a field/fields that mirror the fields my application web page. I make no attempt for this to actually look like my page or anything, it's just the data fields and the submit button stuff that I am trying to duplicate, without using HTML/get/post/etc.

Then I create a button that contains the script that would be in my submit cgi (a .lc file) This also contains all of the environment setup stuff that, on the server, would normally be a separate include - it's easier to put it here than to call it as a function or something.

I put all of my library stuff into the stack script as functions. This is all code that when I move to the server is actually in one or more includes that are called from my submit cgi.

I try to separate out as much of the static HTML as I can into text files that my .lc CGI loads and use merge or replace to put my dynamic data into the template.

On the server the CGI would spit back out an HTML page with my results to the browser using the put command, but when I am working on the desktop I just have the CGI spit the raw HTML in the message box (so I don't have to edit the put command later when I deploy). Makes it harder to read while developing, and I suppose I could also create an output field in my stack that I set the htmltext of.

Once I have the livecode portion of the web app working, I start to work on porting it to the server. This involves putting together the web pages, putting in all the get and post stuff that is specific to a web implementation of what I have in the desktop IDE, copying the scripts out of the IDE and commenting out anything that was desktop specific and commenting in anything that was server specific.

If I am really lucky, then when I access the apps web page through the browser I can enter data into the web page, click submit and get a result that duplicates what I had on the desktop. Usually though, I don't and I end up having to spend the next couple of days adding debugging code to the server scripts so they display the values of certain variables during execution so I can see where things are going wrong. Sometimes it's because I made a mistake in the stuff that had to be edited/added to migrate for desktop to server, sometimes it seem to be because server is actually doing something differently than the desktop engine did, so I have to code a server specific work around that becomes another thing to comment in/out when moving between desktop and server.

What I am not sure about is where my technique might be messing me up. For example, is there a difference in the way an include operates versus a bunch of functions in a stack script? There doesn't seem to be in practice, but I am not sure.

What are other people doing? Are there third party tools that you have found useful for working with server? Are there any tips that anyone might have for using Ajax to give webpages some of the dynamic interaction and feedback that normally comes with a livecode desktop application? For example, I have a web app that performs several queries and takes a while to complete and I would love to provide the user with a status bar that updates as the background stuff goes on and I am sure Ajax provide the ability to do this, but I haven't had the time to teach myself javascript and really learn about Ajax yet.

Anyway, I just wanted to sort of open up this topic for people to add to There has been some discussion about his in other threads, But I didn't see anything dedicated to tip and tricks about developing and debugging for deployment on server.

[andyh1234]

What you are doing sounds good, the main thing I've found is to make sure the version of Livecode you are using on the desktop matches the server version, I've had a few scripts fail only to find they worked perfectly as soon as I upgraded the Livecode engine on the server.

I use ajax and live code together all the time now, Jquery makes it all quite easy and if you take any php script that is used to handle ajax interaction, converting that into live code is usually very quick and easy.

[edgore]

That's what has been driving me crazy, I have even tried using 6.1 community server and 6.1 community on the desktop just to make sure I am in sync (even though I have a commercial license) and I still get strange situations where server gives a different result than desktop. I have been able to code around it all, so far in a way that results in something that works on both (with code that removes stray spaces, etc that don't impact the desktop since they aren't there), but I worry that at some point I am going to run into something that requires forking my server and desktop stuff, which will be...bad.

I really need to figure out jQuery and reteach myself enough PHP to steal stuff...

[FourthWorld]

That's what has been driving me crazy, I have even tried using 6.1 community server and 6.1 community on the desktop just to make sure I am in sync (even though I have a commercial license) and I still get strange situations where server gives a different result than desktop. I have been able to code around it all, so far in a way that results in something that works on both (with code that removes stray spaces, etc that don't impact the desktop since they aren't there), but I worry that at some point I am going to run into something that requires forking my server and desktop stuff, which will be...bad.

One of the many reasons I've moved so much of my development to Linux is so I have the same OS working locally as on the server. It's taught me a lot about both, and reduces the number of differences between the two.

That said, I recognize that isn't a solution for everyone, so as you come across differences please feel free to note them here. A a minimum we may be able to find solutions for coping with them, and perhaps they can inspire a library to help smooth out the workflow of moving scripts to the server.

[edgore]

Hey Richard, I've probably mentioned this before, but these varying results are actually occurring on the same machine, running the same version of desktop and server, with the webserver/server engine at localhost. I would file bug reports, becasue they seem to be consistent problems for me, but I am seeing these in situations that are difficult to reproduce - shelling out to make an LDAP query against an internally hosted LDAP directory using the Windows version on the opensource ldapsearch utility, or using the (license purchase required) IBM DB2 ODBC drivers to query against an internally hosted DB2 database. I figure that since they are such weird things to start with, expecting RunRev to duplicate them is probably a pipe dream.

Still, I will try to post here with the particulars of each situation as they come up.

[edgore]

Okay, here is a situation where I getting strange results from server, but not in desktop. Unfortunately, getting it to happen is complicated, and requires things in addition to livecode to be installed. This actually came about because I was trying to document another error, but this is a good example of the weirdness I see...

Desktop and server are running on the same machine, and I have the same version for server and desktop (6.1), so there can't be anything very environmental about this. The only obvious difference I can think of is that the code shown below is a function in the stack script on the desktop, while on the server it's in a required file (which is in a directory apart from the script taht calls it - the files mentioned below all get created in the directory of the calling script, which is what I would expect)

So, I do an ldap lookup by shelling out to ldapsearch (an opensource ldap utility) - the code is complicated because I have to deal with an unknown number of ids and break them up to fit them on a command line. Unfortunately this specific LDAP search probably won't work anywhere else, since it's specific to my environment - I have also blocked out some stuff.

Code: Select all

   repeat for each line theID in theIDList
      put searchType & theID & ")" after theCommand
      if the number of chars of theCommand > 4000 then
         put theCommand & ")" & quote & " givenName nickname sn uid title telephoneNumber mail manager" into theCommand
         put shell(theCommand) & return after LDAPResponse
         put "C:\ldap\ldapsearch -x -b " & quote & "ou=people, o=xxxx, c=US" & quote & " -H ldap://directory.xxxx.com:389 " & quote & "(|" into theCommand
      end if
   end repeat
   
   if theCommand is not "C:\ldap\ldapsearch -x -b " & quote & "ou=people, o=xxxx, c=US" & quote & " -H ldap://directory.xxxx.com:389 " & quote & "(|" then
      put theCommand & ")" & quote & " givenName nickname sn uid title telephoneNumber mail manager" into theCommand
      put LDAPResponse & shell(theCommand) & return after LDAPResponse
   end if
   
   put LDAPResponse
   put LDAPResponse into url("file:LDAPTest.txt")
   
   put "test" into testData
   put testData into url("file:Test.txt")

So, in the scenario above, on the desktop the variable LDAPResponse contains

Code: Select all

# extended LDIF
#
# LDAPv3
# base <ou=people, o=xxx, c=US> with scope subtree
# filter: (|(uid=111111))
# requesting: givenName nickname sn uid title telephoneNumber mail manager 
#

# 111111, people, xxx, US
dn: uid=111111, ou=people, o=fedex, c=US
givenName: Bob
sn: Smith
uid: 111111
title: Project/Process Advisor
telephoneNumber: 555-555-5555
mail: bob.smith@xxx.com
manager: uid=111112, ou=people, o=xxx, c=US

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

And that is what get's written to the file LDAPTest.txt, as well. The lines for writing "test" to text.txt are just there to validate that I have the permissions to write, as you will see.

On the server the "put LDAPresponse" line throws up something similar enough into the browser that I am willing to accept that the variable contents are the same on server and desktop (you know, formatting in the browser makes it hard to be sure - extra spaces, returns, etc).

However, on the server, the file LDAPTest.txt is created, but is empty. The file test.txt however, is created just fine, and contains the word "test", just as you would expect.

So, why does the desktop version write the value of LDAPResult out correctly, while server obviously has the value (or *some* value at least) but will not write it to a file? Why one variable, but not the other? The code is just copied and pasted between desktop and server- no edits or anything.

It's maddening.

[edgore]

And here is another one, the one that led to me finding the strange behavior documented in the last post (I tried saving LDAPResult to a text file so I could see what was actually in it, without interference from browser formatting). Once I have the LDAPResponse I do some processing to make each entry in the response into a single, comma delimited line.

On the desktop I get this data back from LDAP:

Code: Select all

# extended LDIF
#
# LDAPv3
# base <ou=people, o=xxx, c=US> with scope subtree
# filter: (|(uid=111111))
# requesting: givenName nickname sn uid title telephoneNumber mail manager 
#

# 111111, people, xxx, US
dn: uid=111111, ou=people, o=fedex, c=US
givenName: Bob
sn: Smith
uid: 111111
title: Project/Process Advisor
telephoneNumber: 555-555-5555
mail: bob.smith@xxx.com
manager: uid=111112, ou=people, o=xxx, c=US

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

I run it through this script:

Code: Select all

   --remove all the extra cruft from the responses, and turn them into comma delimited lines seperated by returns
   filter LDAPResponse without "#*"
   filter LDAPResponse without "search:*"
   filter LDAPResponse without "result:*"
   filter LDAPResponse without "dn:*"
   replace ", ou=people, o=xxx, c=US" with empty in LDAPResponse
   replace ",ou=people,o=xxx,c=US" with empty in LDAPResponse
   
   --before breaking the entry up into comma separated items, remove any commas in the data
   replace comma with space in LDAPResponse
   
   --process the entry so that if there is a preferred name ("nick") it will be displayed rather than the formal name
   replace return & "given" with "given" in LDAPResponse
   replace return & "nick" with "nick" in LDAPResponse
   put replaceText(LDAPResponse,"givenName:(.*)nickname: ",empty) into LDAPResponse
   
   -- more cruft removal and inserting commas between data elements
   replace "givenname:" & space with empty in LDAPResponse
   replace return & "sn:" with empty in LDAPResponse
   replace return & "uid:" & space with comma in LDAPResponse
   replace return & "title:" & space with comma in LDAPResponse
   --the ~ before the manager ID is used to distinguish the manager field from others with the same data
   replace return & "manager:" & space with comma  in LDAPResponse
   replace "uid=" with empty in LDAPResponse
   replace return & "telephoneNumber:" & space with comma in LDAPResponse
   replace return & "mail:" & space with comma in LDAPResponse
   
   --remove any empty lines
   filter LDAPResponse without empty
   if the last char of LDAPResponse is return then delete the last char of LDAPResponse

Which results in this final result, on the desktop:

Bob Smith,111111,Project/Process Advisor,555-555-5555,bob.smith@xxx.com,111112

On server this is what LDAP result looks like if "put":

Code: Select all

# extended LDIF # # LDAPv3 # base with scope subtree # filter: (|(uid=111111)) # requesting: givenName nickname sn uid title telephoneNumber mail manager # # 111111, people, xxx, US dn: uid=111111, ou=people, o=xxx, c=US givenName: Bob sn: Smith uid: 111111 title: Project/Process Advisor telephoneNumber: 555-555-5555 mail: bob.smith@xxx.com manager: uid=111112, ou=people, o=xxx, c=US # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 

and this is the final result that I get:
Bob Smith ,111111 ,Project/Process Advisor ,555-555-5555 ,bob.smith@xxx.com ,111112 (there is a space at the end of this line)

As near as I can tell, on the server each line in LDAPResponse ends with a space. Since I am running exactly the same shell command, on exactly the same machine, this makes no sense.

EDIT: The stray character is not a space, it's a carriage return (char num 13). So, I assume that this means that the issue is that the server engine doesn't do the same kind of line delimiter magic that the desktop does? Is the what was causing the file write issue above too, I wonder? Though, why would it write nothing to file because it encountered a carriage return? I think that now at least I know enough about what is going on the report a bug on this issue. Keep in mind, this is all on the same machine, so it's the Windows engine in both cases.

[edgore]

Having just found out that the environment function can return "server" (it's not in the docs) I will using this going forward in my scripts. By checking the environment I can use conditionals to have different sections of a script execute or not depending on whether it's running desktop or server. This will eliminate the overhead I have now in commenting stuff in or out - I can use the same script without changes. Just figured I would put this here as a repository of helpful tips when developing for server.

[edgore]

When you are developing on the desktop for eventually deploying on the server it's very easy to remember to get rid of the on mouseUp and end mouseUp at the beginning of your desktop development handler and replace them with <!lc and !> that are needed on the server. However, don't forget that if you have an exit mouseUp in the middle of your script you need to do something to make things work differently on the server. Right now I am using:

Code: Select all

 if the environment is "development" then exit mouseUp
if the environment is "server" then exit to top

there might be a better way to do this - for example, I could just use exit to top in both cases, but I don't like jumping all the way out on the desktop if I don't have to. I don't know of a way on the server to just exit out of the current handler if the handler doesn't have a name, which a handler in a .lc file doesn't have, as far as I know.

If you don't do something to handle this, and the script attempts to execute the exit mouseUp on the server you will get a 500 error in the browser, and a very unhelpful entry in your error log on the server, which can take a while to puzzle through.

Hope this saves someone some trouble.