(libSOAP) 3rd party library best practice

[Clarkey]

Hi folks, Please can someone clarify how best to work with libraries such as libSOAP - especially where it comes to passing variables?

I'm using Revolution Enterprise and am trying to implement the libSOAP library to manage HTTPS SOAP web service calls within a project. I currently have it set under my main stack, with an on preOpenStack command to start using stack "libSOAP". The libSOAP library stack uses a custom property template to build the SOAP message structure, together with various local variables (within the libSOAP stack) to replace elements within the template when the SOAP function is called.

I then have a separate card in my project's main stack that manages calls to the web service and has fields and associated local variables defined for user credentials and the details of the SOAP calls - URL, method, namespace, action, etc. On this web service card, I have created a login button, which includes the libSOAP functions SOAP.AddParam() to (hopefully!) pass username and password to the library's variables and then SOAP.RPCRequest() to post these to the web service.

Here's where I'm confused. The libSOAP example code has the SOAP.AddParam() functions writing to local variables. My understanding is that local variables are script-specific - so how do variables defined in my web services script and the libSOAP library's script interact? Also, how can I debug the calls to the library when its variables are defined outside the script I am stepping through?

This is my first attempt to wok with a library, so I may well have missed something fundamental here.
Any insights gratefully received.
Best,
Keith..

[Clarkey]

I've already learned a great deal from this community's content, so I thought share what I learned on the way to solving this - for any other fools who might be trying to learn development, Revolution, the use of libraries, scripting languages and SOAP at the same time - not recommended!

The key issues were around:
1. The Revolution message path and how libraries augment this, which is explained very well here, with essential diagrams http://www.fourthworld.com/embassy/arti ... _path.html and this lesson also helped http://lessons.runrev.com/spaces/lesson ... er-object-
2. Script naming conventions for different variables, constants and parameters, to know what I was looking at http://www.fourthworld.com/embassy/arti ... style.html.
3. What the target web service was expecting to receive - testing with http://ditchnet.org/soapclient/ helped significantly here.

So, having managed to login via https using the libSOAP library, I'll now see if I can actually get some data!
Best,
Keith..

[mwieder]

Keith-

Is that my libSOAP library you're looking at (the one from RevOnline)? If so, let me know if there's anything specific you need help with. It seems you've gotten a fair ways along as it is, since dealing with https web services was the hard part as I was creating the library.

And yes, you're definitely in the deep end of the pool. <g> Might be better to start with something simpler if you're just starting out with rev - the learning curve is steep enough as it is. Congrats on getting this far.

[Clarkey]

Is that my libSOAP library you're looking at (the one from RevOnline)? If so, let me know if there's anything specific you need help with. It seems you've gotten a fair ways along as it is, since dealing with https web services was the hard part as I was creating the library.

Hi Mark, Thanks for responding and yes, it is indeed your library - and thanks for doing the heavy lifting with updating the old library.

It turned out that sorting the login script - and receiving a "SessionID" for subsequent calls was only the first of a two part logon security challenge for accessing the Salesforce.com API! The second part required two features that I couldn't find (or bend) from the current library:

1. Declaration of additional name servers: into the SOAP Header before the "<SOAP-ENV:Header>" tag (e.g. xmlns:urn="urn:partner.soap.sforce.com").
I couldn't see an obvious way to add nameserver bindings, so I created a variant "SOAP.RPCRequestSforce()" function, with a modified custom property template. This currently has the above Salesforce name server hard-coded but depending on how many name servers I find I need to add (and how widespread this need is for HTTPS connections in general), it might warrant a 'SOAP.AddNameserver()" function addition to the library.

2. Structuring the SOAP authentication header: to create the nested element structure needed for the Salesforce SessionID token from the login call
For now I have defined this in the pHeader definition 'around' the standard "SOAP.AddParam()" of the calling script but if it proves standard for all Salesforce calls, it could migrate to the custom properties.

Feel free to correct my 'hacks' if I've missed some standard libSOAP features - I'm still learning this stuff.

[Clarkey]

Is that my libSOAP library you're looking at (the one from RevOnline)? If so, let me know if there's anything specific you need help with. It seems you've gotten a fair ways along as it is, since dealing with https web services was the hard part as I was creating the library.

Hi Mark, Thanks for responding. Yes, it is indeed your library - and thanks for doing the heavy lifting with updating the old library.

It turned out that sorting the login script - and receiving a "SessionID" for subsequent calls was only the first of a two part logon security challenge for accessing the Salesforce.com API! The second part required two features that I couldn't find (or bend) from the current library:

1. Declaration of additional name servers: into the SOAP Header before the "<SOAP-ENV:Header>" tag (e.g. xmlns:urn="urn:partner.soap.sforce.com").
I couldn't see an obvious way to add nameserver bindings, so I created a variant "SOAP.RPCRequestSforce()" function, with a modified custom property template. This currently has the above Salesforce name server hard-coded but depending on how many name servers I find I need to add (and how widespread this need is for HTTPS connections in general), it might warrant a 'SOAP.AddNameserver()" function addition to the library.

2. Structuring the SOAP authentication header: to create the nested element structure needed for the Salesforce SessionID token from the login call
For now I have defined this in the pHeader definition 'around' the standard "SOAP.AddParam()" of the calling script but if it proves standard for all Salesforce calls, it could migrate to the custom properties.

Feel free to correct my 'hacks' if I've missed some standard libSOAP features - I'm still learning this stuff.

[mwieder]

Thanks - been a while since I've had to deal with this thing and I haven't run into a two-part login challenge before (yikes!), so...

#1 is an excellent idea. I'll look into how to shoehorn a SOAP.AddNamespace() function in there... it's not going to be straightforward, so your best bet right now is probably what you've done - changing the custom property.

#2 - I had to do nested parameters as well, and the SOAP.AddParam() function is designed to do that, if I'm understanding what you're doing properly. If you look at the help example, there are two lines there:

Code: Select all

put SOAP.AddParam("Key", tPartnerKey, "string") into tHeader
put SOAP.AddParam("AuthenticationHeader", tHeader, "", kWSNamespace) into tHeader

The second call to SOAP.AddParam() takes tHeader from the first call and encapsulates it, so you end up with

Code: Select all

<AuthenticationHeader>
  <Key>
    tPartnerKey
  </Key>
</AuthenticationHeader>

[Clarkey]

#1: Salesforce seems to need more than 1 additional namespace for some calls. So, given the variability of possible interpretations/implementations involved, perhaps it might be best to leave the complexity in the calling script. How about placing a "$ExtraNamespace" placeholder in the custom property, together with a mechanism on the SOAP.RPCRequest (et al) to pass through a pExtraNamespace parameter. Another alternative might be a variant on the SOAP.AddParam mechanism.

#2: Truly enlightening! I was so deep in the muck & bullets that I had failed to notice that SOAP.AddParam was actually doing encapsulation - and hence, could be iterated. I've made this change.
Best,
Keith..

[mwieder]

<g> I made AddParam work that way because I needed it to have multiple levels for the authentication mechanism.

The namespace thing is a bit more bothersome: I could put a placeholder in the custom prop, but then what happens when you need *two* extra namespaces? I think a SOAP.AddNamespace() function would be better, but where to place it in the workflow? I'll obviously have to change the RPC stuff, because right now it goes straight from grabbing the custom property to doing some basic replacements and posting in a single handler.

[mwieder]

Keith-

If all went well with my upload I just posted a new version of the library to revOnline with a SOAP.AddNamespace() function and an optional parameter to SOAP.RPCRequest() to add the namespace before making the call. It seems to work over here, but I don't have a real-world example to test it on. Feel free to kick the tires.

[Clarkey]

Mark,
Thanks for putting the effort in on this. I've had a quick play but I'm not sure how to manage the tSOAPEnvelope variable and how, where or when to define the pSOAPEnvelope parameter when calling the stack as a library?!? This may just be me being dumb and/or a lack of coffee!

[mwieder]

Try it now. I've been having trouble with revOnline. I *think* the latest upload worked. If so, there are now two extra parameters you can add to SOAP.RPCRequest: a comma-separated list of namespaces and a comma-separated list of namespace identifiers, as in

"http://www.google.com/authorities/,http ... com/hello/", "abc,xyz"

which should result in

xmlns:abc="http://www.google.com/authorities/"
xmlns:xyz="http://hello.sailor.com/hello/"

in the SOAP envelope.

[Clarkey]

Ah, OK - quite different to what I downloaded earlier (I too have big problems with Rev Online - I don't think I've been able to login through Revolution at all since I got involved with the product, a week or so ago).

If I understand this new approach correctly, the SOAP.AddNamespace() function is different to the SOAP.AddParam() in that it is a private function - working behind the scenes within the library stack, to support the revised SOAP.RPCRequest() function. So, I can ignore its usage notes and Instead, just state the pAuthority and pAuthId parameters directly in the SOAP.RPCRequest() function call. (If this is correct, maybe the function should be renamed as "_SOAP.AddNamespace()", for clarity?)

So, I have dropped the custom property from the line above the <soapenv:Header> tag (xmlns:urn="urn:partner.soap.sforce.com") and added the following variables in my script...

Code: Select all

   put "urn:partner.soap.sforce.com" into tSforceAuthority
   put "urn" into tSforceAuthId 

The main function call now adds the Namespace specific parameters...

Code: Select all

put SOAP.RPCRequest (gSforceServerUrl, tSOAPMethod, gSforceNamespace, tParam, tSOAPAction, tHeader, tSforceAuthority, tSforceAuthId)\
      into tSoapMessage

Unfortunately, this is creating errors (but I don't have a way to trap the outbound SOAP message or a debugging insight to see what's happening in the library stack).
Any clues?

[mwieder]

Well, you're using the function correctly, but I'm concerned about the parameters. It looks recursive to me, from my limited understanding of SOAP namespace identifiers. What you're doing will indeed give you the line

xmlns:urn="urn:partner.soap.sforce.com"

in the header, but I think it should be something like

xmlns:urn="http://partner.soap.sforce.com"

If you want to test the AddNamespace function, try typing this in the message box:

Code: Select all

put SOAP.AddNamespace(the uSOAPEnvelope of stack "libSOAP", "http://partner.soap.sforce.com", "urn")

That will show you the modified envelope with the additional namespace.

[Clarkey]

Thanks for the tip on the message box - I'd not discovered the input side of this before or indeed, sorted the syntax to work across cards! I'm really still only scratching the surface with Revolution.

I really comment on namespace syntax, though I'm sure Salesforce's adoption of standards is no less 'standard' than many other major players! Still, for my project, I have no choice but to accept the API as fixed. So I'm attempting to replicate example working code from the Salesforce developer documentation. FYI, here is such an example - in all its glory, with multiple additional namespaces, nested headers and nested parameters! ...

Code: Select all

POST https://na1.salesforce.com/services/Soap/u/10.0 HTTP/1.1 
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol
1.1.4322.573) 
Content-Type: text/xml; charset=utf-8  
SOAPAction: "" 
Content-Length: 510 
Expect: 100-continue 
Host: na1.salesforce.com 

<?xml version="1.0" encoding="utf-8"?>   
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:urn="urn:partner.soap.sforce.com"
  xmlns:urn1="urn:sobject.partner.soap.sforce.com">
  <soapenv:Header>
     <urn:SessionHeader>
        <urn:sessionId>QwWsHJyTPW.1pd0_jXlNKOSU</urn:sessionId>
     </urn:SessionHeader>
  </soapenv:Header>
  <soapenv:Body>
     <urn:merge>
        <urn:request>
           <urn:masterRecord>
              <urn1:type>Account</urn1:type>
              <urn1:Id>001D000000HSDfA</urn1:Id>
              <!--You may enter ANY elements to update at this point-->
              <Description>Merged with Dupe Account.</Description>
           </urn:masterRecord>
           <urn:recordToMergeIds>001D000000HSDf9</urn:recordToMergeIds>
        </urn:request>
     </urn:merge>
  </soapenv:Body>
</soapenv:Envelope>

So, here's what I get from the message box if I insert the namespace code that I currently have hard-coded in my current (working) custom properties work-around...

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:urn="urn:partner.soap.sforce.com"
>
<SOAP-ENV:Header>
<$HEADER>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<$METHOD><$PARAMS/></$METHOD>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The only difference I can see is the carriage return after the additional Namespace. I wasn't sure if this was important but I tried adding the following IF statement to the loop on the SOAP.AddNamespace() code to prevent this return after the final additional namespace. This sorted the layout but I still get errors, which is weird.

Code: Select all

 -- find the insertion point
    get matchChunk(pSOAPEnvelope, "(?Us)<SOAP-ENV:Envelope.+(>)", tStart, tEnd)
    if pNamespaceNames is not empty then
       put cr into tParam
       put the number of items in pNamespaceNames into tNamespaceNameCount
        repeat with x=1 to tNamespaceNameCount
           put "xmlns:" & item x of pNamespaceIDs & "=" & quote & item x of pNamespaceNames & quote after tParam
           if x <> tNamespaceNameCount then
              put cr after tParam
              end if
            -- this should leave tParam with something like

There may be other nuances in the Salesforce SOAP implementation that are buried in calls that I've yet to use, but the custom properties approach at least provides me with a solution to deliver my project. Also, I don't want to waste your time attacking piecemeal, Salesforce-specific issues that don't add to the generic libSOAP solution. So, whilst it would be good to get a library that covers all eventualities, it may be better to if I continue with the custom properties approach for now - to flush out all the issues - and then consider the clean-up options later?

[mwieder]

Quick reply: if you can get things going with a modified custom property, then by all means do that. No need to hold you back trying to get some of my code working.

The extra carriage return shouldn't matter. It's just xml, which ignores things like that.

But thanks for the salesforce example. That makes things much clearer. Now I see:

There are actually two additional namespace arguments (for urn and urnl namespaces). To deal with this, the easiest thing to do is to place them into variables and pass the variables:

Code: Select all

put "partner.soap.sforce.com,sobject.partner.soap.sforce.com" into tURLs
put "urn,urnl" into tUrlIDs

Then the following call should add both additional namespaces:

Code: Select all

put SOAP.AddNamespace(the uSOAPEnvelope of stack "libSOAP", tURLs, tUrlIDs)

and the same should also work for the namespace parameters to the SOAP.RPCRequest call.

I also see salesforce is using the "soapenv" naming convention instead of "SOAP-ENV". I don't know whether or not this matters. They're using the same namespace authority to define it, so my guess is that it should pass.