Foreign Handlers and Custom Types

[n.allan]

I eventually got my head around some of the basics of foreign handlers in lcb. I can now successfully call some functions from the windows api which is really cool! I also went back to one of my own externals and wrapped it without too much trouble (also really cool). However this involved passing and returning only the primitive types (bool, int, float, real, char * etc...)

How do we plan to handle the passing of structs or complex data types to foreign handlers?
Is there a way of creating a compatible list or array in lcb and passing a pointer to the list or array?
I would expect the layout of the struct in memory would need to be very specific in order to do something like that.

I only ask because one of my old externals involved setting the PIXELFORMAT of a window, this custom type was the passed to the SetPixelFormat() function. After some jiggery pokery, I could then render to the livecode window with OpenGL calls. Fun times.

[LCMark]

@n.allan: I'm glad to hear you've had some success with this - the foreign interconnect in LCB is still in its infancy.

In terms of some of your questions...

We have a plan for a 'foreign record' type which would pack its data members in memory in the appropriate way for the target language. Records would have syntax akin to arrays in LCB.

There needs to be some sort of 'foreign array' concept which allows mapping to C-style arrays - we've still not worked out the details of this.

[n.allan]

Still impressive as it stands. Great work by the team.

Not a single line of c in sight!

[LCMark]

I do wonder whether (for the time being at least), it might be worth adding some functions which allow you to dereference an offset from a pointer as all the native C storage types - that would enable you to build and decompose structs and C-style arrays right now.

Basically, with enough glue-code you should be able to wrap any API with those basic things.

[n.allan]

What's your thoughts? Something like?

variable tInt as Integer
variable tStruct as Pointer
put tInt into (tStruct + 4)

I'm not sure how these structs are laid out in memory or how the compiler knows where each variable type is located in memory.

Would itinvolve knowing exactly which order the variables are created in the complex type?

Not all libraries would allow access to the full spec of the struct and most open source libraries are so abstract that they assume you will never need to know. It would involve a bit of digging into the code maybe.

[peter-b]

Personally, I'd prefer something more like Python's "struct" module:

Code: Select all

variable tStruct as data
pack [1, 2, 3.5] with struct layout "bid" into tStruct

[LCMark]

If the structs involved are private, then the library in question must provide appropriate function-based accessors so that isn't an issue.

If the structs are public then they have the appropriate packing / order implicitly or explicitly specified so it can be mapped.

The issue is where you need to provide packed structs or c-style arrays to functions. The ability missing from the foreign support at the moment is a way to map those low-level structures to high-level types you can directly manipulate in LCB.

I wasn't thinking in terms of pointer arithmetic; rather to add syntax to enable you to do what you should be able to do with typing but currently cannot:

Code: Select all

variable tByteArray as Pointer
put malloc(100) into tByteArray
put byteArrayFetchAtIndex(tByteArray, 1)

With typing support, you'd do something like:

Code: Select all

variable tByteArray as UnsafePointer<Byte>
put malloc(100) into tByteArray
put tByteArray[1]

However, this is unsafe (as marked). There are ways to do the mapping (with types) in a safer way.

Peter has already done some work to manipulate structs in a safe (if slightly slow) way in LCB directly - so that might be a better short term measure until we can do the typing properly.

[LCMark]

And Peter has just reminded me that I played around with a similar concept a while ago - but it is unfinished: https://github.com/livecode/livecode/pull/1863

That approach would probably be a better one in the short-term - allowing you to safely pack / unpack structures and arrays using appropriate format strings.

[n.allan]

I did stumble upon that paticular page on github funnily enough. Still way over my head unfortunately.
Either way I don't envy you guys at it looks like some task.

[trevordevore]

@n.allan: Would you mind posting some of your Windows examples? I've been working with widgets mainly and haven't ventured into foreign handlers yet. I would love to see some more examples as I would like to replace some Windows externals I have.

[monte]

I'd be interested too. I can't seem to get foreign handlers to work with a library in the extension resources but I assume that's a separate issue as I can call system functions fine. Definitely interested in the struct discussion.

[n.allan]

ok here's what I have so far. This is calling a blitz3d sdk library in my syswow64 folder (it's a 32 bit dll) called b3d.dll. It's an old library but it is my "go to" library for testing due to the fact it only uses primative types. Unfortunaltely I still have to get my head around passing pointers back and forth but this covers mose basic types.

LCB Code

Code: Select all

library community.livecode.allan.Blitz3D

metadata title is "Blitz3D Library"
metadata author is "Neil Allan"
metadata version is "1.0.0"

use com.livecode.foreign

foreign handler bbBeginBlitz3D ( ) returns nothing binds to "b3d>bbBeginBlitz3D"
foreign handler bbGraphics3D ( in tWidth as CInt, in tHeight as CInt,in tDepth as CInt, in tMode as CInt ) returns nothing binds to "b3d>bbGraphics3D"
foreign handler bbCreateLight ( in tType as CInt, in tParent as CUInt ) returns CUint binds to "b3d>bbCreateLight"
foreign handler bbCreateCamera ( in tParent as CUint ) returns CUInt binds to "b3d>bbCreateCamera"
foreign handler bbCreateCube ( in tParent as CUint ) returns CUint binds to "b3d>bbCreateCube"
foreign handler bbPositionEntity ( in tEntity as CUInt, in tX as CFloat, in tY as CFloat, in tZ as CFloat,in tGlobal as CBool ) returns nothing binds to "b3d>bbPositionEntity"
foreign handler bbTurnEntity ( in tEntity as CUint, in tPitch as CFloat, in tYaw as CFloat, in tRoll as CFloat, in tGlobal as CBool ) returns nothing binds to "b3d>bbTurnEntity"
foreign handler bbRenderWorld ( ) returns nothing binds to "b3d>bbRenderWorld"
foreign handler bbFlip ( ) returns nothing binds to "b3d>bbFlip"
foreign handler bbSetBlitz3DTitle ( in tTitle as ZStringNative, in tQuitMessage as ZStringNative ) returns nothing binds to "b3d>bbSetBlitz3DTitle"

public handler BeginBlitz3D ( ) returns nothing
	return bbBeginBlitz3D ( )
end handler

public handler Graphics3D ( in tWidth as Integer, in tHeight as Integer,in tDepth as Integer, in tMode as Integer ) returns nothing
	return bbGraphics3D ( tWidth, tHeight, tDepth, tMode )
end handler

public handler CreateLight ( in tType as Integer, in tParent as Integer ) returns Integer
	return bbCreateLight ( tType, tParent )
end handler

public handler CreateCamera ( in tParent as Integer ) returns Integer
	return bbCreateCamera ( tParent )
end handler

public handler CreateCube ( in tParent as Integer ) returns Integer
	return bbCreateCube ( tParent )
end handler
	
public handler PositionEntity ( in tEntity as Integer, in tX as Number, in tY as Number, in tZ as Number, in tGlobal as Boolean ) returns nothing
	return bbPositionEntity ( tEntity, tX, tY, tZ, tGlobal )
end handler

public handler TurnEntity ( in tEntity as CUint, in tPitch as Number, in tYaw as Number, in tRoll as Number, in tGlobal as Boolean ) returns nothing
	return bbTurnEntity ( tEntity, tPitch, tYaw, tRoll, tGlobal)
end handler

public handler RenderWorld ( ) returns nothing
	return bbRenderWorld ( )
end handler

public handler FlipBuffer ( ) returns nothing
	return bbFlip ( )
end handler

public handler SetBlitz3DTitle ( in tTitle as String, in tQuitMessage as String ) returns nothing
	return bbSetBlitz3DTitle ( tTitle, tQuitMessage )
end handler

end library

LCS Code to use it from LCS

Code: Select all

global tCamera
global tLight
global tCube

on DrawCube
   BeginBlitz3D
   Graphics3D 800, 600, 0, 2
   put CreateLight ( 0, 0 ) into tLight
   put CreateCamera ( 0 ) into tCamera
   put CreateCamera ( 0 ) into tCamera
   put CreateCube ( 0 ) into tCube
   PositionEntity tCamera, 0, 0, -4, false
   SpinCube
end DrawCube


on SpinCube
   TurnEntity tCube, 1, 2, 3, true
   RenderWorld
   FlipBuffer
   send "SpinCube" to me in 2 ticks -- approx 30 fps?
end SpinCube

Should result in a new 3d window with a spinnng cube.

[PaulDaMacMan]

Thanks for posting your code Neil!
If you saved any code from exploring the use libFluidSynth (per our FaceBook exchange), please post it as well. T.I.A.

So this library resides in a default/static library location and/or is already linked/loaded into memory. Peter's blog examples use the C standard library or other lib that are always loaded/linked to by LC. Are there any examples of loading an external library from a directory / file path yet?

[PaulDaMacMan]

Note: please be patient with me. I'm likely out of my league here and may not be using the correct terminology for static linking / dynamic linked libraries vs the library's location on disk. I'm more of a scripter /graphic artist /musician and I haven't attempted to learn any non-scripting programming languages for a long time until LCB came along. I'm trying to read up on analyzing C .h header files, static link / dynamic linked libraries, etc. as I play around with LCB. If nothing else I might be a good test subject for LCB's learning curve.

[PaulDaMacMan]

I think the MacOS X function dlopen is what I was looking for:
https://developer.apple.com/library/mac ... unc/dlopen