android externals

[Janschenkel]

@runrevmark
Ah, the joy of leapfrogging comment posts
Again, thanks for clarifying the direction LiveCode is taking with regards to extensions and modules - very insightful.

We historically use a lot of generated code at the day-job and have been moving away from it to more dynamic/reflective code.
There's nothing wrong with generated code when you at least have a chance to step in and fix its shortcomings if needed.
And that's hard when it happens automagically during the standalone build process.

Jan Schenkel.

[Janschenkel]

@runrevmark
Now for the logical next questions...
- is there a timeframe for the modules API?
- is there a design document we can read and provide feedback on?
and far more importantly
- how can we help make this happen?

@monte
If the timeframe is too far out, would you be interested in a stripped down version of the Quartam JVM External which can serve as a bridge to the Dalvik VM, allowing you to call into the Java code and the Java code to call back to the engine using the 'classic' externals API?

Jan Schenkel.

[monte]

@Janschenkel
My gut feeling is we are pretty close to a functioning solution and we should put any available energy into that

[LCMark]

@Janschenkel:
In terms of what I talked about above then that is essentially where the refactor branch is going. This is a huge undertaking as the results of it will appear in stages - the first being an engine that essentially works exactly the same as the one now, but with the new method of binding syntax to 'native' methods (albeit with the current script parser) and the new type architecture. All being well at this point 'externals' will start to creep towards what we want 'modules' to be - they'll use a spec as outlined above, have access to the same rich type conversion layer and the LC API (as present from the lcidlc point of view) will expand to include access to appropriate libfoundation APIs (so that they can manipulate the types the engine will then have in a much more efficient way). The time frame for this first stage is essentially 'as soon as possible'

In terms of the specific case of Android externals, then really the best path we have to get to these now is the not-quite-finished work on the externals-api-v5 branch. It's so close to being functional it's quite annoying. I'll see if I can review where I got to with it today, and let you know what's left to do - sounds like you guys would be more than happy to help out to finish it

[monte]

as soon as possible

I like timeframes like that

It's so close to being functional it's quite annoying

And here was I being polite saying we were pretty close to a functioning solution

sounds like you guys would be more than happy to help out to finish it

more than happy to pitch in

[Janschenkel]

In terms of the specific case of Android externals, then really the best path we have to get to these now is the not-quite-finished work on the externals-api-v5 branch. It's so close to being functional it's quite annoying. I'll see if I can review where I got to with it today, and let you know what's left to do - sounds like you guys would be more than happy to help out to finish it

Always happy to help - especially in this particular area

Jan Schenkel.

[monte]

One question I have is will we be able to use a single lcidl file for both android and iOS... either define a command once and it produces appropriately #ifdefed out stubs for each platform or define the command twice (once as java) and the compiler knows it's not a variant but a separate implementation...

[LCMark]

One question I have is will we be able to use a single lcidl file for both android and iOS... either define a command once and it produces appropriately #ifdefed out stubs for each platform or define the command twice (once as java) and the compiler knows it's not a variant but a separate implementation...

Yes - essentially. At the moment, if you do:

Code: Select all

  java command foobar ...

Then it will expect a native (Obj-C/C++) implementation for all platforms except Android where it will generate the stub that calls the method 'foobar' in the external class in java.

However, ideally we'd add some sort of if clause so different declarations can be made for the different platforms:

Code: Select all

if the platform is android then
  java command foobar
end if
if the platform is not android then
  command foobar
end if

The reason here is that the type mappings you might want for iOS or another platform will likely be different from Android.

This is just a by-product of the fact lcidlc attempts to merge the idea of 'script types' and 'native types' - a good example is that for a function taking an array, you might want an LCArray on non-iOS platforms, a NSDictionary on iOS platforms and a native java hashmap on Android.

[monte]

Hmm... linking java to only android means we can't easily add desktop java externals. I know it's not on the radar right now but thinking ahead...

Also perhaps a different way to map the language and parameter types? What about something like this:

Code: Select all

command hello
     on android use java
     on ios,osx use objective-c
     in pParam
         on android as java-string
         on ios,osx as objc-string
         on others as c-string

Alternatively and perhaps less verbose would be extending the use clauses:

Code: Select all

use objc-objects on ios,osx
use java-objects on android,windows,linux -- just throwing desktops in for the example

Then add some new data types (string,data,array) and these are mapped to known types using the use clause to work out what they should be on which platform, however, you could still use c-string and it would be a c-string on all platforms unless it's a java method...

[monte]

I was just about to post a new topic on adding a use encoding clause but then I thought I'd check what's happening with the java side of things. It seems we have an issue because NewString expects UTF16 host byte order and NSString is using mac roman. This is another one of those moments when I wish for everything UTF8... So... what I'm thinking (in my ignorance of the unicode plans) is a use {native-encoding|utf8-encoding|utf16-encoding} clause could be added and strings converted appropriately...

[Janschenkel]

@monte
Placing an 'on platform use...' switch at every possible location is going to lead to errors that are easy to make and difficult to spot.
I think a fundamental question is in order:
- do we want mixed modules, where some commands/functions are implemented in native, some in java and others in c#?
- or do we prefer separate modules and dependency definitions so that the engine automatically loads the other required modules?
If we take the second door then we need:
- a simple tweak to the 'use-clause' in .lcidl parlance to allow specifying 'use java'
- a system to declare dependencies in the .lcidl file
- an engine change to resolve these dependencies at runtime
However, once we no longer see a module as a self-contained extension, we come in the territory of module versioning.
The guys at Sun/Oracle have been mulling this over for quite some time now, as part of Project Jigsaw - see the latest draft here.

Jan Schenkel.

[monte]

Good point. I think something like my second option but perhaps just:

Code: Select all

use objc on ios,osx
use java on android,windows,linux

Using objc implies objective-c objects... a platform not covered by a use clause would default to C or C++ if the use c++ naming clause has been set.

I don't really think it's necessary to mix languages in the same external on the same platform.. for example, implementing some of an android external in C and some in Java.

[Janschenkel]

At the risk of over-architecting, one could argue that an Extension is a collection of Modules, with one 'Interface' module and one or more 'Implementation' modules. The engine can then pick the best implementation module: native if available and java/c#/... otherwise. In fact, this approach would open up the possibility that one developer provides an Extension with an interface module and a single java implementation module, and other developers step in to provide additional native implementation modules for Desktop and iOS.

Jan Schenkel.

[monte]

Hmm... I would go with the KISS approach of one development team per module...

[Janschenkel]

Heh, did I mention the word 'over-architecting' yet?
Not every idea in this exchange is going to be implemented, the course is set out by RunRev. But we need to consider even the more outlandish ideas and weigh the complexity against the end result to have a solid solution.

Jan Schenkel.