Android/Java externals

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, LCMark

Locked
Janschenkel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 977
Joined: Sat Apr 08, 2006 7:47 am
Location: Aalst, Belgium
Contact:

Android/Java externals

Post by Janschenkel » Sun Jun 09, 2013 3:44 pm

The recent topic hacking desktop externals for mobile touched on the implementation of Android/Java externals for LiveCode.
Now I've been pondering and studying this since the conference, and I promised the other day I'd bundle my thoughts on such an implementation.
Base principle: it should feel natural to Java developers, because to get them on board, we'll have to save them from having to deal with C.
Bonus points if we can get the same glue working on Android + Desktop platforms - iOS being the outsider here (*)

1. A simple example
What would a simple external look like?

Code: Select all

package com.example.external;
import com.runrev.external.LCExternalCommand;
import com.runrev.external.LCExternalFunction;
import com.runrev.external.LCExternalLibrary;
public class ExampleExternal implements LCExternalLibrary {
    @LCExternalCommand
    public void exampleExternalCommand(String[] args) { ... }
    @LCExternalFunction
    public String exampleExternalFunction(String[] args) { ... }
}
where:
  • LCExternalLibrary is an interface to mark this class as a collection of external commands and functions
  • LCExternalCommand is an annotation to mark this method as an external command
  • LCExternalFunction is an annotation to mark this method as an external function
The methods in this class can then be called from your LiveCode script as if it was an every-day external:

Code: Select all

on mouseUp
    exampleExternalCommand "foo", "bar"
    answer exampleExternalFunction("baz")
end mouseUp
2. More extensive examples and features

2.1 Startup and Shutdown
Analogous to the iOS externals, we would also have optional annotations for the Startup and Shutdown of an external:

Code: Select all

package com.example.external;
import com.runrev.external.LCExternalCommand;
import com.runrev.external.LCExternalFunction;
import com.runrev.external.LCExternalLibrary;
import com.runrev.external.LCExternalShutdown;
import com.runrev.external.LCExternalStartup;
public class ExampleExternal implements LCExternalLibrary {
    @LCExternalStartup
    public void exampleExternalStartup(...) { ... }
    @LCExternalShutdown
    public void exampleExternalShutdown() { ... }
    @LCExternalCommand
    public void exampleExternalCommand(String[] args) { ... }
    @LCExternalFunction
    public String exampleExternalFunction(String[] args) { ... }
}
where:
  • LCExternalStartup is an annotation to mark this method as the one to call at startup of the external library
  • LCExternalShutdown is an annotation to mark this method as the one to call at shutdown of the external library
2.2 Name mapping
To prevent name clashes while preserving the Java style of method naming, the LCExternalCommand and LCExternalFunction annotations would also have an 'alias' parameter to allow the engine to map external calls to the right methods

Code: Select all

package com.example.external;
import com.runrev.external.LCExternalCommand;
import com.runrev.external.LCExternalFunction;
import com.runrev.external.LCExternalLibrary;
import com.runrev.external.LCExternalInterface;
import com.runrev.external.LCExternalStartup;
public class ExampleExternal implements LCExternalLibrary {
    @LCExternalStartup
    public void exampleExternalStartup(LCExternalInterface ei) { ... }
    @LCExternalCommand(alias="exampleExternalCommand")
    public void externalCommand(String[] args) { ... }
    @LCExternalFunction(alias="exampleExternalFunction")
    public String externalFunction(String[] args) { ... }
}
3. Engine callbacks
Callbacks would be handled via an object passed into the 'startup' method which the object then stores and uses when necessary.
Note that we could also place all callback methods in a singleton, but this would make evolution of the API harder.

Code: Select all

package com.example.external;
import com.runrev.external.LCExternalCommand;
import com.runrev.external.LCExternalFunction;
import com.runrev.external.LCExternalLibrary;
import com.runrev.external.LCExternalInterface;
import com.runrev.external.LCExternalStartup;
public class ExampleExternal implements LCExternalLibrary {
    @LCExternalStartup
    public void exampleExternalStartup(LCExternalInterface ei) { ... }
    @LCExternalCommand
    public void exampleExternalCommand(String[] args) { ... }
    @LCExternalFunction
    public String exampleExternalFunction(String[] args) { ... }
}
where:
  • LCExternalInterface is an interface (not an implementation class) defining the provided callback methods
By shielding the callbacks behind an interface, we could provide two actual implementations:
  • com.runrev.android.AndroidExternalHost -> Android specific
  • com.runrev.desktop.DesktopExternalHost -> Java generic for desktop
And as things evolve, we could have additional interfaces LCExternalInterfaceLevel2, LCExternalInterfaceLevel3, etc. while remaining compatible.

3.1 Callback methods
Ideally, we would have the best of both the classic desktop and iOS externals callbacks in one unified set of methods.
This is something that I haven't completely mulled over yet, but I do know we'll need Java-like names for them.
For instance, LCRunBlockOnMainThread would become LCExternalInterface.runOnMainThread(Runnable r) and thus an external could use it like this:

Code: Select all

package com.example.external;
import com.runrev.external.LCExternalCommand;
import com.runrev.external.LCExternalFunction;
import com.runrev.external.LCExternalLibrary;
import com.runrev.external.LCExternalInterface;
import com.runrev.external.LCExternalStartup;
import java.lang.Runnable;
public class ExampleExternal implements LCExternalLibrary {
    LCExternalInterface externalInterface;
    @LCExternalStartup
    public void exampleExternalStartup(LCExternalInterface ei) {
        this.externalInterface = ei;
    }
    @LCExternalCommand
    public void exampleExternalCommand(String[] args) {
        externalInterface.runOnMainThread(new Runnable() {
            @Override
            public void run() {
                // whatever
            }
        });
    }
    @LCExternalFunction
    public String exampleExternalFunction(String[] args) { ... }
}
Of course the above looks silly when the example command isn't running on a a different thread, but you catch my drift...

There will probably also need to be a LCCallback interface for those cases where we need to shuffle context data back and forth.
That context data should probably take the form of a java.io.Serializable object.

3.2 LCContext and LCObject references
One of the nice features of he iOS externals API is the LCObjectRef weak handle that you can get back by calling LCContextMe and friends.
Definitely needed, though a Java makeover will have to happen for the API - to be discussed.

3.3 Variable setting and getting
One thing present in the Desktop externals API but missing in the iOS externals API is the ability to get and set variables.
We're going to need a consistent API which also covers arrays (both traditional and nested arrays, with conversion from/to java.util.Map<K,V> instances) and binary data (presumably using java.nio.ByteBuffer instances).
Again, this needs to be fleshed out.

4. Loading externals

Java 6 introduced the ServiceLoader class as a simple service-provider loading facility.
The idea is that you include a file in your .jar file that is read by the class loader and allows it to find at runtime the classes which implement a certain interface.
In our case, we'd have a single file

Code: Select all

META-INF/services/com.runrev.external.LCExternalInterface
With content

Code: Select all

com.example.external.ExampleExternal
And if your jar contains multiple externals, all their fully qualified class names would be added there.

Code: Select all

com.quartam.livecode.android.ZeroConf
com.quartam.livecode.android.XslTransformer
Android adopted this approach and included their own implementation of the ServiceLoader class in API Level 9 (Android 2.3 Gingerbread). If we want to continue support for Android 2.2 Froyo, we'll have to whip up our own implementation but that should be reasonably straightforward.

We could provide an AnnotationProcessor which automatically populates this file, and any other files that the engine might need to make this work (e.g. the .lcidl file). The main point is that we should make life easy and let the annotations do the work.

5. Complicating factors on the desktop
Traditionally, Sun/Oracle has provided the 32-bit implementation for Windows, with 64-bit versions since Java 6.
In the Linux world, RedHat has been the driving force behind IcedTea, and everything is converging towards OpenJDK these days.

The OS X situation is more complicated:
Apple used to provide its own implementations of Java for OS X, tightly coupled to the operating system version - Technical Note 2110: Identifying Java on OS X provides a complete overview.
Oracle only provides 64-bit Java implementations for OS X 10.7 Lion and 10.8 Mountain Lion. There are no 32-bit implementations, and given that Oracle doesn't support 10.6 Snow Leopard, chances are good that 10.7 Lion will be dropped as soon as 10.9 <insert feline> comes out.
And the Mac App Store won't allow apps which rely on a built-in Java - you must bundle a JRE yourself.

So a lot depends on which Java version we want and which operating system versions we still want to support.

(*) boot note regarding iOS
Technically, there are no restrictions against running Java on iOS platforms (except that it ought to be AOT compilation instead of JIT)
Both Avian and RoboVM may be of interest for anyone wanting to take this route.
But in the current situation, it would definitely be out of scope for the mainstream LiveCode engine.

These are my thoughts - I realise they're incomplete and things have to be fleshed out.
So have at! :-)

Jan Schenkel.
Quartam Reports & PDF Library for LiveCode
www.quartam.com

Janschenkel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 977
Joined: Sat Apr 08, 2006 7:47 am
Location: Aalst, Belgium
Contact:

Re: Android/Java externals

Post by Janschenkel » Sun Jun 09, 2013 6:30 pm

Some implementation notes
On the C side, we could have a class MCJavaExternalProxy which inherits from MCExternal and acts as a go-between for the classic externals interface and the JNI part, so the rest of the engine can be happily ignorant of the fact that the actual implementation is done in Java.
We should also channel all JNI interaction (if not for everything then at least for the externals) via a single class MCJNIBridge which taps into the LCExternalInterface implementation class instance running inside the JVM.

On the Java side, we would have an implementation of the LCExternalInterface interface which has two jobs:
  1. find and instantiate the Java classes which implement LCExternalLibrary; and call the appropriate methods at the request of MCJNIBridge
  2. provide all the callbacks for the Java classes; turning around and tapping into MCJNIBridge
If this all seems too abstract, I'll see about ways to make it more concrete.
I've been contemplating sharing the source code of the Quartam JVM External I wrote 5 years ago. It was never finished, partially because I ran into problems with event handling on OS X when using AWT/Swing, and partially because ExternalsV4 failed to materialise. But if it can help move things forward here, I'm happy to publish it.

Jan Schenkel.
Last edited by Janschenkel on Sun Jun 30, 2013 9:14 pm, edited 3 times in total.
Quartam Reports & PDF Library for LiveCode
www.quartam.com

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: Android/Java externals

Post by monte » Sun Jun 09, 2013 11:43 pm

Interesting... have you had a chance to look at @runrevmark's latest stuff? One thought I have is lcidl parser -> JNI method stubs is probably going to be an easier way to go for open language externals... Or perhaps the the lcidl could just parse into some syntax description stuff...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

Janschenkel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 977
Joined: Sat Apr 08, 2006 7:47 am
Location: Aalst, Belgium
Contact:

Re: Android/Java externals

Post by Janschenkel » Mon Jun 10, 2013 9:33 am

Additional information
In other topics on this forum, I've commented on things that are also relevant to this discussion. So I figured I had better include them here as well...

Android classpath
There is no requirement for static linking on Android, and all .dex files in the .apk package are automatically put on the classpath. This means we don't need to manually load those libraries in order to instantiate the LCExternalLibrary classes.

Android vs Standard Java
While Android and Standard Java share a lot of core libraries, there are also differences. This means developers may have to provide platform-specific externals depending on Android API or Standard Java API - especially when it comes to UI elements. But for a lot of externals, developers could very well write their external once, and have the Android building process convert the .jar files to .dex files.

Jan Schenkel.
Quartam Reports & PDF Library for LiveCode
www.quartam.com

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1208
Joined: Thu Apr 11, 2013 11:27 am

Re: Android/Java externals

Post by LCMark » Mon Jun 10, 2013 9:53 am

@Janschenkel

I really like the idea of us trying to make the engine as easy to extend in other (embeddable - i.e. easy to launch a VM) languages as possible (Java is definitely a good first target, given the breadth of functionality available). However, I think it would be best to evaluate the way forward in the context of the refactoring project and the work going on there rather than base it on what is currently present in the engine - mainly because what is currently there is going to change, and the new direction provides for a much richer type-system.

I'll start a new topic on the more general idea of 'language bindings' explaining the general path we are currently taking.
I've been contemplating sharing the source code of the Quartam JVM External I wrote 5 years ago. It was never finished, partially because I ran into problems with event handling on OS X when using AWT/Swing, and partially because ExternalsV4 failed to materialise. But if it can help move things forward here, I'm happy to publish it.
The problems you ran into here you'd still run into - it's to do with the architecture of the engine and the 'wait' command (and the fact that Mac has a single UI-event loop per app / whereas Windows has a single UI-event loop per thread). Similar problems arise in Android and iOS, which we solved by using two threads. In the short-term, this is something we're planning on changing when we do the platform API (i.e. make desktop platforms run on a system and script thread). In the longer term, the architecture of the engine needs to change so that functions-that-can-call-wait (i.e. anything that runs script) work via a callback system, rather than blocking.

Janschenkel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 977
Joined: Sat Apr 08, 2006 7:47 am
Location: Aalst, Belgium
Contact:

Re: Android/Java externals

Post by Janschenkel » Mon Jun 10, 2013 10:25 am

runrevmark wrote: I really like the idea of us trying to make the engine as easy to extend in other (embeddable - i.e. easy to launch a VM) languages as possible (Java is definitely a good first target, given the breadth of functionality available). However, I think it would be best to evaluate the way forward in the context of the refactoring project and the work going on there rather than base it on what is currently present in the engine - mainly because what is currently there is going to change, and the new direction provides for a much richer type-system.

I'll start a new topic on the more general idea of 'language bindings' explaining the general path we are currently taking.
Looking forward to it as I'd rather help with the implementation of Android/Java externals support once and get it right the first time around :-)
runrevmark wrote:
janschenkel wrote:I've been contemplating sharing the source code of the Quartam JVM External I wrote 5 years ago. It was never finished, partially because I ran into problems with event handling on OS X when using AWT/Swing, and partially because ExternalsV4 failed to materialise. But if it can help move things forward here, I'm happy to publish it.
The problems you ran into here you'd still run into - it's to do with the architecture of the engine and the 'wait' command (and the fact that Mac has a single UI-event loop per app / whereas Windows has a single UI-event loop per thread). Similar problems arise in Android and iOS, which we solved by using two threads. In the short-term, this is something we're planning on changing when we do the platform API (i.e. make desktop platforms run on a system and script thread). In the longer term, the architecture of the engine needs to change so that functions-that-can-call-wait (i.e. anything that runs script) work via a callback system, rather than blocking.
Before I release the code into the wild, I'll probably just force the JavaVM to start with option '-Djava.awt.headless=true' to avoid the UI threading shenanigans I had encountered. A good callback system and clear threading rules would definitely help, as part of a unified Externals API.

Jan Schenkel.
Last edited by Janschenkel on Mon Jun 10, 2013 6:39 pm, edited 1 time in total.
Quartam Reports & PDF Library for LiveCode
www.quartam.com

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: Android/Java externals

Post by mwieder » Mon Jun 10, 2013 5:48 pm

Looking forward to it as I'd rather help with the implementation of Android/Java externals support once and get it right the first time around :-)
Buttons still don't work, so this is me pressing a virtual "Like" button... :-)

Janschenkel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 977
Joined: Sat Apr 08, 2006 7:47 am
Location: Aalst, Belgium
Contact:

Re: Android/Java externals

Post by Janschenkel » Sun Jun 30, 2013 5:21 pm

To give this topic some much-needed oxygen, I've shared the Quartam JVM External code on github.
For now, it's all still in the 'develop' branch as I need to go back in and sort out the windows side.
Not to mention add some explanations on how to use it.
Anyway, you can take a look at https://github.com/quartamsoftware/qrtjvm/tree/develop

Jan Schenkel.
Quartam Reports & PDF Library for LiveCode
www.quartam.com

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: Android/Java externals

Post by monte » Mon Jul 01, 2013 12:09 am

Nice!

Your message contains 5 characters. The minimum number of characters you need to enter is 10.
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

Locked

Return to “Engine Contributors”