android externals

Moderators: Klaus, FourthWorld, heatherlaine, kevinmiller, LCMark

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: android externals

Post by monte » Thu Jul 11, 2013 9:53 am

;-)

A scheme that avoids having to declare the command multiple times is the main thing I think because it will make the file much less readable... And break my documentation parser...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: android externals

Post by monte » Thu Jul 11, 2013 11:20 am

pretty excited to see @runrevmark's latest commit... just compiled the test external and android engine... now I need to work out how to get the two together... runtime folder in user extensions? or is there a better way?
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: android externals

Post by monte » Thu Jul 11, 2013 11:59 am

ooo... big button...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

Janschenkel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 977
Joined: Sat Apr 08, 2006 7:47 am
Location: Aalst, Belgium
Contact:

Re: android externals

Post by Janschenkel » Thu Jul 11, 2013 9:33 pm

@monte
Look at it this way: 'revdb' would be an 'interface' module, with multiple 'implementation' modules.
The 'revdb' .lcidl file would describe the syntax for all 'revdb-driver' implementations.
Now if I wanted to write a 'qrtjdbc' driver, its .lcmdl definition could be as short as

Code: Select all

implements revdb-driver for database-type "jdbc" using java
and the engine would know enough to glue the pieces together...

Jan Schenkel.
Quartam Reports & PDF Library for LiveCode
www.quartam.com

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: android externals

Post by monte » Thu Jul 11, 2013 10:24 pm

Hmm... externals for externals... revdb could sure use something like this because at the moment the list of drivers is hard coded in so to add a custom driver you would need to submit a pull request to add it to revdb... there is a "new database layer" coming though... I know you were thinking beyond that use case though. I think this would be good but it's probably something that can be tacked on later... an interface block in the externals lcidl and an implementation block in the lcidl of the implementation.

BTW I've finally been able to implement the command we've all been waiting for... mergPopToast.... mmm... yum ;-)
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: android externals

Post by monte » Fri Jul 12, 2013 12:28 am

BTW I see what's going on now with character encoding for java strings... so I guess it's equivalent to NSString... what I find a bit annoying is in order to use unicode strings we need to use c-data and convert to a known encoding... also we probably need to create variants of our commands that require unicode... or add an encoding parameter to each command... is there any way we can make this whole thing more transparent on both the scripter side and the external side?
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

bn
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3281
Joined: Sun Jan 07, 2007 9:12 pm
Location: Bochum, Germany

Re: android externals

Post by bn » Fri Jul 12, 2013 12:44 am

mergPopToast.... mmm... yum
wasn't that mergPanCake ?
:)
Kind regards
Bernd

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2820
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: android externals

Post by mwieder » Fri Jul 12, 2013 1:59 am

All right - I'm in.
Breakfast at Monte's. :-P

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: android externals

Post by monte » Fri Jul 12, 2013 3:55 am

Mmm... mergPancakeMachine.... mmm
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

Janschenkel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 977
Joined: Sat Apr 08, 2006 7:47 am
Location: Aalst, Belgium
Contact:

Re: android externals

Post by Janschenkel » Fri Jul 12, 2013 1:37 pm

We ought to standardize on UTF-8 when crossing the boundaries from LiveCode to other technologies (be it C, Java or C#).
Or maybe the .lcidl definition could be expanded with an encoding clause at the same level as the use clauses?

Jan Schenkel.
Quartam Reports & PDF Library for LiveCode
www.quartam.com

Martin Koob
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 237
Joined: Sun May 27, 2007 8:19 pm

Re: android externals

Post by Martin Koob » Fri Jul 12, 2013 2:18 pm

mergPopToast, mergPancakeMachine, once Monte releases his mApp_liance framework we can write code for any kitchen appliance. Code once, cook anywhere.

Sean O'Connor
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 21
Joined: Sat Mar 16, 2013 7:06 pm

Re: android externals

Post by Sean O'Connor » Fri Jul 12, 2013 7:02 pm

time to get working on that jellybean sandwich i think.
when the going gets tough the tough get going.

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: android externals

Post by monte » Fri Jul 12, 2013 10:20 pm

It seems people are hungry ;-)

@Janshenkel I wish the whole platform would standardise to UTF8. I never understood why our unicode implementation in such a cross platform tool was one that required byte ordering rather than one that was totally cross platform but I guess that ship has sailed. I know @runrevmark has mentioned in passing that moving the whole platform to UTF8 would be very bad. There was some reason and I must not have understood it because I can't remember it so I'm still left thinking that it would be a one time translation of a stackfile and from then on we can forget about unicode this and that unless we are actually reading or writing non-UTF8 data... anyway @runrevmark has a plan which he will fill us in on when he has time but I'd be interested to know if that plan extends to modules.

One idea might be to extend the functionality of the useUnicode property to govern how strings are encoded when they go to and from externals... not sure how much of my stuff would break though.. certainly mergJSON which relies on UTF8 encoding would break... I guess if it's only for these NSString / java String type conversions then it's less likely to break things in the external...

The other idea which we've both mentioned now is adding a use encoding clause so the generated file knows the input is utf8 or unicode or native and translates to/from accordingly.
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1561
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: android externals

Post by monte » Sun Jul 14, 2013 3:35 am

OK so now that I've implemented the fairly trivial example (mergPopToast)... I did this in the revtestexternal... I'd like to integrate that into my currently iOS only mergPop external. In this case this external would have a set of iOS only commands and an android only command.. I guess I could make a completely separate external but the command fits the general function of the external.

The first stumbling block I hit is I'm using objective c objects so I can't just add java to my commands in the lcidl... so I guess I need to change all the objc-string etc to c-string then use [NSString stringWithCString:myParam encoding:NSMacOSRomanStringEncoding]...

Then it occurs to me that if I'm changing all my parameters anyway I might as well take the time to deal with the encoding issue... What I'm thinking at the moment is to have a command in all my externals..eg.. mergPopUseEncoding {"UTF8"|"UTF16"|"native"} and then I will need to use c-data parameters for everything and do all the conversion in my external implementation rather than what I have been doing in relying on the lcidl magic to handle this stuff for me...

The other thing is Xcode seems to treat java files as just text with no IDE support... so I'm trying to work out if I can use an IDE for these things. Eclipse seemed very difficult to work with so I'm trying IntelliJ IDEA CE which seems much easier to work with and it also looks like if we pump out the correct files in the right places it will open projects properly which is something which seemed difficult without writing a custom plugin for eclipse. Interestingly this will change what lcidl needs to do with respect to the generation of LC.java. The file would be generated at project creation time rather than at each compile... How we integrate that with SDK updates I'm not sure... perhaps it could overwrite the file or there might be an SDK update process.
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1033
Joined: Thu Apr 11, 2013 11:27 am

Re: android externals

Post by LCMark » Mon Jul 15, 2013 2:29 pm

A scheme that avoids having to declare the command multiple times is the main thing I think because it will make the file much less readable... And break my documentation parser...
In the current context, my suggestions about an 'if' clause were just to get lcidlc working well enough to allow android externals now - to be fair, in this interim period it probably isn't necessary (although see below...). This also why I'm talking in terms of Android externals and not Java more generally at the moment.
Look at it this way: 'revdb' would be an 'interface' module, with multiple 'implementation' modules.
This is essentially what I was talking about above post-lcidlc. The 'script level' spec (which will eventually include syntax and docs) represents the 'interface' of the module - it is then up to the engine to determine what language the implementation is in, do the relevant type mapping and call the appropriate method using the appropriate language runtime. i.e. You specify the interface to script once, and then the 'script-level' methods are bound to their 'native' implementations (whether that be C/C++/Obj-C, Java or some other language).
BTW I see what's going on now with character encoding for java strings... so I guess it's equivalent to NSString... what I find a bit annoying is in order to use unicode strings we need to use c-data and convert to a known encoding... also we probably need to create variants of our commands that require unicode... or add an encoding parameter to each command... is there any way we can make this whole thing more transparent on both the scripter side and the external side?
Yes - Java strings (at the moment) work just like NSString's do for iOS externals - the generated code converts the native encoding to a Java string, just like it converts the native encoding to an NSString. Post-lcidlc it becomes completely transparent...
We ought to standardize on UTF-8 when crossing the boundaries from LiveCode to other technologies (be it C, Java or C#).
Or maybe the .lcidl definition could be expanded with an encoding clause at the same level as the use clauses?
There's no need for this... At the script-level you specify a method as taking a 'string' - this is an abstract concept meaning a sequence of characters. It is in the implementation methods that a particular type of string will be presented as an argument. For example, in the C bindings you'll be able to specify these:
  • MCStringRef - the encoding-transparent string value (basically the same idea as CFStringRef / NSString*)
  • stringz_t - a NUL-terminated native encoded string
  • unistringz_t - a NUL-terminated UTF-16 encoded string
  • utf8stringz_t - a NUL-terminated UTF-8 encoded string
  • string_t - a counted native encoded string (i.e. a ptr and a length)
  • unistring_t - a counted UTF-16 encoded string
  • utf8string_t - a counted UTF-8 encoded string
  • (Mac) CFStringRef - a CoreFoundation/Obj-C string (NSString* is toll-free-bridged to CFStringRef)
  • (Windows) BSTR - the type used by a lot of COM methods and API calls on Windows (its actually a UTF-16 encoded string but with special memory management requirements)
The point here is that its the native implementation that dictates what it wants to make things as simple to implement as possible, from the script point of view its just an (abstract) string - the engine can manage and store these as it likes (indeed, as MCStringRef's).
For Java then there would be two options:
  • A wrapper around the native MCStringRef type
  • java.lang.String - the standard Java string type
I wish the whole platform would standardise to UTF8. I never understood why our unicode implementation in such a cross platform tool was one that required byte ordering rather than one that was totally cross platform but I guess that ship has sailed.
UTF-8 is an implementation detail - it isn't something that most people will want to deal with at the script level. In particular, UTF-8 is a multibyte encoding - if 'char' at the script level were a UTF-8 char then *all* string operations become O(n) (i.e. proportional to the length of the string) rather than O(1) (i.e. constant time). Now, it is true that UTF-16 is also a multi-byte encoding, but mostly not - if a UTF-16 string does not use surrogates then it's a 1-1 mapping and so 'char <n> of ...' remains constant time. The point is that we are a high-level language so really, at that level strings should be sequences of (indivisible) chars - the only time you should ever need to be concerned about encodings is when performing I/O (where you need to represent the 'abstract' notion of string in some concrete form).
Now, the 'abstract' notion of string will be codified as an MCStringRef - essentially the same sort of beast as CFStringRef (NSString*). It's an opaque type which (therefore) can carry much more information than a (ptr, length) pair which is how the current engine treats strings. In particular, it can know whether its currently a natively encoded string, or unicode string, whether (if it is a unicode string) it contains surrogates etc. This abstraction means we can transition to Unicode with virtually no performance penalty and very little changes (if any) at the user script level.
So, the 'plan' certainly does extend to modules - the engine will know how to map its idea of a string to whatever a 'native' method in a module has requested and vice-versa. Indeed, modules will be able to use the same opaque set of types that the core engine evaluation system will run on if they wish.
OK so now that I've implemented the fairly trivial example (mergPopToast)... I did this in the revtestexternal... I'd like to integrate that into my currently iOS only mergPop external. In this case this external would have a set of iOS only commands and an android only command.. I guess I could make a completely separate external but the command fits the general function of the external.
This was the motivation for suggesting an 'if' clause... As mentioned before, there's a difference between 'script' level types and 'native' types - at the script level there is only the concept of 'string', at the native level what a 'string' needs to be passed as can and will vary. On iOS, you might want it as an objc-string or a c-string, whereas for Android (Java) you will want it as a Java string.

How about a slightly modified version of what @monte suggested above... We introduce 'abstract' types 'string', 'data', 'array' and 'dictionary'. These can only be used in command/function clauses that specify the implementation. e.g.

Code: Select all

  command foobar
    on ios, mac use objc
    on android use java
    on windows, linux use c
    in a as string
    in b as data
    in c as array
    in d as dictionary
Then the abstract types map depending on the 'on' clause:
  • objc - string -> NSString, data -> NSData, array -> NSArray, dictionary -> NSDictionary
  • java - string -> Java String, data -> byte[], array -> Java Array, dictionary -> Java hash table equivalent
  • c - string -> c-string, data -> c-data, array -> c-array, dictionary -> c-dictionary
Here 'c-array' would be struct {const char **elements; int element_count}, and 'c-dictionary' would be struct {const char **keys, **elements; int element_count}.

Post Reply

Return to “Engine Contributors”