android externals

[monte]

A scheme that avoids having to declare the command multiple times is the main thing I think because it will make the file much less readable... And break my documentation parser...

[monte]

pretty excited to see @runrevmark's latest commit... just compiled the test external and android engine... now I need to work out how to get the two together... runtime folder in user extensions? or is there a better way?

[monte]

ooo... big button...

[Janschenkel]

@monte
Look at it this way: 'revdb' would be an 'interface' module, with multiple 'implementation' modules.
The 'revdb' .lcidl file would describe the syntax for all 'revdb-driver' implementations.
Now if I wanted to write a 'qrtjdbc' driver, its .lcmdl definition could be as short as

Code: Select all

implements revdb-driver for database-type "jdbc" using java

and the engine would know enough to glue the pieces together...

Jan Schenkel.

[monte]

Hmm... externals for externals... revdb could sure use something like this because at the moment the list of drivers is hard coded in so to add a custom driver you would need to submit a pull request to add it to revdb... there is a "new database layer" coming though... I know you were thinking beyond that use case though. I think this would be good but it's probably something that can be tacked on later... an interface block in the externals lcidl and an implementation block in the lcidl of the implementation.

BTW I've finally been able to implement the command we've all been waiting for... mergPopToast.... mmm... yum

[monte]

BTW I see what's going on now with character encoding for java strings... so I guess it's equivalent to NSString... what I find a bit annoying is in order to use unicode strings we need to use c-data and convert to a known encoding... also we probably need to create variants of our commands that require unicode... or add an encoding parameter to each command... is there any way we can make this whole thing more transparent on both the scripter side and the external side?

[bn]

mergPopToast.... mmm... yum

wasn't that mergPanCake ?

Kind regards
Bernd

[mwieder]

All right - I'm in.
Breakfast at Monte's.

[monte]

Mmm... mergPancakeMachine.... mmm

[Janschenkel]

We ought to standardize on UTF-8 when crossing the boundaries from LiveCode to other technologies (be it C, Java or C#).
Or maybe the .lcidl definition could be expanded with an encoding clause at the same level as the use clauses?

Jan Schenkel.

[Martin Koob]

mergPopToast, mergPancakeMachine, once Monte releases his mApp_liance framework we can write code for any kitchen appliance. Code once, cook anywhere.

[Sean O'Connor]

time to get working on that jellybean sandwich i think.

[monte]

It seems people are hungry

@Janshenkel I wish the whole platform would standardise to UTF8. I never understood why our unicode implementation in such a cross platform tool was one that required byte ordering rather than one that was totally cross platform but I guess that ship has sailed. I know @runrevmark has mentioned in passing that moving the whole platform to UTF8 would be very bad. There was some reason and I must not have understood it because I can't remember it so I'm still left thinking that it would be a one time translation of a stackfile and from then on we can forget about unicode this and that unless we are actually reading or writing non-UTF8 data... anyway @runrevmark has a plan which he will fill us in on when he has time but I'd be interested to know if that plan extends to modules.

One idea might be to extend the functionality of the useUnicode property to govern how strings are encoded when they go to and from externals... not sure how much of my stuff would break though.. certainly mergJSON which relies on UTF8 encoding would break... I guess if it's only for these NSString / java String type conversions then it's less likely to break things in the external...

The other idea which we've both mentioned now is adding a use encoding clause so the generated file knows the input is utf8 or unicode or native and translates to/from accordingly.

[monte]

OK so now that I've implemented the fairly trivial example (mergPopToast)... I did this in the revtestexternal... I'd like to integrate that into my currently iOS only mergPop external. In this case this external would have a set of iOS only commands and an android only command.. I guess I could make a completely separate external but the command fits the general function of the external.

The first stumbling block I hit is I'm using objective c objects so I can't just add java to my commands in the lcidl... so I guess I need to change all the objc-string etc to c-string then use [NSString stringWithCString:myParam encoding:NSMacOSRomanStringEncoding]...

Then it occurs to me that if I'm changing all my parameters anyway I might as well take the time to deal with the encoding issue... What I'm thinking at the moment is to have a command in all my externals..eg.. mergPopUseEncoding {"UTF8"|"UTF16"|"native"} and then I will need to use c-data parameters for everything and do all the conversion in my external implementation rather than what I have been doing in relying on the lcidl magic to handle this stuff for me...

The other thing is Xcode seems to treat java files as just text with no IDE support... so I'm trying to work out if I can use an IDE for these things. Eclipse seemed very difficult to work with so I'm trying IntelliJ IDEA CE which seems much easier to work with and it also looks like if we pump out the correct files in the right places it will open projects properly which is something which seemed difficult without writing a custom plugin for eclipse. Interestingly this will change what lcidl needs to do with respect to the generation of LC.java. The file would be generated at project creation time rather than at each compile... How we integrate that with SDK updates I'm not sure... perhaps it could overwrite the file or there might be an SDK update process.

[LCMark]

A scheme that avoids having to declare the command multiple times is the main thing I think because it will make the file much less readable... And break my documentation parser...

In the current context, my suggestions about an 'if' clause were just to get lcidlc working well enough to allow android externals now - to be fair, in this interim period it probably isn't necessary (although see below...). This also why I'm talking in terms of Android externals and not Java more generally at the moment.

Look at it this way: 'revdb' would be an 'interface' module, with multiple 'implementation' modules.

This is essentially what I was talking about above post-lcidlc. The 'script level' spec (which will eventually include syntax and docs) represents the 'interface' of the module - it is then up to the engine to determine what language the implementation is in, do the relevant type mapping and call the appropriate method using the appropriate language runtime. i.e. You specify the interface to script once, and then the 'script-level' methods are bound to their 'native' implementations (whether that be C/C++/Obj-C, Java or some other language).

BTW I see what's going on now with character encoding for java strings... so I guess it's equivalent to NSString... what I find a bit annoying is in order to use unicode strings we need to use c-data and convert to a known encoding... also we probably need to create variants of our commands that require unicode... or add an encoding parameter to each command... is there any way we can make this whole thing more transparent on both the scripter side and the external side?

Yes - Java strings (at the moment) work just like NSString's do for iOS externals - the generated code converts the native encoding to a Java string, just like it converts the native encoding to an NSString. Post-lcidlc it becomes completely transparent...

We ought to standardize on UTF-8 when crossing the boundaries from LiveCode to other technologies (be it C, Java or C#).
Or maybe the .lcidl definition could be expanded with an encoding clause at the same level as the use clauses?

There's no need for this... At the script-level you specify a method as taking a 'string' - this is an abstract concept meaning a sequence of characters. It is in the implementation methods that a particular type of string will be presented as an argument. For example, in the C bindings you'll be able to specify these:

• MCStringRef - the encoding-transparent string value (basically the same idea as CFStringRef / NSString*)
• stringz_t - a NUL-terminated native encoded string
• unistringz_t - a NUL-terminated UTF-16 encoded string
• utf8stringz_t - a NUL-terminated UTF-8 encoded string
• string_t - a counted native encoded string (i.e. a ptr and a length)
• unistring_t - a counted UTF-16 encoded string
• utf8string_t - a counted UTF-8 encoded string
• (Mac) CFStringRef - a CoreFoundation/Obj-C string (NSString* is toll-free-bridged to CFStringRef)
• (Windows) BSTR - the type used by a lot of COM methods and API calls on Windows (its actually a UTF-16 encoded string but with special memory management requirements)

The point here is that its the native implementation that dictates what it wants to make things as simple to implement as possible, from the script point of view its just an (abstract) string - the engine can manage and store these as it likes (indeed, as MCStringRef's).
For Java then there would be two options:

• A wrapper around the native MCStringRef type
• java.lang.String - the standard Java string type

I wish the whole platform would standardise to UTF8. I never understood why our unicode implementation in such a cross platform tool was one that required byte ordering rather than one that was totally cross platform but I guess that ship has sailed.

UTF-8 is an implementation detail - it isn't something that most people will want to deal with at the script level. In particular, UTF-8 is a multibyte encoding - if 'char' at the script level were a UTF-8 char then *all* string operations become O(n) (i.e. proportional to the length of the string) rather than O(1) (i.e. constant time). Now, it is true that UTF-16 is also a multi-byte encoding, but mostly not - if a UTF-16 string does not use surrogates then it's a 1-1 mapping and so 'char <n> of ...' remains constant time. The point is that we are a high-level language so really, at that level strings should be sequences of (indivisible) chars - the only time you should ever need to be concerned about encodings is when performing I/O (where you need to represent the 'abstract' notion of string in some concrete form).
Now, the 'abstract' notion of string will be codified as an MCStringRef - essentially the same sort of beast as CFStringRef (NSString*). It's an opaque type which (therefore) can carry much more information than a (ptr, length) pair which is how the current engine treats strings. In particular, it can know whether its currently a natively encoded string, or unicode string, whether (if it is a unicode string) it contains surrogates etc. This abstraction means we can transition to Unicode with virtually no performance penalty and very little changes (if any) at the user script level.
So, the 'plan' certainly does extend to modules - the engine will know how to map its idea of a string to whatever a 'native' method in a module has requested and vice-versa. Indeed, modules will be able to use the same opaque set of types that the core engine evaluation system will run on if they wish.

OK so now that I've implemented the fairly trivial example (mergPopToast)... I did this in the revtestexternal... I'd like to integrate that into my currently iOS only mergPop external. In this case this external would have a set of iOS only commands and an android only command.. I guess I could make a completely separate external but the command fits the general function of the external.

This was the motivation for suggesting an 'if' clause... As mentioned before, there's a difference between 'script' level types and 'native' types - at the script level there is only the concept of 'string', at the native level what a 'string' needs to be passed as can and will vary. On iOS, you might want it as an objc-string or a c-string, whereas for Android (Java) you will want it as a Java string.

How about a slightly modified version of what @monte suggested above... We introduce 'abstract' types 'string', 'data', 'array' and 'dictionary'. These can only be used in command/function clauses that specify the implementation. e.g.

Code: Select all

  command foobar
    on ios, mac use objc
    on android use java
    on windows, linux use c
    in a as string
    in b as data
    in c as array
    in d as dictionary

Then the abstract types map depending on the 'on' clause:

• objc - string -> NSString, data -> NSData, array -> NSArray, dictionary -> NSDictionary
• java - string -> Java String, data -> byte[], array -> Java Array, dictionary -> Java hash table equivalent
• c - string -> c-string, data -> c-data, array -> c-array, dictionary -> c-dictionary

Here 'c-array' would be struct {const char **elements; int element_count}, and 'c-dictionary' would be struct {const char **keys, **elements; int element_count}.