Custom keyboard

Interested in using the many environmental add-ons and extensions from LiveCode and third parties to the flexible LiveCode environment? This is the place to talk about it.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: Custom keyboard

Post by bogs » Tue Nov 12, 2019 1:36 am

FourthWorld wrote:
Tue Nov 12, 2019 12:59 am
If it were simple for an app to control input into another app everyone would be doing it, and the outcome in many cases wouldn't be pretty.
I think this is where we are having a gap in communications. I'm not (and neither is the OP) talking about controling just any random app, but instead, sending keystrokes to applications built for receiving just that kind of input. Mousepad. Whats app I am guessing messenger side of things. Not, for instance, a game or drawing application, where a keyboard would likely have minimal business sending anything (although I remember some early paint programs that were designed for just that).

In any case, I'd have to guess that 99% of text and mesenger applications have been setup for keyboard or string input. I don't see what the sin would be in being able to make your own keyboard to send a string, such as Richmond's (going to mangle this, sorry Richmond) devangali thingie, to what essentially is a string oriented application. I sure can't sign you up for 'Boys life' from mousepad, or even from VIM :P
Image

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Custom keyboard

Post by FourthWorld » Tue Nov 12, 2019 1:46 am

bogs wrote:
Tue Nov 12, 2019 1:36 am
FourthWorld wrote:
Tue Nov 12, 2019 12:59 am
If it were simple for an app to control input into another app everyone would be doing it, and the outcome in many cases wouldn't be pretty.
I think this is where we are having a gap in communications. I'm not (and neither is the OP) talking about controling just any random app, but instead, sending keystrokes to applications built for receiving just that kind of input. Mousepad. Whats app I am guessing messenger side of things. Not, for instance, a game or drawing application, where a keyboard would likely have minimal business sending anything (although I remember some early paint programs that were designed for just that).

In any case, I'd have to guess that 99% of text and mesenger applications have been setup for keyboard or string input. I don't see what the sin would be in being able to make your own keyboard to send a string, such as Richmond's (going to mangle this, sorry Richmond) devangali thingie, to what essentially is a string oriented application. I sure can't sign you up for 'Boys life' from mousepad, or even from VIM :P
There's no sin, just no support in LC Script.

It's not a sin to use SSH to log into a remote computer. In fact, it's very useful; for some, essential. But it is a crime to use SSH without authorization.

And so safeguards exist in systems that run SSH to minimize the risk if unauthorized use. This adds to the complexity of making such systems, and they are not completely unhackable. But better to try than not.

Similarly, the ability to drive the inputs of one software from another software made by someone else can be useful, but is also inherently risky.

So OS vendors design their APIs as deployment systems for such things to allow useful implementations, while mitigating risks from unauthorized use.

This APIs are not trivially easy for any novice hacker to exploit. They are imperfect, but aim to limit API use to those the user intended.

This lower level APIs cannot be addressed through LC Script directly, but will need Builder to access.

Further, I'm not familiar with packaging requirements, which may be different for system extensions like keyboards then they are for applications. If that's the case, even with Builder it would be difficult or perhaps impossible to make a custom keyboard for mobile with LC.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7215
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Custom keyboard

Post by jacque » Tue Nov 12, 2019 5:29 am

I'm not (and neither is the OP) talking about controling just any random app, but instead, sending keystrokes to applications built for receiving just that kind of input.
Well, there's receiving direct input and there's receiving remote input.

If the app is built to receive out-of-app input, it would have an Applescript dictionary (Mac) or whatever Windows uses (is it still VB?) If it doesn't invite outside input then the OS will put up it's barriers for safety's sake. I don't think I'd want it any other way.

If WhatsApp has an Applescript dictionary then LC can send text to it, no keyboard required.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Custom keyboard

Post by FourthWorld » Tue Nov 12, 2019 6:28 am

Is AppleScript available on iOS?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7215
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Custom keyboard

Post by jacque » Tue Nov 12, 2019 7:17 am

FourthWorld wrote:
Tue Nov 12, 2019 6:28 am
Is AppleScript available on iOS?
No. Sorry, I was thinking of desktop. For mobile you'd need a keyboard, and I guess if we're talking about WhatsApp we're talking about mobile. My mind wanders this time of night.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

Thierry
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 875
Joined: Wed Nov 22, 2006 3:42 pm

Re: Custom keyboard

Post by Thierry » Tue Nov 12, 2019 7:58 am

jacque wrote: .....
My mind wanders this time of night.
Hi,

Don't worry, your mind is still bright and fine :)

Check this to re-assure yourself:

https://www.whatsapp.com/download/


Thierry
!
SUNNY-TDZ.COM doesn't belong to me since 2021.
To contact me, use the Private messages. Merci.
!

richmond62
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 9287
Joined: Fri Feb 19, 2010 10:17 am
Location: Bulgaria

Re: Custom keyboard

Post by richmond62 » Tue Nov 12, 2019 8:22 am

you are just not devious and bone headed enough
Now the question is whether I should take that as a compliment or an insult. :D
-
Picture 1.png
Picture 1.png (18.07 KiB) Viewed 17644 times
-
Lumpy custard!
-
Picture 2.png

richmond62
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 9287
Joined: Fri Feb 19, 2010 10:17 am
Location: Bulgaria

Re: Custom keyboard

Post by richmond62 » Tue Nov 12, 2019 11:00 am


bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: Custom keyboard

Post by bogs » Tue Nov 12, 2019 11:06 am

richmond62 wrote:
Tue Nov 12, 2019 8:22 am
you are just not devious and bone headed enough
Now the question is whether I should take that as a compliment or an insult. :D
Well, it wasn't meant as either, more like self reflection. I was told by many people that playing with Mc wasn't worth the time, energy, or a number of other things, but I am so obtuse as to make a circle appear sharp by comparison. You, by comparison, are not ;)

About the only thing I still have left in my favor is an overwhelming curiosity (my wife calls it something else and doesn't say it in a nice way most times :twisted: ), so when I see something that was available and isn't anymore, I go look for why. Most other people just take it as face value. I think that is fine and probably safer in the long term, remind me some day to tell you about the first tv I took apart without knowing what I was doing :shock:

In any case, now that you have found it, just move it somewhere nice and only set its visibility to true when that tab is selected, and your there :mrgreen:
jacque wrote:
Tue Nov 12, 2019 5:29 am
If the app is built to receive out-of-app input, it would have an Applescript dictionary (Mac) or whatever Windows uses (is it still VB?) If it doesn't invite outside input then the OS will put up it's barriers for safety's sake. I don't think I'd want it any other way.
I completely agree with that, but I suspect there is a way, similar to the way that the Os'es own virt. kb works. Otherwise, the app could shut out the Os'es virt kb and then where would you be? I also tend to doubt that the Os'es virt kb is part and parcel to the Os, just for the reasons you mentioned earlier, custom virt kb's are all over the place and they seem to send text to apps just fine.

The question then becomes a matter of 'how'. I can't do the things I would have done in Delphi (FPC today), because as Richard mentions, Lc doesn't reach quite deep enough into the guts of an Os, but the Os itself provides apis that can be tapped. I'll skip widgets though, since I don't tend to play in the later versions.

As I said earlier, I may well fail (that happens a lot). It is of no consequence in my life other than as a curiosity, but it is fun to think over in my vanishingly rare moments of free time available at the moment :mrgreen:
Image

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7215
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Custom keyboard

Post by jacque » Tue Nov 12, 2019 6:30 pm

I went back and read the original post but I'm not clear on the goal. Is the question whether LC can create a custom keyboard for use system-wide like those in the App Store, or is the question whether LC can send keystrokes to another app from a LC app?

The former can't be done in LCS, the latter is the security issue we've been talking about. Actually, all custom keyboards could have malware which is why Android puts up a scary warning when you install one.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Custom keyboard

Post by FourthWorld » Tue Nov 12, 2019 8:00 pm

jacque wrote:
Tue Nov 12, 2019 6:30 pm
I went back and read the original post but I'm not clear on the goal. Is the question whether LC can create a custom keyboard for use system-wide like those in the App Store, or is the question whether LC can send keystrokes to another app from a LC app?

The former can't be done in LCS, the latter is the security issue we've been talking about. Actually, all custom keyboards could have malware which is why Android puts up a scary warning when you install one.
^ this.

If I could learn to write as succinctly I'd be a happy man. :)
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

PaulDaMacMan
Posts: 617
Joined: Wed Apr 24, 2013 4:53 pm
Contact:

Re: Custom keyboard

Post by PaulDaMacMan » Sat Mar 28, 2020 5:09 pm

Since I've been reviving zombie posts, and reading many of the down-the-rabid-hole conversations they inspired months ago, I thought I'd share my experience in making a custom keyboard of sorts on macOS in LiveCode here:
https://www.youtube.com/watch?v=s_knX8mJg1c

In this video I'm playing the alpha-numeric laptop keyboard with each row of keys mapped to frets/notes of a bass guitar. In the LC scripts I'm trapping for rawKeyboard messages, and then using an array of those raw values (originally used delimited list, but arrays are faster) with corresponding arrays of other values that are then used to send messages to other handlers for highlighting buttons, sending MIDI notes messages out though a note playing engine (I've used various ways to do that in the past but in this video it's using my CoreMIDI LCB Library), and putting the value into a queue for later checking and turn-off messages. It's a pretty fast method and no mile long case/switch! The interesting thing is that it continues to work even when my stack is not in the foreground and I'm in other apps, which I guess might be considered a security flaw, but for this project it's a really nice feature! It's kind of fun when you leave it on and you can hear the text you're typing in another app, like a text editor, sort-of translated into music, lol. But anyway... I imagine this could be useful for someone trying to make system-wide custom-keyboard thing on macOS for other reasons, probably even some nefarious reasons like capturing confidential text input, although I know the OP was talking about a mobile onscreen-keyboard.
My GitHub Repos: https://github.com/PaulMcClernan/
Related YouTube Videos: PlayList

jiml
Posts: 336
Joined: Sat Dec 09, 2006 1:27 am
Location: Los Angeles

Re: Custom keyboard

Post by jiml » Mon Mar 30, 2020 4:30 pm

Very nice!

richmond62
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 9287
Joined: Fri Feb 19, 2010 10:17 am
Location: Bulgaria

Re: Custom keyboard

Post by richmond62 » Tue Apr 07, 2020 4:37 pm

What does
spilling instrument
mean?

PaulDaMacMan
Posts: 617
Joined: Wed Apr 24, 2013 4:53 pm
Contact:

Re: Custom keyboard

Post by PaulDaMacMan » Wed Apr 08, 2020 10:43 pm

richmond62 wrote:
Tue Apr 07, 2020 4:37 pm
What does
spilling instrument
mean?
Is this a trick question? Typo somewhere?
My GitHub Repos: https://github.com/PaulMcClernan/
Related YouTube Videos: PlayList

Post Reply

Return to “Using Plugins for Customizing the LiveCode IDE”