handling of mouse messages in widgets

LiveCode Builder is a language for extending LiveCode's capabilities, creating new object types as Widgets, and libraries that access lower-level APIs in OSes, applications, and DLLs.

Moderators: LCMark, LCfraser

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: handling of mouse messages in widgets

Post by monte » Mon Jun 01, 2015 7:46 am

Hmm... don't you want widgets to reliably get the messages whether they are handled by the script or not? Unless I'm misunderstanding your post you are introducing the limitation with behaviours that then required the introduction of before/after handlers.

Also it seems a bit odd to depend on handling and not passing mouseDown in order to handle mouseUp. I would have thought the simplest solution would be to have a way for the widget author to disable normal mouse and touch events
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: handling of mouse messages in widgets

Post by LCMark » Mon Jun 01, 2015 9:06 am

@monte: The above scheme does provide a reliable way for widgets to get mouse messages.

The problem with the current system is that mouse events are seen as individual things - they should not be. When you mouseDown you actually start a gesture - if a widget is never told about a particular click gesture it can have no effect on its state; it would be as if the click never happened which is precisely the semantic you want if script decides it wants to completely handle the mouse click event.

Allowing widgets to choose how script interacts with the core mouse messages means that widgets will behave inconsistently. By giving script first pass at the 'gestures' (sequences of down / up type events) it means that all widgets work uniformly with script; and (because things are viewed as gestures, rather than individual events) you can't break a widget by blocking a gesture in script (since the widget won't even be aware it will ever have happened).

Note: I've used 'mouse click gestures' as an example here, but it applies to key events, touch events and any sequence of actionable UI events. (By actionable here I mean events which explicitly indicate a user has performed an action rather than passive notificiation events such as mouseMove / Enter / Leave).

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: handling of mouse messages in widgets

Post by monte » Mon Jun 01, 2015 12:03 pm

Perhaps I'm not understanding what you are proposing. How can a widget author reliably handle an event if a script can handle and not pass it? This was one of the worst things about behaviors.
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: handling of mouse messages in widgets

Post by LCMark » Mon Jun 01, 2015 6:59 pm

Unreliability only comes about when messages are partially blocked - for example if a mouseEnter is blocked by a script but not the mouseLeave, or if a mouseDown is blocked by script but not the mouseUp. If we think of sequences of related events as gestures - which can be blocked as a whole or not at all then I'm pretty sure those problems go away.

If script has an element of control over what gestures a widget receives to process then it means that widgets become a great deal more adaptable in their (LCS) environment - it seems to me that the script of an object should be the natural master of the gestures that a widget receives as the widgets are embedded in the LCS environment rather than the other way round. This also makes sense from a point of view of consistency - if LCS can block gestures that a widget receives then it means that, by default, widgets all work in a consistent way from the script point of view and event handling.

The reason why I don't think there's any problem with 'reliability' is simply that if a widget never sees a mouse click gesture (i.e. mouseDown ... mouseUp) then it can't have had any impact on its state as it has no way of knowing that such a thing ever occurred so it is forever none-the-wiser. However, that ability from a script point of view is actually quite powerful - it means that controls can be adapted from script without having to dig into / customize / plead with the widget author to add a specific feature (in terms of event handling etc.) they might need.

Another argument that things should be this way round is also provided by wanting to integrate native views into LiveCode in a natural way. Any interception of events that are targetted at native views has to be before they are dispatched to the view - this is the only point at which the engine could pass them through script because as as soon as they get delivered to the native view they disappear into that native view's rabbit hole. Thus with things the proposed way round we should be able to make widgets work identically regardless of whether they are based on native views or not.

I'm pretty sure the proposal actually adds quite a strong feature which isn't entirely there (or consistent) at the moment but we see in some places such as the ability to block a key event inserting a character into a field by blocking the appropriate key message.

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: handling of mouse messages in widgets

Post by monte » Mon Jun 01, 2015 9:16 pm

Hmm... This scares me a bit. I'm imagining a heap of bug reports from people that break the expected behavior of a control after implementing a handler. Imagine a button that should change its appearance on mouseDown etc.

Perhaps the corresponding widget handler could have a passed parameter so it's up to the widget author how to handle whether the event was trapped???

A good test for this idea comes in whether you think you could implement a consistent behavior for legacy controls. Imagine be quantity of buttons that break from unpassed mouseDowns alone...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: handling of mouse messages in widgets

Post by LCMark » Tue Jun 02, 2015 9:01 am

The question to ask is this: why would you put a mouseDown (or mouseUp) handler on a widget that was designed to be a 'push button'?

Such a widget is highly specialized for a specific role - it would have a hover state, a depressed state, and dispatch a 'clicked' message when it received a mouse click gesture. In the above world, if someone puts a 'mouseDown' handler in such an object's script then they must have good reason to - part of the reason being they want to handle the mouse click in script rather than letting the widget handle it, and if they do want the widget to still handle the mouseDown and gesture they just need to pass it.

I don't think the comparison to legacy controls is actually all that useful here: people use the existing controls in the ways they need to in order to achieve their ends working with the way they currently work thus I'd fully expect scripts to break if we changed the behavior there - (legacy) buttons are used for all kinds of things because they are not specialized.

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: handling of mouse messages in widgets

Post by monte » Tue Jun 02, 2015 10:56 am

Maybe it's just me but if I were to implement a button I'd like my users to be able to use the regular mouse event messages and any other common type of event message in the usual way without breaking the widget functionality. Even for personal use I don't want to have to remember a different api when the standard messages would do.
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

trevordevore
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1005
Joined: Sat Apr 08, 2006 3:06 pm
Location: Overland Park, Kansas
Contact:

Re: handling of mouse messages in widgets

Post by trevordevore » Tue Jun 02, 2015 12:08 pm

@Monte - My feeling is that standard messages, like legacy controls, are overloaded. As developers we have had to deal with these objects which had to be all things to all people. We have to understand how the events work and then figure out how to best use them to customize behavior. Yesterday I swapped a legacy custom control group used as a navigation bar out for a widget. I had to handle navigation in different mouse events based on whether it was Mac or Windows in order to make the control work correctly. Now I just respond to a <navigate> message and the widget works great everywhere. Much simpler and much more understandable.

With widgets we are entering a world where a push button is just a push button, nothing more. It isn't a menu, a checkbox, or a radio button. When an end user clicks on a push button it has defined visual behavior but only one event where the application should respond. That is when the button is <clicked>. If I were designing a menu widget that the developer could configure right before the being displayed I might send a <configureMenu> message. No need to teach the developer that you can configure a menu in mouseDown right before it is displayed. To me that makes so much more sense.

I think that handling the individual messages of a mouse click gesture will be an outlying case that does something out of the ordinary. If you need to do it great, just pass the mouseDown message and your widget gets the event and works 100% correctly. Don't pass it and the widget has no idea the mouse button was pushed down and released.

Widgets are an opportunity to rid ourselves of this notion of limited controls that must do everything. I don't think that the way widgets work needs to conform to this antiquated approach to controls that we currently have in LC. My feeling is that if widgets behave a certain way just because that is how legacy controls behave then we've lost a golden opportunity to create a sane development system. Once 8 comes out legacy controls (most likely) won't be used to create new projects. It will be all widgets. For those of us who've been around a long time there will be a small learning curve. I don't think it will be a difficult one, however. For new users I think widgets will make much more sense. They will come to the application knowing that you just look in the docs for the messages the widget sends and respond to those.
Trevor DeVore
ScreenSteps - https://www.screensteps.com

LiveCode Repos - https://github.com/search?q=user%3Atrevordevore+topic:livecode
LiveCode Builder Repos - https://github.com/search?q=user%3Atrevordevore+topic:livecode-builder

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: handling of mouse messages in widgets

Post by monte » Tue Jun 02, 2015 1:14 pm

How would your nav bar work if someone implements an unpassed mouseDown handler for some bizarre reason? Would you want that to break your seemingly unrelated navigate handler and any appearance changes? Lets imagine they wanted to blur the background from mouseDown until the navigation is complete or something. Your original widget never considered such an edge case but suddenly things are broken for the user.
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: handling of mouse messages in widgets

Post by LCMark » Tue Jun 02, 2015 1:24 pm

@monte: Isn't there a confusion there between 'user' and 'developer'. The *developer* wants to perform an action on mouseDown for the navbar - so the *developer* puts in a mouseDown handler which passes the message as its augmenting the gesture, not blocking it. Remember that we are talking about a general mechanism pertinent to all widgets - I think people will learn quite quickly (should the blocking semantic be implemented) that you need to pass mouse events if you want widgets to act on them.

Note: The point I'm trying to make here is that the 'developer' is using the widget in their stack and so one would assume would have familiarised themselves with how widgets work.

trevordevore
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1005
Joined: Sat Apr 08, 2006 3:06 pm
Location: Overland Park, Kansas
Contact:

Re: handling of mouse messages in widgets

Post by trevordevore » Tue Jun 02, 2015 1:28 pm

@Monte - developers can already alter behavior of controls by not passing messages. That is part of the engine design. With the proposed behavior there are no unbalanced messages which is really the important part. In your hypothetical the developer would see that the toolbar wasn't responding to anything. No visual feedback, no navigate message. To un-break it they would just pass mouseDown.

It doesn't really make sense to blur the background in a mouseDown that is trapped prior to the toolbar control though. You blur the UI when the user takes an action that changes the UI. That action would occur in <navigate> for the tab control, not mouseDown.

That being said, I understand the point you are trying to make. I think it can be difficult to determine which old engine behaviors we need to hold onto and what the proper behavior should be going forward.
Last edited by trevordevore on Tue Jun 02, 2015 1:37 pm, edited 1 time in total.
Trevor DeVore
ScreenSteps - https://www.screensteps.com

LiveCode Repos - https://github.com/search?q=user%3Atrevordevore+topic:livecode
LiveCode Builder Repos - https://github.com/search?q=user%3Atrevordevore+topic:livecode-builder

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: handling of mouse messages in widgets

Post by LCMark » Tue Jun 02, 2015 1:34 pm

I should also say that there is a choice here...

I think it is better to think of script as wrapping widgets so the message order:

mouseDown to script
OnMouseDown to widget
OnMouseUp to widget
mouseUp to script

Seems to be correct to me.

The choice that comes about is whether script should have the option to block messages. The 'legacy' controls do this somewhat inconsistently, but we have a chance with widgets to make it 100% consistent. It seems to me that (in terms of adaptability) having a blocking ability actually adds a potentially useful feature which we don't really have at the moment.

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: handling of mouse messages in widgets

Post by monte » Tue Jun 02, 2015 9:40 pm

Hmm... well it doesn't seem like I'm going to sway anybody here. I still think I'd rather the widget author have control of the impact of an unpassed event rather than the scripter. I guess if this were platform wide I'd cope but I suspect that will be impossible. Unpassed resizeStack stops a window from resizing? So much will break. The idea that this concept will only be applied to widgets so they are 100% consistent but 99% inconsistent with the rest of the platform doesn't appeal that much. Maybe the big script translation thing that's happening later might help here. Any unpassed messages that didn't have special meaning for being unpassed in the engine already could have a new line included at the end of the handler with a new command `pass <message> to engine` or something like that.

@trevordevore I did say the blur thing was an edge case ;-) To be honest I struggled to come up with something which probably means I don't have much of a point...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

Martin Koob
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 256
Joined: Sun May 27, 2007 8:19 pm

Re: handling of mouse messages in widgets

Post by Martin Koob » Wed Jun 03, 2015 5:50 pm

I started this thread but haven't commented much as the level of the discussion has gone above my pay grade :D

However I just want to respond to Monte's point regarding the bug reports to widget authors that could result from allowing the 'developer' (using Mark's distinction from 'user') to trap the mouse message in the widget script thus breaking the intended function of the widget. I think developers will be aware if they add a mouseDown or mouseUp handler to the widget script they will alter the behaviour of the widget possibly breaking it. For the most part I think people will use the widget for the intended function and likely won't add code that might break it. If they do so they should know they are the ones who broke it (not the author of the widget) and they are the ones who have to fix it by passing the mouse messages.

One use case that I can think of that demonstrates this issue of adding mouseDown, mouseUp mouseMove and mouseEnter handlers to add features not intended by the widget author would be using to the forthcoming cross platform player widget. The player widget will be one with lots of moving parts, play/stop button, slider, volume control etc. Suppose I wanted to have a way to know if the user interacted with the player in any way by clicking on one of the controls or not so I can indicate that interaction with the player by having an outerGlow around the widget that changes colours.

So in the script of the player widget I would have

Code: Select all

on mouseEnter
   set the outerglow["color"] of player 1 to blue
   pass mouseEnter
end mouseEnter

on mouseLeave
   set the outerglow of player 1 to empty
   pass mouseLeave
end mouseLeave

on mouseDown
   set the outerglow["color"] of player 1 to "dark green"
   pass mouseDown
end mouseDown

on mouseUp
   set the outerglow["color"] of player 1 to "green"
   pass mouseUp
end mouseUp
If, with how widgets work in LC8 DP2, the widget's script does not receive the mouseUp etc. message as the author of the widget had not included the onMouseup() handler with pass mouseUp onMouseDown()...etc. in the widget's implementation, then I would not be able to do that.

If Mark's suggestion is implemented and I can add the mouse Message handlers to the widget script, as the developer I would know that for the controls within the player widget to work properly I would have to pass whatever mouse messages that I handle in the widget or those widget controls would cease to work as the script has now taken control of the mouse gesture.

@Mark one question I have is that once there is a mouse message handled in the widget script by any one of the mouse messages i.e. mouseUp, and the script takes control of the entire mouse click gesture would I have to pass just the mouseUp message that I handled to have the widget implantation receive all the mouse messages in the gesture or would I have to pass each one in the gesture i.e. mouseDown, mouseStilldown, mouseUp/mouseRelase?

@Mark a second question is would I have to pass mouseEnter and mouseLeave messages?

One thing I noticed while I was working on this is that if you put the above code in the player control in LC 6.7 the start/stop controls etc work without passing the mouseUp, mouseDown etc. So there would be a difference from the current behaviour for existing LiveCode users to learn. I think that is not as bad as having a widget where the mouse messages are not received by the widget script. That was the thing that I found disconcerting when I was trying out the widgets.

Martin

trevordevore
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1005
Joined: Sat Apr 08, 2006 3:06 pm
Location: Overland Park, Kansas
Contact:

Re: handling of mouse messages in widgets

Post by trevordevore » Fri Aug 14, 2015 10:07 pm

@LCMark - have widgets been updated to use the new event model you proposed in this thread? I'm using the latest in develop and it doesn't seem that mouseMove is being sent to the widget instance. I added a mouseMove handler to a widget using LC Script and it isn't triggered. Should it be?
Trevor DeVore
ScreenSteps - https://www.screensteps.com

LiveCode Repos - https://github.com/search?q=user%3Atrevordevore+topic:livecode
LiveCode Builder Repos - https://github.com/search?q=user%3Atrevordevore+topic:livecode-builder

Post Reply

Return to “LiveCode Builder”